Noise-Aware Hybrid Compression of Deep Models with Zero-Shot Denoising and Failure Prediction
Abstract
1. Introduction
- We identify, characterize, and quantify the universal vulnerability of deep image compression models to high-noise, high-entropy, and high-frequency images, which is consistent across different model architectures.
- The High-Frequency Vulnerability Index (HFVI) is proposed as a metric that integrates frequency energy, encoder Jacobian sensitivity, and texture entropy to enable compression failure prediction.
- A noise-aware hybrid compression pipeline is designed, which integrates zero-shot denoising and a lightweight selector, which selectively improves compression quality without retraining.
- Experiments on synthetic noise, NIND, and artificial noisy datasets, along with cross-validation, and intensity-profile studies, validating both effectiveness and practicality.
2. Related Work
2.1. Image Compression and Learning-Based Codecs
2.2. Lossy Compression of Noisy Images
2.3. Image Denoising and Zero-Shot Denoising
2.4. Selection of Datasets
3. Methodology and Proposed Algorithm
3.1. Noise Synthesis
3.2. Compression Failure Characterization
3.3. High-Frequency Vulnerability Index (HFVI)
3.4. Failure Mechanisms in Deep Image Compression
3.5. Zero-Shot Denoising as Preprocessing
3.6. Noise-Aware Compression Selector
4. Experimental Results and Discussion
4.1. Comparison of Traditional and Deep Compression Models on Clean and Noisy Data
4.2. Threshold Definition and Ablation of HFVI
4.3. Pixel-Level Intensity Profile Analysis
4.4. Compression Behavior Across Synthetic Noise and Real-World Scenarios
4.5. Experiments of Dynamic Denoising Decisions
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Zhou, L.; Cai, C.; Gao, Y.; Su, S.; Wu, J. Variational autoencoder for low bit-rate image compression. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, UT, USA, 18–22 June 2018; pp. 2617–2620. [Google Scholar]
- Ballé, J.; Minnen, D.; Singh, S.; Hwang, S.J.; Johnston, N. Variational image compression with a scale hyperprior. In Proceedings of the International Conference on Learning Representations (ICLR), Vancouver, BC, Canada, 30 April–3 May 2018. [Google Scholar]
- Cheng, Z.; Sun, H.; Takeuchi, M.; Katto, J. Deep convolutional autoencoder-based lossy image compression. In Proceedings of the Picture Coding Symposium (PCS), San Francisco, CA, USA, 24–27 June 2018; pp. 253–257. [Google Scholar]
- Choi, Y.; El-Khamy, M.; Lee, J. Variable rate deep image compression with a conditional autoencoder. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea, 27 October–2 November 2019; pp. 3146–3154. [Google Scholar]
- Liu, J.; Sun, H.; Katto, J. Learned image compression with mixed transformer-CNN architectures. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, QC, Canada, 17–24 June 2023; pp. 14388–14397. [Google Scholar]
- Han, L.; Shaohui, L.; Wenrui, D.; Chenglin, L.; Zou, J.; Xiong, H. Frequency-aware transformer for learned image compression. In Proceedings of the Twelfth International Conference on Learning Representations, Vienna, Austria, 7–11 May 2024. [Google Scholar]
- ISO/IEC 10918-1:1992; Information Technology-Digital Compression and Coding of Continuous-Tone Still Images: Requirements and Guidelines. ISO: Geneva, Switzerland, 1992.
- ISO/IEC 15444-1:2000; Information Technology-JPEG 2000 Image Coding System: Core Coding System. ISO: Geneva, Switzerland, 2000.
- Google Inc. WebP Compression Format. Available online: https://developers.google.com/speed/webp/ (accessed on 18 October 2025).
- Kodak True-Color Image Suite. Available online: http://r0k.us/graphics/kodak/ (accessed on 18 October 2025).
- Challenge on Learned Image Compression (CLIC). Available online: https://archive.compression.cc/2024/tasks/ (accessed on 18 October 2025).
- Kuznetsova, A.; Rom, H.; Alldrin, N.; Uijlings, J.; Krasin, I.; Pont-Tuset, J.; Kamali, S.; Popov, S.; Malloci, M.; Kolesnikov, A.; et al. The open images dataset V4: Unified image classification, object detection, and visual relationship detection at scale. Int. J. Comput. Vis. 2020, 128, 1956–1981. [Google Scholar] [CrossRef]
- Alshina, E.; Ascenso, J.; Ebrahimi, T. JPEG AI: The first international standard for image coding based on an end-to-end learning-based approach. IEEE MultiMedia 2024, 31, 60–69. [Google Scholar] [CrossRef]
- Brummer, B.; De Vleeschouwer, C. Natural image noise dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA, 16–17 June 2019. [Google Scholar]
- Duan, Z.; Lu, M.; Ma, Z.; Zhu, F. Lossy image compression with quantized hierarchical VAEs. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, 2–7 January 2023; pp. 198–207. [Google Scholar]
- Lu, J.; Zhang, L.; Zhou, X.; Li, M.; Li, W.; Gu, S. Learned image compression with dictionary-based entropy model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 13–15 June 2025; pp. 12850–12859. [Google Scholar]
- Ge, Z.; Ma, S.; Gao, W.; Pan, J.; Jia, C. NLIC: Non-uniform quantization-based learned image compression. IEEE Trans. Circuits Syst. Video Technol. 2024, 34, 9647–9663. [Google Scholar] [CrossRef]
- Cheng, K.L.; Xie, Y.; Chen, Q. Optimizing image compression via joint learning with denoising. In Proceedings of the European Conference on Computer Vision (ECCV), Glasgow, UK, 23–28 August 2022; Springer: Cham, Switzerland, 2022; pp. 56–73. [Google Scholar]
- Cai, S.; Liang, X.; Cao, S.; Yan, L.; Zhong, S.; Chen, L.; Ziu, X. Powerful lossy compression for noisy images. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME), Niagara Falls, ON, Canada, 15–19 July 2024; pp. 1–6. [Google Scholar]
- Qian, S.E.; Bergeron, M.; Cunningham, I.; Gagnon, L.; Hollinger, A. Near lossless data compression onboard a hyperspectral satellite. IEEE Trans. Aerosp. Electron. Syst. 2006, 42, 851–866. [Google Scholar] [CrossRef]
- Zhang, J.; Zhang, H.; Shi, X.; Peng, X.; Geng, S. Compressed sensing for high-noise astronomical image recovery. J. Electron. Imaging 2019, 28, 053026. [Google Scholar] [CrossRef]
- Zhang, L.; Zhou, Q.; Liu, R.; Huyan, L.; Liu, J.; Zhang, Y. The blind spot of deep image compression: Why JPEG2000 still wins in high-entropy high-frequency regions. Chin. J. Stereol. Image Anal. 2025, 30, 187–197. [Google Scholar]
- Fabrice, B. Better Portable Graphics Image Format. Available online: https://bellard.org/bpg/ (accessed on 18 October 2025).
- Alliance for Open Media. AV1 Image File Format (AVIF). Available online: https://aomediacodec.github.io/av1-avif/ (accessed on 18 October 2025).
- Bross, B.; Chen, J.; Ohmm, J.R.; Sullivan, G.J.; Wang, Y.-K. Developments in international video coding standardization after AVC, with an overview of versatile video coding (VVC). Proc. IEEE 2021, 109, 1463–1493. [Google Scholar] [CrossRef]
- Cheng, Z.; Zhou, M.; Guo, J.; Yuan, J.; Ji, Y.; Zhang, Y. Steering one-step diffusion model with fidelity-rich decoder for fast image compression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA, 13–19 June 2020; pp. 7939–7948. [Google Scholar]
- International Telecommunication Union (ITU). Recommendation ITU-T T.840.1. JPEG AI Learning-Based Image Coding System (JPEG-AI). Available online: https://gitlab.com/wg1/jpeg-ai/jpeg-ai-reference-software.git (accessed on 18 October 2025).
- Wei, H.; Zhou, Y.; Jia, Y.; Ge, C.; Anwar, S.; Mian, A. A lightweight model for perceptual image compression via implicit priors. Neural Netw. 2025, 108279, 0893–6080. [Google Scholar] [CrossRef]
- Zeng, F.; Tang, H.; Shao, Y.; Chen, S.; Shao, L.; Wang, Y. MambaIC: State space models for high-performance learned image compression. arXiv 2025, arXiv:2503.12461. [Google Scholar]
- Al-Shaykh, O.K.; Mersereau, R.M. Lossy compression of noisy images. IEEE Trans. Image Process. 1998, 7, 1641–1652. [Google Scholar] [CrossRef]
- Chang, S.G.; Yu, B.; Vetterli, M. Adaptive wavelet thresholding for image denoising and compression. IEEE Trans. Image Process. 2000, 9, 1532–1546. [Google Scholar] [CrossRef] [PubMed]
- Brummer, B.; De Vleeschouwer, C. On the importance of denoising when learning to compress images. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, 2–7 January 2023; pp. 2440–2448. [Google Scholar]
- Donoho, D.L. De-noising by soft-thresholding. IEEE Trans. Inf. Theory 2002, 41, 613–627. [Google Scholar] [CrossRef]
- Buades, A.; Coll, B.; Morel, J.M. A review of image denoising algorithms, with a new one. Multiscale Model. Simul. 2005, 4, 490–530. [Google Scholar] [CrossRef]
- Zhang, K.; Zuo, W.; Chen, Y.; Meng, D.; Zhang, L. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE Trans. Image Process. 2017, 26, 3142–3155. [Google Scholar] [CrossRef]
- Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 5–9 October 2015; pp. 234–241. [Google Scholar]
- Chang, X.; Wang, X.; Huang, X.; Yan, M.; Cheng, L. Multi-scale differentiated network with spatial-spectral co-operative attention for hyperspectral image denoising. Appl. Sci. 2025, 15, 8648. [Google Scholar] [CrossRef]
- Ma, Q.; Jiang, J.; Zhou, X.; Liang, P.; Liu, X. Pixel2pixel: A pixelwise approach for zero-shot single image denoising. IEEE Trans. Pattern Anal. Mach. Intell. 2025, 47, 4614–4629. [Google Scholar] [CrossRef]
- Huang, T.; Li, S.; Jia, X.; Lu, H.; Lu, J. Neighbor2neighbor: Self-supervised denoising from single noisy images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 20–25 June 2021; pp. 14781–14790. [Google Scholar]
- Mansour, Y.; Heckel, R. Zero-shot noise2noise: Efficient image denoising without any data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 17–23 June 2023; pp. 14018–14027. [Google Scholar]
- Lin, T.Y.; Maire, M.; Belongie, S.; Bourdev, L.; Girshick, R.; Hays, J.; Perona, P.; Ramanan, D.; Zitnick, C.L.; Dollár, P. Microsoft COCO: Common objects in context. In Proceedings of the European Conference on Computer Vision (ECCV), Zurich, Switzerland, 6–12 September 2014; pp. 740–755. [Google Scholar]
- Machado, G.; Ferreira, E.; Nogueira, K.; Oliveira, H.; Brito, M.; Gama, P.H.T.; dos Santos, J.A. AiRound and CV-BrCT: Novel multiview datasets for scene classification. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2020, 14, 488–503. [Google Scholar] [CrossRef]
- Abdelhamed, A.; Lin, S.; Brown, M.S. A high-quality denoising dataset for smartphone cameras. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, 18–23 June 2018; pp. 1692–1700. [Google Scholar]
- Abdelhamed, A.; Afifi, M.; Timofte, R.; Brown, M.S.; Cao, Y.; Zhang, Z.; Zuo, W.; Zhang, X.; Liu, J.; Chen, W.; et al. NTIRE 2020 challenge on real image denoising: Dataset, methods and results. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA, 14–19 June 2020; pp. 496–497. [Google Scholar]
- Ignatov, A.; Kobyshev, N.; Timofte, R. Noise-aware training for deep image compression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA, 14–19 June 2020. [Google Scholar]
- Mildenhall, B.; Barron, J.T.; Chen, J.; Sharlet, D.; Ng, R.; Carroll, R. Burst denoising with kernel prediction networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, 18–23 June 2018; pp. 2502–2510. [Google Scholar]
- Finlay, C.; Jacobsen, J.H.; Nurbekyan, L.; Oberman, A.M. How to train your neural ODE: The world of Jacobian and kinetic regularization. In Proceedings of the International Conference on Machine Learning (ICML), Vienna, Austria, 12–18 July 2020; pp. 3154–3164. [Google Scholar]
- Shamir, L.; Wolkow, C.A.; Goldberg, I.G. Quantitative measurement of aging using image texture entropy. Bioinformatics 2009, 25, 3060–3063. [Google Scholar] [CrossRef]
- Chen, T.; Guestrin, C. XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, 13–17 August 2016; pp. 785–794. [Google Scholar]
- Wang, Z.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 2004, 13, 600–612. [Google Scholar] [CrossRef]
- Zhang, R.; Isola, P.; Efros, A.A.; Shechtman, E.; Wang, O. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA, 18–23 June 2018; pp. 586–595. [Google Scholar]
- Ding, K.; Ma, K.; Wang, S.; Simoncelli, E.P. Image quality assessment: Unifying structure and texture similarity. IEEE Trans. Pattern Anal. Mach. Intell. 2020, 44, 2567–2581. [Google Scholar] [CrossRef]
- Sheikh, H.R.; Bovik, A.C. A visual information fidelity approach to video quality assessment. First Int. Workshop Video Process. Qual. Metrics Consum. Electron. 2005, 7, 2117–2128. [Google Scholar]
- Laparra, V.; Ballé, J.; Berardino, A.; Simoncelli, E.P. Perceptual image quality assessment using a normalized Laplacian pyramid. Electron. Imaging 2016, 28, 1–6. [Google Scholar] [CrossRef]
- Manoj, D.; Prabhishek, S. CT image denoising using multivariate model and method noise thresholding in non-subsampled shearlet domain. Biomed. Signal Process. Control 2020, 101754, 1746–8094. [Google Scholar]









| Dataset | #Images | Resolution | Characteristics | Purpose |
|---|---|---|---|---|
| Kodak [10] | 24 | 768 × 512 | High-quality natural photographs | Classical benchmark for compression |
| CLIC 2024 [11] | 63 | 2–6 MP * | Diverse high-resolution natural images | Evaluation of high-resolution performance |
| COCO subset [41] | 10k+ | Variable | Diverse natural scenes, large scale | Evaluation of generalization |
| AiRound [42] | 11k+ | 500 × 500 | Remote sensing satellite and aerial images | Cross-domain validation for remote sensing |
| NIND [14] | 836 | Variable | Real noisy-clean image pairs at different ISO levels | Evaluation of robustness to sensor noise |
| SIDD [43,44] | 320 | 5328 × 3000 | Smartphone noise dataset with real captures | Benchmark for denoising-oriented evaluation |
| Alg. | Data | bpp | PSNR ↑ | SSIM ↑ | LPIPS-VGG ↓ | DISTS ↓ |
|---|---|---|---|---|---|---|
| JPEG [7] | Clean GT | 1.842 | 36.580 | 0.956 | 0.285 | 0.210 |
| Noisy Image | 5.437 | 19.389 | 0.680 | 0.307 | 0.295 | |
| JPEG2000 [8] | Clean GT | 2.132 | 37.963 | 0.963 | 0.214 | 0.118 |
| Noisy Image | 5.557 | 29.617 | 0.951 | 0.264 | 0.112 | |
| WebP [9] | Clean GT | 1.851 | 38.240 | 0.967 | 0.220 | 0.125 |
| Noisy Image | 5.466 | 19.502 | 0.653 | 0.295 | 0.188 | |
| Bmshj [2] | Clean GT | 1.681 | 34.342 | 0.964 | 0.224 | 0.138 |
| Noisy Image | 5.143 | 18.868 | 0.609 | 0.294 | 0.169 | |
| Cheng [26] | Clean GT | 1.782 | 38.060 | 0.969 | 0.223 | 0.130 |
| Noisy Image | 1.934 | 17.281 | 0.470 | 0.255 | 0.214 | |
| Qres [15] | Clean GT | 1.923 | 40.240 | 0.976 | 0.213 | 0.111 |
| Noisy Image | 5.528 | 20.827 | 0.698 | 0.245 | 0.127 | |
| JPEG AI [27] | Clean GT | 2.081 | 40.401 | 0.971 | 0.255 | 0.129 |
| Noisy Image | 5.934 | 15.772 | 0.698 | 0.365 | 0.267 | |
| FTIC [6] | Clean GT | 1.051 | 41.914 | 0.971 | 0.178 | 0.116 |
| Noisy Image | 3.634 | 25.941 | 0.810 | 0.212 | 0.155 | |
| ICISP [28] | Clean GT | 0.251 | 36.082 | 0.950 | 0.273 | 0.130 |
| Noisy Image | 0.525 | 20.723 | 0.518 | 0.220 | 0.195 | |
| DCAE [16] | Clean GT | 0.829 | 37.346 | 0.954 | 0.159 | 0.137 |
| Noisy Image | 2.346 | 21.673 | 0.625 | 0.287 | 0.146 |
| Threshold | AUC ↑ | Precision @25% ↑ | Recall @25% ↑ |
|---|---|---|---|
| −0.2 | 0.298 | 0.955 | 0.253 |
| −0.3 | 0.435 | 0.886 | 0.239 |
| −0.5 | 0.471 | 0.864 | 0.235 |
| −1.0 | 0.446 | 0.773 | 0.222 |
| Feature/Variant | Spearman vs. | Kendall | AUC (Failure Detection) ↑ | MAE of | Precision @25% ↑ | Recall @25% ↑ |
|---|---|---|---|---|---|---|
| E (High-frequency Energy) | −0.58 | −0.41 | 0.39 | 0.67 dB | 0.91 | 0.27 |
| H (Texture Entropy) | −0.34 | −0.23 | 0.31 | 0.81 dB | 0.72 | 0.21 |
| J (Jacobian Sensitivity) | −0.27 | −0.19 | 0.29 | 0.84 dB | 0.69 | 0.19 |
| (2-component) | −0.63 | −0.45 | 0.42 | 0.61 dB | 0.88 | 0.26 |
| (2-component) | −0.61 | −0.43 | 0.41 | 0.63 dB | 0.85 | 0.25 |
| (2-component) | 0.46 | 0.32 | 0.35 | 0.76 dB | 0.78 | 0.22 |
| Full HFVI (E + H + J) | −0.71 | −0.52 | 0.47 | 0.55 dB | 0.86 | 0.32 |
| Algorithms | Description | PSNR ↑ | SSIM ↑ | VIF ↓ | NLPD ↓ | LPIPS-Alex ↓ | LPIPS-VGG ↓ |
|---|---|---|---|---|---|---|---|
| DC [18] | Joint Denoising | 26.1360 | 0.7212 | 0.3874 | 1.0630 | 0.2446 | 0.3212 |
| Joint-IC [19] | & Compression | 26.2241 | 0.7260 | 0.3906 | 1.0919 | 0.2407 | 0.3151 |
| JPEG2000 [8] | 24.9971 | 0.6978 | 0.3976 | 2.4521 | 0.2319 | 0.2965 | |
| JPEG AI [27] | 16.0108 | 0.4151 | 0.3003 | 7.5903 | 0.6665 | 0.4939 | |
| Qres [15] | Only | 26.3275 | 0.7371 | 0.4358 | 1.2123 | 0.2051 | 0.2913 |
| FTIC [6] | Compression | 24.4286 | 0.6841 | 0.3679 | 1.2476 | 0.2695 | 0.2839 |
| ICISP [28] | 21.9919 | 0.5384 | 0.2443 | 1.5891 | 0.2811 | 0.3387 | |
| DCAE [16] | 25.7127 | 0.7329 | 0.4122 | 1.0435 | 0.2275 | 0.2880 | |
| P2PA & JPEG2000 | Always | 26.0370 | 0.7025 | 0.4104 | 1.3991 | 0.2784 | 0.3117 |
| P2PA & Qres | Denoising | 26.3802 | 0.7055 | 0.4156 | 1.1854 | 0.3488 | 0.3650 |
| P2PA & DCAE | & Compression | 26.1770 | 0.6976 | 0.4087 | 1.4052 | 0.3617 | 0.4086 |
| Proposed | Hybrid | 27.5016 | 0.7418 | 0.3526 | 0.9928 | 0.2211 | 0.2619 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zhang, L.; Zhou, Q.; Liu, R.; Huyan, L.; Liu, J.; Zhang, Y. Noise-Aware Hybrid Compression of Deep Models with Zero-Shot Denoising and Failure Prediction. Appl. Sci. 2025, 15, 12882. https://doi.org/10.3390/app152412882
Zhang L, Zhou Q, Liu R, Huyan L, Liu J, Zhang Y. Noise-Aware Hybrid Compression of Deep Models with Zero-Shot Denoising and Failure Prediction. Applied Sciences. 2025; 15(24):12882. https://doi.org/10.3390/app152412882
Chicago/Turabian StyleZhang, Lizhe, Quan Zhou, Ruihua Liu, Lang Huyan, Juanni Liu, and Yi Zhang. 2025. "Noise-Aware Hybrid Compression of Deep Models with Zero-Shot Denoising and Failure Prediction" Applied Sciences 15, no. 24: 12882. https://doi.org/10.3390/app152412882
APA StyleZhang, L., Zhou, Q., Liu, R., Huyan, L., Liu, J., & Zhang, Y. (2025). Noise-Aware Hybrid Compression of Deep Models with Zero-Shot Denoising and Failure Prediction. Applied Sciences, 15(24), 12882. https://doi.org/10.3390/app152412882

