# Adaptive Lossless Image Data Compression Method Inferring Data Entropy by Applying Deep Neural Network

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

- Applying a DNN that infers a data compression algorithm from a data pattern of a data block divided from the original data, we have developed a novel method for lossless data compression that achieves a better compression ratio than the worst case of available data compression programs.
- We have applied the proposed method to an image data compression and achieved a better compression ratio than the case when we select the worst compression program available in a computer system to the entire image data.

## 2. Backgrounds and Definitions

#### 2.1. Lossless Data Compression Algorithms

#### 2.2. Lossless Data Compression with Neural Network

## 3. Lossless Image Data Compression with Predicting Entropy by Neural Network

#### 3.1. System Overview

#### 3.2. Deep Neural Network to Predict Data Entropy

## 4. Experimental Evaluations

## 5. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Reddy, B.V.; Reddy, P.B.; Kumar, P.S.; Reddy, A.S. Lossless Compression of Medical Images for Better Diagnosis. In Proceedings of the 2016 IEEE 6th International Conference on Advanced Computing (IACC), Bhimavaram, India, 27–28 February 2016; pp. 404–408. [Google Scholar] [CrossRef]
- Gómez-Brandón, A.; Paramá, J.R.; Villalobos, K.; Illarramendi, A.; Brisaboa, N.R. Lossless compression of industrial time series with direct access. Comput. Ind.
**2021**, 132, 103503. [Google Scholar] [CrossRef] - Gia, T.N.; Qingqing, L.; Queralta, J.P.; Tenhunen, H.; Zou, Z.; Westerlund, T. Lossless Compression Techniques in Edge Computing for Mission-Critical Applications in the IoT. In Proceedings of the 2019 Twelfth International Conference on Mobile Computing and Ubiquitous Network (ICMU), Kathmandu, Nepal, 4–6 November 2019; pp. 1–2. [Google Scholar] [CrossRef]
- Huang, W.; Wang, W.; Xu, H. A Lossless Data Compression Algorithm for Real-time Database. In Proceedings of the 2006 6th World Congress on Intelligent Control and Automation, Dalian, China, 21–23 June 2006; Volume 2, pp. 6645–6648. [Google Scholar] [CrossRef]
- Nivedha, B.; Priyadharshini, M.; Thendral, E.; Deenadayalan, T. Lossless Image Compression in Cloud Computing. In Proceedings of the 2017 International Conference on Technical Advancements in Computers and Communications (ICTACC), Melmaurvathur, India, 10–11 April 2017; pp. 112–115. [Google Scholar] [CrossRef]
- Routray, S.K.; Javali, A.; Sharmila, K.P.; Semunigus, W.; Pappa, M.; Ghosh, A.D. Lossless Compression Techniques for Low Bandwidth Networks. In Proceedings of the 2020 3rd International Conference on Intelligent Sustainable Systems (ICISS), Thoothukudi, India, 3–5 December 2020; pp. 823–828. [Google Scholar] [CrossRef]
- Li, M.; Vitányi, P.M. An Introduction to Kolmogorov Complexity and Its Applications; Springer: New York, NY, USA, 2008. [Google Scholar]
- Willems, F.; Shtarkov, Y.; Tjalkens, T. The context-tree weighting method: Basic properties. IEEE Trans. Inf. Theory
**1995**, 41, 653–664. [Google Scholar] [CrossRef][Green Version] - Cleary, J.; Witten, I. Data Compression Using Adaptive Coding and Partial String Matching. IEEE Trans. Commun.
**1984**, 32, 396–402. [Google Scholar] [CrossRef][Green Version] - Storer, J.A.; Szymanski, T.G. Data Compression via Textual Substitution. J. ACM
**1982**, 29, 928–951. [Google Scholar] [CrossRef] - Oord, A.V.; Kalchbrenner, N.; Kavukcuoglu, K. Pixel Recurrent Neural Networks. In Proceedings of the 33rd International Conference on Machine Learning, New York, NY, USA, 19–24 June 2016; Volume 48, pp. 1747–1756. [Google Scholar]
- Oord, A.v.d.; Kalchbrenner, N.; Vinyals, O.; Espeholt, L.; Graves, A.; Kavukcuoglu, K. Conditional Image Generation with PixelCNN Decoders. In Proceedings of the 30th International Conference on Neural Information Processing Systems, Barcelona, Spain, 5–10 December 2016; Curran Associates Inc.: Red Hook, NY, USA, 2016; pp. 4797–4805. [Google Scholar]
- Mentzer, F.; Agustsson, E.; Tschannen, M.; Timofte, R.; Gool, L.V. Practical Full Resolution Learned Lossless Image Compression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, 16–20 June 2019. [Google Scholar]
- Mahoney, M.V. Fast Text Compression with Neural Networks. In Proceedings of the Thirteenth International Florida Artificial Intelligence Research Society Conference, Orlando, FL, USA, 17–19 May 2000; AAAI Press: Menlo Park, CA, USA, 2000; pp. 230–234. [Google Scholar]
- Storer, J.A.; Szymanski, T.G. Syntactically Informed Text Compression with Recurrent Neural Networks. arXiv
**2016**, arXiv:1608.02893. [Google Scholar] - Mahoney, M. Adaptive Weighing of Context Models for Lossless Data Compression; Technical Report of Florida Institute of Technology: Melbourne, FL, USA, 2016. [Google Scholar]
- Goyal, M.; Tatwawadi, K.; Chandak, S.; Ochoa, I. DeepZip: Lossless Data Compression Using Recurrent Neural Networks. In Proceedings of the 2019 Data Compression Conference (DCC), Snowbird, UT, USA, 26–29 March 2019; p. 575. [Google Scholar] [CrossRef][Green Version]
- CMIX. Available online: https://github.com/byronknoll/cmix (accessed on 8 February 2022).
- Nagoor, O.H.; Whittle, J.; Deng, J.; Mora, B.; Jones, M.W. Lossless Compression For Volumetric Medical Images Using Deep Neural Network With Local Sampling. In Proceedings of the 2020 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, UAE, 25–28 September 2020; pp. 2815–2819. [Google Scholar] [CrossRef]
- Lu, G.; Yang, R.; Wang, S.; Liu, S.; Timofte, R. Deep Learning for Visual Data Compression. In Proceedings of the 29th ACM International Conference on Multimedia, New York, NY, USA, 8–12 March 2021; Association for Computing Machinery: New York, NY, USA, 2021; pp. 5683–5685. [Google Scholar]
- Zhang, C.; Zhang, S.; Carlucci, F.M.; Li, Z. OSOA: One-Shot Online Adaptation of Deep Generative Models for Lossless Compression. arXiv
**2021**, arXiv:2111.01662. [Google Scholar] - Schiopu, I.; Munteanu, A. Deep-Learning-Based Lossless Image Coding. IEEE Trans. Circuits Syst. Video Technol.
**2020**, 30, 1829–1842. [Google Scholar] [CrossRef] - Luo, J.; Wu, J.; Zhao, S.; Wang, L.; Xu, T. Lossless compression for hyperspectral image using deep recurrent neural networks. Int. J. Mach. Learn. Cybern.
**2019**, 10, 2619–2629. [Google Scholar] [CrossRef] - Yamagiwa, S.; Ichinomiya, Y. Stream-Based Visually Lossless Data Compression Applying Variable Bit-Length ADPCM Encoding. Sensors
**2021**, 21, 4602. [Google Scholar] [CrossRef] [PubMed] - Mercat, A.; Viitanen, M.; Vanne, J. UVG Dataset: 50/120fps 4K Sequences for Video Codec Analysis and Development. In Proceedings of the 11th ACM Multimedia Systems Conference MMSys’20, Istanbul, Turkey, 8–11 June 2020; ACM: New York, NY, USA, 2020; pp. 297–302. [Google Scholar] [CrossRef]
- Ultra Video Group. Available online: http://ultravideo.fi/ (accessed on 8 February 2022).

**Figure 6.**Comparisons of compression ratios with/without the DNN inference in the case of Bosphorus.

**Figure 7.**Comparisons of compression ratios with/without the DNN inference in the case of ReadySetGo.

**Figure 8.**Comparisons among distributions of compression programs used for training and inferred by DNN (Sky).

**Figure 9.**Comparisons among distributions of compression programs used for training and inferred by DNN (Beauty).

**Figure 10.**Comparisons among distributions of compression programs used for training and inferred by DNN (Bosphorus).

**Figure 11.**Comparisons among distributions of compression programs used for training and inferred by DNN (ReadySetGo).

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Yamagiwa, S.; Yang, W.; Wada, K. Adaptive Lossless Image Data Compression Method Inferring Data Entropy by Applying Deep Neural Network. *Electronics* **2022**, *11*, 504.
https://doi.org/10.3390/electronics11040504

**AMA Style**

Yamagiwa S, Yang W, Wada K. Adaptive Lossless Image Data Compression Method Inferring Data Entropy by Applying Deep Neural Network. *Electronics*. 2022; 11(4):504.
https://doi.org/10.3390/electronics11040504

**Chicago/Turabian Style**

Yamagiwa, Shinichi, Wenjia Yang, and Koichi Wada. 2022. "Adaptive Lossless Image Data Compression Method Inferring Data Entropy by Applying Deep Neural Network" *Electronics* 11, no. 4: 504.
https://doi.org/10.3390/electronics11040504