Low-Light Image Enhancement via Retinex-Style Decomposition of Denoised Deep Image Prior
Abstract
:1. Introduction
- We propose a novel noise-added RetinexDIP method to enhance images.
- Three components are generated by the DIP network.
- The zero-reference process avoids the risk of overfitting and improves generalization.
- The experimental results show that our method significantly outperforms some current state-of-the-art methods.
2. Materials and Methods
Algorithm 1: our algorithm |
|
3. Experiment
3.1. Settings
3.2. Performance Criteria
3.3. Results
4. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Hu, G.; Yang, Y.; Yi, D.; Kittler, J.; Christmas, W.; Li, S.Z.; Hospedales, T. When face recognition meets with deep learning: An evaluation of convolutional neural networks for face recognition. In Proceedings of the IEEE International Conference on Computer Vision Workshops, Santiago, Chile, 7–13 December 2015; pp. 142–150. [Google Scholar]
- Koch, C.; Georgieva, K.; Kasireddy, V.; Akinci, B.; Fieguth, P. A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure. Adv. Eng. Inform. 2015, 29, 196–210. [Google Scholar] [CrossRef] [Green Version]
- Qayyum, A.; Anwar, S.M.; Awais, M.; Majid, M. Medical image retrieval using deep convolutional neural network. Neurocomputing 2017, 266, 8–20. [Google Scholar] [CrossRef] [Green Version]
- Buch, N.; Velastin, S.A.; Orwell, J. A review of computer vision techniques for the analysis of urban traffic. IEEE Trans. Intell. Transp. Syst. 2011, 12, 920–939. [Google Scholar] [CrossRef]
- Cheng, Z.; Bai, F.; Xu, Y.; Zheng, G.; Pu, S.; Zhou, S. Focusing attention: Towards accurate text recognition in natural images. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 5076–5084. [Google Scholar]
- Ancuti, C.; Ancuti, C.O.; Haber, T.; Bekaert, P. Enhancing underwater images and videos by fusion. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, 16–21 June 2012; pp. 81–88. [Google Scholar]
- Lyu, G.; Huang, H.; Yin, H.; Luo, S.; Jiang, X. A novel visual perception enhancement algorithm for high-speed railway in the low light condition. In Proceedings of the 2014 12th International Conference on Signal Processing (ICSP), Hangzhou, China, 19–23 October 2014; pp. 1022–1025. [Google Scholar]
- Cho, Y.; Jeong, J.; Kim, A. Model-assisted multiband fusion for single image enhancement and applications to robot vision. IEEE Robot. Autom. Lett. 2018, 3, 2822–2829. [Google Scholar]
- Ibrahim, H.; Kong, N.S.P. Brightness preserving dynamic histogram equalization for image contrast enhancement. IEEE Trans. Consum. Electron. 2007, 53, 1752–1758. [Google Scholar] [CrossRef]
- Abdullah-Al-Wadud, M.; Kabir, M.H.; Dewan, M.A.A.; Chae, O. A dynamic histogram equalization for image contrast enhancement. IEEE Trans. Consum. Electron. 2007, 53, 593–600. [Google Scholar] [CrossRef]
- Arici, T.; Dikbas, S.; Altunbasak, Y. A histogram modification framework and its application for image contrast enhancement. IEEE Trans. Image Process. 2009, 18, 1921–1935. [Google Scholar] [CrossRef]
- Pizer, S.M.; Amburn, E.P.; Austin, J.D.; Cromartie, R.; Geselowitz, A.; Greer, T.; ter Haar Romeny, B.; Zimmerman, J.B.; Zuiderveld, K. Adaptive histogram equalization and its variations. Comput. Vision, Graph. Image Process. 1987, 39, 355–368. [Google Scholar] [CrossRef]
- Pisano, E.D.; Zong, S.; Hemminger, B.M.; DeLuca, M.; Johnston, R.E.; Muller, K.; Braeuning, M.P.; Pizer, S.M. Contrast limited adaptive histogram equalization image processing to improve the detection of simulated spiculations in dense mammograms. J. Digit. Imaging 1998, 11, 193–200. [Google Scholar] [CrossRef] [Green Version]
- Celik, T.; Tjahjadi, T. Contextual and variational contrast enhancement. IEEE Trans. Image Process. 2011, 20, 3431–3441. [Google Scholar] [CrossRef] [Green Version]
- Lee, C.; Lee, C.; Kim, C.S. Contrast enhancement based on layered difference representation of 2D histograms. IEEE Trans. Image Process. 2013, 22, 5372–5384. [Google Scholar] [CrossRef] [PubMed]
- Land, E.H. The retinex theory of color vision. Sci. Am. 1977, 237, 108–129. [Google Scholar] [CrossRef] [PubMed]
- Jobson, D.J.; Rahman, Z.U.; Woodell, G.A. Properties and performance of a center/surround retinex. IEEE Trans. Image Process. 1997, 6, 451–462. [Google Scholar] [CrossRef]
- Jobson, D.J.; Rahman, Z.U.; Woodell, G.A. A multiscale retinex for bridging the gap between color images and the human observation of scenes. IEEE Trans. Image Process. 1997, 6, 965–976. [Google Scholar] [CrossRef] [Green Version]
- Wang, S.; Zheng, J.; Hu, H.M.; Li, B. Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Trans. Image Process. 2013, 22, 3538–3548. [Google Scholar] [CrossRef] [PubMed]
- Fu, X.; Zeng, D.; Huang, Y.; Liao, Y.; Ding, X.; Paisley, J. A fusion-based enhancing method for weakly illuminated images. Signal Process. 2016, 129, 82–96. [Google Scholar] [CrossRef]
- Guo, X.; Li, Y.; Ling, H. LIME: Low-light image enhancement via illumination map estimation. IEEE Trans. Image Process. 2016, 26, 982–993. [Google Scholar] [CrossRef] [PubMed]
- Kimmel, R.; Elad, M.; Shaked, D.; Keshet, R.; Sobel, I. A variational framework for retinex. Int. J. Comput. Vis. 2003, 52, 7–23. [Google Scholar] [CrossRef]
- Fu, X.; Zeng, D.; Huang, Y.; Ding, X.; Zhang, X.P. A variational framework for single low light image enhancement using bright channel prior. In Proceedings of the 2013 IEEE Global Conference on Signal and Information Processing, Austin, TX, USA, 3–5 December 2013; pp. 1085–1088. [Google Scholar]
- Park, S.; Yu, S.; Moon, B.; Ko, S.; Paik, J. Low-light image enhancement using variational optimization-based retinex model. IEEE Trans. Consum. Electron. 2017, 63, 178–184. [Google Scholar] [CrossRef]
- Fu, G.; Duan, L.; Xiao, C. A hybrid L2-Lp variational model for single low-light image enhancement with bright channel prior. In Proceedings of the 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan, 22–25 September 2019; pp. 1925–1929. [Google Scholar]
- Zhang, Y.; Di, X.; Zhang, B.; Wang, C. Self-supervised image enhancement network: Training with low light images only. arXiv 2020, arXiv:2002.11300. [Google Scholar]
- Lore, K.G.; Akintayo, A.; Sarkar, S. LLNet: A deep autoencoder approach to natural low-light image enhancement. Pattern Recognit. 2017, 61, 650–662. [Google Scholar] [CrossRef] [Green Version]
- Tao, L.; Zhu, C.; Xiang, G.; Li, Y.; Jia, H.; Xie, X. LLCNN: A convolutional neural network for low-light image enhancement. In Proceedings of the 2017 IEEE Visual Communications and Image Processing (VCIP), St. Petersburg, FL, USA, 10–13 December 2017; pp. 1–4. [Google Scholar]
- Ignatov, A.; Kobyshev, N.; Timofte, R.; Vanhoey, K.; Van Gool, L. Dslr-quality photos on mobile devices with deep convolutional networks. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 3277–3285. [Google Scholar]
- Shen, L.; Yue, Z.; Feng, F.; Chen, Q.; Liu, S.; Ma, J. Msr-net: Low-light image enhancement using deep convolutional network. arXiv 2017, arXiv:1711.02488. [Google Scholar]
- Gharbi, M.; Chen, J.; Barron, J.T.; Hasinoff, S.W.; Durand, F. Deep bilateral learning for real-time image enhancement. ACM Trans. Graph. 2017, 36, 1–12. [Google Scholar] [CrossRef]
- Wei, C.; Wang, W.; Yang, W.; Liu, J. Deep retinex decomposition for low-light enhancement. arXiv 2018, arXiv:1808.04560. [Google Scholar]
- Wang, W.; Wei, C.; Yang, W.; Liu, J. Gladnet: Low-light enhancement network with global awareness. In Proceedings of the 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi’an, China, 15–19 May 2018; pp. 751–755. [Google Scholar]
- Chen, C.; Chen, Q.; Xu, J.; Koltun, V. Learning to see in the dark. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 3291–3300. [Google Scholar]
- Chen, Y.S.; Wang, Y.C.; Kao, M.H.; Chuang, Y.Y. Deep photo enhancer: Unpaired learning for image enhancement from photographs with gans. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 6306–6314. [Google Scholar]
- Zhang, Y.; Zhang, J.; Guo, X. Kindling the darkness: A practical low-light image enhancer. In Proceedings of the 27th ACM International Conference on Multimedia, Nice, France, 21–25 October 2019; pp. 1632–1640. [Google Scholar]
- Wang, R.; Zhang, Q.; Fu, C.W.; Shen, X.; Zheng, W.S.; Jia, J. Underexposed photo enhancement using deep illumination estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019; pp. 6849–6857. [Google Scholar]
- Jiang, Y.; Gong, X.; Liu, D.; Cheng, Y.; Fang, C.; Shen, X.; Yang, J.; Zhou, P.; Wang, Z. Enlightengan: Deep light enhancement without paired supervision. IEEE Trans. Image Process. 2021, 30, 2340–2349. [Google Scholar] [CrossRef]
- Yang, W.; Wang, S.; Fang, Y.; Wang, Y.; Liu, J. From fidelity to perceptual quality: A semi-supervised approach for low-light image enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020; pp. 3063–3072. [Google Scholar]
- Lv, F.; Liu, B.; Lu, F. Fast enhancement for non-uniform illumination images using light-weight CNNs. In Proceedings of the 28th ACM International Conference on Multimedia, Seattle, WA, USA, 12–16 October 2020; pp. 1450–1458. [Google Scholar]
- Wang, L.W.; Liu, Z.S.; Siu, W.C.; Lun, D.P. Lightening network for low-light image enhancement. IEEE Trans. Image Process. 2020, 29, 7984–7996. [Google Scholar] [CrossRef]
- Zhu, M.; Pan, P.; Chen, W.; Yang, Y. Eemefn: Low-light image enhancement via edge-enhanced multi-exposure fusion network. Proc. Aaai Conf. Artif. Intell. 2020, 34, 13106–13113. [Google Scholar] [CrossRef]
- Liu, R.; Ma, L.; Zhang, J.; Fan, X.; Luo, Z. Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 20–25 June 2021; pp. 10561–10570. [Google Scholar]
- Li, J.; Feng, X.; Hua, Z. Low-light image enhancement via progressive-recursive network. IEEE Trans. Circ. Syst. Video Technol. 2021, 31, 4227–4240. [Google Scholar] [CrossRef]
- Zhang, F.; Li, Y.; You, S.; Fu, Y. Learning temporal consistency for low light video enhancement from single images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 20–25 June 2021; pp. 4967–4976. [Google Scholar]
- Fu, Y.; Hong, Y.; Chen, L.; You, S. LE-GAN: Unsupervised low-light image enhancement network using attention module and identity invariant loss. Knowl. Based Syst. 2022, 240, 108010. [Google Scholar] [CrossRef]
- Zhao, Z.; Xiong, B.; Wang, L.; Ou, Q.; Yu, L.; Kuang, F. Retinexdip: A unified deep framework for low-light image enhancement. IEEE Trans. Circ. Syst. Video Technol. 2021, 32, 1076–1088. [Google Scholar] [CrossRef]
- Liu, S.; Long, W.; He, L.; Li, Y.; Ding, W. Retinex-based fast algorithm for low-light image enhancement. Entropy 2021, 23, 746. [Google Scholar] [CrossRef] [PubMed]
- Liang, H.; Yu, A.; Shao, M.; Tian, Y. Multi-Feature Guided Low-Light Image Enhancement. Appl. Sci. 2021, 11, 5055. [Google Scholar] [CrossRef]
- Li, Q.; Wu, H.; Xu, L.; Wang, L.; Lv, Y.; Kang, X. Low-light image enhancement based on deep symmetric encoder—decoder convolutional networks. Symmetry 2020, 12, 446. [Google Scholar] [CrossRef] [Green Version]
- Han, S.; Lee, T.B.; Heo, Y.S. Deep Image Prior for Super Resolution of Noisy Image. Electronics 2021, 10, 2014. [Google Scholar] [CrossRef]
- Ai, S.; Kwon, J. Extreme low-light image enhancement for surveillance cameras using attention U-Net. Sensors 2020, 20, 495. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Zhao, B.; Gong, X.; Wang, J.; Zhao, L. Low-Light Image Enhancement Based on Multi-Path Interaction. Sensors 2021, 21, 4986. [Google Scholar] [CrossRef] [PubMed]
- Zhu, J.Y.; Park, T.; Isola, P.; Efros, A.A. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 2223–2232. [Google Scholar]
- Guo, C.; Li, C.; Guo, J.; Loy, C.C.; Hou, J.; Kwong, S.; Cong, R. Zero-reference deep curve estimation for low-light image enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020; pp. 1780–1789. [Google Scholar]
- Ulyanov, D.; Vedaldi, A.; Lempitsky, V. Deep image prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 9446–9454. [Google Scholar]
- Ng, M.K.; Wang, W. A total variation model for Retinex. SIAM J. Imaging Sci. 2011, 4, 345–365. [Google Scholar] [CrossRef]
- Yue, H.; Yang, J.; Sun, X.; Wu, F.; Hou, C. Contrast enhancement based on intrinsic image decomposition. IEEE Trans. Image Process. 2017, 26, 3981–3994. [Google Scholar] [CrossRef]
- Lee, C.; Lee, C.; Lee, Y.Y.; Kim, C.S. Power-constrained contrast enhancement for emissive displays based on histogram equalization. IEEE Trans. Image Process. 2011, 21, 80–93. [Google Scholar]
- Mittal, A.; Soundararajan, R.; Bovik, A.C. Making a “completely blind" image quality analyzer. IEEE Signal Process. Lett. 2012, 20, 209–212. [Google Scholar] [CrossRef]
- Gu, K.; Lin, W.; Zhai, G.; Yang, X.; Zhang, W.; Chen, C.W. No-reference quality metric of contrast-distorted images based on information maximization. IEEE T. Cybern. 2017, 47, 4559–4565. [Google Scholar] [CrossRef] [PubMed]
- Gu, K.; Tao, D.; Qiao, J.F.; Lin, W. Learning a no-reference quality assessment model of enhanced images with big data. IEEE Trans. Neural Netw. Learn. Syst. 2017, 29, 1301–1313. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Fu, X.; Zeng, D.; Huang, Y.; Zhang, X.P.; Ding, X. A weighted variational model for simultaneous reflectance and illumination estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 2782–2790. [Google Scholar]
Method | DICM | Fusion | LIME | MEF | NPE | VV | Average |
---|---|---|---|---|---|---|---|
LIME | 3.5360 | 3.9183 | 4.1423 | 3.7022 | 4.2625 | 2.7475 | 3.5442 |
NPE | 3.4530 | 3.8883 | 3.9031 | 3.5155 | 3.9501 | 3.0290 | 3.4928 |
SRIE | 3.5768 | 3.9741 | 3.7868 | 3.4742 | 3.9883 | 3.1357 | 3.5668 |
KinD | 4.2691 | 4.1027 | 4.3525 | 4.1318 | 3.9589 | 3.4255 | 4.0752 |
Zero-DCE | 3.6091 | 4.2421 | 3.9354 | 3.4044 | 4.0944 | 3.2245 | 3.6332 |
RetinexDIP | 3.7612 | 4.2308 | 3.6355 | 3.2721 | 4.1012 | 2.4890 | 3.5363 |
Ours | 3.7911 | 4.0628 | 3.7615 | 3.2363 | 4.0426 | 2.4604 | 3.5294 |
Method | DICM | Fusion | LIME | MEF | NPE | VV | Average |
---|---|---|---|---|---|---|---|
LIME | 5.3397 | 5.3686 | 5.4956 | 5.4168 | 5.4480 | 5.5805 | 5.4121 |
NPE | 5.0895 | 4.5802 | 4.6168 | 4.8610 | 5.1738 | 5.2655 | 5.0104 |
SRIE | 4.9990 | 4.3568 | 4.5032 | 4.7045 | 5.1848 | 5.3021 | 4.9246 |
KinD | 4.6155 | 4.5248 | 4.6841 | 4.6725 | 4.5766 | 4.8159 | 4.6511 |
Zero-DCE | 4.8984 | 4.7346 | 5.0678 | 5.0504 | 5.1068 | 5.3614 | 5.0062 |
RetinexDIP | 4.9912 | 4.4449 | 4.7830 | 5.0151 | 5.3222 | 5.3915 | 5.0126 |
Ours | 5.0093 | 4.5210 | 4.7996 | 5.0761 | 5.2931 | 5.4138 | 5.0398 |
Method | DICM | Fusion | LIME | MEF | NPE | VV | Average |
---|---|---|---|---|---|---|---|
LIME | 0.8986 | 0.9642 | 1.0882 | 1.0385 | 0.9844 | 0.9555 | 0.9515 |
NPE | 0.9139 | 0.9705 | 1.0812 | 1.0372 | 1.0228 | 0.9557 | 0.9609 |
SRIE | 0.9056 | 1.0094 | 1.1121 | 1.0967 | 1.0258 | 0.9629 | 0.9721 |
KinD | 0.7459 | 0.8148 | 0.8336 | 0.7877 | 0.8007 | 0.7418 | 0.7670 |
Zero-DCE | 0.7818 | 0.8820 | 0.9803 | 0.9461 | 0.8578 | 0.8396 | 0.8415 |
RetinexDIP | 0.9999 | 1.0680 | 1.1595 | 1.1088 | 1.0411 | 1.0525 | 1.0436 |
Ours | 1.0038 | 1.0787 | 1.1585 | 1.0926 | 1.0524 | 1.0445 | 1.0437 |
Method | () | () | () | () | () | () | () | () |
---|---|---|---|---|---|---|---|---|
LIME | 0.1133 | 0.4196 | 1.0148 | 1.5713 | 2.3901 | 3.3302 | 4.4058 | 5.7054 |
NPE | 5.8861 | 26.6340 | 58.5019 | 104.8345 | 163.9938 | 235.7513 | 326.0996 | 427.1531 |
SRIE | 4.7643 | 33.6684 | 121.5802 | 343.9839 | 726.5981 | 386.7066 | 544.0660 | 865.0404 |
KinD | 0.1554 | 0.0464 | - | - | - | - | - | - |
Zero-DCE | 0.12559 | 0.1390 | 0.2539 | 0.4051 | 0.83371 | - | - | - |
RetinexDIP | 15.2482 | 15.5945 | 31.4575 | 52.1564 | 102.5126 | 139.4568 | 182.4861 | 212.1594 |
Ours | 15.3655 | 15.4910 | 30.8131 | 48.9527 | 94.5498 | 122.6732 | 154.7097 | 189.1483 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Gao, X.; Zhang, M.; Luo, J. Low-Light Image Enhancement via Retinex-Style Decomposition of Denoised Deep Image Prior. Sensors 2022, 22, 5593. https://doi.org/10.3390/s22155593
Gao X, Zhang M, Luo J. Low-Light Image Enhancement via Retinex-Style Decomposition of Denoised Deep Image Prior. Sensors. 2022; 22(15):5593. https://doi.org/10.3390/s22155593
Chicago/Turabian StyleGao, Xianjie, Mingliang Zhang, and Jinming Luo. 2022. "Low-Light Image Enhancement via Retinex-Style Decomposition of Denoised Deep Image Prior" Sensors 22, no. 15: 5593. https://doi.org/10.3390/s22155593
APA StyleGao, X., Zhang, M., & Luo, J. (2022). Low-Light Image Enhancement via Retinex-Style Decomposition of Denoised Deep Image Prior. Sensors, 22(15), 5593. https://doi.org/10.3390/s22155593