Low-Light Image Enhancement Integrating Retinex-Inspired Extended Decomposition with a Plug-and-Play Framework
Abstract
:1. Introduction
- We decompose the input image into three layers: illumination, reflectance, and color layer, allowing for a more precise and detailed representation of each component.
- Instead of traditional penalty functions, an adaptive iterative reweighting method is used to regularize the illumination component, allowing gentle smoothing near edges and bright areas while stronger smoothing in darker regions.
- The Plug-and-Play framework is incorporated into the reflectance restoration process, utilizing several off-the-shelf image denoising filters to retain essential details and eliminate noise during image enhancement.
2. Related Work
2.1. Retinex-Based Methods
2.2. Simultaneous Enhancement and Denoising
2.3. Plug-and-Play Framework
3. Proposed Model and Algorithm
3.1. Extended Decomposition Approach
3.2. Numerical Algorithm with the Plug-and-Play Framework
- -subproblem
- -subproblem
Algorithm 1 The numerical algorithm for the minimization problem (5) |
1. Input: convert the input image into the logarithmic domain and denote it as , initialize , choose a group of initial parameters. 2. Conduct the following alternating iterative process. For k = 1, 2……, do update by (13); update by (14); update by (18); End above iteration when the stopping criterion is satisfied. 3. Gamma correct. 4. Output: and . |
4. Experimental Results and Discussions
4.1. Experiment Setting
4.2. Experimental Results
4.2.1. Qualitative Comparison
4.2.2. Quantitative Comparison
4.3. Discussions
4.3.1. The Effectiveness of Denoiser
4.3.2. Evaluations on the Extended Decomposition Method
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Lozano-Vázquez, L.V.; Miura, J.; Rosales-Silva, A.J.; Luviano-Juárez, A.; Mújica-Vargas, D. Analysis of different image enhancement and feature extraction methods. Mathematics 2022, 10, 2407. [Google Scholar] [CrossRef]
- Kumar, A.; Rastogi, P.; Srivastava, P. Design and FPGA implementation of DWT, image text extraction technique. Procedia Comput. Sci. 2015, 57, 1015–1025. [Google Scholar] [CrossRef]
- Muslim, H.S.M.; Khan, S.A.; Hussain, S.; Jamal, A.; Qasim, H.S.A. A knowledge-based image enhancement and denoising approach. Comput. Math. Organ. Theory 2019, 25, 108–121. [Google Scholar] [CrossRef]
- Wang, W.; Wu, X.; Yuan, X.; Gao, Z. An experiment-based review of low-light image enhancement methods. IEEE Access 2020, 8, 87884–87917. [Google Scholar] [CrossRef]
- Wu, W.; Weng, J.; Zhang, P.; Wang, X.; Yang, W.; Jiang, J. Uretinex-net: Retinex-based deep unfolding network for low-light image enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022; pp. 5901–5910. [Google Scholar]
- Yang, S.; Zhou, D.; Cao, J.; Guo, Y. LightingNet: An integrated learning method for low-light image enhancement. IEEE Trans. Comput. Imaging 2023, 9, 29–42. [Google Scholar] [CrossRef]
- Bharati, S.; Khan, T.Z.; Podder, P.; Hung, N.Q. A comparative analysis of image denoising problem: Noise models, denoising filters and applications. Cogn. Internet Med. Things Smart Healthc. Serv. Appl. 2021, 311, 49–66. [Google Scholar]
- Chen, X.; Zhan, S.; Ji, D.; Xu, L.; Wu, C.; Li, X. Image denoising via deep network based on edge enhancement. J. Ambient Intell. Humaniz. Comput. 2023, 14, 14795–14805. [Google Scholar] [CrossRef]
- Dhal, K.G.; Das, A.; Ray, S.; Gálvez, J.; Das, S. Histogram equalization variants as optimization problems: A review. Arch. Comput. Methods Eng. 2021, 28, 1471–1496. [Google Scholar] [CrossRef]
- Patel, S.; Bharath, K.P.; Balaji, S.; Muthu, R.K. Comparative study on histogram equalization techniques for medical image enhancement. In Soft Computing for Problem Solving: SocProS 2018; Springer: Singapore, 2020; Volume 1, pp. 657–669. [Google Scholar]
- Jha, K.; Sakhare, A.; Chavhan, N.; Lokulwar, P.P. A Review on Image Enhancement Techniques using Histogram Equalization. Grenze Int. J. Eng. Technol. 2024, 497–502. [Google Scholar]
- Ng, M.K.; Wang, W. A total variation model for retinex. SIAM J. Imaging Sci. 2011, 4, 345–365. [Google Scholar] [CrossRef]
- Guo, X.; Li, Y.; Ling, H. LIME: Low-light image enhancement via illumination map estimation. IEEE Trans. Image Process. 2016, 26, 982–993. [Google Scholar] [CrossRef] [PubMed]
- Park, S.; Yu, S.; Moon, B.; Ko, S.; Paik, J. Low-light image enhancement using variational optimization-based retinex model. IEEE Trans. Consum. Electron. 2017, 63, 178–184. [Google Scholar] [CrossRef]
- Fu, X.; Zeng, D.; Huang, Y.; Zhang, X.P.; Ding, X. A weighted variational model for simultaneous reflectance and illumination estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 2782–2790. [Google Scholar]
- Xu, J.; Hou, Y.; Ren, D.; Liu, L.; Zhu, F.; Yu, M.; Wang, H.; Shao, L. Star: A structure and texture aware retinex model. IEEE Trans. Image Process. 2020, 29, 5022–5037. [Google Scholar] [CrossRef]
- Jia, X.; Feng, X.; Wang, W.; Zhang, L. An extended variational image decomposition model for color image enhancement. Neurocomputing 2018, 322, 216–228. [Google Scholar] [CrossRef]
- Zheng, C.; Shi, D.; Shi, W. Adaptive unfolding total variation network for low-light image enhancement. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada, 10–17 October 2021; pp. 4439–4448. [Google Scholar]
- Li, J.; Li, J.; Fang, F.; Li, F.; Zhang, G. Luminance-aware pyramid network for low-light image enhancement. IEEE Trans. Multimed. 2020, 23, 3153–3165. [Google Scholar] [CrossRef]
- Liu, R.; Ma, L.; Zhang, J.; Fan, X.; Luo, Z. Retinex-inspired unrolling with cooperative prior architecture search for low-light image enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 20–25 June 2021; pp. 10561–10570. [Google Scholar]
- Guo, C.; Li, C.; Guo, J.; Loy, C.C.; Hou, J.; Kwong, S.; Cong, R. Zero-reference deep curve estimation for low-light image enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020; pp. 1780–1789. [Google Scholar]
- Li, C.; Guo, J.; Porikli, F.; Pang, Y. LightenNet: A convolutional neural network for weakly illuminated image enhancement. Pattern Recognit. Lett. 2018, 104, 15–22. [Google Scholar] [CrossRef]
- Pan, Z.; Yuan, F.; Lei, J.; Li, W.; Ling, N.; Kwong, S. MIEGAN: Mobile image enhancement via a multi-module cascade neural network. IEEE Trans. Multimed. 2021, 24, 519–533. [Google Scholar] [CrossRef]
- Liang, L.; Zharkov, I.; Amjadi, F.; Joze, H.R.V.; Pradeep, V. Guidance network with staged learning for image enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 20–25 June 2021; pp. 836–845. [Google Scholar]
- Yu, W.; Zhao, L.; Zhong, T. Unsupervised Low-Light Image Enhancement Based on Generative Adversarial Network. Entropy 2023, 25, 932. [Google Scholar] [CrossRef] [PubMed]
- Ma, L.; Liu, R.; Zhang, J.; Fan, X.; Luo, Z. Learning deep context-sensitive decomposition for low-light image enhancement. IEEE Trans. Neural Netw. Learn. Syst. 2021, 33, 5666–5680. [Google Scholar] [CrossRef] [PubMed]
- Venkatakrishnan, S.V.; Bouman, C.A.; Wohlberg, B. Plug-and-play priors for model based reconstruction. In Proceedings of the 2013 IEEE Global Conference on Signal and Information Processing, Austin, TX, USA, 3–5 December 2013; pp. 945–948. [Google Scholar]
- Chan, S.H.; Wang, X.; Elgendy, O.A. Plug-and-play ADMM for image restoration: Fixed-point convergence and applications. IEEE Trans. Comput. Imaging 2016, 3, 84–98. [Google Scholar] [CrossRef]
- Wu, T.; Wu, W.; Yang, Y.; Fan, F.L.; Zeng, T. Retinex image enhancement based on sequential decomposition with a plug-and-play framework. IEEE Trans. Neural Netw. Learn. Syst. 2024, 35, 14559–14572. [Google Scholar] [CrossRef] [PubMed]
- Zhang, K.; Li, Y.; Zuo, W.; Zhang, L.; Van Gool, L.; Timofte, R. Plug-and-play image restoration with deep denoiser prior. IEEE Trans. Pattern Anal. Mach. Intell. 2021, 44, 6360–6376. [Google Scholar] [CrossRef]
- Zhu, Y.; Zhang, K.; Liang, J.; Cao, J.; Wen, B.; Timofte, R.; Van Gool, L. Denoising diffusion models for plug-and-play image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 17–24 June 2023; pp. 1219–1229. [Google Scholar]
- Lin, Y.H.; Lu, Y.C. Low-light enhancement using a plug-and-play Retinex model with shrinkage mapping for illumination estimation. IEEE Trans. Image Process. 2022, 31, 4897–4908. [Google Scholar] [CrossRef] [PubMed]
- Cai, B.; Xu, X.; Guo, K.; Jia, K.; Hu, B.; Tao, D. A joint intrinsic-extrinsic prior model for retinex. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 4000–4009. [Google Scholar]
- Du, S.; Zhao, M.; Liu, Y.; You, Z.; Shi, Z.; Li, J.; Xu, Z. Low-light image enhancement and denoising via dual-constrained Retinex model. Appl. Math. Model. 2023, 116, 1–15. [Google Scholar] [CrossRef]
- Ren, X.; Yang, W.; Cheng, W.H.; Liu, J. LR3M: Robust low-light enhancement via low-rank regularized retinex model. IEEE Trans. Image Process. 2020, 29, 5862–5876. [Google Scholar] [CrossRef] [PubMed]
- Kurihara, K.; Yoshida, H.; Iiguni, Y. Low-light image enhancement via adaptive shape and texture prior. In Proceedings of the 2019 15th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS), Sorrento, Italy, 26–29 November 2019; pp. 74–81. [Google Scholar]
- Chien, C.C.; Kinoshita, Y.; Shiota, S.; Kiya, H. A retinex-based image enhancement scheme with noise aware shadow-up function. Int. Workshop Adv. Image Technol. (IWAIT) 2019, 11049, 501–506. [Google Scholar]
- Ma, Q.; Wang, Y.; Zeng, T. Retinex-based variational framework for low-light image enhancement and denoising. IEEE Trans. Multimed. 2022, 25, 5580–5588. [Google Scholar] [CrossRef]
- Zhao, C.; Yue, W.; Xu, J.; Chen, H. Joint Low-Light Image Enhancement and Denoising via a New Retinex-Based Decomposition Model. Mathematics 2023, 11, 3834. [Google Scholar] [CrossRef]
- Wei, C.; Wang, W.; Yang, W.; Liu, J. Deep retinex decomposition for low-light enhancement. arXiv 2018, arXiv:1808.04560. [Google Scholar]
- Ma, Q.; Wang, Y.; Zeng, T. Variational Low-Light Image Enhancement Based on Fractional-Order Differential. Commun. Comput. Phys. 2024, 35, 139–159. [Google Scholar]
- Wang, Z.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P. Image quality assessment: From error visibility to structural similarity. IEEE Trans. Image Process. 2004, 13, 600–612. [Google Scholar] [CrossRef] [PubMed]
- Mittal, A.; Soundararajan, R.; Bovik, A.C. Making a completely blind image quality analyzer. IEEE Signal Process 2012, 20, 209–212. [Google Scholar] [CrossRef]
- Gu, K.; Zhai, G.; Lin, W.; Yang, X.; Zhang, W. No-reference image sharpness assessment in autoregressive parameter space. IEEE Trans. Image Process. 2015, 24, 3218–3231. [Google Scholar] [PubMed]
Methods | 5.png | 9.png | ||||||
---|---|---|---|---|---|---|---|---|
PSNR | SSIM | NIQE | ARISM | PSNR | SSIM | NIQE | ARISM | |
JIEP | 20.4562 | 0.8401 | 4.3768 | 3.8856 | 20.0129 | 0.8321 | 4.3768 | 3.8089 |
EVID | 20.1943 | 0.8561 | 4.5880 | 3.7864 | 20.4001 | 0.8687 | 4.5880 | 3.7901 |
STAR | 21.0998 | 0.8654 | 4.0742 | 3.5698 | 20.9985 | 0.8701 | 4.0742 | 3.7866 |
RBVF | 20.9813 | 0.8675 | 4.0816 | 3.6001 | 21.0981 | 0.8699 | 4.0816 | 3.5800 |
SDPF | 21.8102 | 0.8991 | 3.8690 | 3.4251 | 21.9020 | 0.9001 | 3.8690 | 3.4521 |
REDM | 21.9698 | 0.8922 | 3.8894 | 3.5002 | 21.9005 | 0.9000 | 3.8894 | 3.4866 |
OURS | 22.8030 | 0.9100 | 3.8962 | 3.4002 | 22.0330 | 0.9089 | 3.8601 | 3.4200 |
Methods | 3002.jpg | 3007.jpg | ||||||
---|---|---|---|---|---|---|---|---|
PSNR | SSIM | NIQE | ARISM | PSNR | SSIM | NIQE | ARISM | |
JIEP | 20.2581 | 0.8269 | 4.5986 | 3.9621 | 19.9867 | 0.8465 | 4.7888 | 4.0201 |
EVID | 20.0876 | 0.8499 | 4.6091 | 3.8510 | 20.0066 | 0.8723 | 4.4213 | 3.9017 |
STAR | 21.0001 | 0.8586 | 4.1936 | 3.6901 | 20.5862 | 0.8793 | 4.0001 | 3.8676 |
RBVF | 20.9012 | 0.8501 | 4.1652 | 3.5984 | 21.0599 | 0.8662 | 4.0699 | 3.5252 |
SDPF | 21.0268 | 0.9001 | 3.9021 | 3.4555 | 22.0221 | 0.9135 | 3.6778 | 3.4365 |
REDM | 21.6875 | 0.8901 | 3.9120 | 3.4600 | 21.9965 | 0.9054 | 3.7852 | 3.4123 |
OURS | 22.0002 | 0.8965 | 3.8900 | 3.4166 | 22.3177 | 0.9199 | 3.7001 | 3.4000 |
Datasets | Index | BM3D | NLM | RF | TV |
---|---|---|---|---|---|
LOL | PSNR | 22.8256 | 22.6801 | 22.3487 | 22.4330 |
SSIM | 0.9168 | 0.9023 | 0.8967 | 0.9002 | |
NIQE | 3.8651 | 3.8953 | 3.9752 | 3.8922 | |
ARISM | 3.3540 | 3.3854 | 3.4874 | 3.6522 | |
Nikon | PSNR | 22.1540 | 22.0625 | 21.8650 | 21.0002 |
SSIM | 0.8948 | 0.8941 | 0.8877 | 0.8965 | |
NIQE | 3.8057 | 3.8657 | 3.8701 | 3.8659 | |
ARISM | 3.4025 | 3.4076 | 3.4512 | 3.4762 |
Image | JIEP | STAR | OURS |
---|---|---|---|
Image1 | 0.0561 | 0.0392 | 0.0218 |
Image2 | 0.0588 | 0.0412 | 0.0229 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Zhao, C.; Yue, W.; Wang, Y.; Wang, J.; Luo, S.; Chen, H.; Wang, Y. Low-Light Image Enhancement Integrating Retinex-Inspired Extended Decomposition with a Plug-and-Play Framework. Mathematics 2024, 12, 4025. https://doi.org/10.3390/math12244025
Zhao C, Yue W, Wang Y, Wang J, Luo S, Chen H, Wang Y. Low-Light Image Enhancement Integrating Retinex-Inspired Extended Decomposition with a Plug-and-Play Framework. Mathematics. 2024; 12(24):4025. https://doi.org/10.3390/math12244025
Chicago/Turabian StyleZhao, Chenping, Wenlong Yue, Yingjun Wang, Jianping Wang, Shousheng Luo, Huazhu Chen, and Yan Wang. 2024. "Low-Light Image Enhancement Integrating Retinex-Inspired Extended Decomposition with a Plug-and-Play Framework" Mathematics 12, no. 24: 4025. https://doi.org/10.3390/math12244025
APA StyleZhao, C., Yue, W., Wang, Y., Wang, J., Luo, S., Chen, H., & Wang, Y. (2024). Low-Light Image Enhancement Integrating Retinex-Inspired Extended Decomposition with a Plug-and-Play Framework. Mathematics, 12(24), 4025. https://doi.org/10.3390/math12244025