PerNet: Progressive and Efficient All-in-One Image-Restoration Lightweight Network
Abstract
:1. Introduction
- We design a simple and efficient adaptive progressive network architecture, which has excellent progressive stability and can be easily plugged with any module.
- We devise a PPELAM, composed of multiple ELAUs, which can fully exploit the continuous spatial correlation of degradation in both horizontal and vertical directions, thereby achieving a high match with different types of degradation.
- Our method shows excellent recovery effects on seven types of image degradation datasets, and our model achieves good lightweight effects.
2. Related Works
2.1. Image-Restoration
2.2. Lightweight Image-Restoration
2.3. Attention-Based Image-Restoration
2.4. All-in-One Image-Restoration
3. Method
3.1. Overall Network Architecture
3.2. Plug-and-Play Efficient Local Attention Module
3.3. Loss Function and Evaluation Metrics
4. Experiments and Analysis
4.1. Experimental Setup
4.1.1. Datasets
4.1.2. Execution Details
4.2. Subjective Evaluation
4.2.1. Deraining Task
4.2.2. Desnowing Task
4.2.3. Dehazing Task
4.2.4. Underwater Enhancement Task
4.3. Objective Evaluation
4.3.1. Deraining Task
4.3.2. Desnowing Task
4.3.3. Dehazing Task
4.3.4. Underwater Enhancement Task
4.4. Lightweight Experiment
4.5. Ablation Experiment
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Nomenclatures
DSC | Discriminative Sparse Coding |
DiG-CoM | Directional Gradient, Constraints based Model |
SPD-Net | Structure-Preserving Deraining Network |
NLCL | Non-Local Contrastive Learning |
SIRR | Single Image Rain Removal |
JRGB | Joint Rain Generation and removal for Both the real and synthetic image |
DDN | Deep Detail Network |
Air-Net | Auxiliary image reconstruction Network |
DID-MDN | DensIty-aware De-raining using a Multi-stream Dense Network |
RESCAN | RecurrEnt Squeeze-and-excitation Context Aggregation Network |
PreNet | Progressive image deraining Networks |
MSPFN | Multi-Scale Progressive Fusion Network |
JSTASR | Joint Size and Transparency-Aware Snow Removal |
HDCW-Net | Hierarchical Dual-tree Complex Wavelet representation Network |
DDMSNet | Deep Dense Multi-Scale Network |
MPRNet | Multi-stage Progressive image-restoration Net |
SMGARN | Snow Mask Guided Adaptive Residual Network |
TKL | Two-stage Knowledge Learning |
MSP-Former | Multi-Scale Projection transFormer |
Uformer | U-shaped transformer |
NAFNet | Nonlinear Activation Free Network |
DGUNet | Deep Generalized Unfolding Networks |
Cycle-SNSPGAN | Cycle Spectral Normalized Soft likelihood estimation Patch GAN |
ZID | Zero-shot Image Dehazing |
FCTF-Net | First-Coarse-Then-Fine Network |
FFA-Net | Feature Fusion Attention Network |
TCN | Triple-Convolutional Network |
EVPM | dEhazing Values Prior Model |
IDeRs | Iterative Dehazing method for single Remote sensing image |
GRS-HTM | Ground Radiance Suppressed Haze Thickness Map |
SDCP | Sphere model improved Dark Channel Prior |
UHD | Ultra-High-Definition |
DeHamer | DeHazing transformer |
STD | Structure layer according To the Distribution |
Zero-restore | Zero-shot single image-restoration |
ROP | Rank-One Prior |
PRWNet | Progressively Refine Wavelet Network |
ShallowUW | Shallow UnderWater |
UWCNN | UnderWater image enhancement Convolutional Neural Network |
FunIE-GAN | Fast underwater Image Enhancement Generative Adversarial Network |
UT-UIE | U-shape Transformer for Underwater Image Enhancement |
Water-Net | underWater image enhancement Network |
RAUNE-Net | Residual and Attention-driven Underwater eNhancEment Network |
CPDM | Content-Preserving Diffusion Model |
SyreaNet | Synthetic and real images Network |
SGUIE-Net | Semantic attention Guided Underwater Image Enhancement Network |
Cycle-GAN | Cycle-consistent Generative Adversarial Networks |
CSD | Comprehensive Snow Dataset |
RSID | Remote Sensing Image Dataset |
EUVP | Enhancing Underwater Visual Perception |
References
- Liu, Q.; Liu, Y.; Lin, D. Revolutionizing Target Detection in Intelligent Traffic Systems: YOLOv8-SnakeVision. Electronics 2023, 12, 4970. [Google Scholar] [CrossRef]
- Zhou, X.; Duan, Y.; Ding, R.; Wang, Q.; Wang, Q.; Qin, J.; Liu, H. Bit-Weight Adjustment for Bridging Uniform and Non-Uniform Quantization to Build Efficient Image Classifiers. Electronics 2023, 12, 5043. [Google Scholar] [CrossRef]
- Hirschmuller, H.; Scharstein, D. Evaluation of cost functions for stereo matching. In Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA, 17–22 June 2007; pp. 1–8. [Google Scholar]
- Hu, H.; Zhang, Z.; Xie, Z.; Lin, S. Local relation networks for image recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea, 27 October–2 November 2019; pp. 3464–3473. [Google Scholar]
- Li, P.; Tian, J.; Tang, Y.; Wang, G.; Wu, C. Model-based deep network for single image deraining. IEEE Access 2020, 8, 14036–14047. [Google Scholar] [CrossRef]
- Kamilaris, A.; Prenafeta-Boldú, F.X. Deep learning in agriculture: A survey. Comput. Electron. Agric. 2018, 147, 70–90. [Google Scholar] [CrossRef]
- Guo, Y.; Liu, Y.; Oerlemans, A.; Lao, S.; Wu, S.; Lew, M.S. Deep learning for visual understanding: A review. Neurocomputing 2016, 187, 27–48. [Google Scholar] [CrossRef]
- LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 521, 436–444. [Google Scholar] [CrossRef] [PubMed]
- Zhang, J.; Zhao, D.; Xiong, R.; Ma, S.; Gao, W. Image-restoration using joint statistical modeling in a space-transform domain. IEEE Trans. Circuits Syst. Video Technol. 2014, 24, 915–928. [Google Scholar] [CrossRef]
- Chambolle, A.; Lions, P.L. Image recovery via total variation minimization and related problems. Numer. Math. 1997, 76, 167–188. [Google Scholar] [CrossRef]
- Podilchuk, C.I.; Mammone, R.J. Image recovery by convex projections using a least-squares constraint. JOSA A 1990, 7, 517–521. [Google Scholar] [CrossRef]
- Chen, Y.; Pock, T. Trainable nonlinear reaction diffusion: A flexible framework for fast and effective Image-restoration. IEEE Trans. Pattern Anal. Mach. Intell. 2016, 39, 1256–1272. [Google Scholar] [CrossRef]
- Liu, Q.; Wang, S.; Ying, L.; Peng, X.; Zhu, Y.; Liang, D. Adaptive dictionary learning in sparse gradient domain for image recovery. IEEE Trans. Image Process. 2013, 22, 4652–4663. [Google Scholar] [CrossRef]
- Yu, H.; Yuan, X.; Jiang, R.; Feng, H.; Liu, J.; Li, Z. Feature Reduction Networks: A Convolution Neural Network-Based Approach to Enhance Image Dehazing. Electronics 2023, 12, 4984. [Google Scholar] [CrossRef]
- Haris, M.; Shakhnarovich, G.; Ukita, N. Deep back-projection networks for super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–22 June 2018; pp. 1664–1673. [Google Scholar]
- Zhang, Y.; Tian, Y.; Kong, Y.; Zhong, B.; Fu, Y. Residual dense network for image super-resolution. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–22 June 2018; pp. 2472–2481. [Google Scholar]
- Fu, X.; Liang, B.; Huang, Y.; Ding, X.; Paisley, J. Lightweight pyramid networks for image deraining. IEEE Trans. Neural Netw. Learn. Syst. 2019, 31, 1794–1807. [Google Scholar] [CrossRef] [PubMed]
- Hu, M.; Yang, J.; Ling, N.; Liu, Y.; Fan, J. Lightweight single image deraining algorithm incorporating visual saliency. IET Image Process. 2022, 16, 3190–3200. [Google Scholar] [CrossRef]
- Mou, C.; Zhang, J.; Fan, X.; Liu, H.; Wang, R. COLA-Net: Collaborative attention network for Image-restoration. IEEE Trans. Multimed. 2021, 24, 1366–1377. [Google Scholar] [CrossRef]
- Deng, S.; Wei, M.; Wang, J.; Feng, Y.; Liang, L.; Xie, H.; Wang, F.L.; Wang, M. Detail-recovery image deraining via context aggregation networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 14–19 June 2020; pp. 14560–14569. [Google Scholar]
- Yu, J.; Lin, Z.; Yang, J.; Shen, X.; Lu, X.; Huang, T.S. Generative image inpainting with contextual attention. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–22 June 2018; pp. 5505–5514. [Google Scholar]
- Wang, Y.; Tao, X.; Qi, X.; Shen, X.; Jia, J. Image inpainting via generative multi-column convolutional neural networks. Adv. Neural Inf. Process. Syst. 2018, 329–338. [Google Scholar]
- Siddiqua, M.; Belhaouari, S.B.; Akhter, N.; Zameer, A.; Khurshid, J. MACGAN: An all-in-one Image-restoration under adverse conditions using multidomain attention-based conditional GAN. IEEE Access 2023, 11, 70482–70502. [Google Scholar] [CrossRef]
- Mei, Y.; Fan, Y.; Zhang, Y.; Yu, J.; Zhou, Y.; Liu, D.; Fu, Y.; Huang, T.S.; Shi, H. Pyramid attention network for image-restoration. Int. J. Comput. Vis. 2023, 131, 3207–3225. [Google Scholar] [CrossRef]
- Chen, S.; Ye, T.; Liu, Y.; Chen, E. Dual-former: Hybrid self-attention transformer for efficient image restoration. Digit. Signal Process. 2024, 149, 104485. [Google Scholar] [CrossRef]
- Liu, G.; Reda, F.A.; Shih, K.J.; Wang, T.C.; Tao, A.; Catanzaro, B. Image inpainting for irregular holes using partial convolutions. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 85–100. [Google Scholar]
- Yang, W.; Tan, R.T.; Feng, J.; Liu, J.; Guo, Z.; Yan, S. Deep joint rain detection and removal from a single image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 1357–1366. [Google Scholar]
- Zhang, H.; Sindagi, V.; Patel, V.M. Image de-raining using a conditional generative adversarial network. IEEE Trans. Circuits Syst. Video Technol. 2019, 30, 3943–3956. [Google Scholar] [CrossRef]
- Liu, Y.F.; Jaw, D.W.; Huang, S.C.; Hwang, J.N. Desnownet: Context-aware deep network for snow removal. IEEE Trans. Image Process. 2018, 27, 3064–3073. [Google Scholar] [CrossRef] [PubMed]
- Chen, W.T.; Fang, H.Y.; Hsieh, C.L.; Tsai, C.C.; Chen, I.; Ding, J.J.; Kuo, S.Y. All snow removed: Single image desnowing algorithm using hierarchical dual-tree complex wavelet representation and contradict channel loss. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada, 11–17 October 2021; pp. 4196–4205. [Google Scholar]
- Zhang, L.; Wang, S. Dense haze removal based on dynamic collaborative inference learning for remote sensing images. IEEE Trans. Geosci. Remote Sens. 2022, 60, 5631016. [Google Scholar] [CrossRef]
- Islam, M.J.; Xia, Y.; Sattar, J. Fast underwater image enhancement for improved visual perception. IEEE Robot. Autom. Lett. 2020, 5, 3227–3234. [Google Scholar] [CrossRef]
- Luo, Y.; Xu, Y.; Ji, H. Removing rain from a single image via discriminative sparse coding. In Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile, 7–13 December 2015; pp. 3397–3405. [Google Scholar]
- Ran, W.; Yang, Y.; Lu, H. Single image rain removal boosting via directional gradient. In Proceedings of the 2020 IEEE International Conference on Multimedia and Expo (ICME), London, UK, 6–10 July 2020; pp. 1–6. [Google Scholar]
- Wei, Y.; Zhang, Z.; Wang, Y.; Xu, M.; Yang, Y.; Yan, S.; Wang, M. Deraincyclegan: Rain attentive cyclegan for single image deraining and rainmaking. IEEE Trans. Image Process. 2021, 30, 4788–4801. [Google Scholar] [CrossRef] [PubMed]
- Yi, Q.; Li, J.; Dai, Q.; Fang, F.; Zhang, G.; Zeng, T. Structure-preserving deraining with residue channel prior guidance. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada, 11–17 October 2021; pp. 4238–4247. [Google Scholar]
- Ye, Y.; Yu, C.; Chang, Y.; Zhu, L.; Zhao, X.L.; Yan, L.; Tian, Y. Unsupervised deraining: Where contrastive learning meets self-similarity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 19–24 June 2022; pp. 5821–5830. [Google Scholar]
- Yasarla, R.; Sindagi, V.A.; Patel, V.M. Syn2real transfer learning for image deraining using gaussian processes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 14–19 June 2020; pp. 2726–2736. [Google Scholar]
- Wang, H.; Yue, Z.; Xie, Q.; Zhao, Q.; Zheng, Y.; Meng, D. From rain generation to rain removal. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 19–25 June 2021; pp. 14791–14801. [Google Scholar]
- Ye, Y.; Chang, Y.; Zhou, H.; Yan, L. Closing the loop: Joint rain generation and removal via disentangled image translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 19–25 June 2021; pp. 2053–2062. [Google Scholar]
- Fu, X.; Huang, J.; Zeng, D.; Huang, Y.; Ding, X.; Paisley, J. Removing rain from single images via a deep detail network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 3855–3863. [Google Scholar]
- Gui, D.; Song, Q.; Song, B.; Li, H.; Wang, M.; Min, X.; Li, A. AIR-Net: A novel multi-task learning method with auxiliary image reconstruction for predicting EGFR mutation status on CT images of NSCLC patients. Comput. Biol. Med. 2022, 141, 105157. [Google Scholar] [CrossRef]
- Zhang, H.; Patel, V.M. Density-aware single image de-raining using a multi-stream dense network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–22 June 2018; pp. 695–704. [Google Scholar]
- Li, X.; Wu, J.; Lin, Z.; Liu, H.; Zha, H. Recurrent squeeze-and-excitation context aggregation net for single image deraining. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 254–269. [Google Scholar]
- Wei, M.; Shen, Y.; Wang, Y.; Xie, H.; Qin, J.; Wang, F.L. Raindiffusion: When unsupervised learning meets diffusion models for real-world image deraining. arXiv 2023, arXiv:2301.09430. [Google Scholar]
- Ren, D.; Zuo, W.; Hu, Q.; Zhu, P.; Meng, D. Progressive image deraining networks: A better and simpler baseline. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019; pp. 3937–3946. [Google Scholar]
- Jiang, K.; Wang, Z.; Yi, P.; Chen, C.; Huang, B.; Luo, Y.; Ma, J.; Jiang, J. Multi-scale progressive fusion network for single image deraining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 14–19 June 2020; pp. 8346–8355. [Google Scholar]
- Zhu, J.Y.; Park, T.; Isola, P.; Efros, A.A. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 22–29 October 2017; pp. 2223–2232. [Google Scholar]
- Li, R.; Tan, R.T.; Cheong, L.F. All in one bad weather removal using architectural search. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 14–19 June 2020; pp. 3175–3185. [Google Scholar]
- Chen, W.T.; Fang, H.Y.; Ding, J.J.; Tsai, C.C.; Kuo, S.Y. JSTASR: Joint size and transparency-aware snow removal algorithm based on modified partial convolution and veiling effect removal. In Proceedings of the Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, 23–28 August 2020; Proceedings, Part XXI 16. Springer: Berlin/Heidelberg, Germany, 2020; pp. 754–770. [Google Scholar]
- Zhang, K.; Li, R.; Yu, Y.; Luo, W.; Li, C. Deep dense multi-scale network for snow removal using semantic and depth priors. IEEE Trans. Image Process. 2021, 30, 7419–7431. [Google Scholar] [CrossRef] [PubMed]
- Zamir, S.W.; Arora, A.; Khan, S.; Hayat, M.; Khan, F.S.; Yang, M.H.; Shao, L. Multi-stage progressive image-restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 19–25 June 2021; pp. 14821–14831. [Google Scholar]
- Valanarasu, J.M.J.; Yasarla, R.; Patel, V.M. Transweather: Transformer-based restoration of images degraded by adverse weather conditions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022; pp. 2353–2363. [Google Scholar]
- Cheng, B.; Li, J.; Chen, Y.; Zeng, T. Snow mask guided adaptive residual network for image snow removal. Comput. Vis. Image Underst. 2023, 236, 103819. [Google Scholar] [CrossRef]
- Chen, W.T.; Huang, Z.K.; Tsai, C.C.; Yang, H.H.; Ding, J.J.; Kuo, S.Y. Learning multiple adverse weather removal via two-stage knowledge learning and multi-contrastive regularization: Toward a unified model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022; pp. 17653–17662. [Google Scholar]
- Özdenizci, O.; Legenstein, R. Restoring vision in adverse weather conditions with patch-based denoising diffusion models. IEEE Trans. Pattern Anal. Mach. Intell. 2023, 45, 10346–10357. [Google Scholar] [CrossRef]
- Chen, S.; Ye, T.; Liu, Y.; Liao, T.; Jiang, J.; Chen, E.; Chen, P. Msp-former: Multi-scale projection transformer for single image desnowing. In Proceedings of the ICASSP 2023–2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Rhodes Island, Greece, 4–10 June 2023; pp. 1–5. [Google Scholar]
- Wang, Z.; Cun, X.; Bao, J.; Zhou, W.; Liu, J.; Li, H. Uformer: A general u-shaped transformer for image-restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022; pp. 17683–17693. [Google Scholar]
- Zamir, S.W.; Arora, A.; Khan, S.; Hayat, M.; Khan, F.S.; Yang, M.H. Restormer: Efficient transformer for high-resolution image restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022; pp. 5728–5739. [Google Scholar]
- Chen, L.; Chu, X.; Zhang, X.; Sun, J. Simple baselines for image-restoration. In Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel, 23–27 October 2022; Springer: Berlin/Heidelberg, Germany, 2022; pp. 17–33. [Google Scholar]
- Mou, C.; Wang, Q.; Zhang, J. Deep generalized unfolding networks for image-restoration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022; pp. 17399–17410. [Google Scholar]
- Wang, T.; Zhang, K.; Shao, Z.; Luo, W.; Stenger, B.; Lu, T.; Kim, T.K.; Liu, W.; Li, H. Gridformer: Residual dense transformer with grid structure for image restoration in adverse weather conditions. arXiv 2023, arXiv:2305.17863. [Google Scholar] [CrossRef]
- Wang, Y.; Yan, X.; Guan, D.; Wei, M.; Chen, Y.; Zhang, X.P.; Li, J. Cycle-snspgan: Towards real-world image dehazing via cycle spectral normalized soft likelihood estimation patch gan. IEEE Trans. Intell. Transp. Syst. 2022, 23, 20368–20382. [Google Scholar] [CrossRef]
- Li, B.; Gou, Y.; Liu, J.Z.; Zhu, H.; Zhou, J.T.; Peng, X. Zero-shot image dehazing. IEEE Trans. Image Process. 2020, 29, 8457–8466. [Google Scholar] [CrossRef] [PubMed]
- Li, Y.; Chen, X. A coarse-to-fine two-stage attentive network for haze removal of remote sensing images. IEEE Geosci. Remote Sens. Lett. 2020, 18, 1751–1755. [Google Scholar] [CrossRef]
- Qin, X.; Wang, Z.; Bai, Y.; Xie, X.; Jia, H. FFA-Net: Feature fusion attention network for single image dehazing. In Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA, 7–12 February 2020; Volume 34, pp. 11908–11915. [Google Scholar]
- Shin, J.; Park, H.; Paik, J. Region-based dehazing via dual-supervised triple-convolutional network. IEEE Trans. Multimed. 2021, 24, 245–260. [Google Scholar] [CrossRef]
- Han, J.; Zhang, S.; Fan, N.; Ye, Z. Local patchwise minimal and maximal values prior for single optical remote sensing image dehazing. Inf. Sci. 2022, 606, 173–193. [Google Scholar] [CrossRef]
- Xu, L.; Zhao, D.; Yan, Y.; Kwong, S.; Chen, J.; Duan, L.Y. IDeRs: Iterative dehazing method for single remote sensing image. Inf. Sci. 2019, 489, 50–62. [Google Scholar] [CrossRef]
- Liu, Q.; Gao, X.; He, L.; Lu, W. Haze removal for a single visible remote sensing image. Signal Process. 2017, 137, 33–43. [Google Scholar] [CrossRef]
- Li, J.; Hu, Q.; Ai, M. Haze and thin cloud removal via sphere model improved dark channel prior. IEEE Geosci. Remote Sens. Lett. 2018, 16, 472–476. [Google Scholar] [CrossRef]
- Zheng, Z.; Ren, W.; Cao, X.; Hu, X.; Wang, T.; Song, F.; Jia, X. Ultra-high-definition image dehazing via multi-guided bilateral learning. In Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 19–25 June 2021; pp. 16180–16189. [Google Scholar]
- Guo, C.L.; Yan, Q.; Anwar, S.; Cong, R.; Ren, W.; Li, C. Image dehazing transformer with transmission-aware 3d position embedding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022; pp. 5812–5820. [Google Scholar]
- Li, R.; Pan, J.; Li, Z.; Tang, J. Single image dehazing via conditional generative adversarial network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–22 June 2018; pp. 8202–8211. [Google Scholar]
- Mi, Z.; Li, Y.; Jin, J.; Liang, Z.; Fu, X. A generalized enhancement framework for hazy images with complex illumination. IEEE Geosci. Remote Sens. Lett. 2021, 19, 3079456. [Google Scholar] [CrossRef]
- Kar, A.; Dhara, S.K.; Sen, D.; Biswas, P.K. Zero-shot single image-restoration through controlled perturbation of koschmieder’s model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 19–25 June 2021; pp. 16205–16215. [Google Scholar]
- Liu, J.; Liu, R.W.; Sun, J.; Zeng, T. Rank-one prior: Real-time scene recovery. IEEE Trans. Pattern Anal. Mach. Intell. 2022, 45, 8845–8860. [Google Scholar] [CrossRef]
- Huo, F.; Li, B.; Zhu, X. Efficient wavelet boost learning-based multi-stage progressive refinement network for underwater image enhancement. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, BC, Canada, 11–17 October 2021; pp. 1944–1952. [Google Scholar]
- Naik, A.; Swarnakar, A.; Mittal, K. Shallow-uwnet: Compressed model for underwater image enhancement (student abstract). In Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada, 2–9 February 2021; Volume 35, pp. 15853–15854. [Google Scholar]
- Li, C.; Anwar, S.; Porikli, F. Underwater scene prior inspired deep underwater image and video enhancement. Pattern Recognit. 2020, 98, 107038. [Google Scholar] [CrossRef]
- Peng, L.; Zhu, C.; Bian, L. U-shape transformer for underwater image enhancement. IEEE Trans. Image Process. 2023, 29, 4376–4389. [Google Scholar]
- Li, C.; Guo, C.; Ren, W.; Cong, R.; Hou, J.; Kwong, S.; Tao, D. An underwater image enhancement benchmark dataset and beyond. IEEE Trans. Image Process. 2019, 29, 4376–4389. [Google Scholar] [CrossRef] [PubMed]
- Peng, W.; Zhou, C.; Hu, R.; Cao, J.; Liu, Y. RAUNE-Net: A Residual and Attention-Driven Underwater Image Enhancement Method. arXiv 2023, arXiv:2311.00246. [Google Scholar]
- Shi, X.; Wang, Y.G. CPDM: Content-Preserving Diffusion Model for Underwater Image Enhancement. arXiv 2024, arXiv:2401.15649. [Google Scholar]
- Wen, J.; Cui, J.; Zhao, Z.; Yan, R.; Gao, Z.; Dou, L.; Chen, B.M. Syreanet: A physically guided underwater image enhancement framework integrating synthetic and real images. In Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA), London, UK, 29 May–2 June 2023; pp. 5177–5183. [Google Scholar]
- Qi, Q.; Li, K.; Zheng, H.; Gao, X.; Hou, G.; Sun, K. SGUIE-Net: Semantic attention guided underwater image enhancement with multi-scale perception. IEEE Trans. Image Process. 2022, 31, 6816–6830. [Google Scholar] [CrossRef]
- Li, C.; Guo, J.; Guo, C. Emerging from water: Underwater image color correction based on weakly supervised color transfer. IEEE Signal Process. Lett. 2018, 25, 323–327. [Google Scholar] [CrossRef]
- Woo, S.; Park, J.; Lee, J.Y.; Kweon, I.S. Cbam: Convolutional block attention module. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 3–19. [Google Scholar]
- Hu, J.; Shen, L.; Sun, G. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–22 June 2018; pp. 7132–7141. [Google Scholar]
- Chen, X.; Li, H.; Li, M.; Pan, J. Learning a sparse transformer network for effective image deraining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 18–22 June 2023; pp. 5896–5905. [Google Scholar]
Datasets | Training Set/Pairs | Test Set/Pairs |
---|---|---|
Rain200H [27] | 1800 | 200 |
Rain200L [27] | 1800 | 200 |
Rain800 [28] | 700 | 100 |
Snow100K [29] | 50,000 | 50,000 |
CSD [30] | 7000 | 1000 |
RSID [31] | 900 | 100 |
EUVP [32] | 11435 | 515 |
Methods | Rain200L | Rain200H | Rain800 | |||
---|---|---|---|---|---|---|
PSNR↑ | SSIM↑ | PSNR↑ | SSIM↑ | PSNR↑ | SSIM↑ | |
DSC [33] | 27.163 | 0.866 | 14.735 | 0.382 | 14.935 | 0.468 |
DiG-CoM [34] | 30.782 | 0.854 | 19.332 | 0.767 | 22.535 | 0.833 |
DerainCycleGAN [35] | 31.491 | 0.936 | 24.321 | 0.842 | 24.293 | 0.859 |
SPD-Net [36] | 31.591 | 0.919 | 26.071 | 0.857 | 24.372 | 0.861 |
NLCL [37] | 31.741 | 0.935 | 22.312 | 0.728 | 24.461 | 0.821 |
Syn2Real [38] | 34.391 | 0.965 | 25.761 | 0.837 | 23.741 | 0.799 |
SIRR [39] | 34.471 | 0.969 | 26.551 | 0.846 | 24.361 | 0.859 |
JRGB [40] | 34.512 | 0.967 | 24.621 | 0.849 | 24.621 | 0.828 |
DDN [41] | 34.683 | 0.976 | 26.053 | 0.806 | 24.234 | 0.468 |
Air-Net [42] | 34.901 | 0.969 | 25.482 | 0.829 | 23.771 | 0.833 |
DID-MDN [43] | 35.401 | 0.961 | 25.612 | 0.854 | 21.891 | 0.795 |
RESCAN [44] | 36.094 | 0.970 | 26.751 | 0.835 | 24.332 | 0.823 |
RainDiffusion [45] | 36.851 | 0.972 | 26.021 | 0.862 | 26.491 | 0.875 |
PReNet [46] | 37.802 | 0.866 | 14.735 | 0.382 | 14.935 | 0.468 |
MSPFN [47] | 38.581 | 0.983 | 29.361 | 0.903 | 23.332 | 0.803 |
PerNet | 39.591 | 0.989 | 29.582 | 0.912 | 25.993 | 0.889 |
Methods | Snow100K-S | Snow100K-L | CSD | |||
---|---|---|---|---|---|---|
PSNR↑ | SSIM↑ | PSNR↑ | SSIM↑ | PSNR↑ | SSIM↑ | |
CycleGAN [48] | 28.513 | 0.902 | 23.596 | 0.883 | 20.981 | 0.801 |
RESCAN [44] | 31.512 | 0.903 | 26.080 | 0.811 | 22.031 | 0.812 |
DesnowNet [29] | 32.332 | 0.950 | 27.173 | 0.898 | 20.131 | 0.812 |
ALL in one [49] | 31.231 | 0.923 | 28.331 | 0.882 | 26.312 | 0.873 |
JSTASR [50] | 31.401 | 0.901 | 25.321 | 0.808 | 27.961 | 0.883 |
HDCW-Net [30] | 31.542 | 0.952 | 27.236 | 0.886 | 29.061 | 0.910 |
DDMSNet [51] | 34.342 | 0.995 | 28.851 | 0.877 | 30.201 | 0.923 |
MPRNet [52] | 35.872 | 0.962 | 31.023 | 0.913 | 33.981 | 0.972 |
TransWeather [53] | 32.512 | 0.934 | 29.312 | 0.888 | 31.761 | 0.932 |
SMGARN [54] | 33.854 | 0.950 | 29.312 | 0.890 | 31.931 | 0.952 |
TKL [55] | 35.213 | 0.963 | 31.001 | 0.919 | 33.891 | 0.963 |
WeatherDiff128 [56] | 35.023 | 0.952 | 29.582 | 0.849 | 33.463 | 0.968 |
MSP-Former [57] | 35.421 | 0.936 | 30.312 | 0.913 | 33.751 | 0.961 |
Uformer [58] | 35.512 | 0.963 | 31.301 | 0.923 | 33.801 | 0.961 |
WeatherDiff64 [56] | 35.831 | 0.957 | 30.092 | 0.904 | 33.631 | 0.962 |
Restormer [59] | 36.081 | 0.959 | 30.281 | 0.912 | 35.431 | 0.972 |
SnowDiff128 [56] | 36.092 | 0.955 | 30.283 | 0.900 | 35.134 | 0.974 |
NAFNet [60] | 36.123 | 0.970 | 31.263 | 0.924 | 35.132 | 0.973 |
DGUNet [61] | 36.312 | 0.971 | 31.204 | 0.922 | 34.741 | 0.973 |
SnowDiff64 [56] | 36.591 | 0.963 | 30.431 | 0.915 | 35.231 | 0.976 |
GridFormer-S [62] | 36.681 | 0.960 | 30.782 | 0.917 | 33.903 | 0.963 |
PerNet | 36.982 | 0.974 | 31.623 | 0.937 | 35.861 | 0.979 |
Methods | R100 | |
---|---|---|
PSNR↑ | SSIM↑ | |
Cycle-SNSPGAN [63] | 18.344 | 0.729 |
ZID [64] | 18.992 | 0.727 |
FCTF-Net [65] | 19.306 | 0.856 |
FFA-Net [66] | 24.052 | 0.899 |
TCN [67] | 14.208 | 0.606 |
EVPM [68] | 15.579 | 0.689 |
IDeRs [69] | 13.604 | 0.644 |
GRS-HTM [70] | 14.800 | 0.519 |
SDCP [71] | 16.055 | 0.691 |
UHD [72] | 26.659 | 0.923 |
DeHamer [73] | 23.752 | 0.899 |
Dehaze-cGAN [74] | 18.703 | 0.743 |
STD [75] | 16.258 | 0.559 |
Zero-restore [76] | 16.648 | 0.717 |
ROP [77] | 15.575 | 0.750 |
PerNet | 26.794 | 0.935 |
Methods | EUVP(515) | |
---|---|---|
PSNR↑ | SSIM↑ | |
PRWNet [78] | 25.441 | 0.843 |
ShallowUW [79] | 24.551 | 0.852 |
UWCNN [80] | 17.725 | 0.704 |
FunIE-GAN [32] | 24.077 | 0.794 |
UT-UIE [81] | 25.214 | 0.813 |
Water-Net [82] | 25.285 | 0.833 |
RAUNE-Net [83] | 26.331 | 0.845 |
CPDM [84] | 23.243 | 0.901 |
SyreaNet [85] | 17.721 | 0.743 |
SGUIE-Net [86] | 19.187 | 0.760 |
Cycle-GAN [87] | 17.963 | 0.709 |
PerNet | 25.592 | 0.913 |
ELAU = 8 | ELAU = 16 | ELAU = 24 | ELAU = 32 | |
---|---|---|---|---|
T | 25.362 | 25.444 | 25.597 | 25.662 |
B | 25.271 | 25.685 | 25.768 | 25.832 |
S | 25.021 | 25.791 | 25.862 | 25.883 |
L | 24.992 | 25.993 | 26.012 | 26.241 |
ELAU = 8 | ELAU = 16 | ELAU = 24 | ELAU = 32 | |
---|---|---|---|---|
T | 0.852 | 0.856 | 0.862 | 0.868 |
B | 0.848 | 0.869 | 0.879 | 0.887 |
S | 0.844 | 0.873 | 0.884 | 0.894 |
L | 0.843 | 0.889 | 0.893 | 0.899 |
PSNR↑ | SSIM↑ | |
---|---|---|
ELAU | 25.993 | 0.889 |
SE | 25.586 | 0.852 |
CBAM | 25.675 | 0.856 |
ELAU+ST | 26.291 | 0.912 |
SE+ST | 26.021 | 0.901 |
CBAM+ST | 26.186 | 0.908 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Li, W.; Zhou, G.; Lin, S.; Tang, Y. PerNet: Progressive and Efficient All-in-One Image-Restoration Lightweight Network. Electronics 2024, 13, 2817. https://doi.org/10.3390/electronics13142817
Li W, Zhou G, Lin S, Tang Y. PerNet: Progressive and Efficient All-in-One Image-Restoration Lightweight Network. Electronics. 2024; 13(14):2817. https://doi.org/10.3390/electronics13142817
Chicago/Turabian StyleLi, Wentao, Guang Zhou, Sen Lin, and Yandong Tang. 2024. "PerNet: Progressive and Efficient All-in-One Image-Restoration Lightweight Network" Electronics 13, no. 14: 2817. https://doi.org/10.3390/electronics13142817
APA StyleLi, W., Zhou, G., Lin, S., & Tang, Y. (2024). PerNet: Progressive and Efficient All-in-One Image-Restoration Lightweight Network. Electronics, 13(14), 2817. https://doi.org/10.3390/electronics13142817