Underwater Image Enhancement Based on Multi-Scale Fusion and Global Stretching of Dual-Model
Abstract
:1. Introduction
2. Methodology
2.1. Color Correction Based on White-Balancing
2.2. Multi-Scale Fusion with Updated Saliency Weight
2.3. Global Stretching of Dual-Model
3. Results and Discussion
4. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Abbreviations
MFGS | Multi-scale fusion and global stretching of dual-model |
RGB | Red, green, blue |
CIE-Lab | Commission International Eclairage-Lab |
UIEBD | Underwater image enhancement benchmark dataset |
IBLA | Image blurring and light absorption |
GDCP | Generalized dark channel prior |
RAHIM | Recursive adaptive histogram modification |
HSV | Hue, saturation, value |
RGHS | Relative global histogram stretching |
GLHDF | Global and local equalization of the histogram and dual-image multi-scale fusion |
CNN | Convolutional neural networks |
GAN | Generative adversarial network |
UCM | Unsupervised color correction method |
ICM-RD | Integrated color model-Rayleigh distribution |
RD | Rayleigh distribution |
PCA | Principal component analysis |
UIQM | Underwater image quality measure |
PCQI | Patch-based contrast quality index |
UCIQE | Underwater color image quality evaluation |
References
- Mazel, C.H. In situ measurement of reflectance and fluorescence spectra to support hyperspectral remote sensing and marine biology research. In Proceedings of the OCEANS 2006, Boston, MA, USA, 18–21 September 2006; pp. 1–4. [Google Scholar]
- Hou, G.J.; Luan, X.; Song, D.L.; Ma, X.Y. Underwater man-made object recognition on the basis of color and shape features. J. Coast. Res. 2016, 32, 1135–1141. [Google Scholar] [CrossRef]
- Foresti, G. Visual inspection of sea bottom structures by an autonomous underwater vehicle. IEEE Trans. Syst. Man Cybern. Part B (Cybernetics) 2001, 31, 691–705. [Google Scholar] [CrossRef] [PubMed]
- Lu, H.; Li, Y.; Zhang, L.; Serikawa, S. Contrast enhancement for images in turbid water. JOSA A 2015, 32, 886–893. [Google Scholar] [CrossRef] [Green Version]
- Lu, H.; Guna, J.; Zhou, Q. Preface: Optical imaging for extreme environment. Opt. Laser Technol. 2019, 110, 1. [Google Scholar] [CrossRef]
- Jaffe, J. Computer modeling and the design of optimal underwater imaging systems. IEEE J. Ocean. Eng. 1990, 15, 101–111. [Google Scholar] [CrossRef]
- Peng, Y.T.; Cosman, P.C. Underwater image restoration based on image blurriness and light absorption. IEEE Trans. Image Process. 2017, 26, 1579–1594. [Google Scholar] [CrossRef]
- Peng, Y.T.; Cao, K.; Cosman, P.C. Generalization of the Dark Channel Prior for Single Image Restoration. IEEE Trans. Image Process. 2018, 27, 2856–2868. [Google Scholar] [CrossRef] [PubMed]
- Wang, Y.; Song, W.; Fortino, G.; Qi, L.-Z.; Zhang, W.; Liotta, A. An Experimental-Based Review of Image Enhancement and Image Restoration Methods for Underwater Imaging. IEEE Access 2019, 7, 140233–140251. [Google Scholar] [CrossRef]
- Schettini, R.; Corchs, S. Underwater image processing: State of the art of restoration and image enhancement methods. EURASIP J. Adv. Signal Process. 2010, 2010, 1–14. [Google Scholar] [CrossRef] [Green Version]
- Ghani, A.S.A.; Isa, N.A.M. Automatic system for improving underwater image contrast and color through recursive adaptive histogram modification. Comput. Electron. Agric. 2017, 141, 181–195. [Google Scholar] [CrossRef] [Green Version]
- Ancuti, C.O.; Ancuti, C.; De Vleeschouwer, C.; Bekaert, P. Color Balance and Fusion for Underwater Image Enhancement. IEEE Trans. Image Process. 2017, 27, 379–393. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Huang, D.; Wang, Y.; Song, W.; Sequeira, J.; Mavromatis, S. Shallow-water image enhancement using relative global histogram stretching based on adaptive parameter acquisition. In International Conference on Multimedia Modeling; Springer: Cham, Switzerland, 2018; pp. 453–465. [Google Scholar]
- Bai, L.; Zhang, W.; Pan, X.; Zhao, C. Underwater image enhancement based on global and local equalization of histogram and dual-image multi-scale fusion. IEEE Access 2020, 8, 128973–128990. [Google Scholar] [CrossRef]
- Han, F.; Yao, J.; Zhu, H.; Wang, C. Underwater Image Processing and Object Detection Based on Deep CNN Method. J. Sens. 2020, 2020, 1–20. [Google Scholar] [CrossRef]
- Li, J.; Skinner, K.A.; Eustice, R.M.; Johnson-Roberson, M. WaterGAN: Unsupervised generative network to enable real-time color correction of monocular underwater images. IEEE Robot. Autom. Lett. 2017, 3, 387–394. [Google Scholar] [CrossRef] [Green Version]
- Wang, N.; Zhou, Y.; Han, F.; Zhu, H.; Zheng, Y. UWGAN: Underwater GAN for real-world underwater color restoration and dehazing. arXiv 2019, arXiv:1912.10269. [Google Scholar]
- Hou, G.; Zhao, X.; Pan, Z.; Yang, H.; Tan, L.; Li, J. Benchmarking Underwater Image Enhancement and Restoration, and Beyond. IEEE Access 2020, 8, 122078–122091. [Google Scholar] [CrossRef]
- Li, C.; Guo, C.; Ren, W.; Cong, R.; Hou, J.; Kwong, S.; Tao, D. An Underwater Image Enhancement Benchmark Dataset and Beyond. IEEE Trans. Image Process. 2020, 29, 4376–4389. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Anwar, S.; Li, C. Diving deeper into underwater image enhancement: A survey. Signal Process. Image Commun. 2020, 89, 115978. [Google Scholar] [CrossRef]
- Iqbal, K.; Odetayo, M.; James, A.; Salam, R.A.; Talib, A.Z.H. Enhancing the low quality images using unsupervised colour correction method. In Proceedings of the 2010 IEEE International Conference on Systems, Man and Cybernetics, Istanbul, Turkey, 10–13 October 2010; pp. 1703–1709. [Google Scholar]
- Ghani, A.S.A.; Isa, N.A.M. Underwater image quality enhancement through integrated color model with Rayleigh distribution. Appl. Soft Comput. 2015, 27, 219–230. [Google Scholar] [CrossRef]
- Fu, H.; Cao, X.; Tu, Z. Cluster-based co-saliency detection. IEEE Trans. Image Process. 2013, 22, 3766–3778. [Google Scholar] [CrossRef] [Green Version]
- Cheng, D.; Prasad, D.K.; Brown, M.S. Illuminant estimation for color constancy: Why spatial-domain methods work and the role of the color distribution. JOSA A 2014, 31, 1049–1058. [Google Scholar] [CrossRef]
- Ebner, M. White Patch Retinex. In Color Constancy; John Wiley & Sons Ltd.: Chichester, UK, 2007; pp. 104–105. ISBN 978-0-470-05829-9. [Google Scholar]
- Ebner, M. The Gray World Assumption. In Color Constancy; John Wiley & Sons: Chichester, UK, 2007; pp. 106–108. ISBN 978-0-470-05829-9. [Google Scholar]
- Grundland, M.; Vohra, R.; Williams, G.P.; Dodgson, N.A. Cross Dissolve Without Cross Fade: Preserving Contrast, Color and Salience in Image Compositing. Comput. Graph. Forum 2010, 25, 577–586. [Google Scholar] [CrossRef] [Green Version]
- Ancuti, C.; Bekaert, P. Effective Single Image Dehazing by Fusion. In Proceedings of the IEEE International Conference on Image Processing, Hong Kong, China, 26–29 September 2010. [Google Scholar]
- Bennett, E.P.; Mason, J.L.; Mcmillan, L. Multispectral Bilateral Video Fusion; IEEE Press: New York, NY, USA, 2007. [Google Scholar]
- Ancuti, C.; Ancuti, C.O.; Haber, T.; Bekaert, P. Enhancing underwater images and videos by fusion. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, 16–21 June 2012; pp. 81–88. [Google Scholar]
- Achanta, R.; Hemami, S.; Estrada, F.; Susstrunk, S. Frequency-tuned salient region detection. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20–25 June 2009; pp. 1597–1604. [Google Scholar]
- Burt, P.J.; Adelson, E.H. The Laplacian pyramid as a compact image code. In Readings in Computer Vision; Morgan Kaufmann: Burlington, MA, USA, 1987; pp. 671–679. [Google Scholar]
- Mangeruga, M.; Cozza, M.; Bruno, F. Evaluation of underwater image enhancement algorithms under different environmental conditions. J. Mar. Sci. Eng. 2018, 6, 10. [Google Scholar] [CrossRef] [Green Version]
- Mangeruga, M.; Bruno, F.; Cozza, M.; Agrafiotis, P.; Skarlatos, D. Guidelines for Underwater Image Enhancement Based on Benchmarking of Different Methods. Remote Sens. 2018, 10, 1652. [Google Scholar] [CrossRef] [Green Version]
- Panetta, K.; Gao, C.; Agaian, S. Human-Visual-System-Inspired Underwater Image Quality Measures. IEEE J. Ocean. Eng. 2016, 41, 541–551. [Google Scholar] [CrossRef]
- Wang, S.; Ma, K.; Yeganeh, H.; Wang, Z.; Lin, W. A Patch-Structure Representation Method for Quality Assessment of Contrast Changed Images. IEEE Signal Process. Lett. 2015, 22, 2387–2390. [Google Scholar] [CrossRef]
- Yang, M.; Sowmya, A. An underwater color image quality evaluation metric. IEEE Trans. Image Process. 2015, 24, 6062–6071. [Google Scholar] [CrossRef] [PubMed]
Data A | RD [22] | RGHS [13] | IBLA [7] | ||||||
---|---|---|---|---|---|---|---|---|---|
UIQM | PCQI | UCIQE | UIQM | PCQI | UCIQE | UIQM | PCQI | UCIQE | |
1 | 1.986 | 1.340 | 34.383 | 2.273 | 1.110 | 31.028 | 2.386 | 1.073 | 13.808 |
2 | 2.952 | 1.156 | 34.328 | 3.091 | 1.190 | 31.589 | 2.419 | 1.056 | 28.186 |
3 | 2.983 | 1.042 | 34.117 | 3.142 | 1.121 | 30.180 | 2.676 | 1.009 | 20.676 |
4 | 2.666 | 1.287 | 32.500 | 2.048 | 1.304 | 31.315 | 1.914 | 1.113 | 13.040 |
5 | 2.500 | 1.260 | 35.688 | 2.254 | 1.315 | 31.859 | 1.636 | 1.108 | 17.175 |
6 | 2.745 | 1.058 | 35.087 | 2.034 | 0.967 | 31.491 | 2.502 | 0.952 | 17.076 |
7 | 1.336 | 1.240 | 38.017 | 1.629 | 1.295 | 32.355 | 1.713 | 1.078 | 8.190 |
Average | 2.453 | 1.197 | 34.874 | 2.353 | 1.186 | 31.402 | 2.178 | 1.055 | 16.879 |
Data A | GLHDF [14] | Fusion [12] | Our Method | ||||||
UIQM | PCQI | UCIQE | UIQM | PCQI | UCIQE | UIQM | PCQI | UCIQE | |
1 | 2.549 | 1.270 | 33.709 | 2.494 | 1.227 | 15.905 | 2.499 | 1.237 | 31.477 |
2 | 2.826 | 1.193 | 32.767 | 3.139 | 1.216 | 18.166 | 3.213 | 1.260 | 31.189 |
3 | 2.799 | 1.101 | 32.099 | 3.048 | 1.162 | 14.192 | 3.135 | 1.134 | 30.207 |
4 | 2.505 | 1.314 | 32.871 | 3.201 | 1.273 | 13.343 | 3.190 | 1.329 | 29.357 |
5 | 2.588 | 1.284 | 34.755 | 2.992 | 1.314 | 12.406 | 2.750 | 1.303 | 29.697 |
6 | 2.624 | 1.148 | 28.952 | 2.897 | 1.153 | 13.667 | 2.880 | 1.113 | 29.289 |
7 | 2.670 | 1.301 | 36.889 | 1.756 | 1.309 | 16.604 | 1.660 | 1.261 | 34.998 |
Average | 2.652 | 1.230 | 33.149 | 2.790 | 1.236 | 14.898 | 2.761 | 1.234 | 30.888 |
Data B/C | RD [22] | RGHS [13] | IBLA [7] | ||||||
---|---|---|---|---|---|---|---|---|---|
UIQM | PCQI | UCIQE | UIQM | PCQI | UCIQE | UIQM | PCQI | UCIQE | |
1 | 2.749 | 1.160 | 31.477 | 2.753 | 1.200 | 30.347 | 1.640 | 0.956 | 35.010 |
2 | 3.413 | 1.225 | 34.612 | 3.319 | 1.230 | 28.946 | 0.860 | 0.727 | 36.087 |
3 | 2.362 | 1.101 | 33.240 | 2.440 | 1.089 | 31.440 | 0.652 | 0.638 | 35.764 |
4 | 3.139 | 1.083 | 33.813 | 2.997 | 1.097 | 30.929 | 1.286 | 0.706 | 36.287 |
5 | 2.240 | 1.217 | 35.177 | 2.034 | 1.150 | 32.355 | 1.423 | 0.816 | 39.603 |
6 | 1.803 | 1.077 | 35.512 | 1.836 | 1.099 | 34.175 | 2.395 | 1.055 | 34.993 |
7 | 1.629 | 1.014 | 37.796 | 1.417 | 1.001 | 36.915 | 0.983 | 1.004 | 35.528 |
Average | 2.476 | 1.125 | 34.518 | 2.399 | 1.124 | 32.158 | 1.320 | 0.843 | 36.182 |
Data B/C | GLHDF [14] | Fusion [12] | Our method | ||||||
UIQM | PCQI | UCIQE | UIQM | PCQI | UCIQE | UIQM | PCQI | UCIQE | |
1 | 2.942 | 1.177 | 30.530 | 3.083 | 1.196 | 18.335 | 2.773 | 1.234 | 29.653 |
2 | 3.492 | 1.181 | 32.849 | 3.503 | 1.218 | 19.584 | 3.323 | 1.301 | 31.501 |
3 | 2.776 | 1.028 | 31.055 | 2.688 | 1.193 | 16.249 | 2.623 | 1.200 | 31.853 |
4 | 3.033 | 1.086 | 31.370 | 3.160 | 1.069 | 28.215 | 3.203 | 1.151 | 30.497 |
5 | 2.792 | 1.114 | 33.074 | 2.780 | 1.285 | 15.220 | 2.274 | 1.272 | 32.042 |
6 | 2.491 | 1.068 | 34.115 | 2.105 | 1.198 | 23.082 | 2.001 | 1.131 | 30.447 |
7 | 2.316 | 0.946 | 37.858 | 1.810 | 1.120 | 18.632 | 1.554 | 1.067 | 29.719 |
Average | 2.835 | 1.086 | 32.979 | 2.733 | 1.183 | 19.902 | 2.536 | 1.194 | 30.816 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Song, H.; Wang, R. Underwater Image Enhancement Based on Multi-Scale Fusion and Global Stretching of Dual-Model. Mathematics 2021, 9, 595. https://doi.org/10.3390/math9060595
Song H, Wang R. Underwater Image Enhancement Based on Multi-Scale Fusion and Global Stretching of Dual-Model. Mathematics. 2021; 9(6):595. https://doi.org/10.3390/math9060595
Chicago/Turabian StyleSong, Huajun, and Rui Wang. 2021. "Underwater Image Enhancement Based on Multi-Scale Fusion and Global Stretching of Dual-Model" Mathematics 9, no. 6: 595. https://doi.org/10.3390/math9060595
APA StyleSong, H., & Wang, R. (2021). Underwater Image Enhancement Based on Multi-Scale Fusion and Global Stretching of Dual-Model. Mathematics, 9(6), 595. https://doi.org/10.3390/math9060595