Illuminant Adaptive Wideband Image Synthesis Using Separated Base-Detail Layer Fusion Maps
Abstract
:1. Introduction
2. Related works
2.1. Synthesis of Visible and Near-Infrared Images
2.2. Joint Bilateral filter
3. Proposed method
3.1. Single Camera Module Time-Division Image Acquisition System
3.2. Base-Detail Layer Decomposition
3.3. Base Layer Synthesis
3.4. Detail Layer Synthesis
3.5. Color Compensation and Adjustment
4. Simulation Results
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Conflicts of Interest
References
- Kwon, H.J.; Lee, S.H. Visible and Near-Infrared Image Acquisition and Fusion for Night Surveillance. Chemosensors 2021, 9, 75. [Google Scholar] [CrossRef]
- Vanmali, A.V.; Gadre, V.M. Visible and NIR Image Fusion Using Weight Map Guided Laplacian Gaussian Pyramid for Improving Scene Visibility. Sādhanā 2017, 42, 1063–1082. [Google Scholar] [CrossRef]
- Li, H.; Wu, X.J. DenseFuse: A Fusion Approach to Infrared and Visible Images. IEEE Trans. Image Processing 2018, 28, 2614–2623. [Google Scholar] [CrossRef] [PubMed]
- Kil, T.; Cho, N.I. Image Fusion using RGB and Near Infrared Image. J. Broadcast Eng. 2016, 21, 515–524. [Google Scholar] [CrossRef]
- Sadeghipoor, Z.; Lu, Y.M.; Süsstrunk, S. Correlation-Based Joint Acquisition and Demosaicing of Visible and Near-Infrared Images. In Proceedings of the IEEE International Conference on Image Processing, Brussels, Belgium, 11–14 September 2011; pp. 3165–3168. [Google Scholar]
- Li, H.; Wu, X.J. Infrared and Visible Image Fusion Using Latent Low-Rank Representation. arXiv 2018, arXiv:1804.08992. [Google Scholar]
- Son, D.M.; Kwon, H.J.; Lee, S.H. Visible and Near-Infrared Image Synthesis Using PCA Fusion of Multiscale Layers. Appl. Sci. 2020, 10, 8702. [Google Scholar] [CrossRef]
- Son, D.M.; Kwon, H.J.; Lee, S.H. Visible and Near Infrared Image Fusion Using Base Tone-Compression and Detail Transform Fusion. Chemosensors 2022, 10, 124. [Google Scholar] [CrossRef]
- Paris, S.; Kornprobst, P.; Tumblin, J.; Durand, F. A Gentle Introduction to Bilateral Filtering and its Applications. In ACM SIGGRAPH; ACM Press: New York, NY, USA, 2007; p. 1. [Google Scholar]
- Tomasi, C.; Manduchi, R. Bilateral filtering for gray and color images. In Proceedings of the Sixth International Conference on Computer Vision(IEEE Cat. No. 98CH36271), Bombay, India, 7 January 1998; pp. 836–846. [Google Scholar]
- Petschnigg, G.; Szeliski, R.; Agrawala, M.; Cohen, M.; Hoppe, H.; Toyama, K. Digital Photography with Flash and No-Flash Image Pairs. ACM Trans. Graph. 2004, 23, 664–672. [Google Scholar] [CrossRef]
- Eisemann, E.; Durand, F. Flash Photography Enhancement via Intrinsic Relighting. ACM Trans. Graph. 2004, 23, 673–678. [Google Scholar] [CrossRef]
- Durand, F.; Dorsey, J. Fast bilateral filtering for the display of high dynamic range images. In Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques, San Antonio, TX, USA, 21–26 July 2002; pp. 257–266. [Google Scholar]
- EPFL 2015 EPFL Database. Available online: http://ivrg.epfl.ch/supplementary_material/cvpr11/index.html (accessed on 5 November 2015).
- Vu, C.T.; Chandler, D.M. S3: A spectral and spatial sharpness measure. In Proceedings of the 2009 First International Conference on Advances in Multimedia IEEE, Colmar, France, 20–25 July 2009; pp. 37–43. [Google Scholar]
- Han, Y.; Cai, Y.; Cao, Y.; Xu, X. A new image fusion performance metric based on visual information fidlity. Inf. Fusion 2013, 14, 127–135. [Google Scholar] [CrossRef]
- Eskicioglu, A.M.; Fisher, P.S. Image quality measures and their performance. IEEE Trans. Commun. 1995, 43, 2959–2965. [Google Scholar] [CrossRef] [Green Version]
- Hassen, R.; Wang, Z.; Salama, M.M. Image sharpness assessment based on local phase coherence. IEEE Trans. Image Processing 2013, 22, 2798–2810. [Google Scholar] [CrossRef] [PubMed]
- Fajalingam, B.; Priya, R. Hybrid nultimodality medical image fusion technique for feature enhancement in medical diagnosis. Int. J. Eng. Sci. Invent. 2018, 2, 52–60. [Google Scholar]
- Cui, G.; Feng, H.; Xu, Z.; Li, Q.; Chen, Y. Detail preserved fusion of visible and infrared images using regional saliency extraction and multi-scale image decomposition. Opt. Commun. 2015, 341, 199–209. [Google Scholar] [CrossRef]
Half Mixed | Low Rank | Dense Fusion | Vanmali | PCA Fusion | Proposed | |
---|---|---|---|---|---|---|
1 | 0.174178 | 0.181196 | 0.197622 | 0.233407 | 0.289241 | 0.298935 |
2 | 0.199256 | 0.209302 | 0.204024 | 0.242589 | 0.256526 | 0.324236 |
3 | 0.267765 | 0.293025 | 0.286023 | 0.312739 | 0.361716 | 0.442621 |
4 | 0.141611 | 0.153234 | 0.151603 | 0.184209 | 0.290325 | 0.246327 |
5 | 0.086545 | 0.099137 | 0.102088 | 0.118752 | 0.271392 | 0.146197 |
6 | 0.295408 | 0.313109 | 0.295229 | 0.326159 | 0.296745 | 0.450269 |
7 | 0.121372 | 0.198892 | 0.120727 | 0.191929 | 0.162646 | 0.197848 |
8 | 0.091421 | 0.149966 | 0.089868 | 0.134533 | 0.12752 | 0.149905 |
9 | 0.089133 | 0.148288 | 0.088384 | 0.141575 | 0.149946 | 0.139303 |
10 | 0.078376 | 0.156636 | 0.077333 | 0.136656 | 0.175766 | 0.129637 |
11 | 0.057808 | 0.1386 | 0.05744 | 0.16435 | 0.136583 | 0.126464 |
12 | 0.037551 | 0.131233 | 0.037351 | 0.13783 | 0.139274 | 0.088884 |
13 | 0.061953 | 0.141537 | 0.060919 | 0.150945 | 0.151045 | 0.110261 |
Average | 0.130952 | 0.178012 | 0.136047 | 0.190436 | 0.216056 | 0.219299 |
Half Mixed | Low Rank | Dense Fusion | Vanmali | PCA Fusion | Proposed | |
---|---|---|---|---|---|---|
1 | 0.745553 | 0.70428 | 0.891771 | 0.747645 | 0.998658 | 0.993871 |
2 | 0.649207 | 0.67164 | 0.673752 | 0.694098 | 0.681961 | 0.853196 |
3 | 0.590613 | 0.590075 | 0.668474 | 0.673026 | 0.833315 | 0.923921 |
4 | 0.571183 | 0.628726 | 0.635068 | 0.799879 | 0.743241 | 0.936521 |
5 | 0.486169 | 0.430399 | 0.63737 | 0.580149 | 0.665135 | 0.711031 |
6 | 0.705222 | 0.741236 | 0.705064 | 0.760859 | 0.718529 | 0.911802 |
7 | 0.827273 | 0.738855 | 0.822458 | 0.869483 | 0.842554 | 0.958695 |
8 | 0.851614 | 0.786688 | 0.836366 | 0.856757 | 0.829625 | 0.940923 |
9 | 0.803353 | 0.710701 | 0.793571 | 0.829319 | 0.809903 | 0.907141 |
10 | 0.778532 | 0.66576 | 0.768575 | 0.745719 | 0.812205 | 0.871855 |
11 | 0.68633 | 0.637163 | 0.68143 | 0.808235 | 0.719192 | 0.814258 |
12 | 0.694986 | 0.57363 | 0.691165 | 0.690373 | 0.768138 | 0.802015 |
13 | 0.663064 | 0.645432 | 0.653592 | 0.752853 | 0.692664 | 0.799822 |
Average | 0.696392 | 0.655737 | 0.727589 | 0.754492 | 0.778086 | 0.87885 |
Half Mixed | Low Rank | Dense Fusion | Vanmali | PCA Fusion | Proposed | |
---|---|---|---|---|---|---|
1 | 12.54686 | 14.14278 | 13.94147 | 15.9371 | 19.472 | 24.33865 |
2 | 12.4484 | 15.05581 | 12.68151 | 16.60634 | 17.24116 | 23.4287 |
3 | 17.34853 | 20.25724 | 18.68782 | 21.34845 | 26.16536 | 36.68218 |
4 | 10.14903 | 11.66165 | 10.66109 | 13.74866 | 19.43001 | 21.46389 |
5 | 7.238538 | 9.097301 | 8.040366 | 10.16365 | 18.85362 | 13.88883 |
6 | 20.75647 | 24.89999 | 20.74539 | 23.60679 | 21.74731 | 38.34033 |
7 | 16.09507 | 19.39597 | 15.99705 | 20.85826 | 18.50416 | 25.57275 |
8 | 13.67963 | 17.51154 | 13.48037 | 15.63094 | 16.24731 | 20.72835 |
9 | 12.45933 | 15.2962 | 12.35908 | 14.98069 | 14.04758 | 18.13125 |
10 | 14.35344 | 19.68436 | 14.16031 | 17.66891 | 20.58637 | 21.32011 |
11 | 11.42254 | 16.26538 | 11.313 | 16.53255 | 17.70956 | 18.81093 |
12 | 9.273672 | 16.03667 | 9.219986 | 13.74854 | 19.35755 | 13.9302 |
13 | 12.02791 | 20.40927 | 11.83664 | 17.52521 | 21.03067 | 19.64098 |
Average | 13.06149 | 16.90109 | 13.31724 | 16.79662 | 19.26097 | 22.79055 |
Half Mixed | Low Rank | Dense Fusion | Vanmali | PCA Fusion | Proposed | |
---|---|---|---|---|---|---|
1 | 0.955372 | 0.964734 | 0.960554 | 0.95496 | 0.960093 | 0.963412 |
2 | 0.951973 | 0.96309 | 0.952825 | 0.96345 | 0.962063 | 0.955613 |
3 | 0.940863 | 0.954856 | 0.945167 | 0.94834 | 0.94615 | 0.961987 |
4 | 0.933977 | 0.95032 | 0.937741 | 0.937994 | 0.940312 | 0.958902 |
5 | 0.912469 | 0.906715 | 0.922604 | 0.928372 | 0.937174 | 0.954875 |
6 | 0.947806 | 0.955256 | 0.94787 | 0.947966 | 0.947871 | 0.964539 |
7 | 0.963834 | 0.962881 | 0.96352 | 0.970573 | 0.96687 | 0.973125 |
8 | 0.956019 | 0.962156 | 0.955143 | 0.955262 | 0.952006 | 0.968308 |
9 | 0.956409 | 0.961442 | 0.95589 | 0.957297 | 0.955774 | 0.966371 |
10 | 0.973693 | 0.966751 | 0.973387 | 0.972475 | 0.972709 | 0.982896 |
11 | 0.958526 | 0.964662 | 0.958383 | 0.969902 | 0.968848 | 0.978657 |
12 | 0.95942 | 0.959802 | 0.95985 | 0.957279 | 0.94973 | 0.958169 |
13 | 0.955009 | 0.965001 | 0.954265 | 0.957888 | 0.950951 | 0.966305 |
Average | 0.951182 | 0.956744 | 0.952861 | 0.95552 | 0.954658 | 0.965628 |
Half Mixed | Low Rank | Dense Fusion | Vanmali | PCA Fusion | Proposed | |
---|---|---|---|---|---|---|
1 | 34.9795 | 38.2708 | 38.66148 | 42.32002 | 49.30071 | 64.73625 |
2 | 43.57379 | 51.91313 | 44.37411 | 55.37315 | 56.45746 | 77.99232 |
3 | 60.13544 | 67.86473 | 64.65341 | 73.72576 | 88.08473 | 121.8725 |
4 | 32.05627 | 34.73319 | 33.67106 | 44.62118 | 53.37883 | 65.84561 |
5 | 23.02605 | 26.08361 | 25.7862 | 30.6574 | 46.29512 | 41.06304 |
6 | 70.21422 | 79.1646 | 70.18694 | 81.35604 | 76.28592 | 127.8647 |
7 | 49.07052 | 61.13808 | 48.81656 | 63.21881 | 58.45182 | 73.80415 |
8 | 46.28894 | 57.17705 | 45.60989 | 53.38763 | 53.79862 | 66.33241 |
9 | 42.9718 | 53.02003 | 42.60357 | 53.82957 | 49.14437 | 58.22496 |
10 | 37.33375 | 51.14391 | 36.75772 | 47.23395 | 59.97219 | 58.27842 |
11 | 30.13547 | 44.43486 | 29.98567 | 51.42781 | 49.36001 | 50.71271 |
12 | 25.76822 | 42.30395 | 25.8431 | 42.59128 | 53.73518 | 41.73756 |
13 | 32.84225 | 52.98035 | 32.29235 | 51.9274 | 58.67464 | 56.20844 |
Average | 40.64586 | 50.78679 | 41.48016 | 53.20538 | 57.91843 | 69.59024 |
Half Mixed | Low Rank | Dense Fusion | Vanmali | PCA Fusion | Proposed | |
---|---|---|---|---|---|---|
1 | 3.880816 | 4.153757 | 4.299817 | 5.010667 | 6.093581 | 7.418859 |
2 | 4.516674 | 5.255388 | 4.598725 | 5.873233 | 6.03323 | 8.274568 |
3 | 6.478177 | 7.300047 | 6.966816 | 7.961987 | 9.733417 | 13.46038 |
4 | 3.322676 | 3.596682 | 3.48355 | 4.629657 | 6.418822 | 6.852565 |
5 | 2.443045 | 2.811208 | 2.712147 | 3.256962 | 5.96494 | 4.292975 |
6 | 7.359794 | 8.171345 | 7.356862 | 8.552003 | 7.804351 | 13.56777 |
7 | 4.66935 | 5.926646 | 4.645744 | 6.104861 | 5.591356 | 6.992199 |
8 | 4.355526 | 5.472404 | 4.293732 | 5.063539 | 5.076561 | 6.208429 |
9 | 4.132718 | 5.207001 | 4.098049 | 5.251618 | 4.762244 | 5.555363 |
10 | 3.414311 | 4.782535 | 3.362169 | 4.379015 | 5.719537 | 5.342915 |
11 | 2.798207 | 4.212974 | 2.785703 | 4.854306 | 4.732941 | 4.743542 |
12 | 2.377223 | 4.019897 | 2.387551 | 4.012972 | 5.201723 | 3.872744 |
13 | 3.028285 | 5.025304 | 2.978305 | 4.864133 | 5.689062 | 5.233637 |
Average | 4.059754 | 5.071938 | 4.151475 | 5.370381 | 6.063213 | 7.062765 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Park, C.-W.; Kwon, H.-J.; Lee, S.-H. Illuminant Adaptive Wideband Image Synthesis Using Separated Base-Detail Layer Fusion Maps. Appl. Sci. 2022, 12, 9441. https://doi.org/10.3390/app12199441
Park C-W, Kwon H-J, Lee S-H. Illuminant Adaptive Wideband Image Synthesis Using Separated Base-Detail Layer Fusion Maps. Applied Sciences. 2022; 12(19):9441. https://doi.org/10.3390/app12199441
Chicago/Turabian StylePark, Cheul-Woo, Hyuk-Ju Kwon, and Sung-Hak Lee. 2022. "Illuminant Adaptive Wideband Image Synthesis Using Separated Base-Detail Layer Fusion Maps" Applied Sciences 12, no. 19: 9441. https://doi.org/10.3390/app12199441
APA StylePark, C.-W., Kwon, H.-J., & Lee, S.-H. (2022). Illuminant Adaptive Wideband Image Synthesis Using Separated Base-Detail Layer Fusion Maps. Applied Sciences, 12(19), 9441. https://doi.org/10.3390/app12199441