Achieving Higher Resolution Lake Area from Remote Sensing Images Through an Unsupervised Deep Learning Super-Resolution Method
Abstract
:1. Introduction
- (1)
- The deep learning-based SR technique is first introduced to the lake area extraction process to improve the spatial resolution and generate a finer lake area.
- (2)
- A new unsupervised SR model UDGN is proposed based on a deep residual network in this paper. It does not require pretraining and can be adapted to different settings of images, such as different image sizes and channels.
- (3)
- The features of the gradient map are extracted and fused in the network to provide more geographic details in HR images.
- (4)
- We verify the effectiveness of our method with two data sets, the results demonstrate the superiority of our method in improving the spatial resolution of lake area extraction.
2. Materials and Methods
2.1. Study Area and Data
2.2. Overview of the Proposed Method
2.3. Unsupervised Super-Resolution Mechanism
2.4. The Structure of the UDGN Model
- I.
- Deep: it can efficiently extract deep features and complete multi-spectral RS-SR tasks.
- II.
- Geographic information preservation: By the fusion of the gradient information to enhance the original image, more geoinformation such as terrain and texture can be preserved, which provides a good foundation for subsequent LAE.
- III.
- Adaptive: it can super-resolve RS images/products of different image sizes and channels.
2.5. Evaluation Criteria
3. Results
3.1. Implementation Details
3.1.1. Architecture Details of UDGN
3.1.2. Training Data Extraction
3.2. Results of Two Strategies
3.3. Comparison with Different SR Methods
3.4. Results of Lake Extraction from MODIS Data
4. Discussion
5. Conclusions
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- O’Reilly, C.M.; Rowley, R.J.; Schneider, P.; Lenters, J.D.; Mcintyre, P.B.; Kraemer, B.M. Rapid and highly variable warming of lake surface waters around the globe. Geophys. Res. Lett. 2015, 42, 10773–10781. [Google Scholar] [CrossRef] [Green Version]
- Adrian, R.; O’Reilly, C.M.; Zagarese, H.; Baines, S.B.; Hessen, D.O.; Keller, W.; Livingstone, D.M.; Sommaruga, R.; Straile, D.; Van Donk, E.; et al. Lakes as sentinels of climate change. Limnol. Oceanogr. 2009, 54, 2283–2297. [Google Scholar] [CrossRef] [PubMed]
- Zhang, G.; Yao, T.; Chen, W.; Zheng, G.; Shum, C.K.; Yang, K.; Piao, S.; Sheng, Y.; Yi, S.; Li, J.; et al. Regional differences of lake evolution across China during 1960s–2015 and its natural and anthropogenic causes. Remote Sens. Environ. 2019, 221, 386–404. [Google Scholar] [CrossRef]
- Qiao, B.; Zhu, L.; Yang, R. Temporal-spatial differences in lake water storage changes and their links to climate change throughout the Tibetan Plateau. Remote Sens. Environ. 2019, 222, 232–243. [Google Scholar] [CrossRef]
- Lei, Y.; Yao, T.; Yang, K.; Sheng, Y.; Kleinherenbrink, M.; Yi, S.; Bird, B.W.; Zhang, X.; Zhu, L.; Zhang, G. Lake seasonality across the Tibetan Plateau and their varying relationship with regional mass changes and local hydrology. Geophys. Res. Lett. 2017, 44, 892–900. [Google Scholar] [CrossRef] [Green Version]
- Song, C.; Huang, B.; Ke, L. Modeling and analysis of lake water storage changes on the Tibetan Plateau using multi-mission satellite data. Remote Sens. Environ. 2013, 135, 25–35. [Google Scholar] [CrossRef]
- Lei, Y.; Yang, K.; Wang, B.; Sheng, Y.; Bird, B.W.; Zhang, G.; Tian, L. Response of inland lake dynamics over the Tibetan Plateau to climate change. Clim. Change 2014, 125, 281–290. [Google Scholar] [CrossRef]
- Zhang, G.; Yao, T.; Piao, S.; Bolch, T.; Xie, H.; Chen, D.; Gao, Y.; O’Reilly, C.M.; Shum, C.K.; Yang, K.; et al. Extensive and drastically different alpine lake changes on Asia’s high plateaus during the past four decades. Geophys. Res. Lett. 2017, 44, 252–260. [Google Scholar] [CrossRef] [Green Version]
- Li, L.; Chen, Y.; Xu, T.; Liu, R.; Shi, K.; Huang, C. Super-resolution mapping of wetland inundation from remote sensing imagery based on integration of back-propagation neural network and genetic algorithm. Remote Sens. Environ. 2015, 164, 142–154. [Google Scholar] [CrossRef]
- Xu, Z.; Chen, Z.; Yi, W.; Gui, Q.; Hou, W.; Ding, M. Deep gradient prior network for DEM super-resolution: Transfer learning from image to DEM. ISPRS J. Photogramm. Remote Sens. 2019, 150, 80–90. [Google Scholar] [CrossRef]
- Xu, Z.; Wang, X.; Chen, Z.; Xiong, D.; Ding, M.; Hou, W. Nonlocal similarity based DEM super resolution. ISPRS J. Photogramm. Remote Sens. 2015, 110, 48–54. [Google Scholar] [CrossRef]
- Ma, W.; Pan, Z.; Guo, J.; Lei, B. Achieving super-resolution remote sensing images via the wavelet transform combined with the recursive Res-Net. IEEE Trans. Geosci. Remote Sens. 2019, 57, 3512–3527. [Google Scholar] [CrossRef]
- Qin, M.; Mavromatis, S.; Hu, L.; Zhang, F.; Liu, R. Remote Sensing Single-Image Resolution Improvement Using A Deep Gradient-Aware Network with Image-Specific Enhancement. Remote Sens. 2020, 12, 758. [Google Scholar] [CrossRef] [Green Version]
- Lanaras, C.; Bioucas-Dias, J.; Galliani, S.; Baltsavias, E.; Schindler, K. Super-resolution of Sentinel-2 images: Learning a globally applicable deep neural network. ISPRS J. Photogramm. Remote Sens. 2018, 146, 305–319. [Google Scholar] [CrossRef] [Green Version]
- Xie, W.; Li, Y.; Lei, J. Hyperspectral image super-resolution using deep feature matrix factorization. IEEE Trans. Geosci. Remote Sens. 2019, 57, 6055–6067. [Google Scholar] [CrossRef]
- Han, W.; Chang, S.; Liu, D.; Yu, M.; Witbrock, M.; Huang, T.S. Image Super-Resolution via Dual-State Recurrent Networks. Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 2018, 1654–2013. [Google Scholar] [CrossRef] [Green Version]
- Haris, M.; Shakhnarovich, G.; Ukita, N. Deep back-projection networks for super-resolution. Proc. IEEE Conf. Comput. Vis. pattern Recognit. 2018, 1664–1673. [Google Scholar] [CrossRef] [Green Version]
- Yamanaka, J.; Kuwashima, S.; Kurita, T. Fast and Accurate Image Super Resolution by Deep CNN with Skip Connection and Network in Network. Int. Conf. Neural Inf. Process. 2017, 217–225. [Google Scholar] [CrossRef] [Green Version]
- Kim, J.; Lee, J.K.; Lee, K.M. Accurate image super-resolution using very deep convolutional networks. Proc. IEEE Conf. Comput. Vis. pattern Recognit. 2016, 1646–1654. [Google Scholar] [CrossRef] [Green Version]
- Tian, C.; Xu, Y.; Fei, L.; Yan, K. Deep learning for image super-resolution: A survey. IEEE Trans. Pattern Anal. Mach. Int. 2018, 1–23. [Google Scholar] [CrossRef] [Green Version]
- Shocher, A.; Cohen, N.; Irani, M.; Math, A. “Zero-Shot” super-resolution using deep internal learning. Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 2018. [Google Scholar] [CrossRef] [Green Version]
- Sun, J.; Xu, Z.; Shum, H.-Y. Image super-resolution using gradient profile prior. In Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska, 23–28 June 2008; pp. 1–8. [Google Scholar]
- Irani, M.; Peleg, S. Improving resolution by image registration. CVGIP Graph. Model. image Process. 1991, 53, 231–239. [Google Scholar] [CrossRef]
- Huang, J.-B.; Singh, A.; Ahuja, N. Single image super-resolution from transformed self-exemplars. Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 2015, 5197–5206. [Google Scholar] [CrossRef]
- Haut, J.M.; Fernandez-Beltran, R.; Paoletti, M.E.; Plaza, J.; Plaza, A.; Pla, F. A new deep generative network for unsupervised remote sensing single-image super-resolution. IEEE Trans. Geosci. Remote Sens. 2018, 11, 6792–6810. [Google Scholar] [CrossRef]
- Bulat, A.; Yang, J.; Tzimiropoulos, G. To learn image super-resolution, use a GAN to learn how to do image degradation first. In Proceedings of the The European Conference on Computer Vision (ECCV), Munich, Germany, 13–18 September 2018. [Google Scholar]
- Yuan, Y.; Liu, S.; Zhang, J.; Zhang, Y.; Dong, C.; Lin, L. Unsupervised image super-resolution using cycle-in-cycle generative adversarial networks. Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Work. 2018, 814–823. [Google Scholar] [CrossRef] [Green Version]
- Yasuhiro, S. Journal of Geophysical Research: Preface. J. Geophys. Res. Atmos. 2015, 120, 4764–4782. [Google Scholar]
- Wan, W.; Long, D.; Hong, Y.; Ma, Y.; Yuan, Y.; Xiao, P.; Duan, H.; Han, Z.; Gu, X. A lake data set for the Tibetan Plateau from the 1960s, 2005, and 2014. Sci. Data 2016, 3, 1–13. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Tao, S.; Fang, J.; Zhao, X.; Zhao, S.; Shen, H.; Hu, H.; Tang, Z.; Wang, Z. Rapid loss of lakes on the Mongolian Plateau. 2015, 112, 2281–2286. [Google Scholar] [CrossRef] [Green Version]
- Xu, H. Modification of normalised difference water index (NDWI) to enhance open water features in remotely sensed imagery. Int. J. Remote Sens. 2006, 14, 3025–3033. [Google Scholar] [CrossRef]
- Mcfeeters, S.K. The use of the Normalized Difference Water Index (NDWI) in the delineation of open water features. Int. J. Remote Sens. 1996, 17, 1425–1432. [Google Scholar] [CrossRef]
- Zontak, M.; Irani, M. Internal Statistics of a Single Natural Image. CVPR 2011, 977–984. [Google Scholar] [CrossRef]
- Glasner, D.; Bagon, S.; Irani, M. Super-Resolution from a Single Image. In Proceedings of the 2009 IEEE 12th international conference on computer vision, Kyoto, Japan, 29 September–2 October 2009; pp. 349–356. [Google Scholar]
- Sobel, I. An Isotropic 3x3 Image Gradient Operator. Present. Stanford A.I. Proj. 1968, 271–272. [Google Scholar]
- Shi, W.; Caballero, J.; Huszár, F.; Totz, J.; Aitken, A.P.; Bishop, R.; Rueckert, D.; Wang, Z. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 2016, 1874–1883. [Google Scholar] [CrossRef] [Green Version]
- Huynh-Thu, Q.; Ghanbari, M. Scope of validity of PSNR in image/video quality assessment. Electron. Lett. 2008, 44, 800–801. [Google Scholar] [CrossRef]
- Wang, Z.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P. Image quality assessment: From error visibility to structural similarity. IEEE Trans. image Process. 2004, 13, 600–612. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Yuhas, R.H.; Goetz, A.F.; Boardman, J.W. Discrimination among semi-arid landscape endmembers using the Spectral Angle Mapper (SAM) algorithm. Proc. Summ. Annu. JPL Airborne Geosci. Work. 1992, 147–149. [Google Scholar]
- Wang, X.; Ling, F.; Yao, H.; Liu, Y.; Xu, S. Unsupervised Sub-Pixel Water Body Mapping with Sentinel-3 OLCI Image. Remote Sens. 2019, 11, 327. [Google Scholar] [CrossRef] [Green Version]
- Li, L.; Xu, T.; Chen, Y. Improved urban flooding mapping from remote sensing images using generalized regression neural network-based super-resolution algorithm. Remote Sens. 2016, 8, 625. [Google Scholar] [CrossRef] [Green Version]
- Dong, C.; Loy, C.C.; He, K.; Tang, X. Learning a Deep Convolutional Network for Image Super-Resolution; Springer: Berlin/Heidelberg, Germany, 2014; pp. 184–199. [Google Scholar]
Properties | Study area1 | Study area2 | ||
---|---|---|---|---|
Location | 28.727°-29.203N°, 90.365°-91.085°E | 32.672°-33.278°N, 87.572°-88.480°E | 32.675°-33.274°N, 87.574°-88.478°E | 32.849°-33.277°N, 88.047°-88.497°E |
Image Data | Landsat 8 OLI image | Landsat 8 OLI image | MODIS image | Sentinel 2 image |
Date | October 15, 2014 | October 13, 2014 August 18, 2017 | October 13, 2014 August 18, 2017 | August 11, 2017 |
Image size | 2688 × 1760 | 3354 × 2220 | 202 × 135 | 5015 × 4767 |
Image resolution | 30m | 30m | 500m | 10m |
Layer | Kernel Size | Number of Kernels | Output Size | Stride | |
---|---|---|---|---|---|
Feature Fusion | CNN1 | 32 | 1 | ||
CNN2 | 32 | 1 | |||
Deep feature extraction | 64 | 1 | |||
ResBlock1 | 64 | 1 | |||
ResBlock2 | 64 | 1 | |||
ResBlock3 | 64 | 1 | |||
ResBlock4 | 64 | 1 | |||
ResBlock5 | 64 | 1 | |||
64 | 1 | ||||
Upsampling | 64 | 1 | |||
pixel-shuffle | |||||
pixel-shuffle | |||||
Reconstruction | Interpolation | ||||
Element-wise sum |
Evaluation Criteria | Upscale Factor | Strategy 1 | Strategy 2 |
---|---|---|---|
OA | 2 | 0.9930 | 0.9986 |
AUA | 2 | 0.9993 | 0.9942 |
APA | 2 | 0.9627 | 0.9978 |
kappa | 2 | 0.9764 | 0.9951 |
OA | 4 | 0.9902 | 0.9972 |
AUA | 4 | 0.9987 | 0.9869 |
APA | 4 | 0.9487 | 0.9972 |
kappa | 4 | 0.9670 | 0.9904 |
OA | 8 | 0.9809 | 0.9918 |
AUA | 8 | 0.9977 | 0.9722 |
APA | 8 | 0.9046 | 0.9813 |
kappa | 8 | 0.9372 | 0.9718 |
Upscale Factors | Evaluation Criteria | BCI | IBP | TSR | SRCNN | ZSSR | UDGN |
---|---|---|---|---|---|---|---|
2 | OA | 0.9901 | 0.9803 | 0.9871 | 0.9871 | 0.9866 | 0.9930 |
AUA | 0.9998 | 0.9997 | 0.9995 | 0.9983 | 0.9991 | 0.9993 | |
APA | 0.9472 | 0.9005 | 0.9327 | 0.9337 | 0.9306 | 0.9627 | |
kappa | 0.9667 | 0.9355 | 0.9570 | 0.9571 | 0.9554 | 0.9764 | |
PSNR | 33.4038 | 34.4224 | 37.2856 | 34.9307 | 39.0759 | 39.3095 | |
SSIM | 0.9745 | 0.9750 | 0.9819 | 0.9741 | 0.9839 | 0.9858 | |
NRMSE | 0.0214 | 0.0190 | 0.0137 | 0.0179 | 0.0111 | 0.0108 | |
SAM | 0.0662 | 0.0588 | 0.0423 | 0.0551 | 0.0344 | 0.0335 | |
4 | OA | 0.9783 | 0.9631 | 0.9694 | 0.9801 | 0.9715 | 0.9902 |
AUA | 0.9996 | 0.9996 | 0.9996 | 0.9983 | 0.9988 | 0.9987 | |
APA | 0.8914 | 0.8280 | 0.8534 | 0.9006 | 0.8626 | 0.9487 | |
kappa | 0.9291 | 0.8830 | 0.9019 | 0.9348 | 0.9082 | 0.9670 | |
PSNR | 29.4002 | 30.1131 | 33.3130 | 29.8395 | 34.5430 | 35.1123 | |
SSIM | 0.9481 | 0.9387 | 0.9600 | 0.9481 | 0.9628 | 0.9726 | |
NRMSE | 0.0339 | 0.0312 | 0.0216 | 0.0322 | 0.0187 | 0.0176 | |
SAM | 0.1050 | 0.0967 | 0.0669 | 0.0986 | 0.0556 | 0.0543 | |
8 | OA | 0.9562 | 0.9284 | 0.9390 | 0.9594 | 0.9386 | 0.9809 |
AUA | 0.9996 | 0.9995 | 0.9994 | 0.9980 | 0.9992 | 0.9977 | |
APA | 0.8022 | 0.7126 | 0.7444 | 0.8150 | 0.7434 | 0.9046 | |
kappa | 0.8631 | 0.7881 | 0.8158 | 0.8723 | 0.8148 | 0.9372 | |
PSNR | 25.9517 | 26.5812 | 29.1790 | 23.8941 | 29.5749 | 30.0654 | |
SSIM | 0.9081 | 0.8873 | 0.9232 | 0.9146 | 0.9234 | 0.9459 | |
NRMSE | 0.0504 | 0.0469 | 0.0348 | 0.0639 | 0.0332 | 0.0314 | |
SAM | 0.1565 | 0.1456 | 0.1078 | 0.1698 | 0.1029 | 0.0973 |
Lake Name | Pongyin Co | Timachaka | Noname Lake |
---|---|---|---|
MODIS image (km2) | 88.25 | 6.75 | 0 |
Predicted HR image (km2) | 85.3515 | 7.5051 | 0.2664 |
Reference data (km2) | 75.594858 | 7.439273 | 1.104846 |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Qin, M.; Hu, L.; Du, Z.; Gao, Y.; Qin, L.; Zhang, F.; Liu, R. Achieving Higher Resolution Lake Area from Remote Sensing Images Through an Unsupervised Deep Learning Super-Resolution Method. Remote Sens. 2020, 12, 1937. https://doi.org/10.3390/rs12121937
Qin M, Hu L, Du Z, Gao Y, Qin L, Zhang F, Liu R. Achieving Higher Resolution Lake Area from Remote Sensing Images Through an Unsupervised Deep Learning Super-Resolution Method. Remote Sensing. 2020; 12(12):1937. https://doi.org/10.3390/rs12121937
Chicago/Turabian StyleQin, Mengjiao, Linshu Hu, Zhenhong Du, Yi Gao, Lianjie Qin, Feng Zhang, and Renyi Liu. 2020. "Achieving Higher Resolution Lake Area from Remote Sensing Images Through an Unsupervised Deep Learning Super-Resolution Method" Remote Sensing 12, no. 12: 1937. https://doi.org/10.3390/rs12121937