Next Article in Journal
Estimating the Product of the X-ray Spectrum and Quantum Detection Efficiency of a CT System and Its Application to Beam Hardening Correction
Previous Article in Journal
IMU-Based Effects Assessment of the Use of Foot Orthoses in the Stance Phase during Running and Asymmetry between Extremities
Article

Non-Local and Multi-Scale Mechanisms for Image Inpainting

by and *
School of Microelectronics and Communication Engineering, Chongqing University, Chongqing 400044, China
*
Author to whom correspondence should be addressed.
Academic Editor: Ludovic Macaire
Sensors 2021, 21(9), 3281; https://doi.org/10.3390/s21093281
Received: 2 April 2021 / Revised: 30 April 2021 / Accepted: 6 May 2021 / Published: 10 May 2021
(This article belongs to the Section Sensing and Imaging)
Recently, deep learning-based techniques have shown great power in image inpainting especially dealing with squared holes. However, they fail to generate plausible results inside the missing regions for irregular and large holes as there is a lack of understanding between missing regions and existing counterparts. To overcome this limitation, we combine two non-local mechanisms including a contextual attention module (CAM) and an implicit diversified Markov random fields (ID-MRF) loss with a multi-scale architecture which uses several dense fusion blocks (DFB) based on the dense combination of dilated convolution to guide the generative network to restore discontinuous and continuous large masked areas. To prevent color discrepancies and grid-like artifacts, we apply the ID-MRF loss to improve the visual appearance by comparing similarities of long-distance feature patches. To further capture the long-term relationship of different regions in large missing regions, we introduce the CAM. Although CAM has the ability to create plausible results via reconstructing refined features, it depends on initial predicted results. Hence, we employ the DFB to obtain larger and more effective receptive fields, which benefits to predict more precise and fine-grained information for CAM. Extensive experiments on two widely-used datasets demonstrate that our proposed framework significantly outperforms the state-of-the-art approaches both in quantity and quality. View Full-Text
Keywords: image inpainting; implicit diversified Markov random fields; dense connection of dilated convolution; contextual attention mechanism image inpainting; implicit diversified Markov random fields; dense connection of dilated convolution; contextual attention mechanism
Show Figures

Figure 1

MDPI and ACS Style

He, X.; Yin, Y. Non-Local and Multi-Scale Mechanisms for Image Inpainting. Sensors 2021, 21, 3281. https://doi.org/10.3390/s21093281

AMA Style

He X, Yin Y. Non-Local and Multi-Scale Mechanisms for Image Inpainting. Sensors. 2021; 21(9):3281. https://doi.org/10.3390/s21093281

Chicago/Turabian Style

He, Xu, and Yong Yin. 2021. "Non-Local and Multi-Scale Mechanisms for Image Inpainting" Sensors 21, no. 9: 3281. https://doi.org/10.3390/s21093281

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop