Conv-Former: A Novel Network Combining Convolution and Self-Attention for Image Quality Assessment

Han, Lintao; Lv, Hengyi; Zhao, Yuchen; Liu, Hailong; Bi, Guoling; Yin, Zhiyong; Fang, Yuqiang

doi:10.3390/s23010427

Open AccessArticle

Conv-Former: A Novel Network Combining Convolution and Self-Attention for Image Quality Assessment

by

Lintao Han

^1,2

,

Hengyi Lv

^1,*

,

Yuchen Zhao

¹,

Hailong Liu

¹,

Guoling Bi

¹,

Zhiyong Yin

³ and

Yuqiang Fang

³

¹

Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China

²

College of Materials Science and Opto-Electronic Technology, University of Chinese Academy of Sciences, Beijing 100049, China

³

Department of Electrical and Optical Engineering, Space Engineering University, Beijing 101416, China

^*

Author to whom correspondence should be addressed.

Sensors 2023, 23(1), 427; https://doi.org/10.3390/s23010427

Submission received: 7 November 2022 / Revised: 21 December 2022 / Accepted: 26 December 2022 / Published: 30 December 2022

(This article belongs to the Special Issue Computer Vision and Sensing Technologies for Industrial Quality Inspection)

Download

Browse Figures

Versions Notes

Abstract

To address the challenge of no-reference image quality assessment (NR-IQA) for authentically and synthetically distorted images, we propose a novel network called the Combining Convolution and Self-Attention for Image Quality Assessment network (Conv-Former). Our model uses a multi-stage transformer architecture similar to that of ResNet-50 to represent appropriate perceptual mechanisms in image quality assessment (IQA) to build an accurate IQA model. We employ adaptive learnable position embedding to handle images with arbitrary resolution. We propose a new transformer block (TB) by taking advantage of transformers to capture long-range dependencies, and of local information perception (LIP) to model local features for enhanced representation learning. The module increases the model’s understanding of the image content. Dual path pooling (DPP) is used to keep more contextual image quality information in feature downsampling. Experimental results verify that Conv-Former not only outperforms the state-of-the-art methods on authentic image databases, but also achieves competing performances on synthetic image databases which demonstrate the strong fitting performance and generalization capability of our proposed model.

Keywords: image quality assessment; vision transformer; self-attention; neural network; deep model interation

Share and Cite

MDPI and ACS Style

Han, L.; Lv, H.; Zhao, Y.; Liu, H.; Bi, G.; Yin, Z.; Fang, Y. Conv-Former: A Novel Network Combining Convolution and Self-Attention for Image Quality Assessment. Sensors 2023, 23, 427. https://doi.org/10.3390/s23010427

AMA Style

Han L, Lv H, Zhao Y, Liu H, Bi G, Yin Z, Fang Y. Conv-Former: A Novel Network Combining Convolution and Self-Attention for Image Quality Assessment. Sensors. 2023; 23(1):427. https://doi.org/10.3390/s23010427

Chicago/Turabian Style

Han, Lintao, Hengyi Lv, Yuchen Zhao, Hailong Liu, Guoling Bi, Zhiyong Yin, and Yuqiang Fang. 2023. "Conv-Former: A Novel Network Combining Convolution and Self-Attention for Image Quality Assessment" Sensors 23, no. 1: 427. https://doi.org/10.3390/s23010427

APA Style

Han, L., Lv, H., Zhao, Y., Liu, H., Bi, G., Yin, Z., & Fang, Y. (2023). Conv-Former: A Novel Network Combining Convolution and Self-Attention for Image Quality Assessment. Sensors, 23(1), 427. https://doi.org/10.3390/s23010427

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Menu

Conv-Former: A Novel Network Combining Convolution and Self-Attention for Image Quality Assessment

Abstract

Share and Cite

Article Metrics

Article Access Statistics

Further Information

Guidelines

MDPI Initiatives

Follow MDPI