Next Article in Journal
Surface Plasmon Resonance and Bending Loss-Based U-Shaped Plastic Optical Fiber Biosensors
Next Article in Special Issue
Archetype-Based Modeling of Persona for Comprehensive Personality Computing from Personal Big Data
Previous Article in Journal
Flexible, Stretchable Sensors for Wearable Health Monitoring: Sensing Mechanisms, Materials, Fabrication Strategies and Features
Previous Article in Special Issue
A Single RF Emitter-Based Indoor Navigation Method for Autonomous Service Robots
Article Menu
Issue 2 (February) cover image

Export Article

Open AccessArticle
Sensors 2018, 18(2), 646; https://doi.org/10.3390/s18020646

Social Image Captioning: Exploring Visual Attention and User Attention

1
College of Computer & Communication Engineering, China University of Petroleum (East China), Qingdao 266555, China
2
First Research Institute of the Ministry of Public Security of PRC, Beijing 100048, China
3
School of Information and Communication Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China
*
Author to whom correspondence should be addressed.
Received: 1 December 2017 / Revised: 9 February 2018 / Accepted: 12 February 2018 / Published: 22 February 2018
(This article belongs to the Special Issue Sensing, Data Analysis and Platforms for Ubiquitous Intelligence)
Full-Text   |   PDF [820 KB, uploaded 22 February 2018]   |  

Abstract

Image captioning with a natural language has been an emerging trend. However, the social image, associated with a set of user-contributed tags, has been rarely investigated for a similar task. The user-contributed tags, which could reflect the user attention, have been neglected in conventional image captioning. Most existing image captioning models cannot be applied directly to social image captioning. In this work, a dual attention model is proposed for social image captioning by combining the visual attention and user attention simultaneously.Visual attention is used to compress a large mount of salient visual information, while user attention is applied to adjust the description of the social images with user-contributed tags. Experiments conducted on the Microsoft (MS) COCO dataset demonstrate the superiority of the proposed method of dual attention. View Full-Text
Keywords: social image captioning; user-contributed tags; user attention; visual attention social image captioning; user-contributed tags; user attention; visual attention
Figures

Graphical abstract

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. (CC BY 4.0).
SciFeed

Share & Cite This Article

MDPI and ACS Style

Wang, L.; Chu, X.; Zhang, W.; Wei, Y.; Sun, W.; Wu, C. Social Image Captioning: Exploring Visual Attention and User Attention. Sensors 2018, 18, 646.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Related Articles

Article Metrics

Article Access Statistics

1

Comments

[Return to top]
Sensors EISSN 1424-8220 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top