Next Article in Journal
Sensor Fusion and Convolutional Neural Networks for Indoor Occupancy Prediction Using Multiple Low-Cost Low-Resolution Heat Sensor Data
Next Article in Special Issue
Classification of Critical Levels of CO Exposure of Firefigthers through Monitored Heart Rate
Previous Article in Journal
A Generalization Performance Study Using Deep Learning Networks in Embedded Systems
Previous Article in Special Issue
DLNR-SIQA: Deep Learning-Based No-Reference Stitched Image Quality Assessment
Due to scheduled maintenance work on our core network, there may be short service disruptions on this website between 16:00 and 16:30 CEST on September 25th.
Article

Intelligent Video Highlights Generation with Front-Camera Emotion Sensing

Department of Electrical Engineering, Columbia University, New York, NY 10027, USA
*
Author to whom correspondence should be addressed.
Academic Editor: Manuel José Cabral dos Santos Reis
Sensors 2021, 21(4), 1035; https://doi.org/10.3390/s21041035
Received: 31 December 2020 / Revised: 20 January 2021 / Accepted: 25 January 2021 / Published: 3 February 2021
(This article belongs to the Special Issue Data, Signal and Image Processing and Applications in Sensors)
In this paper, we present HOMER, a cloud-based system for video highlight generation which enables the automated, relevant, and flexible segmentation of videos. Our system outperforms state-of-the-art solutions by fusing internal video content-based features with the user’s emotion data. While current research mainly focuses on creating video summaries without the use of affective data, our solution achieves the subjective task of detecting highlights by leveraging human emotions. In two separate experiments, including videos filmed with a dual camera setup, and home videos randomly picked from Microsoft’s Video Titles in the Wild (VTW) dataset, HOMER demonstrates an improvement of up to 38% in F1-score from baseline, while not requiring any external hardware. We demonstrated both the portability and scalability of HOMER through the implementation of two smartphone applications. View Full-Text
Keywords: mobile computing; emotion recognition; image processing; signal processing algorithms mobile computing; emotion recognition; image processing; signal processing algorithms
Show Figures

Figure 1

MDPI and ACS Style

Meyer, H.; Wei, P.; Jiang, X. Intelligent Video Highlights Generation with Front-Camera Emotion Sensing. Sensors 2021, 21, 1035. https://doi.org/10.3390/s21041035

AMA Style

Meyer H, Wei P, Jiang X. Intelligent Video Highlights Generation with Front-Camera Emotion Sensing. Sensors. 2021; 21(4):1035. https://doi.org/10.3390/s21041035

Chicago/Turabian Style

Meyer, Hugo, Peter Wei, and Xiaofan Jiang. 2021. "Intelligent Video Highlights Generation with Front-Camera Emotion Sensing" Sensors 21, no. 4: 1035. https://doi.org/10.3390/s21041035

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop