Next Article in Journal
Research on Energy Efficiency of NOMA–SWIPT Cooperative Relay Network Using GS-DinkelBach Algorithm
Next Article in Special Issue
A Manycore Vision Processor for Real-Time Smart Cameras
Previous Article in Journal
The Internet of Drones: Requirements, Taxonomy, Recent Advances, and Challenges of Research Trends
Previous Article in Special Issue
Transformations in the Photogrammetric Co-Processing of Thermal Infrared Images and RGB Images
 
 
Article

DisCaaS: Micro Behavior Analysis on Discussion by Camera as a Sensor

1
Department of Computer Science, University of Kaiserslautern & DFKI GmbH, 67663 Kaiserslautern, Germany
2
Graduate School of Science and Technology, Nara Institute of Science and Technology, Nara 630-0192, Japan
3
Department of Information Science and Technology, Graduate School and Faculty of Information Science and Electrical Engineering, Kyushu University, Fukuoka 819-0395, Japan
*
Author to whom correspondence should be addressed.
Academic Editors: Peter Corcoran and Saraju P. Mohanty
Sensors 2021, 21(17), 5719; https://doi.org/10.3390/s21175719
Received: 30 July 2021 / Revised: 19 August 2021 / Accepted: 20 August 2021 / Published: 25 August 2021
(This article belongs to the Collection Camera as a Smart-Sensor (CaaSS))
The emergence of various types of commercial cameras (compact, high resolution, high angle of view, high speed, and high dynamic range, etc.) has contributed significantly to the understanding of human activities. By taking advantage of the characteristic of a high angle of view, this paper demonstrates a system that recognizes micro-behaviors and a small group discussion with a single 360 degree camera towards quantified meeting analysis. We propose a method that recognizes speaking and nodding, which have often been overlooked in existing research, from a video stream of face images and a random forest classifier. The proposed approach was evaluated on our three datasets. In order to create the first and the second datasets, we asked participants to meet physically: 16 sets of five minutes data from 21 unique participants and seven sets of 10 min meeting data from 12 unique participants. The experimental results showed that our approach could detect speaking and nodding with a macro average f1-score of 67.9% in a 10-fold random split cross-validation and a macro average f1-score of 62.5% in a leave-one-participant-out cross-validation. By considering the increased demand for an online meeting due to the COVID-19 pandemic, we also record faces on a screen that are captured by web cameras as the third dataset and discussed the potential and challenges of applying our ideas to virtual video conferences. View Full-Text
Keywords: digital camera; camera as a smart sensor; human action recognition; meeting analysis; 3D pose estimation; RGB sensors digital camera; camera as a smart sensor; human action recognition; meeting analysis; 3D pose estimation; RGB sensors
Show Figures

Figure 1

MDPI and ACS Style

Watanabe, K.; Soneda, Y.; Matsuda, Y.; Nakamura, Y.; Arakawa, Y.; Dengel, A.; Ishimaru, S. DisCaaS: Micro Behavior Analysis on Discussion by Camera as a Sensor. Sensors 2021, 21, 5719. https://doi.org/10.3390/s21175719

AMA Style

Watanabe K, Soneda Y, Matsuda Y, Nakamura Y, Arakawa Y, Dengel A, Ishimaru S. DisCaaS: Micro Behavior Analysis on Discussion by Camera as a Sensor. Sensors. 2021; 21(17):5719. https://doi.org/10.3390/s21175719

Chicago/Turabian Style

Watanabe, Ko, Yusuke Soneda, Yuki Matsuda, Yugo Nakamura, Yutaka Arakawa, Andreas Dengel, and Shoya Ishimaru. 2021. "DisCaaS: Micro Behavior Analysis on Discussion by Camera as a Sensor" Sensors 21, no. 17: 5719. https://doi.org/10.3390/s21175719

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop