Next Article in Journal
Exploring a Novel Multiple-Query Resistive Grid-Based Planning Method Applied to High-DOF Robotic Manipulators
Next Article in Special Issue
Classification of Diffuse Glioma Subtype from Clinical-Grade Pathological Images Using Deep Transfer Learning
Previous Article in Journal
Electrical Tomography Reconstruction Using Reconfigurable Waveforms in a FPGA
Previous Article in Special Issue
Gated Dehazing Network via Least Square Adversarial Learning
Article

Automatic vs. Human Recognition of Pain Intensity from Facial Expression on the X-ITE Pain Database

1
Department of Neuro-Information Technology, Institute for Information Technology and Communications, Otto-von-Guericke University Magdeburg, 39106 Magdeburg, Germany
2
Department of Medical Psychology, Ulm University, 89081 Ulm, Germany
*
Author to whom correspondence should be addressed.
Academic Editors: Chen Chen and Bogdan Smolka
Sensors 2021, 21(9), 3273; https://doi.org/10.3390/s21093273
Received: 31 March 2021 / Revised: 26 April 2021 / Accepted: 5 May 2021 / Published: 10 May 2021
(This article belongs to the Special Issue Sensors and Deep Learning for Digital Image Processing)
Prior work on automated methods demonstrated that it is possible to recognize pain intensity from frontal faces in videos, while there is an assumption that humans are very adept at this task compared to machines. In this paper, we investigate whether such an assumption is correct by comparing the results achieved by two human observers with the results achieved by a Random Forest classifier (RFc) baseline model (called RFc-BL) and by three proposed automated models. The first proposed model is a Random Forest classifying descriptors of Action Unit (AU) time series; the second is a modified MobileNetV2 CNN classifying face images that combine three points in time; and the third is a custom deep network combining two CNN branches using the same input as for MobileNetV2 plus knowledge of the RFc. We conduct experiments with X-ITE phasic pain database, which comprises videotaped responses to heat and electrical pain stimuli, each of three intensities. Distinguishing these six stimulation types plus no stimulation was the main 7-class classification task for the human observers and automated approaches. Further, we conducted reduced 5-class and 3-class classification experiments, applied Multi-task learning, and a newly suggested sample weighting method. Experimental results show that the pain assessments of the human observers are significantly better than guessing and perform better than the automatic baseline approach (RFc-BL) by about 1%; however, the human performance is quite poor due to the challenge that pain that is ethically allowed to be induced in experimental studies often does not show up in facial reaction. We discovered that downweighting those samples during training improves the performance for all samples. The proposed RFc and two-CNNs models (using the proposed sample weighting) significantly outperformed the human observer by about 6% and 7%, respectively. View Full-Text
Keywords: pain recognition; facial expression; CNN; multi task learning; random forest pain recognition; facial expression; CNN; multi task learning; random forest
Show Figures

Figure 1

MDPI and ACS Style

Othman, E.; Werner, P.; Saxen, F.; Al-Hamadi, A.; Gruss, S.; Walter, S. Automatic vs. Human Recognition of Pain Intensity from Facial Expression on the X-ITE Pain Database. Sensors 2021, 21, 3273. https://doi.org/10.3390/s21093273

AMA Style

Othman E, Werner P, Saxen F, Al-Hamadi A, Gruss S, Walter S. Automatic vs. Human Recognition of Pain Intensity from Facial Expression on the X-ITE Pain Database. Sensors. 2021; 21(9):3273. https://doi.org/10.3390/s21093273

Chicago/Turabian Style

Othman, Ehsan, Philipp Werner, Frerk Saxen, Ayoub Al-Hamadi, Sascha Gruss, and Steffen Walter. 2021. "Automatic vs. Human Recognition of Pain Intensity from Facial Expression on the X-ITE Pain Database" Sensors 21, no. 9: 3273. https://doi.org/10.3390/s21093273

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop