Thermal Imaging Based Affective Computing for Educational Robot †
Abstract
:1. Introduction
2. Material and Methods
2.1. Participants
2.2. Materials and Data Acquisition
2.3. Procedure
2.4. Computational Psychophysiology Module—CPM
Thermal Data Extraction and Classification
- -
- Face detection algorithm, applied on the visible image. In detail, the frontal face detector is based on histogram of oriented gradients (HOG) features and a linear SVM classifier [10]. Faces that appeared rotated off-axis were specifically excluded, to preserve the quality of the signals extracted in the later steps.
- -
- Facial landmarks calculation, using an implementation of One Millisecond Face Alignment with an Ensemble of Regression Trees [11].
- -
- Distance between the face and the cameras estimation. The distance was evaluated by comparing an average anatomical model of a face with the observed data from the calibrated visible camera.
- -
- ROIs calculation based on the landmarks’ geometry and signal extraction by taking basic image statistics for each ROI (minimum and maximum, mean and standard deviation of the temperatures of the pixel in the ROI). The assessed ROIs were the tip of the nose, nostrils, glabella and perioral areas.
3. Results
4. Discussion
Author Contributions
Funding
Acknowledgments
Conflicts of Interest
References
- Breazeal, C.; Takanishi, A.; Kobayashi, T. Social Robots that Interact with People. In Springer Handbook of Robotics; Siciliano, B., Khatib, O., Eds.; Springer: Berlin/Heidelberg, Germany, 2008; pp. 1349–1369. [Google Scholar]
- Tanaka, F.; Matsuzoe, S. Children teach a care-receiving robot to promote their learning: Field experiments in a classroom for vocabulary learning. J. Hum.-Robot Interact. 2012, 1, 78–95. [Google Scholar] [CrossRef]
- Leyzberg, D.; Avrunin, E.; Liu, J.; Scassellati, B. Robots that express emotion elicit better human teaching. In Proceedings of the 6th International Conference on Human-Robot Interaction, HRI 2011, Lausanne, Switzerland, 6–9 March 2011; ACM: New York, USA, 2011; pp. 347–354. [Google Scholar]
- Merla, A. Thermal expression of intersubjectivity offers new possibilities to human–machine and technologically mediated interactions. Front. Psychol. 2014, 5, 802. [Google Scholar] [CrossRef] [PubMed]
- Ioannou, S.; Ebisch, S.; Aureli, T.; Bafunno, D.; Ioannides, H.A.; Cardone, D.; Manini, B.; Romani, G.L.; Gallese, V.; Merla, A. The autonomic signature of guilt in children: A thermal infrared imaging study. PLoS ONE 2013, 8, e79440. [Google Scholar] [CrossRef] [PubMed]
- Buddharaju, P.; Dowdall, J.; Tsiamyrtzis, P.; Shastri, D.; Pavlidis, I.; Frank, M.G. Automatic thermal monitoring system (ATHEMOS) for deception detection. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), San Diego, CA, USA, USA, 20–25 June 2005; IEEE: New York, NY, USA, 2005. [Google Scholar]
- Dowdall, J.; Pavlidis, I.T.; Tsiamyrtzis, P. Coalitional tracking. Comput. Vis. Image Underst. 2007, 106, 205–219. [Google Scholar] [CrossRef]
- Merla, A.; Cardone, D.; Di Carlo, L.; Di Donato, L.; Ragnoni, A.; Visconti, A. Noninvasive system for monitoring driver’s physical state. In Proceedings of the 11th AITA Advanced Infrared Technology and Applications, L’Aquila, Italy, 7–9 September 2011; Atti della Fondazione Giorgio Ronchi: Florence, Italy, 2011. [Google Scholar]
- Bradski, G. The opencv library. Dr Dobb’s J. Software Tools 2000, 25, 120–125. [Google Scholar]
- King, D.E. Dlib-ml: A machine learning toolkit. J. Mach. Learn. Res. 2009, 10, 1755–1758. [Google Scholar]
- Kazemi, V.; Sullivan, J. One millisecond face alignment with an ensemble of regression trees. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 24–27 June 2014; IEE: New York, NY, USA, 2014. [Google Scholar]
- Ioannou, S.; Gallese, V.; Merla, A. Thermal infrared imaging in psychophysiology: Potentialities and limits. Psychophysiology 2014, 51, 951–963. [Google Scholar] [CrossRef] [PubMed]
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Filippini, C.; Spadolini, E.; Cardone, D.; Merla, A. Thermal Imaging Based Affective Computing for Educational Robot. Proceedings 2019, 27, 27. https://doi.org/10.3390/proceedings2019027027
Filippini C, Spadolini E, Cardone D, Merla A. Thermal Imaging Based Affective Computing for Educational Robot. Proceedings. 2019; 27(1):27. https://doi.org/10.3390/proceedings2019027027
Chicago/Turabian StyleFilippini, Chiara, Edoardo Spadolini, Daniela Cardone, and Arcangelo Merla. 2019. "Thermal Imaging Based Affective Computing for Educational Robot" Proceedings 27, no. 1: 27. https://doi.org/10.3390/proceedings2019027027