Contextual Emotion Appraisal Based on a Sentential Cognitive System for Robots
Abstract
:Featured Application
Abstract
1. Introduction
2. Related Work
3. Contextual Emotion Appraisal Model
3.1. Emotion Appraisal Model by Simplifying OCC Model
- Primacy condition: although the semantics of emotion can be interpreted variously, the primary meaning of the emotion is used.
- Valence condition: rather than classifying all aspects of emotions, they are represented with the valences of primary emotional states.
- Self-centered condition: only emotions focused on the robot itself are modeled.
- Cognitive condition: only emotions that can be analyzed by the robot’s sensory and behavioral information are used.
3.2. Emotional Transition Model
4. An Emotion Appraisal System on a Sentential Cognitive System
4.1. An SCS for Emotion Appraisal
4.2. Contextual Emotion Appraisal Based on the SCS
5. Experiments and Results
5.1. Implementation
5.2. Experimental Results
6. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Conflicts of Interest
References
- Hashimoto, M.; Yokogawa, C.; Sadoyama, T. Development and control of a face robot imitating human muscular structures. In Proceedings of the 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, Beijing, China, 9–15 October 2006; pp. 1855–1860. [Google Scholar]
- Lee, H.-S.; Park, J.-W.; Chung, M.-J. A linear affective space model based on the facial expressions for mascot-type robots. In Proceedings of the 2006 SICE-ICASE International Joint Conference, Busan, Korea, 18–21 October 2006; pp. 5367–5372. [Google Scholar]
- Breazeal, C. Emotive qualities in lip synchronized robot speech. Adv. Robot. 2003, 17, 97–113. [Google Scholar] [CrossRef]
- Park, G.-Y.; Lee, S.-I.; Kwon, W.-Y.; Park, J.-B. Neurocognitive affective system for an emotive robot. In Proceedings of the 2006 IEEE/RSJ International Conference on Intelligent Robot and Systems, Beijing, China, 9–15 October 2006; pp. 2595–2600. [Google Scholar]
- Arkin, R.C.; Fujita, M.; Takagi, T.; Hasegawa, R. An ethological and emotional basis for human-robot interaction. Robot. Auton. Syst. 2003, 43, 191–201. [Google Scholar] [CrossRef]
- Russell, J. Evidence for a three-factor theory of emotions. J. Res. Personal. 1977, 11, 273–294. [Google Scholar] [CrossRef]
- Ortony, A.; Clore, G.L.; Collins, A. The Cognitive Structure of Emotion; Cambridge University Press: Cambridge, UK, 1988. [Google Scholar]
- Oh, S.; Gratch, J.; Woo, W. Explanatory style for socially interactive agents. In Proceedings of the 2nd International Conference on Affective Computing and Intelligent Interaction, Lisbon, Portugal, 12–14 September 2007; pp. 534–545. [Google Scholar]
- Marsella, S.; Gratch, J.; Petta, P. Computational models of emotion. In Blueprint for Affective Computing: A Source Book, 1st ed.; Scherer, K.R., Bänziger, T., Roesch, E.B., Eds.; Oxford University Press: Oxford, UK, 2010. [Google Scholar]
- Ikeda, Y.; Horie, R.; Sugaya, M. Estimating emotion with biological information for robot interaction. Procedia Comput. Sci. 2017, 112, 1589–1600. [Google Scholar] [CrossRef]
- Rincon, J.A.; Martin, A.; Costa, A.G.A.S.; Novais, P.; Julian, V.; Carrascosa, C. EmIR: An Emotional Intelligent Robot Assistant. In Proceedings of the AfCAI, Valencia, Spain, 19–20 April 2018. [Google Scholar]
- Cid, F.; Manso, L.J.; Nunez, P. A novel multimodal emotion recognition approach for affective human robot interaction. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Hamburg, Germany, 28 September–2 October 2015; pp. 1–9. [Google Scholar]
- Martinez-Hernandez, U.; Prescott, T.J. Expressive touch: Control of robot emotional expression by touch. In Proceedings of the 25th IEEE International Symposium on Robot and Human Interactive Communication, New York, NY, USA, 26–31 August 2016; pp. 974–979. [Google Scholar]
- Kim, H.-R.; Koo, S.-Y.; Kwon, D.-S. Designing reactive emotion generation model for interactive robots. In Proceedings of the 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, Taipei, Taiwan, 18–22 October 2010; pp. 18–22. [Google Scholar]
- Lee, W.H.; Park, J.-W.; Kim, W.H.; Kim, J.C.; Chung, M.J. Robot’s emotion generation model for transition and diversity using energy, entropy, and homeostasis concepts. In Proceedings of the 2010 IEEE International Conference on Robotics and Biomimetics, Tianjin, China, 14–18 December 2010; pp. 555–560. [Google Scholar]
- Jitviriya, W.; Hayashi, E. Design of emotion generation model and action selection for robots using a selforganizing map. In Proceedings of the 11th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON), Nakhon Ratchasima, Thailand, 14–17 May 2014; pp. 1–6. [Google Scholar]
- Samani, H.A.; Saadatian, E. A multidisciplinary artificial intelligence model of an affective robot. Int. J. Adv. Robot. Syst. 2012, 9, 1–11. [Google Scholar] [CrossRef]
- Park, J.-C.; Song, H.; Koo, S.; Kim, Y.-M.; Kwon, D.-S. Robot’s behavior expressions according to the sentence types and emotions with modification by personality. In Proceedings of the 2010 IEEE Workshop on Advanced Robotics and its Social Impacts, Seoul, Korea, 26–28 October 2010; pp. 105–110. [Google Scholar]
- Hakamata, A.; Ren, F.; Tsuchiya, S. Human emotion model based on discourse sentence for expression generation of conversation agent. In Proceedings of the 2008 International Conference on Natural Language Processing and Knowledge Engineering, Beijing, China, 19–22 October 2008. [Google Scholar]
- Zhang, X.; Alves, S.; Nejat, G.; Benhabib, B. A robot emotion model with history. In Proceedings of the 2017 IEEE International Symposium on Robotics and Intelligent Sensors, Ottawa, ON, Canada, 5–7 October 2017; pp. 230–235. [Google Scholar]
- Kinoshita, S.; Takenouchi, H.; Tokumaru, M. An emotion-generation model for a robot that reacts after considering the dialogist. In Proceedings of the 2014 International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment and Management (HNICEM), Palawan, Philippines, 12–16 November 2014; pp. 1–6. [Google Scholar]
- Kirby, R.; Forlizzi, J.; Simmons, R. Affective social robots. Robot. Auton. Syst. 2010, 58, 322–332. [Google Scholar] [CrossRef] [Green Version]
- Itoh, C.; Kato, S.; Itoh, H. Mood-transition-based emotion generation model for the robot’s personality. In Proceedings of the 2009 IEEE International Conference on Systems, Man, and Cybernetics, San Antonio, TX, USA, 11–14 October 2009; pp. 2878–2883. [Google Scholar]
- Trafton, J.; Hiatt, L.M.; Harrison, A.M.; Tamborello, F.P.; Khemlani, S.S.; Schultz, A.C. ACT-R/E: An Embodied Cognitive Architecture for Human-Robot Interaction. J. Hum. Robot Interact. 2013, 2, 30–55. [Google Scholar] [CrossRef]
- Laird, J.; Kinkade, K.R.; Mohan, S.; Xu, J.Z. Cognitive Robotics using the SOAR Cognitive Architecture. In Proceedings of the 2012 AAAI Workshop, CogRob@AAAI 2012, Toronto, ON, Canada, 22–23 July 2012. [Google Scholar]
- Cominelli, L.; Mazzei, D.; Rossi, D. SEAI: Social Emotional Artificial Intelligence Based on Damasio’s Theory of Mind. Frontiers in Robotics and AI 2018, 5, 6. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Damasio, A. The Feeling of What Happens: Body and Emotion in the Making of Consciousness; Spektrum Der Wiss: Heidelberg, Germany, 2000; p. 104. [Google Scholar]
- Marcus, M.P.; Santorini, B.; Marcinkiewicz, M.A. Building a large annotated corpus of English: The Penn Treebank. Comput. Linguist. 1993, 19, 313–330. [Google Scholar]
- Garnsey, S.M.; Pearlmuttter, N.J.; Myers, E.; Lotocky, M.A. The contributions of verb bias and plausibility to the comprehension of temporarily ambiguous sentences. J. Mem. Lang. 1997, 37, 58–93. [Google Scholar] [CrossRef]
- Ren, F. Affective Information processing and recognizing human emotion. Electron. Notes Theor. Comput. Sci. 2009, 225, 39–50. [Google Scholar] [CrossRef] [Green Version]
- Ahn, H. A sentential cognitive system of robots for conversational human–robot interaction. J. Intell. Fuzzy Syst. 2018, 35, 6047–6059. [Google Scholar] [CrossRef]
- Ahn, H.; Ko, H. Natural-language-based robot action control using a hierarchical behavior model. IEIE Trans. Smart Process. Comput. 2012, 1, 192–200. [Google Scholar]
- Link Grammar. Available online: http://www.abisource.com/projects/link-grammar (accessed on 1 July 2018).
- Kinect for Windows. Available online: http://developer.microsoft.com/en-us/windows/kinect (accessed on 1 July 2018).
- Redmon, J.; Farhadi, A. YOLO9000: Better, Faster, Stronger, CVPR. 2017. Available online: https://pjreddie.com/publications/ (accessed on 1 September 2019).
# | Emotion | Appraisal Context | Dichotomy |
---|---|---|---|
1 | Love | Aspects of target | Positive |
2 | Hate | Negative | |
3 | Pride | Action of self | Positive |
4 | Shame | Negative | |
5 | Gratitude | Action of agent | Positive |
6 | Anger | Negative | |
7 | Joy | Consequence of event/prospect irrelevant | Positive |
8 | Distress | Negative | |
9 | Hope | Consequence of event/prospect relevant | Positive |
10 | Fear | Negative |
# | Emotions | Sentences | Targets of Emotions |
---|---|---|---|
1 | Love | (S (NP I) (VP love (NP the object))), | (NP the object), (NP the agent) |
(S (NP I) (VP love (NP the agent) (PP by (NP the event)))) | |||
2 | Hate | (S (NP I) (VP hate (NP the object))), | |
(S (NP I I) (VP hate (NP the agent) (PP by (NP the event)))) | |||
3 | Pride | (S (NP I) (VP boast (NP the event))) | (NP the event) |
4 | Shame | (S (NP I) (VP am (VP shamed (PP for (NP the event))))) | |
5 | Gratitude | (S (NP I) (VP thank (NP the agent) (PP for (NP the event)))) | (NP the agent) |
6 | Anger | (S (NP I) (VP anger to (NP the agent) (PP by (NP the event)))) | |
7 | Joy | (S (NP I) (VP rejoice (PP at (NP the event)))) | (NP the event) |
8 | Distress | (S (NP I) (VP am (VP distressed (PP by (NP the event))))) | |
9 | Hope | (S (NP I) (VP hope (PP for (NP the event)))) | (NP the event) |
10 | Fear | (S (NP I) (VP fear (NP the event))) |
# | Emotions | Targets | Sources of Appraisal | Emotional Valence | |||
---|---|---|---|---|---|---|---|
Object | Agent | Event | A priori | Contextual | |||
1 | Love–Hate (LH) | ○ | ○ | △ | ○ | ○ | −1.0–1.0 |
2 | Pride–Shame (PS) | ○ | ○ | −1.0–1.0 | |||
3 | Gratitude–Anger (GA) | ○ | △ | ○ | −1.0–1.0 | ||
4 | Joy–Distress (JD) | ○ | ○ | −1.0–1.0 | |||
5 | Hope–Fear (HF) | ○ | ○ | −1.0–1.0 |
# | Mod. | Sentences of Events | Emotional Valences | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Type | Verb | Arg_1 | Arg_2 | Space | Time | Adj. | Oi (LH) | Aj (LH, GA) | Ek (PS, JD, HF) | ||
S1 | V | Declarative | appeared | (NP Oi) | - | (PP at (NP x, y, z)) | - | (PP by (NP Aj)) | (l) | (l, g) | (p, j, h) |
S2 | V | Declarative | disappeared | (NP Oi) | - | - | - | (PP by (NP Aj)) | (l) | (l, g) | (p, j, h) |
S3 | V | Declarative | appeared | (NP Aj) | - | (PP at (NP the front)) | - | - | - | (l, g) | (p, j, h) |
S4 | V | Declarative | disappeared | (NP Aj) | - | - | - | - | - | (l, g) | (p, j, h) |
S5 | U | What | is | (NP your name) | - | - | - | - | - | - | - |
S6 | L | Declarative | am | (NP I) | (NP Aj) | - | - | - | - | - | - |
Objects | A Priori LH_O Valences |
---|---|
apples | 0.5 |
orange | 0.3 |
bananas | −0.2 |
carrot | −0.3 |
Targets | Parameters of E(t) of (1) and (2) | ||
---|---|---|---|
σ | s | d | |
LH_O | 1 | 1 | 0 |
LH_A | 2 | 0.5 | 0.002 |
GA_A | 1.6 | 0.1 | 0.002 |
JD_E | 1.9 | 0.2 | 0.002 |
HF_E | 1 | 0.8 | 0 |
# | Time (hh:mm:ss) | Module | Sentences of Events | Emotional Valences of Targets | ||
---|---|---|---|---|---|---|
A1 (LH, GA) | A2 (LH, GA) | Events (PS, JD, HF) | ||||
S1 | 10:13:02 | V | (S (NP A new agent) (VP appeared (PP at (NP the front)))) | (0.0, 0.0) | (0.0, 0.0) | (0.0, 0.0, 0.0) |
S2 | 10:13:05 | U | (S What (S (VP is (NP your name)))) | (0.0, 0.0) | (0.0, 0.0) | (0.0, 0.0, 0.0) |
S3 | 10:13:09 | L | (S (NP I) (VP am (NP John))) | (0.0, 0.0) | (0.0, 0.0) | (0.0, 0.0, 0.0) |
S4 | 10:13:25 | V | (S (NP An apple) (VP appeared (PP at (NP (173, 296))))) | (0.0, 0.41) | (0.0, 0.0) | (0.0, 0.45, 0.0) |
S5 | 10:13:28 | U | (S (VP Thank (NP you))) | (0.0, 0.41 | (0.0, 0.0) | (0.0, 0.45, 0.0) |
S6 | 10:14:06 | V | (S (NP An orange) (VP appeared (PP at (NP (343, 162))))) | (0.49 0.62) | (0.0, 0.0) | (0.0, 1.0, 0.0) |
S7 | 10:14:10 | U | (S (VP Thank (NP you))) | (0.49, 0.61) | (0.0, 0.0) | (0.0, 1.0, 0.0) |
S8 | 10:14:14 | U | (S (NP I) (VP love (NP you))) | (0.49, 0.60) | (0.0, 0.0) | (0.0, 1.0, 0.0) |
S9 | 10:14:52 | L | (S (VP see (NP you) (NP tomorrow))) | (0.47, 0.50) | (0.0, 0.0) | (0.0, 0.90, 0.38) |
S10 | 10:14:57 | V | (S (NP John) (VP disappeared)) | (0.47, 0.48) | (0.0, 0.0) | (0.0, 0.97, 0.38) |
S11 | 10:16:03 | V | (S (NP A new agent) (VP appeared (PP at (NP the front)))) | (0.40, 0.26) | (0.0, 0.0) | (0.0, 0.57, 0.34) |
S12 | 10:16:06 | U | (S What (S (VP is (NP your name)))) | (0.40, 0.25) | (0.0, 0.0) | (0.0, 0.56, 0.34) |
S13 | 10:16:11 | L | (S (NP I) (VP am (NP Tom))) | (0.39, 0.24) | (0.0, 0.0) | (0.0, 0.54, 0.34) |
S14 | 10:17:07 | V | (S (NP A banana) (VP appeared (PP at (NP (635, 393))))) | (0.32 0.12) | (0.0, −0.16) | (0.0, 0.19, 0.31) |
S15 | 10:17:23 | V | (S (NP An apple) (VP disappeared)) | (0.31 0.10) | (0.0, −0.57) | (0.0, −0.29, 0.31) |
S16 | 10:17:26 | U | (S (NP I) (VP am (ADJP angry))) | (0.31, 0.10) | (0.0, −0.56) | (0.0, −0.30, 0.31) |
S17 | 10:17:32 | V | (S (NP An orange) (VP disappeared)) | (0.30, 0.09) | (−0.64, −0.80) | (0.0, −1.0, 0.31) |
S18 | 10:17:35 | U | (S (NP I) (VP am (ADJP angry))) | (0.30 0.09) | (−0.64, −0.80) | (0.0, −1.0, 0.31) |
S19 | 10:17:38 | U | (S (NP I) (VP hate (NP you))) | (0.30, 0.09) | (−0.64, −0.80) | (0.0, −1.0, 0.31) |
S20 | 10:19:02 | V | (S (NP A carrot) (VP appeared (PP at (NP (201, 278))))) | (0.25, 0.06) | (−0.57, −0.67) | (0.0, −0.90, 0.30) |
S21 | 10:19:05 | U | (S (NP I) (VP am (ADJP angry))) | (0.25, 0.06) | (−0.57, −0.65) | (0.0, −0.88, 0.30) |
S22 | 10:20:38 | L | (S (VP see (NP you) (NP tomorrow))) | (0.24, 0.06) | (−0.49, −0.27) | (0.0, −0.42, −0.09) |
S23 | 10:20:46 | L | (S (NP Tom) (VP disappeared)) | (0.24, 0.06) | (−0.48, −0.25) | (0.0, −0.38, −0.09) |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Ahn, H.; Park, S. Contextual Emotion Appraisal Based on a Sentential Cognitive System for Robots. Appl. Sci. 2021, 11, 2027. https://doi.org/10.3390/app11052027
Ahn H, Park S. Contextual Emotion Appraisal Based on a Sentential Cognitive System for Robots. Applied Sciences. 2021; 11(5):2027. https://doi.org/10.3390/app11052027
Chicago/Turabian StyleAhn, Hyunsik, and Sung Park. 2021. "Contextual Emotion Appraisal Based on a Sentential Cognitive System for Robots" Applied Sciences 11, no. 5: 2027. https://doi.org/10.3390/app11052027
APA StyleAhn, H., & Park, S. (2021). Contextual Emotion Appraisal Based on a Sentential Cognitive System for Robots. Applied Sciences, 11(5), 2027. https://doi.org/10.3390/app11052027