SAET: The Non-Verbal Measurement Tool in User Emotional Experience
Abstract
:1. Introduction
2. Related Work
2.1. Emotion and Emotional Space
2.2. Non-Verbal Emotion Measurement Tool
3. The Design of the SAET
3.1. Emotion Set
3.2. The Emotion Design of SAET
4. Validation of the SAET Images
4.1. Pilot Study
4.1.1. Participants
4.1.2. Apparatus
4.1.3. Procedure
4.1.4. Results
4.1.5. Discussion
4.2. Validation Study
4.2.1. Participants
4.2.2. Apparatus
4.2.3. Procedure
4.2.4. Result
4.3. Conclusions
5. Conclusions and Discussion
5.1. Methods of Application
5.2. Limitations and Future Research
5.3. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Zajonc, R.B. Feeling and thinking: Preferences need no inferences. Am. Psychol. 1980, 35, 151–175. [Google Scholar] [CrossRef]
- Scherer, K.R. What are emotions? And how can they be measured? Soc. Sci. Inf. 2005, 44, 695–729. [Google Scholar] [CrossRef]
- Hassenzahl, M.; Diefenbach, S.; Göritz, A. Needs, affect, and interactive products—Facets of user experience. Interact. Comput. 2010, 22, 353–362. [Google Scholar] [CrossRef]
- Law, E.L.; Roto, V.; Hassenzahl, M.; Vermeeren, A.P.; Kort, J. Understanding, scoping and defining user experience: A survey approach. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Boston, MA, USA, 4–9 April 2009; Association for Computing Machinery: New York, NY, USA, 2009; pp. 719–728. [Google Scholar]
- Forlizzi, J.; Battarbee, K. Understanding experience in interactive systems. In Proceedings of the 5th Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques, Cambridge, MA, USA, 1–4 August 2004; Association for Computing Machinery: New York, NY, USA, 2004; pp. 261–268. [Google Scholar]
- Mahlke, S.; Minge, M. Consideration of Multiple Components of Emotions in Human-Technology Interaction. In Affect and Emotion in Human-Computer Interaction: From Theory to Applications; Springer: Berlin/Heidelberg, Germany, 2008; pp. 51–62. [Google Scholar]
- Thüring, M.; Mahlke, S. Usability, aesthetics and emotions in human–technology interaction. Int. J. Psychol. 2007, 42, 253–264. [Google Scholar] [CrossRef]
- Huisman, G.; Van Hout, M.; Van Dijk, E.; Van Der Geest, T.; Heylen, D. LEMtool: Measuring emotions in visual interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Paris, France, 27 April–2 May 2013; Association for Computing Machinery: New York, NY, USA, 2013; pp. 351–360. [Google Scholar]
- Ortony, A.; Clore, G.L.; Collins, A. The Cognitive Structure of Emotions; Cambridge University Press: Cambridge, MA, USA, 1988. [Google Scholar]
- Mehrabian, A. Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in Temperament. Curr. Psychol. 1996, 14, 261–292. [Google Scholar] [CrossRef]
- Kleinginna, P.R.; Kleinginna, A.M. A categorized list of emotion definitions, with suggestions for a consensual definition. Motiv. Emot. 1981, 5, 345–379. [Google Scholar] [CrossRef]
- Shiota, M.N.; Kalat, J.W. Emotion, 2nd ed.; Wadsworth Cengage Learning: San Francisco, CA, USA, 2012. [Google Scholar]
- Russell, J.A.; Mehrabian, A. Evidence for a three-factor theory of emotions. J. Res. Pers. 1977, 11, 273–294. [Google Scholar] [CrossRef]
- Gebhard, P. ALMA: A layered model of affect. In Proceedings of the 4th International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS 2005), Utrecht, The Netherlands, 25–29 July 2005. [Google Scholar]
- Desmet, P.; Hekkert, P.; Jacobs, J. When a car makes you smile: Development and application of an instrument to measure prod-uct emotions. Adv. Consum. Res. 2000, 27, 111–117. [Google Scholar]
- Laurans, G.F.G.; Desmet, P.M.A. Introducing PrEmo2 new directions for the non-verbal measurement of emotion in design. In Out of Control: Proceedings of the 8th International Conference on Design and Emotion, London, UK, 11–14 September 2012; Central Saint Martins College of the Arts and the Design and Emotion Society: London, UK, 2012. [Google Scholar]
- Bradley, M.M.; Lang, P.J. Measuring emotion: The self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry 1994, 25, 49–59. [Google Scholar] [CrossRef]
- Desmet, P.; Laurans, G. Developing 14 animated characters for non-verbal self-report of categorical emotions. J. Des. Res. 2017, 15, 214–233. [Google Scholar] [CrossRef] [Green Version]
- Vastenburg, M.; Romero Herrera, N.; Van Bel, D.; Desmet, P. PMRI: Development of a pictorial mood reporting instrument. In CHI ’11 Extended Abstracts on Human Factors in Computing Systems; Association for Computing Machinery: Vancouver, BC, Canada, 2011; pp. 2155–2160. [Google Scholar]
- Bassili, J.N. Emotion recognition: The role of facial movement and the relative importance of upper and lower areas of the face. J. Pers. Soc. Psychol. 1979, 37, 2049–2058. [Google Scholar] [CrossRef]
- Blow, M.; Dautenhahn, K.; Appleby, A.; Nehaniv, C.L.; Lee, D.C. Perception of Robot Smiles and Dimensions for Human-Robot Interaction Design. In Proceedings of the ROMAN 2006—The 15th IEEE International Symposium on Robot and Human Interactive Communication, Hatfield, UK, 6–8 September 2006; pp. 469–474. [Google Scholar]
- Kobayashi, H.; Suzuki, S.; Takahashi, H. Automatic extraction of facial organs and recognition of facial expressions. In Proceedings of the 8th IEEE International Workshop on Robot and Human Interaction. RO-MAN ’99 (Cat. No.99TH8483), Pisa, Italy, 27–29 September 1999. [Google Scholar]
- Zhao, Y. A study of audience psychology in emojis. Art Des. Res. 2016, 6, 46–49. [Google Scholar]
- Shao, Z. Visual Humor Representation of Cartoon Character Design—Taking Tom and Jerry as an Example. Decorate 2016, 4, 138–139. [Google Scholar]
- Ekman, P. About Brows: Emotional and Conversational Signals; Cambridge University Press: Cambridge, UK, 1979. [Google Scholar]
Reproach | Pride | Love | Admiration | Gratitude | Resentment | Hope | Relief | Satisfaction | Fear | |
---|---|---|---|---|---|---|---|---|---|---|
P | −0.50 | 0.72 | 0.73 | 0.48 | 0.50 | −0.53 | 0.65 | 0.30 | 0.66 | −0.64 |
A | 0.25 | 0.57 | 0.55 | 0.34 | 0.35 | 0.09 | 0.55 | −0.07 | 0.33 | 0.45 |
D | 0.43 | 0.55 | 0.36 | −0.13 | −0.10 | −0.35 | 0.38 | 0.26 | 0.45 | −0.60 |
Disappointment | Mildness | Pity | Distress | Anger | Shame | Happy-For | Hate | Boredom | Gloating | |
P | −0.65 | 0.18 | −0.57 | −0.75 | −0.74 | −0.62 | 0.67 | −0.58 | −0.22 | 0.12 |
A | −0.28 | −0.15 | −0.25 | −0.27 | 0.36 | 0.08 | 0.49 | 0.12 | −0.49 | 0.13 |
D | −0.54 | 0.08 | −0.49 | −0.55 | 0.26 | −0.55 | 0.43 | 0.07 | −0.24 | 0.03 |
Word | |||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Reproach | Pride | Love | Admiration | Gratitude | Resentment | Hope | Relief | Satisfaction | Fear | Disappointment | Mildness | Pity | Distress | Anger | Shame | Happy-For | Hate | Boredom | Gloating | ||
Image | Reproach | 0.27 | 1.27 | 1.02 | 0.90 | 0.82 | 0.47 | 0.78 | 0.72 | 0.98 | 0.54 | 0.63 | 0.74 | 0.42 | 0.92 | 0.50 | 0.64 | 0.47 | 0.38 | 0.69 | 0.49 |
Pride | 1.07 | 0.54 | 0.24 | 0.17 | 0.27 | 1.17 | 0.36 | 0.56 | 0.12 | 1.27 | 1.38 | 0.72 | 1.13 | 1.48 | 1.15 | 1.41 | 0.21 | 1.18 | 1.12 | 0.47 | |
Love | 0.99 | 0.64 | 0.24 | 0.16 | 0.12 | 1.15 | 0.20 | 0.61 | 0.31 | 1.15 | 1.34 | 0.75 | 1.12 | 1.40 | 1.16 | 1.31 | 0.25 | 1.17 | 1.12 | 0.42 | |
Admiration | 1.06 | 0.52 | 0.16 | 0.06 | 0.21 | 1.14 | 0.32 | 0.60 | 0.18 | 1.23 | 1.38 | 0.76 | 1.14 | 1.45 | 1.12 | 1.38 | 0.13 | 1.18 | 1.15 | 0.45 | |
Gratitude | 1.14 | 0.43 | 0.07 | 0.09 | 0.28 | 1.21 | 0.39 | 0.70 | 0.20 | 1.31 | 1.48 | 0.85 | 1.24 | 1.55 | 1.19 | 1.47 | 0.04 | 1.28 | 1.25 | 0.55 | |
Resentment | 0.72 | 1.59 | 1.26 | 1.13 | 0.99 | 0.94 | 0.90 | 0.64 | 1.14 | 0.67 | 0.33 | 0.55 | 0.17 | 0.67 | 0.98 | 0.58 | 1.24 | 0.30 | 0.19 | 0.68 | |
Hope | 1.09 | 0.56 | 0.16 | 0.06 | 0.15 | 1.16 | 0.26 | 0.61 | 0.22 | 1.20 | 1.38 | 0.76 | 1.14 | 1.44 | 1.15 | 1.36 | 0.16 | 1.19 | 1.15 | 0.44 | |
Relief | 0.91 | 1.06 | 0.69 | 0.57 | 0.41 | 1.08 | 0.32 | 0.15 | 0.74 | 0.60 | 0.83 | 0.25 | 0.63 | 0.84 | 0.76 | 0.76 | 0.73 | 0.66 | 0.75 | 0.23 | |
Satisfaction | 0.60 | 0.72 | 0.33 | 0.21 | 0.05 | 1.09 | 0.11 | 0.46 | 0.29 | 1.08 | 1.22 | 0.61 | 0.99 | 1.31 | 1.09 | 1.21 | 0.32 | 1.05 | 0.98 | 0.29 | |
Fear | 0.97 | 1.46 | 1.18 | 1.08 | 0.98 | 0.58 | 1.11 | 1.01 | 1.22 | 0.31 | 0.75 | 1.03 | 0.68 | 0.52 | 0.68 | 0.47 | 1.16 | 0.65 | 0.93 | 0.71 | |
Disappointment | 0.84 | 1.81 | 1.48 | 1.35 | 1.20 | 0.90 | 0.19 | 0.96 | 1.41 | 0.43 | 0.20 | 0.89 | 0.33 | 0.36 | 0.98 | 0.23 | 1.45 | 0.74 | 0.52 | 0.89 | |
Mildness | 0.98 | 0.66 | 0.28 | 0.16 | 0.11 | 1.05 | 0.19 | 0.55 | 0.30 | 1.08 | 1.26 | 0.69 | 1.03 | 1.32 | 1.05 | 1.23 | 0.27 | 1.07 | 1.05 | 0.32 | |
Pity | 0.66 | 1.64 | 1.32 | 1.19 | 1.05 | 0.87 | 0.97 | 0.74 | 1.22 | 0.59 | 0.23 | 0.66 | 0.09 | 0.55 | 0.93 | 0.48 | 1.29 | 0.21 | 0.30 | 0.73 | |
Distress | 0.68 | 1.79 | 1.48 | 1.35 | 1.22 | 0.79 | 1.14 | 1.03 | 1.43 | 0.31 | 0.28 | 0.97 | 0.39 | 0.22 | 0.88 | 0.11 | 1.45 | 0.36 | 0.64 | 0.90 | |
Anger | 0.31 | 1.42 | 1.28 | 1.19 | 1.18 | 0.06 | 1.17 | 1.21 | 1.32 | 0.69 | 0.97 | 1.26 | 0.85 | 0.75 | 0.10 | 0.85 | 1.24 | 0.75 | 1.18 | 0.91 | |
Shame | 0.97 | 0.85 | 0.45 | 0.34 | 0.15 | 1.07 | 0.06 | 0.41 | 0.40 | 0.99 | 1.13 | 0.54 | 0.91 | 1.13 | 1.09 | 1.12 | 0.45 | 0.97 | 0.89 | 0.24 | |
Happy-for | 1.17 | 0.45 | 0.05 | 0.09 | 0.26 | 1.22 | 0.37 | 0.71 | 0.24 | 0.99 | 1.49 | 0.87 | 1.26 | 1.49 | 1.21 | 1.47 | 0.07 | 1.30 | 1.26 | 0.55 | |
Hate | 1.14 | 1.37 | 1.13 | 1.02 | 0.95 | 0.34 | 0.91 | 0.87 | 1.11 | 1.10 | 0.63 | 0.89 | 0.56 | 0.77 | 0.39 | 0.59 | 1.09 | 0.39 | 0.78 | 0.62 | |
Boredom | 0.70 | 1.24 | 0.90 | 0.77 | 0.99 | 0.90 | 0.55 | 0.33 | 0.78 | 0.74 | 0.66 | 0.32 | 0.43 | 0.86 | 0.92 | 0.77 | 0.88 | 0.52 | 0.42 | 0.34 | |
Gloating | 1.22 | 0.33 | 0.07 | 0.19 | 0.38 | 1.26 | 0.49 | 0.81 | 0.30 | 1.39 | 1.58 | 0.97 | 1.35 | 1.59 | 1.24 | 1.56 | 0.29 | 1.38 | 1.36 | 0.42 |
Word | |||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Reproach | Pride | Love | Admiration | Gratitude | Resentment | Hope | Relief | Satisfaction | Fear | Disappointment | Mildness | Pity | Distress | Anger | Shame | Happy-For | Hate | Boredom | Gloating | ||
Image | Reproach | 0.26 | 1.42 | 1.42 | 1.29 | 1.29 | 0.85 | 1.34 | 1.12 | 1.35 | 1.02 | 1.19 | 1.09 | 1.14 | 1.38 | 0.18 | 1.03 | 0.85 | 0.48 | 1.22 | 0.94 |
Pride | 1.24 | 0.06 | 0.23 | 0.78 | 0.74 | 1.61 | 0.21 | 0.84 | 0.31 | 1.77 | 1.95 | 1.04 | 1.85 | 2.02 | 1.48 | 1.79 | 0.19 | 1.45 | 1.64 | 0.92 | |
Love | 1.13 | 0.16 | 0.07 | 0.62 | 0.58 | 1.51 | 0.06 | 0.78 | 0.26 | 1.66 | 1.85 | 0.95 | 1.75 | 1.92 | 1.45 | 1.69 | 0.10 | 1.39 | 1.55 | 0.82 | |
Admiration | 0.95 | 0.46 | 0.38 | 0.43 | 0.40 | 1.18 | 0.32 | 0.42 | 0.27 | 1.40 | 1.49 | 0.58 | 1.39 | 1.57 | 1.18 | 1.37 | 0.32 | 1.06 | 1.17 | 0.46 | |
Gratitude | 0.98 | 0.51 | 0.39 | 0.31 | 0.29 | 1.14 | 0.35 | 0.46 | 0.35 | 1.34 | 1.45 | 0.58 | 1.35 | 1.53 | 1.19 | 1.31 | 0.36 | 1.06 | 1.15 | 0.42 | |
Resentment | 0.84 | 1.58 | 1.49 | 1.04 | 1.07 | 0.37 | 1.44 | 0.86 | 1.37 | 0.82 | 0.40 | 0.65 | 0.31 | 0.48 | 0.87 | 0.50 | 1.44 | 0.52 | 0.29 | 0.67 | |
Hope | 1.15 | 0.24 | 0.12 | 0.52 | 0.48 | 1.40 | 0.07 | 0.67 | 0.21 | 1.57 | 1.73 | 0.83 | 1.63 | 1.81 | 1.37 | 1.58 | 0.10 | 1.29 | 1.43 | 0.70 | |
Relief | 0.88 | 0.96 | 0.87 | 0.52 | 0.53 | 0.87 | 0.83 | 0.23 | 0.82 | 0.89 | 0.97 | 0.09 | 0.88 | 1.04 | 0.87 | 0.82 | 0.85 | 0.65 | 0.78 | 0.17 | |
Satisfaction | 0.97 | 0.40 | 0.32 | 0.44 | 0.41 | 1.27 | 0.28 | 0.45 | 0.19 | 1.49 | 1.57 | 0.62 | 1.47 | 1.65 | 1.28 | 1.45 | 0.26 | 1.16 | 1.23 | 0.53 | |
Fear | 1.10 | 1.63 | 1.52 | 1.06 | 1.09 | 0.38 | 1.67 | 1.23 | 1.52 | 0.20 | 0.72 | 1.07 | 0.68 | 0.73 | 0.83 | 0.36 | 1.51 | 0.67 | 0.99 | 0.88 | |
Disappointment | 1.05 | 1.82 | 1.72 | 1.23 | 1.26 | 0.43 | 0.93 | 1.11 | 1.62 | 0.80 | 0.17 | 0.88 | 0.10 | 0.27 | 1.03 | 0.42 | 1.68 | 1.05 | 0.39 | 0.91 | |
Mildness | 0.79 | 1.08 | 0.98 | 0.58 | 0.59 | 0.68 | 0.93 | 0.38 | 0.86 | 1.03 | 0.90 | 0.19 | 0.80 | 0.98 | 0.95 | 0.86 | 0.93 | 0.67 | 0.56 | 0.19 | |
Pity | 1.04 | 1.85 | 1.75 | 1.27 | 1.29 | 0.39 | 1.70 | 1.15 | 1.65 | 0.75 | 0.11 | 0.93 | 0.04 | 0.20 | 0.99 | 0.37 | 1.71 | 0.66 | 0.46 | 0.94 | |
Distress | 1.19 | 2.06 | 1.96 | 1.47 | 1.50 | 0.50 | 1.91 | 1.38 | 1.87 | 0.74 | 0.13 | 1.17 | 0.23 | 0.05 | 1.08 | 0.39 | 1.92 | 0.80 | 0.68 | 1.16 | |
Anger | 0.40 | 1.55 | 1.53 | 1.31 | 1.32 | 0.69 | 1.45 | 1.19 | 1.47 | 0.82 | 1.03 | 1.12 | 0.99 | 1.02 | 0.09 | 0.84 | 1.47 | 0.39 | 1.15 | 0.96 | |
Shame | 0.84 | 1.55 | 1.44 | 0.96 | 0.99 | 0.13 | 1.39 | 0.99 | 1.39 | 0.46 | 0.48 | 0.80 | 0.41 | 0.53 | 0.78 | 0.26 | 1.41 | 0.50 | 0.65 | 0.68 | |
Happy-for | 1.14 | 0.23 | 0.13 | 0.53 | 0.49 | 1.40 | 0.07 | 0.66 | 0.20 | 0.46 | 1.73 | 0.83 | 1.63 | 1.78 | 1.36 | 1.58 | 0.09 | 1.28 | 1.42 | 0.70 | |
Hate | 1.14 | 1.35 | 1.29 | 0.97 | 0.98 | 0.38 | 1.22 | 0.84 | 1.21 | 1.30 | 0.75 | 0.72 | 0.62 | 0.90 | 0.41 | 0.60 | 1.24 | 0.15 | 0.76 | 0.58 | |
Boredom | 0.97 | 1.61 | 1.52 | 1.08 | 1.07 | 0.58 | 1.47 | 0.83 | 1.38 | 1.01 | 0.50 | 0.61 | 0.42 | 0.60 | 1.04 | 0.68 | 1.46 | 0.69 | 0.09 | 0.72 | |
Gloating | 0.74 | 0.77 | 0.69 | 0.44 | 0.43 | 0.88 | 0.63 | 0.30 | 0.58 | 1.15 | 1.18 | 0.34 | 1.08 | 1.25 | 0.95 | 1.07 | 0.65 | 0.78 | 0.88 | 0.09 |
Image | Hit Rate | Other Options | Hit Rate |
---|---|---|---|
Reproach | 52% | Anger | 38% |
Pride | 74% | None | None |
Love | 92% | None | None |
Admiration | 54% | Satisfaction | 28% |
Gratitude | 44% | Hope | 18% |
Resentment | 40% | Hate | 24% |
Hope | 40% | Admiration | 20% |
Relief | 42% | Mildness | 34% |
Satisfaction | 72% | None | None |
Fear | 82% | None | None |
Disappointment | 50% | Distress | 34% |
Mildness | 44% | Relief | 38% |
Pity | 42% | Disappointment | 36% |
Distress | 92% | None | None |
Anger | 92% | None | None |
Shame | 72% | None | None |
Happy-for | 50% | Love | 24% |
Hate | 54% | Resentment | 14% |
Boredom | 48% | Disappointment | 20% |
Gloating | 82% | None | None |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Wang, J.; Liu, Y.; Wang, Y.; Mao, J.; Yue, T.; You, F. SAET: The Non-Verbal Measurement Tool in User Emotional Experience. Appl. Sci. 2021, 11, 7532. https://doi.org/10.3390/app11167532
Wang J, Liu Y, Wang Y, Mao J, Yue T, You F. SAET: The Non-Verbal Measurement Tool in User Emotional Experience. Applied Sciences. 2021; 11(16):7532. https://doi.org/10.3390/app11167532
Chicago/Turabian StyleWang, Jianmin, Yujia Liu, Yuxi Wang, Jinjing Mao, Tianyang Yue, and Fang You. 2021. "SAET: The Non-Verbal Measurement Tool in User Emotional Experience" Applied Sciences 11, no. 16: 7532. https://doi.org/10.3390/app11167532
APA StyleWang, J., Liu, Y., Wang, Y., Mao, J., Yue, T., & You, F. (2021). SAET: The Non-Verbal Measurement Tool in User Emotional Experience. Applied Sciences, 11(16), 7532. https://doi.org/10.3390/app11167532