A Comparison of Emotion Annotation Approaches for Text
AbstractWhile the recognition of positive/negative sentiment in text is an established task with many standard data sets and well developed methodologies, the recognition of a more nuanced affect has received less attention: there are few publicly available annotated resources and there are a number of competing emotion representation schemes with as yet no clear approach to choose between them. To address this lack, we present a series of emotion annotation studies on tweets, providing methods for comparisons between annotation methods (relative vs. absolute) and between different representation schemes. We find improved annotator agreement with a relative annotation scheme (comparisons) on a dimensional emotion model over a categorical annotation scheme on Ekman’s six basic emotions; however, when we compare inter-annotator agreement for comparisons with agreement for a rating scale annotation scheme (both with the same dimensional emotion model), we find improved inter-annotator agreement with rating scales, challenging a common belief that relative judgements are more reliable. To support these studies and as a contribution in itself, we further present a publicly available collection of 2019 tweets annotated with scores on each of four emotion dimensions: valence, arousal, dominance and surprise, following the emotion representation model identified by Fontaine et al. in 2007. View Full-Text
Share & Cite This Article
Wood, I.D.; McCrae, J.P.; Andryushechkin, V.; Buitelaar, P. A Comparison of Emotion Annotation Approaches for Text. Information 2018, 9, 117.
Wood ID, McCrae JP, Andryushechkin V, Buitelaar P. A Comparison of Emotion Annotation Approaches for Text. Information. 2018; 9(5):117.Chicago/Turabian Style
Wood, Ian D.; McCrae, John P.; Andryushechkin, Vladimir; Buitelaar, Paul. 2018. "A Comparison of Emotion Annotation Approaches for Text." Information 9, no. 5: 117.
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.