A Multilingual and Multidomain Study on Dialog Act Recognition Using Character-Level Tokenization†
AbstractAutomatic dialog act recognition is an important step for dialog systems since it reveals the intention behind the words uttered by its conversational partners. Although most approaches on the task use word-level tokenization, there is information at the sub-word level that is related to the function of the words and, consequently, their intention. Thus, in this study, we explored the use of character-level tokenization to capture that information. We explored the use of multiple character windows of different sizes to capture morphological aspects, such as affixes and lemmas, as well as inter-word information. Furthermore, we assessed the importance of punctuation and capitalization for the task. To broaden the conclusions of our study, we performed experiments on dialogs in three languages—English, Spanish, and German—which have different morphological characteristics. Furthermore, the dialogs cover multiple domains and are annotated with both domain-dependent and domain-independent dialog act labels. The achieved results not only show that the character-level approach leads to similar or better performance than the state-of-the-art word-level approaches on the task, but also that both approaches are able to capture complementary information. Thus, the best results are achieved by combining tokenization at both levels. View Full-Text
Share & Cite This Article
Ribeiro, E.; Ribeiro, R.; de Matos, D.M. A Multilingual and Multidomain Study on Dialog Act Recognition Using Character-Level Tokenization. Information 2019, 10, 94.
Ribeiro E, Ribeiro R, de Matos DM. A Multilingual and Multidomain Study on Dialog Act Recognition Using Character-Level Tokenization. Information. 2019; 10(3):94.Chicago/Turabian Style
Ribeiro, Eugénio; Ribeiro, Ricardo; de Matos, David M. 2019. "A Multilingual and Multidomain Study on Dialog Act Recognition Using Character-Level Tokenization." Information 10, no. 3: 94.
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.