Next Article in Journal
Development of Driftwood Capture Trellis for Capturing Driftwood in Agricultural Drainage Ditches
Next Article in Special Issue
Zero-Shot Learning for Cross-Lingual News Sentiment Classification
Previous Article in Journal
3D Reconstruction of a Single Bubble in Transparent Media Using Three Orthographic Digital Images
Previous Article in Special Issue
Preliminary Results on Different Text Processing Tasks Using Encoder-Decoder Networks and the Causal Feature Extractor
Article

DAWE: A Double Attention-Based Word Embedding Model with Sememe Structure Information

by 1,2, 1, 1,2, 1, 1,2 and 3,*
1
School of Geography and Information Engineering, China University of Geosciences, Wuhan 430074, China
2
National Engineering Research Center for Geographic Information System, University of Geosciences, Wuhan 430074, China
3
School of Computer Science, China University of Geosciences, Wuhan 430074, China
*
Author to whom correspondence should be addressed.
Appl. Sci. 2020, 10(17), 5804; https://doi.org/10.3390/app10175804
Received: 15 July 2020 / Revised: 17 August 2020 / Accepted: 19 August 2020 / Published: 21 August 2020
Word embedding is an important reference for natural language processing tasks, which can generate distribution presentations of words based on many text data. Recent evidence demonstrates that introducing sememe knowledge is a promising strategy to improve the performance of word embedding. However, previous works ignored the structure information of sememe knowledges. To fill the gap, this study implicitly synthesized the structural feature of sememes into word embedding models based on an attention mechanism. Specifically, we propose a novel double attention word-based embedding (DAWE) model that encodes the characteristics of sememes into words by a “double attention” strategy. DAWE is integrated with two specific word training models through context-aware semantic matching techniques. The experimental results show that, in word similarity task and word analogy reasoning task, the performance of word embedding can be effectively improved by synthesizing the structural information of sememe knowledge. The case study also verifies the power of DAWE model in word sense disambiguation task. Furthermore, the DAWE model is a general framework for encoding sememes into words, which can be integrated into other existing word embedding models to provide more options for various natural language processing downstream tasks. View Full-Text
Keywords: natural language processing; word representation learning; word2vec; sememes; attention mechanism; structural information natural language processing; word representation learning; word2vec; sememes; attention mechanism; structural information
Show Figures

Figure 1

MDPI and ACS Style

Li, S.; Chen, R.; Wan, B.; Gong, J.; Yang, L.; Yao, H. DAWE: A Double Attention-Based Word Embedding Model with Sememe Structure Information. Appl. Sci. 2020, 10, 5804. https://doi.org/10.3390/app10175804

AMA Style

Li S, Chen R, Wan B, Gong J, Yang L, Yao H. DAWE: A Double Attention-Based Word Embedding Model with Sememe Structure Information. Applied Sciences. 2020; 10(17):5804. https://doi.org/10.3390/app10175804

Chicago/Turabian Style

Li, Shengwen, Renyao Chen, Bo Wan, Junfang Gong, Lin Yang, and Hong Yao. 2020. "DAWE: A Double Attention-Based Word Embedding Model with Sememe Structure Information" Applied Sciences 10, no. 17: 5804. https://doi.org/10.3390/app10175804

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop