Next Article in Journal
Optimal Containment Control Strategy of the Second Phase of the COVID-19 Lockdown in Morocco
Next Article in Special Issue
Comparison of Deep Learning Models and Various Text Pre-Processing Techniques for the Toxic Comments Classification
Previous Article in Journal
Alzheimer’s Disease: A Thermodynamic Perspective
Previous Article in Special Issue
A Novel Hybrid Model for Cantonese Rumor Detection on Twitter
 
 
Article

Delayed Combination of Feature Embedding in Bidirectional LSTM CRF for NER

1
Department of Big Data Science, University of Science and Technology (UST), Daejeon 34113, Korea
2
Korea Institute of Science and Technology Information (KISTI), Daejeon 34113, Korea
*
Author to whom correspondence should be addressed.
Appl. Sci. 2020, 10(21), 7557; https://doi.org/10.3390/app10217557
Received: 30 July 2020 / Revised: 17 October 2020 / Accepted: 21 October 2020 / Published: 27 October 2020
Named Entity Recognition (NER) plays a vital role in natural language processing (NLP). Currently, deep neural network models have achieved significant success in NER. Recent advances in NER systems have introduced various feature selections to identify appropriate representations and handle Out-Of-the-Vocabulary (OOV) words. After selecting the features, they are all concatenated at the embedding layer before being fed into a model to label the input sequences. However, when concatenating the features, information collisions may occur and this would cause the limitation or degradation of the performance. To overcome the information collisions, some works tried to directly connect some features to latter layers, which we call the delayed combination and show its effectiveness by comparing it to the early combination. As feature encodings for input, we selected the character-level Convolutional Neural Network (CNN) or Long Short-Term Memory (LSTM) word encoding, the pre-trained word embedding, and the contextual word embedding and additionally designed CNN-based sentence encoding using a dictionary. These feature encodings are combined at early or delayed position of the bidirectional LSTM Conditional Random Field (CRF) model according to each feature’s characteristics. We evaluated the performance of this model on the CoNLL 2003 and OntoNotes 5.0 datasets using the F1 score and compared the delayed combination model with our own implementation of the early combination as well as the previous works. This comparison convinces us that our delayed combination is more effective than the early one and also highly competitive. View Full-Text
Keywords: delayed combination; CNN dictionary; named entity recognition; deep learning NER; bidirectional LSTM CRF; CoNLL; OntoNotes delayed combination; CNN dictionary; named entity recognition; deep learning NER; bidirectional LSTM CRF; CoNLL; OntoNotes
Show Figures

Figure 1

MDPI and ACS Style

Ronran, C.; Lee, S.; Jang, H.J. Delayed Combination of Feature Embedding in Bidirectional LSTM CRF for NER. Appl. Sci. 2020, 10, 7557. https://doi.org/10.3390/app10217557

AMA Style

Ronran C, Lee S, Jang HJ. Delayed Combination of Feature Embedding in Bidirectional LSTM CRF for NER. Applied Sciences. 2020; 10(21):7557. https://doi.org/10.3390/app10217557

Chicago/Turabian Style

Ronran, Chirawan, Seungwoo Lee, and Hong Jun Jang. 2020. "Delayed Combination of Feature Embedding in Bidirectional LSTM CRF for NER" Applied Sciences 10, no. 21: 7557. https://doi.org/10.3390/app10217557

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop