Next Article in Journal
Making the Case for a P2P Personal Health Record
Previous Article in Journal
Evaluation of Push and Pull Communication Models on a VANET with Virtual Traffic Lights
Open AccessReview

Survey of Neural Text Representation Models

Center for Artificial Intelligence and Cybersecurity and Department of Informatics, University of Rijeka, 51000 Rijeka, Croatia
*
Author to whom correspondence should be addressed.
Information 2020, 11(11), 511; https://doi.org/10.3390/info11110511
Received: 1 October 2020 / Revised: 25 October 2020 / Accepted: 28 October 2020 / Published: 30 October 2020
(This article belongs to the Section Review)
In natural language processing, text needs to be transformed into a machine-readable representation before any processing. The quality of further natural language processing tasks greatly depends on the quality of those representations. In this survey, we systematize and analyze 50 neural models from the last decade. The models described are grouped by the architecture of neural networks as shallow, recurrent, recursive, convolutional, and attention models. Furthermore, we categorize these models by representation level, input level, model type, and model supervision. We focus on task-independent representation models, discuss their advantages and drawbacks, and subsequently identify the promising directions for future neural text representation models. We describe the evaluation datasets and tasks used in the papers that introduced the models and compare the models based on relevant evaluations. The quality of a representation model can be evaluated as its capability to generalize to multiple unrelated tasks. Benchmark standardization is visible amongst recent models and the number of different tasks models are evaluated on is increasing. View Full-Text
Keywords: deep learning; embedding; neural language model; neural networks; NLP; text representation deep learning; embedding; neural language model; neural networks; NLP; text representation
Show Figures

Figure 1

MDPI and ACS Style

Babić, K.; Martinčić-Ipšić, S.; Meštrović, A. Survey of Neural Text Representation Models. Information 2020, 11, 511.

Show more citation formats Show less citations formats
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Search more from Scilit
 
Search
Back to TopTop