Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
5.8
3.1
Journals
Information
Special Issues
Neural Natural Language Generation
share announcement

Neural Natural Language Generation

A special issue of Information (ISSN 2078-2489). This special issue belongs to the section "Artificial Intelligence".

Deadline for manuscript submissions: closed (15 July 2021) | Viewed by 18194

Special Issue Editors

Dr. Valerio Basile

E-Mail Website
Guest Editor
Content-Centered Computing group, University of Turin, 10124 Torino TO, Italy
Interests: natural language generation; sentiment analysis; language resources; knowledge representation
Dr. Alessandro Mazzei

E-Mail Website
Guest Editor
Computer Science Department, University of Torino, 10124 Torino TO, Italy
Interests: natural language processing; natural language generation; syntactic parsing; dialogue systems

Special Issue Information

Dear Colleagues,

Natural language generation (NLG) has seen a resurgence in recent years, due to the availability of deep neural network models.

While recent technological advances are greatly pushing forward the quality of surface realizations, e.g., in terms of fluency, neural methods struggle to solve high-level, strategic generation tasks. In fact, NLG is one of the few sub-fields of natural language processing where symbolic systems still have predominance in real-world industrial applications.

Fueled by the development of neural NLG models, a new generation of conversational agents is also proliferating, with many systems being published for solving particular tasks, e.g., customer service, but also general-purpose personal assistance and chit-chat.

Another important aspect of neural architectures for NLG is their cognitive plausibility. With many models being proposed as black-box approaches, studying the properties of their mechanisms, such as back propagation, is bound to have significant implications on their interpretability and explainability from a cognitive standpoint.

The goal of this Special Issue of Information is to map the recent advances in neural natural language generation, along three main axes: i) architectural (e.g., end-to-end vs. modular); ii) teleological (e.g., task-based vs. generalist); iii) cognitive  (e.g., plausible vs. agnostic models).

We welcome contributions on topics such as, but not strictly limited to:

  • multilingual neural NLG
  • neural generation in dialogue systems
  • neural generation for machine translation
  • end-to-end neural NLG
  • modular architectures for neural NLG
  • management of hallucinations in neural NLG
  • evaluation of neural NLG systems
  • applications of neural NLG
  • social media content generation
  • summarization of news and social media posts

Dr. Valerio Basile
Dr. Alessandro Mazzei
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Information is an international peer-reviewed open access monthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 1600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • natural language generation
  • deep learning
  • neural architectures
  • cognitive models

Published Papers (5 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

39 pages, 983 KiB  
Article
Decoding Methods in Neural Language Generation: A Survey
by Sina Zarrieß, Henrik Voigt and Simeon Schüz
Information 2021, 12(9), 355; https://doi.org/10.3390/info12090355 - 30 Aug 2021
Cited by 9 | Viewed by 7367
Abstract
Neural encoder-decoder models for language generation can be trained to predict words directly from linguistic or non-linguistic inputs. When generating with these so-called end-to-end models, however, the NLG system needs an additional decoding procedure that determines the output sequence, given the infinite search [...] Read more.
Neural encoder-decoder models for language generation can be trained to predict words directly from linguistic or non-linguistic inputs. When generating with these so-called end-to-end models, however, the NLG system needs an additional decoding procedure that determines the output sequence, given the infinite search space over potential sequences that could be generated with the given vocabulary. This survey paper provides an overview of the different ways of implementing decoding on top of neural network-based generation models. Research into decoding has become a real trend in the area of neural language generation, and numerous recent papers have shown that the choice of decoding method has a considerable impact on the quality and various linguistic properties of the generation output of a neural NLG system. This survey aims to contribute to a more systematic understanding of decoding methods across different areas of neural NLG. We group the reviewed methods with respect to the broad type of objective that they optimize in the generation of the sequence—likelihood, diversity, and task-specific linguistic constraints or goals—and discuss their respective strengths and weaknesses. Full article
(This article belongs to the Special Issue Neural Natural Language Generation)
►▼ Show Figures

Figure 1

17 pages, 411 KiB  
Article
Ranking Algorithms for Word Ordering in Surface Realization
by Alessandro Mazzei, Mattia Cerrato, Roberto Esposito and Valerio Basile
Information 2021, 12(8), 337; https://doi.org/10.3390/info12080337 - 23 Aug 2021
Cited by 1 | Viewed by 1955
Abstract
In natural language generation, word ordering is the task of putting the words composing the output surface form in the correct grammatical order. In this paper, we propose to apply general learning-to-rank algorithms to the task of word ordering in the broader context [...] Read more.
In natural language generation, word ordering is the task of putting the words composing the output surface form in the correct grammatical order. In this paper, we propose to apply general learning-to-rank algorithms to the task of word ordering in the broader context of surface realization. The major contributions of this paper are: (i) the design of three deep neural architectures implementing pointwise, pairwise, and listwise approaches for ranking; (ii) the testing of these neural architectures on a surface realization benchmark in five natural languages belonging to different typological families. The results of our experiments show promising results, in particular highlighting the performance of the pairwise approach, paving the way for a more transparent surface realization from arbitrary tree- and graph-like structures. Full article
(This article belongs to the Special Issue Neural Natural Language Generation)
►▼ Show Figures

Figure 1

16 pages, 2036 KiB  
Article
Goal-Driven Visual Question Generation from Radiology Images
by Mourad Sarrouti, Asma Ben Abacha and Dina Demner-Fushman
Information 2021, 12(8), 334; https://doi.org/10.3390/info12080334 - 20 Aug 2021
Cited by 7 | Viewed by 2597
Abstract
Visual Question Generation (VQG) from images is a rising research topic in both fields of natural language processing and computer vision. Although there are some recent efforts towards generating questions from images in the open domain, the VQG task in the medical domain [...] Read more.
Visual Question Generation (VQG) from images is a rising research topic in both fields of natural language processing and computer vision. Although there are some recent efforts towards generating questions from images in the open domain, the VQG task in the medical domain has not been well-studied so far due to the lack of labeled data. In this paper, we introduce a goal-driven VQG approach for radiology images called VQGRaD that generates questions targeting specific image aspects such as modality and abnormality. In particular, we study generating natural language questions based on the visual content of the image and on additional information such as the image caption and the question category. VQGRaD encodes the dense vectors of different inputs into two latent spaces, which allows generating, for a specific question category, relevant questions about the images, with or without their captions. We also explore the impact of domain knowledge incorporation (e.g., medical entities and semantic types) and data augmentation techniques on visual question generation in the medical domain. Experiments performed on the VQA-RAD dataset of clinical visual questions showed that VQGRaD achieves 61.86% BLEU score and outperforms strong baselines. We also performed a blinded human evaluation of the grammaticality, fluency, and relevance of the generated questions. The human evaluation demonstrated the better quality of VQGRaD outputs and showed that incorporating medical entities improves the quality of the generated questions. Using the test data and evaluation process of the ImageCLEF 2020 VQA-Med challenge, we found that relying on the proposed data augmentation technique to generate new training samples by applying different kinds of transformations, can mitigate the lack of data, avoid overfitting, and bring a substantial improvement in medical VQG. Full article
(This article belongs to the Special Issue Neural Natural Language Generation)
►▼ Show Figures

Figure 1

24 pages, 1303 KiB  
Article
Semantic Systematicity in Connectionist Language Production
by Jesús Calvillo, Harm Brouwer and Matthew W. Crocker
Information 2021, 12(8), 329; https://doi.org/10.3390/info12080329 - 16 Aug 2021
Viewed by 2374
Abstract
Decades of studies trying to define the extent to which artificial neural networks can exhibit systematicity suggest that systematicity can be achieved by connectionist models but not by default. Here we present a novel connectionist model of sentence production that employs rich situation [...] Read more.
Decades of studies trying to define the extent to which artificial neural networks can exhibit systematicity suggest that systematicity can be achieved by connectionist models but not by default. Here we present a novel connectionist model of sentence production that employs rich situation model representations originally proposed for modeling systematicity in comprehension. The high performance of our model demonstrates that such representations are also well suited to model language production. Furthermore, the model can produce multiple novel sentences for previously unseen situations, including in a different voice (actives vs. passive) and with words in new syntactic roles, thus demonstrating semantic and syntactic generalization and arguably systematicity. Our results provide yet further evidence that such connectionist approaches can achieve systematicity, in production as well as comprehension. We propose our positive results to be a consequence of the regularities of the microworld from which the semantic representations are derived, which provides a sufficient structure from which the neural network can interpret novel inputs. Full article
(This article belongs to the Special Issue Neural Natural Language Generation)
►▼ Show Figures

Figure 1

10 pages, 323 KiB  
Article
Pre-Training on Mixed Data for Low-Resource Neural Machine Translation
by Wenbo Zhang, Xiao Li, Yating Yang and Rui Dong
Information 2021, 12(3), 133; https://doi.org/10.3390/info12030133 - 18 Mar 2021
Cited by 4 | Viewed by 2510
Abstract
The pre-training fine-tuning mode has been shown to be effective for low resource neural machine translation. In this mode, pre-training models trained on monolingual data are used to initiate translation models to transfer knowledge from monolingual data into translation models. In recent years, [...] Read more.
The pre-training fine-tuning mode has been shown to be effective for low resource neural machine translation. In this mode, pre-training models trained on monolingual data are used to initiate translation models to transfer knowledge from monolingual data into translation models. In recent years, pre-training models usually take sentences with randomly masked words as input, and are trained by predicting these masked words based on unmasked words. In this paper, we propose a new pre-training method that still predicts masked words, but randomly replaces some of the unmasked words in the input with their translation words in another language. The translation words are from bilingual data, so that the data for pre-training contains both monolingual data and bilingual data. We conduct experiments on Uyghur-Chinese corpus to evaluate our method. The experimental results show that our method can make the pre-training model have a better generalization ability and help the translation model to achieve better performance. Through a word translation task, we also demonstrate that our method enables the embedding of the translation model to acquire more alignment knowledge. Full article
(This article belongs to the Special Issue Neural Natural Language Generation)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-5
Back to TopTop