Review Reports
- Douglas O’Shaughnessy
Reviewer 1: Oleh Berezsky Reviewer 2: Anonymous
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsUser access to information systems is a relevant and timely problem. The use of natural language queries makes it possible to simplify the interaction interface with information systems. Therefore, the analysis of methods and tools that implement natural language interfaces is an актуальна research topic, and the paper addresses an important and relevant subject.
At the same time, in my opinion, the comments and suggestions presented below would improve the quality of the paper.
Suggestions:
- At the end of Section 1 (Introduction), the author should clearly formulate their scientific contribution to the review article. This is consistent with the conventions and expectations of MDPI journals.
- In addition, the author should explain how the proposed review differs from existing review articles on the same topic.
- In Section 2 (Applications), the author should present and describe a hypothetical architecture of an information system, explicitly identifying the interface component and the requirements imposed on it.
- The paper analyzes classical approaches to the design of natural language interfaces; however, the analysis is largely descriptive. Therefore, the author should define explicit criteria for comparing the methods and present the comparison in a tabular form.
- Analogously to point 4, the author should provide a comparative analysis of methods based on large language models (LLMs).
- The author should also describe concrete implementations of natural language interfaces. This would allow readers to better understand the current state of the art in the development of such interfaces.
- The conclusions should be revised: the author should clearly articulate the key trends in the development of natural language interfaces, outline promising directions for future research, and indicate which approaches have the greatest potential for deployment in real-world information systems in the coming years.
Author Response
Author responses to the review comments
(Review comments are preceded by a ‘>’, and then followed, in each case, by my response.)
R1:
> User access to information systems is a relevant and timely problem. The use of natural language queries makes it possible to simplify the interaction interface with information systems. Therefore, the analysis of methods and tools that implement natural language interfaces is an актуальна research topic, and the paper addresses an important and relevant subject.
Thank you for these comments.
> 1. At the end of Section 1 (Introduction), the author should clearly formulate their scientific contribution to the review article. This is consistent with the conventions and expectations of MDPI journals.
Yes, I added text there for my scientific contribution.
> 2. In addition, the author should explain how the proposed review differs from existing review articles on the same topic.
In revised text here, I noted how my review differs from existing review articles (e.g., refs. 1-5) on the same topic.
> 3. In Section 2 (Applications), the author should present and describe a hypothetical architecture of an information system, explicitly identifying the interface component and the requirements imposed on it.
At the end of this section, I added a description for architecture of an information system.
> 4. The paper analyzes classical approaches to the design of natural language interfaces; however, the analysis is largely descriptive. Therefore, the author should define explicit criteria for comparing the methods and present the comparison in a tabular form.
I prefer that the more detailed survey papers I cite provide such details, as the field has a huge number of diverse methods and lacks specific benchmarks that would allow rigorous comparison. Unlike other fields I have examined, surveys of information systems do not indicate explicit system comparisons.
> 5. Analogously to point 4, the author should provide a comparative analysis of methods based on large language models (LLMs).
I have added more analysis, although (as I note above) there is little published comparison of methods.
> 6. The author should also describe concrete implementations of natural language interfaces. This would allow readers to better understand the current state of the art in the development of such interfaces.
I have noted how NL interfaces behave.
> 7. The conclusions should be revised: the author should clearly articulate the key trends in the development of natural language interfaces, outline promising directions for future research, and indicate which approaches have the greatest potential for deployment in real-world information systems in the coming years.
I have revised the conclusions to: a) clearly articulate key trends in the development of NL interfaces, b) outline promising directions for future research, c) indicate which approaches have the more potential for deployment in real-world information systems in the near future.
Reviewer 2 Report
Comments and Suggestions for AuthorsThis paper provides a survey of recent advances in Natural Language Processing (NLP) technologies within information systems, covering areas ranging from fundamental language models to large language models, neural architectures, topic modeling, and fine-tuning methods. The manuscript offers broad coverage of the topic and cites relatively recent literature, providing certain value in organizing the developments within the field. However, there remains significant room for improvement in terms of the logical organization, analytical depth, integration with information systems contexts, and clarity of expression. It is recommended that the manuscript be accepted for publication after the author addresses the following specific comments with substantive revisions. The detailed comments are as follows:
1. It is recommended to strengthen the logical thread and structural visualization of the survey. The current chapter arrangement is somewhat flat and lacks inherent cohesiveness. It is suggested that the author provide a clear logical flow diagram or a framework diagram of technological development in the introduction or within a newly added "survey framework" section. This should clarify the intrinsic evolution and interrelationships from fundamental tasks to core methods, and further to advanced themes and challenges.
2. It is recommended to augment critical analysis and comparative summaries. The current presentation of numerous models and methods leans towards listing rather than providing in-depth comparison and commentary. It is suggested to add summary subsections or comparative tables at the end of key chapters (e.g., Sections 4 and 8) to systematically analyze the strengths, limitations, and applicable scenarios of different technical approaches. For example, a comparison of the effectiveness of encoder-decoder architectures versus decoder-only architectures for information systems tasks could be included.
3. It is recommended to supplement and highlight cutting-edge directions from recent years. The space devoted to certain content (e.g., N-gram models, traditional topic models) seems somewhat disproportionate to the title "Recent Advances." It is advised to condense the discussion of traditional content and significantly enhance the coverage of frontier topics from the last 2-3 years. Examples include: a systematic introduction to Retrieval-Augmented Generation (RAG), AI agents and tool calling, NLP for multimodal information systems, model lightweighting and efficient fine-tuning (e.g., QLoRA), and domain adaptation (e.g., for financial or healthcare information systems).
4. It is recommended to deepen the connection with "Information Systems" application scenarios. While the title and introduction emphasize information systems, the technical descriptions in the main body remain quite generic. It is suggested to either add a dedicated section or weave throughout the paper a discussion on how NLP technologies concretely address core problems in information systems. Examples include: complex query understanding, natural language interfaces for cross-database, real-time updating and Q&A for dynamic knowledge bases, and knowledge extraction and management from enterprise unstructured documents. Relevant applied research should be cited.
5. It is recommended to improve the clarity and rigor of methodological descriptions. The explanations for some key techniques (e.g., DPO, Semantic Role Labeling, Neural Topic Models) are overly brief. It is suggested to supplement key methodological points with concise formulas, flowcharts, or pseudocode to enhance comprehensibility and reproducibility. For instance, a flowchart could be used in Section 6 (Prompting) to illustrate the processes of "discrete optimization" versus "gradient-based soft optimization" methods.
6. It is recommended to thoroughly polish the language and standardize the formatting. The manuscript contains instances of lengthy sentences, redundant expressions, or awkward phrasing that hinder reading fluency. A full-language polish is recommended to strive for conciseness and clarity. Furthermore, please strictly adhere to the journal's formatting guidelines, uniformly and meticulously verify all reference entries (authors, titles, journal/conference, year, page numbers/DOIs) to ensure accuracy.
7. It is recommended to optimize the presentation of figures and experimental results. The current manuscript lacks comprehensive graphical elements. It is suggested to add at least 1-2 synthetic figures or tables. Examples include: a schematic diagram illustrating how NLP technologies integrate and function within a typical information systems architecture, or a summary table comparing the performance of mainstream LLMs on classic information tasks (e.g., QA, summarization). This would significantly enhance the visual appeal and value of the survey.
Author Response
Author responses to the review comments
(Review comments are preceded by a ‘>’, and then followed, in each case, by my response.
————————
R2:
> This paper provides a survey of recent advances in Natural Language Processing (NLP) technologies within information systems, covering areas ranging from fundamental language models to large language models, neural architectures, topic modeling, and fine-tuning methods. The manuscript offers broad coverage of the topic and cites relatively recent literature, providing certain value in organizing the developments within the field.
Thank you for these comments.
>However, there remains significant room for improvement in terms of the logical organization, analytical depth, integration with information systems contexts, and clarity of expression. It is recommended that the manuscript be accepted for publication after the author addresses the following specific comments with substantive revisions.
In this revision, I have improved the work in terms of organization, depth, integration with information system contexts, and its clarity. I reorganized many sections, reducing the number of chapters. More details are below, in response to other specific suggestions.
> 1. It is recommended to strengthen the logical thread and structural visualization of the survey. The current chapter arrangement is somewhat flat and lacks inherent cohesiveness. It is suggested that the author provide a clear logical flow diagram or a framework diagram of technological development in the introduction or within a newly added "survey framework" section. This should clarify the intrinsic evolution and interrelationships from fundamental tasks to core methods, and further to advanced themes and challenges.
I added a survey framework in the introduction, to clarify the relationships of the fundamental tasks and the structure of the paper.
> 2. It is recommended to augment critical analysis and comparative summaries. The current presentation of numerous models and methods leans towards listing rather than providing in-depth comparison and commentary. It is suggested to add summary subsections or comparative tables at the end of key chapters (e.g., Sections 4 and 8) to systematically analyze the strengths, limitations, and applicable scenarios of different technical approaches. For example, a comparison of the effectiveness of encoder-decoder architectures versus decoder-only architectures for information systems tasks could be included.
I added summary subsections at the end of chapters 4 and 8, as requested. I added more critical analysis, e.g., the advantages and disadvantages of various methods. Given the large diversity of the methods, it is difficult to compare methods unless they operate on the same datasets.
>3. It is recommended to supplement and highlight cutting-edge directions from recent years. The space devoted to certain content (e.g., N-gram models, traditional topic models) seems somewhat disproportionate to the title "Recent Advances." It is advised to condense the discussion of traditional content and significantly enhance the coverage of frontier topics from the last 2-3 years. Examples include: a systematic introduction to Retrieval-Augmented Generation (RAG), AI agents and tool calling, NLP for multimodal information systems, model lightweighting and efficient fine-tuning (e.g., QLoRA), and domain adaptation (e.g., for financial or healthcare information systems).
I increased the coverage of several recent topics, including Retrieval-Augmented Generation (RAG), AI agents, tool calling, NLP for multimodal information systems, model light-weighting, efficient fine-tuning, and domain adaptation.
> 4. It is recommended to deepen the connection with "Information Systems" application scenarios. While the title and introduction emphasize information systems, the technical descriptions in the main body remain quite generic. It is suggested to either add a dedicated section or weave throughout the paper a discussion on how NLP technologies concretely address core problems in information systems. Examples include: complex query understanding, natural language interfaces for cross-database, real-time updating and Q&A for dynamic knowledge bases, and knowledge extraction and management from enterprise unstructured documents. Relevant applied research should be cited.
I focussed on complex query understanding, NL interfaces for cross-domain, real-time updating, Q&A for dynamic knowledge bases, and knowledge extraction from unstructured documents. I added relevant citations to recent applied research.
> 5. It is recommended to improve the clarity and rigor of methodological descriptions. The explanations for some key techniques (e.g., DPO, Semantic Role Labeling, Neural Topic Models) are overly brief. It is suggested to supplement key methodological points with concise formulas, flowcharts, or pseudocode to enhance comprehensibility and reproducibility. For instance, a flowchart could be used in Section 6 (Prompting) to illustrate the processes of "discrete optimization" versus "gradient-based soft optimization" methods.
I added more details to these topics, as suggested: DPO, Semantic Role Labeling, Neural Topic Models. I did not wish to include detailed math descriptions, in order to keep the level of the text accessible to non-experts. In general, I find that far too many papers in the field are only accessible to experts, as few of the ideas are explained. In Section 6, I illustrated the processes of "discrete optimization" versus "gradient-based soft optimization" methods.
> 6. It is recommended to thoroughly polish the language and standardize the formatting. The manuscript contains instances of lengthy sentences, redundant expressions, or awkward phrasing that hinder reading fluency. A full-language polish is recommended to strive for conciseness and clarity. Furthermore, please strictly adhere to the journal's formatting guidelines, uniformly and meticulously verify all reference entries (authors, titles, journal/conference, year, page numbers/DOIs) to ensure accuracy.
I apologize for any weaknesses in my writing, and have thoroughly gone through the paper to reduce poorly-phrased expressions, aiming for conciseness and clarity. I also added DOIs to the references (where available), and made sure to follow MDPI formatting guidelines throughout the paper.
> 7. It is recommended to optimize the presentation of figures and experimental results. The current manuscript lacks comprehensive graphical elements. It is suggested to add at least 1-2 synthetic figures or tables. Examples include: a schematic diagram illustrating how NLP technologies integrate and function within a typical information systems architecture, or a summary table comparing the performance of mainstream LLMs on classic information tasks (e.g., QA, summarization). This would significantly enhance the visual appeal and value of the survey.
As I noted elsewhere, formal performance evaluation of mainstream LLMs on classic information tasks is lacking. Indeed, in the many articles I reviewed to prepare this paper, there are few diagrams, as the field does not lend itself well to images, other than lists of methods, which are readily instead described in text form.
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsThe reviewer’s comments regarding both the content and the form of the manuscript have been addressed in the revised version. The manuscript has been improved after revision and can be recommended for publication.
Reviewer 2 Report
Comments and Suggestions for AuthorsACCEPT