Abstract
The aim of this study is to systematically capture, synthesize, and evaluate current research trends related to Automated Multiple-Choice Question Generation as they emerge within the broader landscape of natural language processing (NLP) and large language model (LLM)-based educational and assessment research. A systematic search and selection process was conducted following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, using predefined inclusion and exclusion criteria. A total of 240 eligible publications indexed in the Scopus database were identified and analyzed. To provide a comprehensive overview of this evolving research landscape, a bibliometric analysis was performed utilizing performance analysis and scientific mapping methods, supported by the Bibliometrix (version 4.2.2) R package and VOSviewer (version 1.6.19) software. The findings of the performance analysis indicate a steady upward trend in publications and citations, with significant contributions from leading academic institutions—primarily from the United States—and a strong presence in high quality academic journals. Scientific mapping through co-authorship analysis reveals that, despite the increasing research activity, there remains a need for enhanced collaborative efforts. Bibliographic coupling organizes the analyzed literature into seven thematic clusters, highlighting the main research axes and their diachronic evolution. Furthermore, co-word analysis identifies emerging research trends and underexplored directions, indicating substantial opportunities for future investigation. To the best of our knowledge, this study represents the first systematic bibliometric analysis that examines Automated Multiple-Choice Question Generation research within the context of the broader LLM-driven educational assessment literature. By mapping the relevant scientific production and identifying research gaps and future directions, this work contributes to a more coherent understanding of the field and supports the ongoing development of research at the intersection of generative AI and educational assessment.
1. Introduction
In recent years, the rapid development of natural language processing (NLP) technologies and large language models (LLMs) has radically transformed the field of educational technology, introducing new capabilities for the automatic creation, assessment, and personalization of learning materials. Within this broader landscape of LLM-driven educational and assessment research, the automatic generation of multiple-choice questions (MCQs) from educational texts has emerged as a particularly relevant application, supporting formative assessments and the enhancement of active learning [1,2]. Within this context, automatic question generation is increasingly recognized as a core component of automated assessment systems, addressing the limitations of time-consuming and labor-intensive manual question design. Prior survey-based research highlights that the integration of automatic question generation and answer assessment mechanisms plays a crucial role in identifying learners’ knowledge gaps and supporting scalable, technology-enhanced learning environments [3].
This study focuses on Automated Multiple-Choice Question Generation, as this form of assessment plays a central role in modern educational environments, especially in the context of formative assessments and the support of active learning. Unlike other forms of assessment, such as open-ended questions or development tasks, multiple-choice questions offer the possibility of immediate feedback, automated grading, and easy integration into large-scale digital learning platforms. Furthermore, the automation of multiple-choice question generation addresses a critical practical issue in the educational process, namely the increased workload of teachers related to the design of quality assessment tools. At the same time, the widespread use of MCQs in formal and distance education, combined with their structured nature, makes them particularly suitable for applications of artificial intelligence and machine learning. Therefore, the choice of this specific category of tools is not intended to exclude other forms of automatic assessment, but reflects the importance, widespread application, and high practical utility of multiple-choice questions in modern educational systems.
The existing literature on automated question generation and generative AI in education addresses a wide range of topics, including key-concept extraction, question and distractor generation, as well as broader issues related to assessment design and evaluation, often employing machine learning and neural network methodologies [4,5,6,7,8]. Despite the growing volume of publications, this body of research remains fragmented, with a substantial heterogeneity in datasets, modeling approaches, and evaluation metrics [7]. In recent years, several systematic and bibliometric reviews have examined specific aspects of the broader research landscape related to generative AI-driven educational assessment, within which Automated Multiple-Choice Question Generation is typically addressed as part of wider analytical frameworks rather than as an isolated research focus. For example, Al Faraby et al. [9] provide an extensive systematic review of neural question generation for educational purposes, while Awalurahman and Budi [10] focus on the automatic generation of distractors, identifying prevailing models and their limitations. Other studies, such as Artsi et al. [8], investigate the use of large language models (LLMs), including GPT and BERT, for examination question generation in professional domains such as medicine, highlighting both their potential and associated ethical challenges. Similarly, the bibliometric review by Ahadi et al. [11] analyzes the contribution of text mining techniques to education more broadly, documenting the rapid growth of NLP-related research without explicitly focusing on multiple-choice question generation as a distinct analytical focus.
Although the above studies offer important insights into individual aspects of automated question generation and generative AI in education, there is no comprehensive bibliometric mapping that systematically examines how Automated Multiple-Choice Question Generation research is positioned and evolves within the broader landscape of NLP- and LLM-based educational assessment. Unlike the existing systematic and narrative reviews, which primarily synthesize algorithmic approaches, model architectures, or application cases of automatic question generation [2,3,9,10], the present study adopts a large-scale bibliometric perspective to systematically map the scientific structure and evolution of Automated Multiple-Choice Question Generation research. To the best of our knowledge, no prior study has conducted a comprehensive quantitative bibliometric analysis that situates MCQ generation explicitly within the rapidly expanding landscape of LLM-driven educational assessment. By combining performance analysis with science mapping techniques (co-authorship, bibliographic coupling, and co-word analysis), this review goes beyond descriptive synthesis to reveal publication dynamics, collaboration patterns, dominant thematic clusters, and emerging research directions. In doing so, it offers a macro-level, evidence-based perspective that complements the existing technical reviews and provides a clearer understanding of how this research field has developed, diversified, and positioned itself within the broader ecosystem of artificial intelligence in education. This research gap necessitates a systematic mapping of publication trends, thematic areas, research collaborations, and emerging directions that shape this evolving research landscape. This study adopts the methodology of bibliometric analysis (BA). Through BA, it becomes possible to map the evolution of the knowledge field, identify the most important research components, and reveal thematic correlations and research trends. The study utilizes performance analysis and science mapping techniques, which allow for the visualization and quantitative analysis of publications, citations, authorship collaborations, and thematic foci [12,13]. Specifically, the analysis aims to answer the following research questions (RQ) within the context of LLM-driven educational and assessment research: (RQ1) What are the publication and citation trends in the field under consideration? (RQ2) What are the most important countries (a), scientific sources (b), researchers (c), institutions (d), and articles (e) contributing to the development of the field? (RQ3) What is the level of collaboration among authors in the relevant literature? (RQ4) What are the dominant thematic areas and how have they evolved over time? (RQ5) What are the future directions of research and what research gaps are emerging?
Therefore, following established practices [14,15], we collected data from the Elsevier Scopus database and selected for evaluation relevant studies published in quality journals. We then collected and analyzed bibliometric data, offering a comprehensive mapping of scientific production in the field.
In contrast to the existing systematic and narrative reviews that examine automatic question generation mainly at the level of algorithmic approaches, implementation techniques, and pedagogical applications, the present study adopts a bibliometric approach with the aim of mapping the overall scientific structure of the field. Specifically, through the analysis of publication trends, citation patterns, collaboration networks, and bibliographic coupling, the research captures the evolution, dynamics, and dominant research trends at a macro level.
The contribution of this research is multidimensional. First, to the authors’ knowledge, no other study has attempted a bibliometric analysis by offering a systematic mapping of the research landscape in this field and highlighting its historical and thematic developments. Second, it provides a comprehensive overview of the knowledge structure of the field, identifying the leading research elements, the most productive countries, and thematic convergences. Third, it identifies unexplored areas and suggests future research directions, facilitating the further academic development and practical application of automatic question generation using NLP.
Finally, the paper is organized as follows: Section 2 provides an in-depth literature review discussing prior research on the topic; Section 3 describes in detail the methodology of the BA; Section 4 presents the results of the performance and science mapping analyses; and Section 5 summarizes the main conclusions and discusses the limitations of the study.
2. Literature Review
Research on Automatic Multiple-Choice Question Generation (MCQG) has experienced remarkable development in the last decade, with an emphasis on the utilization of natural language processing (NLP) techniques and, more recently, large language models (LLMs). The first reviews [16,17,18] focused mainly on early methods of extracting questions from texts, offering a general picture of the landscape up to 2014. Later, Ch and Saha [19], as well as Papasalouros and Chatzigiannakou [20], expanded on semi-automatic question generation, with an emphasis on the use of semantic patterns and concept extraction techniques, without, however, quantitatively examining the evolution of the field or the thematic associations of the publications. Kurdi et al. [2] collected 93 studies published between 2015 and 2019, documenting the progress in automatic question generation techniques and their educational application forms. Although their contribution is substantial, this study is limited in time and subject matter, as it does not include the latest developments related to the use of transformers and LLMs, nor does it provide a bibliometric analysis for mapping the intellectual structure of the field.
In recent years (2019–2025), the literature shows a remarkable increase in both research interest and the sophistication of techniques. Contemporary studies, such as those by Bitew et al. [21], Ghanem and Fyshe [22], and Feng et al. [23], have utilized deep neural networks and transformers to optimize distractor generation, while others, such as those by Yu et al. [24] and Cavusoglu et al. [25], have explored the use of pre-training models with knowledge graph integration and retrieval-augmented generation.
In addition to the above, additional reviews and surveys have appeared that focus on individual aspects—for example, Pan et al. [26] and Guo, Liao et al. [27] present a summary of neural approaches to question generation (NQG), while Mulla and Gharpure [28] offer a review with a methodological emphasis (datasets, metrics, applications). Another important development is the emergence of systematic reviews for the sub-field of distractor generation (e.g., [10,29]), which highlight the centrality of the issue of the quality of misleading options.
In parallel, Awalurahman and Budi [10] conducted a systematic literature review exclusively on distractor generation, highlighting the need for unified evaluative criteria and comparative studies. In addition, Lin et al. [30] and Bitew et al. [21] applied fine-tuning and predictive prompting approaches, respectively, to achieve more realistic and pedagogically reliable questions. The recent study by Mucciaccia et al. [31] is one of the first applications of LLMs for the automatic generation of university questions, focusing on the assessment of the quality and validity of the produced results.
Examples of bibliometric/systematic studies in the broader field of NLP application in education that support this observation include work on text mining NLP in education [11], a bibliometric study on automated assessments in programming [32], and a broader review of NLP AI in education [1]. These works show that there are valuable bibliometric approaches in branches of educational technology, but they do not specifically focus on a comprehensive bibliometric mapping of automated MCQ generation as a single field.
Of notable interest is also the research in the article by Dong et al. [33], which proposes an advanced multi-level system for generating diverse and pedagogically useful answer choices. Overall, these new approaches demonstrate the transition of the field from traditional NLP techniques to deeply modeled architectures, which offer greater accuracy and generalizability.
Furthermore, several bibliometric and systematic reviews [34,35,36,37,38] demonstrate that research on automatic question generation (AQG) has grown significantly in recent years, particularly with the incorporation of large language models (LLMs) and technical transformers. However, most studies focus on technical or algorithmic aspects, without comprehensively examining the cognitive, pedagogical, and methodological dimensions of the phenomenon [39,40,41]. Although Ng and Ho [42] and Guo et al. [43] attempt to map the contribution of artificial intelligence to educational evaluation, none of the existing bibliometric analyses focuses exclusively on the NLP–MCQ generation–education interface, as this study seeks to do.
However, despite the rapid increase in publications and the variety of methodologies, a comprehensive bibliometric approach that captures the structure, dynamics, and evolution of research in the field of automatic MCQ generation is lacking. In contrast to previous systematic reviews that methodically document techniques, datasets, and evaluations [2,44], or to specialized reviews on distractors [10], our work offers the first systematic bibliometric mapping that combines quantitative performance analyses (TP, TC, h-index, etc.) with mappings of science (co-authorship, co-citation, co-word analysis). Furthermore, the study updates and extends the time window (2018–2025) and applies strict quality criteria (AJG) to deliver a reproducible, interdisciplinary mapping of trends, collaboration networks, and emerging thematic axes in the field of automatic MCQ generation with NLP techniques. In this way, the study contributes both to theoretical understanding and to the practical routing of future research directions (e.g., evaluation of LLM-generated items, bias and fairness, applications in education).
3. Methodology and Data
This study presents a systematic review and bibliometric analysis conducted in accordance with the PRISMA 2020 guidelines (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) [45]. The review was not prospectively registered in a protocol registry (e.g., PROSPERO). BA was employed as the primary methodological approach, given its significant advancement in recent years as a robust quantitative tool for organizing, analyzing, and mapping extensive bodies of scientific literature [46,47].
Its utility lies in the ability to process extensive, often unstructured data in a systematic and methodologically rigorous manner, offering a high level of objectivity and reproducibility in research findings [48]. BA is particularly useful in rapidly evolving fields where the systematic exploration of emerging thematic directions and the revelation of the intellectual structure of the scientific field are required [49,50].
In this context, BA leverages two main pillars: performance analysis and science mapping. These techniques are well-established approaches in the international literature [12] and have been widely used in recent studies to understand the evolution of scientific fields, such as artificial intelligence, computational linguistics, and educational technology [51,52,53].
Performance analysis is the foundation of BA and examines the contribution of key research components—such as authors, institutions, countries, scientific journals, and subject areas—using criteria such as the number of publications (TP), total citations (TC), and the Hirsch index (h-index) [54,55]. These measures are used to capture productivity and scientific impact, providing an overall picture of the evolution of the research field [56].
In addition, science mapping explores the intellectual and thematic structure of the field. In the context of this study, three main techniques were used: (i) Bibliographic coupling, which is based on the logic that two articles that share common references show thematic and intellectual affinity [57]. This technique aims to identify thematic groups and evolutionary patterns [58]. (ii) Co-authorship analysis, which visualizes the networks of collaboration between authors and organizations [59] and highlights the degree of international collaboration [60]. (iii) Co-word analysis, which examines the associations between terms that appear simultaneously and reveals the conceptual structure and future directions of the field [48].
The three techniques were applied to answer the corresponding research questions: co-authorship analysis for the third question, bibliographic coupling for the fourth, and co-word analysis for the fifth. To perform these analyses, the software Bibliometrix [12] and VOSviewer [13] were used, which are leading platforms for bibliometric analyses and have been adopted in a multitude of recent studies [61,62,63]. The data collection process followed a rigorous, reproducible, and transparent methodological approach, aligned with the PRISMA guidelines [45] and previous relevant studies [64,65,66].
Several established scholarly databases were evaluated for data collection purposes, namely Scopus, Web of Science (WoS), and Google Scholar. Scopus was selected based on its stringent quality control standards [67], its broad interdisciplinary scope, and its extensive use in bibliometric and science mapping studies [68,69]. In comparison with WoS, Scopus demonstrates a wider disciplinary coverage, particularly in the fields of computer science, engineering, and social sciences, which are comparatively less represented in WoS, thereby supporting a more inclusive and multidisciplinary BA [70]. Moreover, Scopus indexes approximately 60% more journals than WoS, further strengthening the depth and completeness of the dataset [71]. Google Scholar was not considered suitable due to limitations related to indexing transparency, the inclusion of discontinued journals, and the high proportion of non-English publications, all of which may affect data reliability and reproducibility in bibliometric research [72]. Overall, Scopus constitutes a robust and comprehensive source for bibliometric inquiry, offering access to over 40,000 peer-reviewed publications across diverse subject areas, along with structured and reliable metadata that support advanced BA and visualization techniques [73,74].
The search was performed using combinations of keywords for the title, abstract, and keywords (TITLE-ABS-KEY), aiming to retrieve as many relevant publications in the field as possible. The final query that was defined and employed was as follows: (“large language model” OR “LLM” OR “GPT” OR “transformer-based model” OR “retrieval-augmented generation” OR RAG OR “fusion-in-decoder”) AND (“chatbot” OR “AI agent” OR “conversational agent” OR “intelligent assistant” OR “virtual assistant” OR “autonomous agent”) AND (“education” OR “educational technology” OR “school administration” OR “teacher support” OR “academic management” OR “payroll” OR “HR system” OR “administrative process” OR “institutional management” OR “database assistant” OR “SQL agent” OR “human evaluation” OR “grounded”). The selection of keywords and data collection time frame was based on the need to capture the most recent and substantial developments in the field of large language models (LLMs), AI agents, and their applications in educational technology. The period 2022–2025 was purposefully chosen, as it represents the phase of explosive development and scientific maturity of LLM technologies and related tools such as GPT and RAG (retrieval-augmented generation). During this period, an exponential increase in publications, research projects, and practical applications of these models in educational settings was observed [75,76].
The selection of specific search terms (e.g., LLM, GPT, transformer-based model, RAG, chatbot, AI agent, intelligent assistant, virtual assistant) was based on two main criteria: (i) scientific accuracy and representativeness, as these terms are the prevailing research labels in the field of AI and education [77]; and (ii) ensuring thematic coverage, so as to include both technological approaches (such as transformer models and NLP techniques), as well as applied pedagogical and administrative uses of AI agents in school and university environments. In addition, the terms “education”, “educational technology”, “school management”, and “administrative processes” were incorporated into the search query to ensure that the results focus on studies that explore the interconnection of LLMs with the educational and organizational sector [78]. This strategy is in line with modern approaches to BA, where keywords are determined based on thematic relevance and the tendency to appear in top journals [79,80]. The period 2022–2025 is considered ideal for this study, as it represents the post-ChatGPT era, where LLMs began to be actively integrated into education platforms, learning support systems, and autonomous administrative systems [81]. Therefore, the choice of this temporal focus allows for the capture of the most modern scientific and technological developments, offering an updated and realistic framework for mapping the cognitive field.
4. Results
4.1. PRISMA Flow Diagram and Study Selection
Figure 1 illustrates the stepwise process of identification, screening, eligibility assessment, and inclusion of publications, in accordance with PRISMA guidelines. The initial search covering the period 2022–2025 (November) yielded 1090 records. Sequential filtering steps were applied to ensure the validity, thematic relevance, and methodological quality of the dataset. Health-related studies were not excluded a priori during the screening process. However, publications focusing exclusively on medical or clinical education, without a direct emphasis on Automated Multiple-Choice Question Generation or related assessment methodologies, were excluded in order to preserve thematic coherence and methodological consistency with the objectives of this bibliometric analysis. Specifically, 40 publications without a defined author were removed, resulting in 1026 papers. The sample was then limited to publications in the fields of computer science, engineering, and social sciences, reducing the number to 725 articles. Subsequently, only journal articles were retained, excluding conference proceedings, book chapters, and other document types, leaving 245 publications. Finally, five non-English publications were excluded, resulting in a final sample of 240 articles for BA. Although concepts such as learning effectiveness, item analysis, and adaptive testing are highly relevant to multiple-choice question design, they appear only sporadically in the retrieved literature. Their limited and fragmented presence in studies focusing on automated MCQ generation indicates an important research gap, particularly with regard to the pedagogical evaluation and learning impact of automatically generated assessment items.
Figure 1.
PRISMA flowchart diagram.
To enhance validity and reduce bias, the relevance assessment was independently performed by three researchers, with a fourth acting as a moderator. Disagreements were resolved through discussion until consensus was reached, following best practices in scientific methodology [82,83]. This rigorous process ensures that the selected dataset is representative of the current research landscape and suitable for systematic bibliometric analysis.
After completing the filtering process, it was observed that the time period of the selected papers finally extended from 2023 to November 2025. This reduction compared to the initial period (2022–2025) is due to the fact that a large part of the 2022 publications did not meet the thematic relevance or quality classification criteria according to the Academic Journal Guide (AJG, 2021), as they focused on traditional or rule-based NLP approaches. Moreover, the shift in the analysis period reflects the rapid evolution of the field after 2023, a period during which there was an explosive increase in interest and research output around the use of LLMs, such as GPT-4 and Claude, in educational technology and automated question generation applications [9,27,29]. In addition, the tightening of selection filters, aimed at ensuring qualitatively reliable and thematically relevant works, contributed to the natural shrinkage of the time range, without, however, affecting the validity or representativeness of the sample [47,48].
The choice to focus on the most recent publications enhances the timeliness and relevance of the analysis, as it reflects the latest technological and scientific developments in the field of automatic question generation and NLP systems in education. A similar methodological approach has been adopted in modern bibliometric studies, which emphasize emerging research trends and thematic clusters [58,84]. Consequently, the selection of the final sample enhances thematic homogeneity, ensures the reproducibility of the results, and provides a realistic depiction of the current research reality in the field.
4.2. Descriptive Analysis Results
The basic information about the research is deployed in this sub-section (Table 1). The bibliometric analysis carried out for the period 2023–2025 includes 240 publications from 149 scientific journals, which demonstrates intense research activity in the field of automatic MCQ generation and, more generally, in the applications of artificial intelligence in education. The impressive annual growth rate of publications, reaching 97.71%, confirms that the field is in a phase of rapid development, mainly due to the rapid progress in LLMs and in neural approaches to natural language processing [27].
Table 1.
Main information about data.
The articles of this period have a very short average age (0.6 years) and a high average number of citations (42.5 per article), which reflects both the modern and scientifically active nature of the field. The increased impact of these studies reflects the broadening interest of the research community in the use of artificial intelligence for automatic question generation, the support for educational assessment, and the development of intelligent learning systems [2].
Regarding authorship, the 1405 authors participating in the writing of the 240 articles demonstrate the highly collaborative nature of the research. No single-author works were identified, while the average number of co-authors per publication (11.7) reveals a high level of interdisciplinary collaboration, with international contributions in 25% of the works. This trend is characteristic of new, rapidly developing research fields, where knowledge is developed through synergies and exchanges between different scientific areas [48].
Regarding thematic content, the 923 plus keywords and 880 author keywords highlight the wide variety of topics covered by the publications. The most frequent terms are associated with concepts such as neural networks, deep learning, automatic content generation, adaptive learning, and NLP-based question generation. This thematic diversity indicates the gradual transition of research from traditional normative questioning methods to advanced approaches that utilize machine learning techniques and transformers.
The reduction in the time frame of the analysis (2023–2025 versus 2022–2025 initially) is explained by the rapid concentration of publications in the last two years, where technological progress in models such as GPT and T5 [85] led to increased interest and the production of new knowledge. This trend is consistent with recent bibliometric studies in related fields of educational technology, where a shift in research interest towards applications of LLMs is observed [86,87].
Overall, the results highlight a dynamic, rapidly evolving, and highly collaborative research ecosystem, which is consolidating its position in the international scientific scene. The present study, by mapping this new phase of development (2023–2025), provides a holistic picture of the course of the field and contributes substantially to the understanding of contemporary research trends, thematic correlations, and future prospects in the field of automatic question generation through NLP and artificial intelligence technologies in education.
4.3. Performance Analysis Results
This subsection focuses on the performance analysis, which examines the overall scientific productivity and influence of the field through key bibliometric indicators. This analysis aims to capture both the quantitative and qualitative dimensions of research activity, highlighting the way in which the relevant research has evolved over time. To this end, five main evaluation criteria were applied: (RQ1a) The temporal evolution of publications, which captures the dynamic development and level of maturity of the research field. (RQ2b) The geographical distribution of research production, in order to identify the regions or countries with the greatest contribution to the bibliography. (RQ3c) The most active and influential authors, as well as their international or institutional collaborations, which reveal the networks of scientific partnership. (RQ4d) The most productive publication sources, i.e., the scientific journals and conferences that shape the research field. (RQ5e) The most cited articles, which constitute the theoretical and methodological pillars of the specific field of knowledge.
In general, the above criteria allow for the formation of a holistic picture of research activity, providing essential information on the structure, key factors, and trends in the diffusion of knowledge in the field of study.
4.3.1. Number of Publications per Year (RQ1)
From Table 2, it is observed that there is a significant variation in the influence of publications by year, based on citations. In 2023, the 33 publications show a high average number of citations per article (257.36) and average number of citations per year (85.79), suggesting that these articles are particularly influential and likely include fundamental or early works that serve as a reference for subsequent research.
Table 2.
Annual scientific production and citation.
For 2024, although the number of articles increases (78), the average number of citations per article decreases significantly (18.78), suggesting that these are mainly newer publications with a limited time to accumulate citations. Accordingly, the average number of citations per year is much lower (9.39). The year 2025 presents the highest number of articles (129), but the average number of citations per article is very low (1.87), reflecting the newness of these publications and the limited time period to accumulate citations.
In conclusion, the trends show that early publications in the field have greater influence and constitute the foundation for newer research, while more recent publications, despite their increased volume (Figure 2), have not yet acquired a significant number of citations, something expected in bibliometric analyses [47,88].
Figure 2.
Publication trends. Source: By authors.
4.3.2. Country-Wise Analysis (RQ2a)
The analysis of the data (Table 3) shows that there is a strong variation between countries in terms of the volume and quality of scientific production. The United States stands out for the largest number of publications, as it accumulates 61 articles and 5836 citations in total. This is followed by China, with 27 articles, Germany, with 19, and the United Kingdom, with 16. These countries are the main producers of scientific work in the field, although the average impact per article (C/A) is not always particularly high. For example, China has a low impact index (9.56), which indicates that, despite the increased production, its publications do not have the same research impact as other countries.
Table 3.
Top 20 contributing countries.
In contrast, some countries with a smaller number of articles have an extremely high average impact. For example, New Zealand, with just four articles, records the highest C/A index (703.75), which indicates that its publications have a very high impact on the scientific community. Correspondingly, Switzerland (C/A 553), Norway (475.67), Italy (456.50), Saudi Arabia (455.50), and Taiwan (406.57) have particularly high impact values, which indicates focused but high-quality research.
In an intermediate category are countries such as Singapore (381.50), Australia (255.43), India (259.17), Spain (240), and Hong Kong (276.55), which combine a satisfactory number of publications with a relatively high impact. These cases show a balance between the quantity and quality of scientific production. In contrast, countries such as Turkey (16.60), Canada (19.38), and Portugal (7.60) show low impact indices, which can be attributed either to the low international visibility of their publications or to their lower participation in international collaboration networks.
Overall, it appears that the volume of publications does not necessarily imply a high scientific influence. Large countries, such as the United States and China, dominate in terms of quantity, while smaller countries, such as New Zealand and Switzerland, are distinguished by the quality and visibility of their research. This picture reveals a field with a strong imbalance, where a few countries produce the largest volume of research, but the highest quality is found in more specialized and research-mature environments.
4.3.3. Most Contributing Journals (RQ2b)
The analysis of publication sources shows that the research field of automated MCQ generation is attracting the interest of a variety of scientific journals active at the intersection of educational technology, artificial intelligence, and medical education (Table 4). At the top of the list is the journal JMIR Medical Education with 24 publications, which indicates the intense research activity around the application of automated assessment methods in medical and clinical education. The high number of publications is also accompanied by a significant number of references (1742 citations), an element that confirms the influence of the journal in this scientific field.
Table 4.
Contributing journals.
Next are Education and Information Technologies (eight articles) and Computers and Education: Artificial Intelligence (seven articles), which function as the pre-eminent international forum for research combining natural language processing (NLP), machine learning, and pedagogical assessment techniques. Also, the presence of journals such as the British Journal of Educational Technology and Frontiers in Education demonstrates the interdisciplinary nature of the field, as they cover topics ranging from learning technology to psychopedagogical assessment.
IEEE Access deserves a special mention, with five publications and a strong technological focus, indicating an involvement of the engineering research community in the development of algorithmic approaches for the automatic production of educational content. The presence of the journals Electronics (Switzerland) and Frontiers in Artificial Intelligence further strengthens the connection of the field with technological and computational sciences.
In terms of the impact of publications, the journal with the most citations is the International Journal of Information Management with 2712 citations, which shows that the relevant works touch on issues of information management, data, and artificial intelligence in organizational or educational contexts. JMIR Medical Education itself, although leading in the number of articles, also ranks very high in citations, which highlights its double influence: both in terms of production volume and scientific awareness. The position of the Journal of Applied Learning and Teaching (1696 citations) is also noteworthy, which shows that the topic of automatic question generation is closely linked to the practical dimension of teaching and educational design. At the same time, the presence of journals such as Education and Information Technologies, Journal of University Teaching and Learning Practice, and European Journal of Engineering Education proves that the research impact of the field is not limited only to technical approaches, but also extends to pedagogical and institutional dimensions.
In summary, the multidimensional nature of the field is reflected in the combination of journals from the fields of education, technology, engineering, and medicine. JMIR Medical Education and the International Journal of Information Management emerge as key journals in both production and citations, acting as research hubs. The strong presence of journals such as Computers and Education: Artificial Intelligence and IEEE Access indicates a shift in research interest towards more automated, technology-oriented solutions. The increasing number of articles in educational technology journals indicates that artificial intelligence in educational assessment is evolving into an emerging scientific field with a broad interdisciplinary reach.
4.3.4. Author-Wise Analysis (RQ2c)
Table 5 captures the ten most productive and influential researchers in the emerging scientific field of artificial intelligence and, in particular, large language models (LLMs), with applications in education, assessment, and cognition. The data demonstrate not only the geographical diversity of the researchers, but also the interdisciplinary nature of the field, where specialties from education, social sciences, computer science, and machine learning converge.
Table 5.
Contributing authors.
At the top of the ranking based on the influence index (citations and h-index) is Rudolph J. from the Kaplan Higher Education Academy (Singapore), with 1696 citations and an h-index of 15, which underlines his systematic contribution to the study of the impacts of genetic artificial intelligence in higher education. Rudolph is one of the most recognized figures in the discussion around the transformation of assessment and learning in the light of AI, as he is the author of some of the most cited articles in the field (e.g., “ChatGPT: Bullshit spewer or the end of traditional assessments in higher education?”).
Tan S., also from Singapore (Institute for Adult Learning), holds a significant position, with 1696 citations, confirming that the Southeast Asian region has emerged as a key research center for investigating the relationship between AI and lifelong learning. The high concentration of citations for researchers from Singapore reflects the country’s strong investment in innovation and educational technology.
Crawford J. (Tasmanian School of Business and Economics, Australia) follows with 515 citations and an h-index of 24, demonstrating his interdisciplinary contribution to the ethical dimension of artificial intelligence and the management of education through digital transformations. Similarly, Allen K.A. (Monash University, Australia), with 483 citations and an h-index of 37, emerges as a key representative of the psychopedagogical approach to artificial intelligence, examining the emotional well-being and adaptability of students in the new era of learning.
In the United States, Demir I. and Sermet Y. (Tulane University), as well as Cwiertny D. (University of Iowa), all with 226 citations, stand out, focusing on the use of LLMs for STEM education and skill assessments. Their presence indicates the increasing research activity of American universities, especially in projects that combine machine learning and pedagogical technology. On the other hand, Manrique R. (FLAGlab, Colombia) and Mondal H. (All India Institute of Medical Sciences, India) contribute with lower impact metrics (h-index 8 and 13); however, their presence demonstrates the expansion of research interest beyond the Western context towards emerging regions, with an increasing activity in educational technology. Finally, Caccavale F. (Technical University of Denmark), with a smaller research output (2 articles, 21 citations), reveals the integration of the field in technological institutions in Europe, which demonstrates the European interest in the interdisciplinary study of AI in education.
From the above, it is found that there is a high level of international collaboration, with researchers from Asia, Oceania, Europe, and the Americas. In addition, a concentration of influence is observed in a few leading authors, mainly from Singapore and Australia. Also, a shift in scientific interest is observed from the technical applications of artificial intelligence (e.g., machine learning) to its pedagogical, ethical, and social dimensions. This distribution reflects a new phase of maturation of the field, where the emphasis is shifting from technological development to the human dimension of education in artificial intelligence.
4.3.5. Contributing Institutions (RQ2d)
The analysis of the leading institutions active in research in the field of Automated Multiple-Choice Question Generation reveals interesting findings both in terms of geographical distribution and level of research impact (Table 6). It is observed that the primacy belongs to institutions in the United States, with the University of Iowa accumulating three publications, confirming its constant presence in issues linking artificial intelligence, education, and automated assessment. Correspondingly, the College of Engineering and the College of Liberal Arts and Sciences (also from the USA) show the same number of publications (three), which indicates a balanced research activity between technological and humanities scientific fields. Their presence confirms that research in the field of automated question generation requires an interdisciplinary approach—combining algorithmic techniques with pedagogical theories and linguistic analysis.
Table 6.
Top ten contributing institutions.
At the international level, the participation of Asian institutions is also notable. The Department of Physiology, Ministry of Health and Family Welfare, India (India), records three publications and h-index = 4, which can be interpreted as a result of the increasing application of natural language processing (NLP) and computational assessment technologies in medical education and training. Similarly, Hannam University, South Korea (with two publications and h-index = 4), shows that the East Asian region is starting to gain research momentum in the subject, especially in the context of English language teaching through intelligent assessment systems.
In terms of citations, we observe that the institutions with the highest impact are Indiana University Bloomington (595 citations), the Director of Research, Singapore (510), and the University of Tasmania (440). This high visibility indicates not only productivity but also a significant influence in the research communities dealing with assessment automation, natural language processing, and educational technology. Also, of interest is the presence of Australian universities (Monash University and CQU University), which seem to be shaping an emerging research stream in the area of computational education.
On the whole, the data suggest that this field is interdisciplinary and globalized, with contributions from universities on different continents. The United States continues to maintain a strong leadership position in the production and dissemination of knowledge. Asia (India, Korea, Singapore) is emerging as a rapidly growing research power, mainly through the application of artificial intelligence technologies in teaching and assessment. Australian university structures occupy significant positions in the citations, reflecting mature networks of international collaboration and research impact.
4.3.6. The Most Cited Global Publications (RQ2e)
Table 7 presents the ten most influential articles in the field of artificial intelligence research, and, in particular, large language models (LLMs), such as ChatGPT, with applications in education, assessment, and society. The data clearly highlight the rapid rise and intense research activity that occurred in 2023 and 2024, years that marked the explosion of scientific engagement with genetic models.
Table 7.
Top cited articles.
The high citation impact of the articles listed in Table 7 can be attributed to their foundational role in shaping the early discourse on large language models in education and assessment. Most of these studies were published in 2023, a pivotal year that marked the rapid diffusion of generative AI tools such as ChatGPT into educational practice. Their multidisciplinary scope, addressing assessment integrity, ethical implications, professional education, and policy considerations, positioned them as reference points for subsequent empirical and conceptual research. As a result, these publications function not only as highly cited works, but also as intellectual anchors that structure the development of the field.
The article by Dwivedi, Kshetri et al. [89], with 2712 total citations and 904 per year, occupies the first place by far, demonstrating its multidisciplinary nature and its importance in setting the research and policy agenda for generative AI systems. This work serves as a reference foundation for the study of the social, ethical, and political implications of generative AI. Next is the study by Gilson et al. [90] with 1357 citations, which examines the relationship of ChatGPT to medical education and its ability to answer professional certification exams (USMLE). This article is a landmark, as it introduced the dialog on the use of LLMs as cognitive learning and assessment tools.
The two studies by Rudolph et al. [91,92], with 1154 and 510 citations, respectively, analyze the impact of LLMs on higher education, focusing on the integrity of assessment and the change in traditional teaching formats. The repeated presence of the same author highlights the consistency and impact of his research contribution. The study by Jeon and Lee [93] (441 citations) examines the complementary relationship between humans and machines in teaching, laying the foundation for pedagogical hybrid learning models, while Crawford et al. [94] highlight the importance of ethical leadership and the responsible use of artificial intelligence in education, showing the shift in interest to ethical issues.
In the health sector, Javaid et al. [95] analyze the potential of ChatGPT in health services, opening new perspectives in diagnostic support and in the education of health professionals. At the same time, Nikolic et al. [96] focus on assessing the validity of AI tools in engineering education, proposing control frameworks to ensure academic integrity. Barrett & Pack [97] examine student and teacher perceptions of the use of generative AI in writing, highlighting issues of critical thinking, authenticity, and creativity. Finally, Sajja et al. [98] is one of the most recent, introducing AI-enabled personalized and adaptive learning, a framework for personalized education that integrates LLMs into adaptive learning systems.
All in all, the analysis shows that the most influential papers focus on three main thematic areas: ethical, social, and pedagogical dimensions of LLMs [89,91,94], applications in education and assessment [90,93,96], and innovative uses in professional fields, such as health [95] and personalized learning [98].
The high concentration of citations in articles from 2023 indicates that this year was a milestone for the foundation of the scientific discussion around LLMs, while the gradual decrease in citations in later papers (2024) reflects the phenomenon of a delayed diffusion of new publications in the literature [12,48].
It should be noted that the identified thematic clusters are not mutually exclusive. Several influential studies span multiple dimensions simultaneously, combining ethical considerations, pedagogical implications, and domain-specific applications. For example, research addressing assessment integrity often intersects both ethical and pedagogical perspectives, while studies conducted in professional domains such as medicine also contribute to broader discussions on educational assessment. This thematic overlap reflects the inherently interdisciplinary and evolving nature of AI-driven research in education.
4.4. Science Mapping
4.4.1. Co-Authorship Analysis of Authors (RQ3)
Co-author analysis provides a comprehensive picture of the collaborative landscape that has emerged in the field of Automated Multiple-Choice Question Generation. The network visualization (Figure 3) captures the dynamic relationships between researchers, highlighting the axes of collaboration and research clusters that have developed in recent years.
Figure 3.
Co-authorship network and cluster density of the author. Labels were adjusted to improve readability. Note(s): Each node represents an author, with node size proportional to the author’s total link strength (TLS), indicating the extent of collaboration. Links between nodes represent co-authorship relations, with thicker links denoting stronger collaborative ties. Colors indicate distinct collaboration clusters identified by VOSviewer. The visualization highlights the collaborative structure and research groupings within the field. Source(s): Figure by authors.
To enhance interpretability, node size and link thickness were adjusted to reflect collaboration intensity, following established visualization practices in bibliometric mapping. Specifically, each node corresponds to an author, with the size of the node or name reflecting its academic productivity or influence, while the color separation indicates the formation of distinct research groups. To capture and analyze this collaborative structure, the VOSviewer tool was used, which is an established choice for mapping bibliometric networks [13]. The analysis revealed a network of 239 authors, who are connected through 660 collaborative ties, with a total connection strength (TLS = 726). The total number of authors was divided into 16 groups, indicating a fragmented but active and interacting research ecosystem. The thickness of the lines connecting the nodes illustrates the degree of collaboration: the thicker the line, the closer and more frequent the intellectual interactions between the researchers involved. The existence of many active links in different subsets of the network highlights the multidimensional and interdisciplinary nature of the field, where artificial intelligence, educational technology, computational linguistics, and data science meet and feed off each other.
Overall, the analysis demonstrates that the level of collaboration in this research field is satisfactory and is steadily growing. The structure of the network indicates not only the existence of central, highly productive authors, but also the presence of many smaller groups that contribute to the continuous evolution and expansion of the literature.
4.4.2. Bibliographic Coupling of Publications (RQ4)
To capture the research structure and thematic directions of a scientific field, the bibliographic coupling method is a particularly suitable tool. This approach is based on the idea that two publications are conceptually related when they draw references from a common bibliographic base, thus forming a “conjugation pair” [47,57]. Through this logic, publications are grouped into clusters that reflect underlying intellectual and thematic affinities.
In the context of the present study, the application of bibliographic coupling to the 240 articles in the sample revealed that 153 of them presented strong connections in terms of their common references. The bibliographic coupling analysis revealed seven distinct thematic areas that capture the structure and research trends in the field of automated MCQs and the use of LLMs in assessment. These themes emerged from the grouping of related publications that share common bibliographic foundations and compose a coherent picture of the field. Table 8 depicts the main focus of each group, accompanied by the most representative publications.
Table 8.
Bibliographic coupling.
Acknowledging that the labeling of clusters always involves an element of interpretative judgment, each group was examined in depth through a close reading of its constituent studies and their citation patterns. The thematic descriptors assigned to each cluster therefore do not merely serve as categorical labels; they encapsulate recurring conceptual motifs, dominant research priorities, and intellectual commonalities that emerged across the included works. The keywords and titles associated with each cluster support the visualization of broader themes in the field and underscore the contributions of the grouped publications to the cumulative knowledge base.
Cluster 1: “AI-Assisted Assessment and Automated MCQ Generation”. The first and largest cluster focuses directly on Automated Multiple-Choice Question Generation. The focus is on comparative studies on the reliability of MCQ generation from GPT-3.5, GPT-4, LLaMA, as well as applications in specialized fields, such as medical education. Works such as those by K. Hwang et al. [99], Ling and Afzaal [100], and Workum et al. [101] highlight issues of quality, alignment with Bloom’s cognitive levels, and the need for human editing. In parallel, some studies [102,103] focus on the comparison between human and automated MCQ generation, demonstrating that LLMs can achieve a comparable quality on well-structured topics. Roumeliotis et al.’s [104] research argues that cryptocurrencies are becoming increasingly popular, but investment sentiment is strongly influenced by news, creating opportunities but also significant risks. The study examines how advanced language models, such as GPT-4, BERT, and FinBERT, can analyze news sentiment about cryptocurrencies, enhancing market understanding and investment decisions. Hamid and Brohi’s [105] work shows that, although LLMs can automate critical processes in the healthcare sector, the existence of vulnerabilities makes it necessary to address serious risks such as data interception and corruption. The research proposes an application taxonomy, a threat model, and a framework for the secure implementation of LLMs in healthcare, filling gaps that have not been addressed in the literature so far.
Cluster 2: “AI in Education Systems, Learning Analytics, and Teacher Support”. The second cluster focuses on the use of LLMs in educational ecosystems, including learning systems, data analytics, and teacher support. The research here shows that LLMs function as digital assistants, capable of analyzing student behavior, generating personalized recommendations, and supporting the development of learning materials, such as mathematical proofs or assessment questions [106]. The discussion of “Education 4.0” [107] is often reoccurring, with researchers arguing that LLMs can extend personalized learning and support teachers in particularly demanding tasks, reducing cognitive load. Moreover, this cluster includes studies that explore how AI fits into the broader education ecosystem: as a digital assistant, as a teacher support system, or as a tool for diagnosing learning patterns. Works such as [85,108,109] highlight cases where LLMs support the creation of learning activities, feedback, and even the analysis of student behavior.
Cluster 3: “Ethical, Responsible, and Philosophical Perspectives of AI in Education”. The third cluster focuses on issues of accountability, transparency, and ethical implications of the use of AI. Publications such as those by [110,111,112,113] analyze the inequalities that may arise from algorithmic processes, the illusion of “personalization” created by LLMs, and the philosophical implications of knowledge production.
Cluster 4: The fourth cluster concerns governance policies and the institutional adoption of artificial intelligence systems. As the introduction of LLMs in universities rapidly progresses, research projects examine how organizations adapt regulations, management practices, and assessment processes to safely and responsibly integrate these systems [114]. The literature highlights the need for clear usage frameworks, quality assurance, data protection, and ongoing staff training, as the adoption of LLMs creates challenges as well as new opportunities. This cluster covers governance and educational policy related to AI systems. Studies such as [115,116] examine academic perceptions of LLMs, the challenges of integrating AI practices into institutional settings, and the emerging regulatory needs.
Cluster 5: “AI Literacy, Technology Adoption, and Public Perception”. In the fifth cluster, issues of technological literacy, technology adoption, and social acceptance of artificial intelligence emerge. Articles in this cluster examine how students and the wider public perceive artificial intelligence tools, what factors influence their intention to use them, and how sociocultural conditions influence the diffusion of these technologies [117]. Research such as that of authors [118,119] shows that the social legitimacy of LLMs in education is closely linked to the level of trust, perceived usefulness, and familiarity with the technology. This cluster brings together studies that examine user attitudes, the diffusion of AI tools, and factors influencing acceptance. For example, Semeler et al. [120] and Kajiwara and Kawabata [121] explore social indicators, digital literacy, subjective utility, and concerns surrounding the use of generative AI.
Cluster 6: “NLP, Natural Language Generation, and Technical Modeling”.
This cluster concerns technical approaches to natural language modeling, dialog generation, and methodologies that enable accurate, coherent, and controlled text production. Works such as [122,123] focus on coherent dialog generation, semantic control, and foundation models for large-scale tasks—which form the technical foundation of automatic MCQ generation.
Cluster 7: “Scientometric and Meta-Analytical Studies of AI in Education”. Finally, the seventh cluster includes scientometric and meta-analytic studies that map the totality of research activity around artificial intelligence in education [124,125,126,127,128,129]. These studies use bibliometric tools to identify research trends, gaps, and shifts in interest in the field. This section is particularly important, as it offers an overall picture of the evolution of the field and contributes to the formulation of strategic directions for future research.
Despite the inherent limitations of bibliographic coupling, such as its dependence on available data and the inability to capture the temporal evolution of knowledge, the results of this analysis have substantial practical value for practitioners and researchers in the field. Specifically, bibliographic coupling allows the identification of thematic clusters and research streams that are actively developing, facilitating an understanding of the structure and dynamics of the research ecosystem. In this way, practitioners can leverage the findings to identify mature and emerging areas of research, monitor key research directions, and support evidence-based decisions at the level of educational design, policy, or technological implementation.
Overall, the bibliographic coupling analysis reveals a rapidly evolving field, where automated MCQ generation is no longer seen as a stand-alone technical capability but as a central element of a broader dialog on reflective learning, artificial intelligence, organizational policies, and social processes. The interdisciplinary nature of the field suggests that future research will need to consider not only technological innovations but also the pedagogical, ethical, and social dimensions associated with the widespread application of LLMs in the educational process.
Beyond the descriptive mapping of publication trends and thematic clusters, the bibliometric patterns identified in this study reflect deeper theoretical and technological transformations in the field of artificial intelligence in education (AIED). The sharp increase in research output after 2023 coincides with the widespread adoption of transformer-based architectures and large language models, which have fundamentally altered approaches to automated content generation and assessment. The dominance of clusters related to AI-assisted assessment and educational support systems indicates a paradigm shift from rule-based NLP methods towards generative, data-driven models capable of supporting formative assessment, adaptive learning, and scalable feedback mechanisms.
Furthermore, the emergence of clusters focusing on ethics, governance, and AI literacy aligns with contemporary theoretical debates in AIED regarding responsible AI, assessment validity, and human–AI collaboration. The relatively limited presence of automated MCQ generation as a standalone theme suggests that this area remains theoretically under-conceptualized, offering opportunities to integrate insights from assessment theory, cognitive taxonomies (e.g., Bloom’s taxonomy), and learning analytics. Overall, the bibliometric findings highlight how technological innovation in AI is increasingly intertwined with pedagogical and theoretical considerations, reinforcing the need for interdisciplinary research that bridges computational advances with educational theory.
4.4.3. Keywords Analysis (RQ5)
The mapping captures the historical evolution of research around Automated Multiple-Choice Question Generation and related technologies. Shades of blue correspond to early publications, while green–yellow highlight more recent studies. The size of the nodes indicates the relative importance of each topic area; for example, AI in education appears as one of the most influential nodes, in contrast to smaller topics such as educational performance.
Strong connecting lines indicate a high level of co-authorship or close thematic correlation. A significant increase in activity is observed in the period 2023–2024, during which terms such as AI, chatbot, ChatGPT, and LLM dominate, reflecting the rapid spread of large language models in educational research. In contrast, smaller nodes—such as Automated Multiple-Choice Question Generation, AI agents, and conversational AI—indicate scientific fields that remain underexplored. The relatively limited literature specifically on automated MCQ generation for educational purposes was one of the main motivations for this study, reinforcing our interest in examining the field in depth and developing a corresponding system.
Figure 4 presents a co-word analysis of the most frequently occurring topics, as they are identified in publications in this scientific field. All keywords that met the frequency criteria were selected, resulting in a fairly extensive network with 7072 links and 8858 TLS. The keywords with the highest frequency and TLS are AI (occ.: 135, TLS: 345), LLM (occ.: 115, TLS: 312), ChatGPT (occ.: 102, TLS: 303), and NLP (occ.: 95, TLS: 295). In addition, related topics such as chatbots, Gemini, machine learning, automated MCQ generation, and educational technology appear. To illustrate the temporal evolution of keywords, an overlay visualization was applied, in which each term is colored based on the average year of its appearance. According to this method, older topics appear in blue shades, while more recent ones appear in green and yellow. Some notable trends emerge from the coexistence network. Early research focused mainly on more general issues, such as artificial intelligence, AI in education, and meaningful learning, supported by LLM-based educational systems that facilitate conceptual understanding, feedback, and adaptive assessment. Over time, new topics began to gain momentum. NLP, machine learning, and automated code generation have been prominent since 2024 and continue to increase in frequency to this day, signaling the broadening and specialization of research interest.
Figure 4.
Co-word analysis (overlay visualization). Labels were adjusted to improve readability. Notes: This figure presents the analysis co-words to illustrate the temporal allocation of authors’ keywords in each thematic group. The colors in the spectrum differ, with blue reflecting the early years and green and yellow the newer ones. Source(s): Figure by authors.
The overall picture of the network highlights that, despite the rapid growth of research around artificial intelligence and large language models, Automated Multiple-Choice Question Generation in education remains a relatively limited and less mature field. This gap creates significant opportunities for new studies, experimentation, and applications, such as the development of question generation systems that can effectively support teachers and students. The ever-expanding research landscape leaves room for deeper analysis, and this study seems to be positioned precisely at this dynamic point.
5. Conclusions
From a theoretical and practical standpoint, the bibliometric findings of this study extend beyond descriptive mapping by explicitly linking observed research patterns to current theoretical and technological developments in artificial intelligence in education. Overall, utilizing bibliometric techniques and approaches from information systems and educational sciences, this study carries out a comprehensive and retrospective assessment of research on automated MCQ generation. Its contribution is structured around three main axes, which substantially strengthen the relevant literature. First, it presents an updated and systematic mapping of the main thematic trends, highlighting their evolution and their place in the broader scientific ecosystem of artificial intelligence in education. Second, it offers an overall assessment of the productivity and scientific impact of the field, highlighting the dynamics that characterize automated MCQ generation. Third, it suggests new, evidence-based directions for future research, with substantial implications for academics, practitioners, and policymakers. The study’s contribution is of particular value to researchers and doctoral candidates, as it offers a coherent and in-depth view of the available body of knowledge, facilitating the identification of emerging thematic areas and new research avenues. At the same time, it helps practitioners understand the challenges and technological developments that may affect the implementation of automated question generation systems in the future. Finally, it provides useful indications to policymakers, contributing to strategic planning and the formulation of regulatory frameworks that can support the adoption and safe utilization of such technologies.
This study constitutes an in-depth retrospective overview of the field, utilizing a combination of various bibliometric methods. Through these techniques, important information was revealed regarding productivity, the structure of scientific knowledge, and the conceptual foundations that support research in Automated Multiple-Choice Question Generation. To achieve our objectives, we answered five research questions.
The first question explored publication and citation trends, revealing a steady upward trend, as well as an increasing diffusion of the topic in high-prestige scientific journals, which demonstrates an increased interest and scientific maturity. The second question focused on identifying the most important research institutions. The findings show that the research core is located mainly in the USA, while the United Kingdom and Australia also play an important role. Then, scientific mapping techniques were utilized to answer the following questions. A literature review was conducted to explore the main thematic axes and their evolution. This analysis revealed that, in recent years, interest has been strongly focused on topics such as artificial intelligence in education, LLMs, NLP techniques, chatbots, ChatGPT, and new models such as Gemini, indicating the close connection of the field with modern developments in computational linguistics. The third question examined the collaborative patterns of authors through a co-authorship analysis. The results indicate that research activity is characterized by a significant degree of collaboration, which reflects the increased interdisciplinarity and dynamic nature of the field. Finally, the fifth question focused on identifying future directions. Through co-word analysis, emerging trends were identified, most notably Automated Multiple-Choice Question Generation itself, which are expected to gain even greater importance, especially when combined with applications in education and the creation of intelligent learning systems.
From a broader theoretical and technological perspective, the findings of this bibliometric analysis indicate that Automated Multiple-Choice Question Generation is increasingly embedded within the evolving ecosystem of AIED. The dominance of themes related to large language models, AI-assisted assessment, and educational support systems reflects a shift in the field towards generative and adaptive paradigms, where assessment is no longer treated as a static evaluative process but as a dynamic component of learning design. At the same time, the relatively limited consolidation of automated MCQ generation as an autonomous research stream suggests that this area remains under-theorized in comparison to other AI-driven educational applications. This gap highlights the need for a stronger integration between technological advances in natural language generation and established educational theories, such as formative assessments, cognitive taxonomies, and learning analytics. Future research is therefore encouraged to move beyond technical performance and address issues of pedagogical validity, cognitive alignment, and the responsible deployment of generative AI in assessment contexts. In this sense, automated MCQ generation can be seen not merely as a technical capability, but as a critical interface between artificial intelligence, educational theory, and instructional practice.
From a practical perspective, the findings of this study offer several implications for key stakeholders in education. For educators, automated MCQ generation systems supported by LLMs can be used to reduce the time and cognitive effort required for assessment design, while enabling more frequent formative assessments and immediate feedback. The bibliometric evidence indicates a growing interest in aligning AI-generated questions with cognitive taxonomies and learning objectives, suggesting that educators may increasingly employ such tools to support adaptive learning, diagnostic assessment, and personalized feedback in both face-to-face and online environments. For system developers, the identified thematic clusters and emerging research trends provide guidance for the design of assessment platforms that integrate natural language generation with learning analytics and instructional design principles. Developers may leverage insights from the literature to enhance question quality, ensure pedagogical validity, and incorporate features such as difficulty control, distractor plausibility, and the transparency of AI-generated outputs. The observed fragmentation of research in automated MCQ generation also highlights opportunities for developing modular, interoperable systems that can be embedded within existing learning management systems. For policy makers and educational administrators, the rapid growth of AI-driven assessment technologies underscores the need for evidence-informed governance frameworks. The results of this study can support policy decisions related to quality assurance, the ethical use of generative AI, data protection, and academic integrity. In particular, bibliometric trends emphasizing ethics, governance, and AI literacy suggest that institutional policies should promote responsible adoption, teacher training, and the continuous evaluation of AI-based assessment tools to ensure fairness, transparency, and pedagogical effectiveness.
Despite its systematic and comprehensive design, this study is subject to several methodological limitations that should be considered when interpreting the results. First, the exclusive use of the Scopus database, while ensuring high-quality and well-structured metadata, may have led to the omission of relevant studies indexed in other sources such as Web of Science or specialized conference proceedings. This choice may partially affect the observed distribution of publications, citations, and geographical contributions. Second, the construction of the search query and the selection of keywords inevitably shaped the scope of the retrieved literature. Although the query was designed to capture recent and impactful developments in automated assessments and large language models in education, certain studies focusing on Automated Multiple-Choice Question Generation using alternative terminology or earlier NLP approaches may not have been fully represented. As a result, some thematic areas may appear less consolidated than they are in practice. Third, the selected time window reflects the rapid evolution of the field in the post-2023 period; however, it also limits the ability to capture long-term citation dynamics and historical continuities. Consequently, highly recent publications may appear underrepresented in terms of citation impact, while earlier influential works may exert a disproportionate influence on bibliometric indicators. Finally, bibliometric techniques inherently emphasize structural and quantitative patterns rather than the qualitative depth of individual contributions. Methods such as bibliographic coupling and co-word analysis depend on citation practices and keyword selection, which may not always fully reflect the conceptual richness of the underlying studies. Future research could address these limitations by integrating bibliometric approaches with qualitative content analysis, expert-driven systematic reviews, and empirical investigations of educational effectiveness. Recognizing these limitations is essential for the appropriate interpretation of the findings and for guiding future research. These implications demonstrate how bibliometric evidence can inform instructional practice, system design, and policy decisions in AI-driven educational assessment.
Author Contributions
Conceptualization, D.M. and S.S.; methodology, D.M.; software, D.M.; validation, D.M. and S.S.; formal analysis, D.M. and S.S.; investigation, D.M.; resources, D.M.; data curation, D.M.; writing—original draft preparation, D.M.; writing—review and editing, D.M. and S.S.; visualization, D.M.; supervision, S.S. All authors have read and agreed to the published version of the manuscript.
Funding
This research received no external funding.
Data Availability Statement
The data we use for our calculations are derived from the Scopus Database.
Conflicts of Interest
The authors declare no conflicts of interest.
References
- Lan, Y.; Li, X.; Du, H.; Lu, X.; Gao, M.; Qian, W.; Zhou, A. Survey of natural language processing for education: Taxonomy, systematic review, and future trends. IEEE Trans. Knowl. Data Eng. 2025, 38, 659–678. [Google Scholar] [CrossRef] [Scilit]
- Kurdi, G.; Leo, J.; Parsia, B.; Sattler, U.; Al-Emari, S. A Systematic Review of Automatic Question Generation for Educational Purposes. Int. J. Artif. Intell. Educ. 2020, 30, 121–204. [Google Scholar] [CrossRef] [Scilit]
- Das, B.; Majumder, M.; Phadikar, S.; Sekh, A.A. Automatic question generation and answer assessment: A survey. Res. Pract. Technol. Enhanc. Learn. 2021, 16, 5. [Google Scholar] [CrossRef] [Scilit]
- Memarzadeh, H.; Ghadiri, N.; Samwald, M.; Lotfi Shahreza, M. Applying unsupervised keyphrase methods on concepts extracted from discharge sheets. Pattern Anal. Appl. 2023, 26, 1715–1727. [Google Scholar] [CrossRef] [Scilit]
- Martínez-Comesaña, M.; Rigueira-Díaz, X.; Larrañaga-Janeiro, A.; Martínez-Torres, J.; Ocarranza-Prado, I.; Kreibel, D. Impact of artificial intelligence on assessment methods in primary and secondary education: Systematic literature review. Rev. Psicodidáctica 2023, 28, 93–103. [Google Scholar] [CrossRef] [Scilit]
- Kotsiantis, S.B. Use of machine learning techniques for educational proposes: A decision support system for forecasting students’ grades. Artif. Intell. Rev. 2012, 37, 331–344. [Google Scholar] [CrossRef] [Scilit]
- Goyal, R.; Kumar, P.; Singh, V.P. A Systematic survey on automated text generation tools and techniques: Application, evaluation, and challenges. Multimed. Tools Appl. 2023, 82, 43089–43144. [Google Scholar] [CrossRef] [Scilit]
- Artsi, Y.; Sorin, V.; Konen, E.; Glicksberg, B.S.; Nadkarni, G.; Klang, E. Large language models for generating medical examinations: Systematic review. BMC Med. Educ. 2024, 24, 354. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Al Faraby, S.; Adiwijaya, A.; Romadhony, A. Review on neural question generation for education purposes. Int. J. Artif. Intell. Educ. 2024, 34, 1008–1045. [Google Scholar] [CrossRef] [Scilit]
- Awalurahman, H.W.; Budi, I. Automatic distractor generation in multiple-choice questions: A systematic literature review. PeerJ Comput. Sci. 2024, 10, e2441. [Google Scholar] [CrossRef] [Scilit]
- Ahadi, A.; Singh, A.; Bower, M.; Garrett, M. Text mining in education—A bibliometrics-based systematic review. Educ. Sci. 2022, 12, 210. [Google Scholar] [CrossRef] [Scilit]
- Aria, M.; Cuccurullo, C. Bibliometrix: An R-tool for comprehensive science mapping analysis. J. Informetr. 2017, 11, 959–975. [Google Scholar] [CrossRef] [Scilit]
- van Eck, N.J.; Waltman, L. Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics 2009, 84, 523–538. [Google Scholar] [CrossRef] [Scilit]
- Dwivedi, Y.K.; Sharma, A.; Rana, N.P.; Giannakis, M.; Goel, P.; Dutot, V. Evolution of artificial intelligence research in Technological Forecasting and Social Change: Research topics, trends, and future directions. Technol. Forecast. Soc. Change 2023, 192, 122579. [Google Scholar] [CrossRef] [Scilit]
- Jain, K.; Tripathi, P.S. Mapping the environmental, social and governance literature: A bibliometric and content analysis. J. Strateg. Manag. 2023, 16, 397–428. [Google Scholar] [CrossRef] [Scilit]
- Kaur, J.; Bathla, A.K. A review on automatic question generation system from a given Hindi text. International. J. Res. Comput. Appl. Robot. (IJRCAR) 2025, 3, 87–92. [Google Scholar]
- Rakangor, S.; Ghodasara, Y.R. Literature review of automatic question generation systems. Int. J. Sci. Res. Publ. 2025, 5, 1–5. [Google Scholar]
- Le, N.-T.; Kojiri, T.; Pinkwart, N. Automatic question generation for educational applications–the state of art. In Advances in Intelligent Systems and Computing, Proceedings of the Advanced Computational Methods for Knowledge Engineering, Budapest, Hungary, 8–9 May 2014; van Do, T., Thi, H., Nguyen, N., Eds.; Springer: Berlin/Heidelberg, Germany, 2014; pp. 325–338. [Google Scholar] [CrossRef] [Scilit]
- Ch, D.R.; Saha, S.K. Automatic multiple choice question generation from text: A survey. IEEE Trans. Learn. Technol. 2018, 13, 14–25. [Google Scholar] [CrossRef] [Scilit]
- Papasalouros, A.; Chatzigiannakou, M. Semantic Web and Question Generation: An Overview of the State of the Art. In Proceedings of the International Conference E-Learning, ERIC, Cape Town, South Africa, 5–6 July 2018; Available online: https://scispace.com/pdf/semantic-web-and-question-generation-an-overview-of-the-3v81u9f0h8.pdf (accessed on 11 October 2025).
- Bitew, S.K.; Deleu, J.; Develder, C.; Demeester, T. Distractor generation for multiple-choice questions with predictive prompting and large language models. In Communications in Computer and Information Science, Proceedings of the International Workshops of ECML PKDD 2023, Turin, Italy, 18–22 September 2023; Springer: Berlin/Heidelberg, Germany, 2023; pp. 48–63. [Google Scholar] [CrossRef] [Scilit]
- Ghanem, B.; Fyshe, A. Disto: Evaluating textual distractors for multi-choice questions using negative sampling based approach. arXiv 2023, arXiv:2304.04881. [Google Scholar] [CrossRef] [Scilit]
- Feng, W.; Lee, J.; McNichols, H.; Scarlatos, A.; Smith, D.; Woodhead, S.; Ornelas, N.O.; Lan, A. Exploring automated distractor generation for math multiple-choice questions via large language models. In Findings of the Association for Computational Linguistics: NAACL; Association for Computational Linguistics: Mexico City, Mexico, 2024; pp. 3067–3082. [Google Scholar] [CrossRef] [Scilit]
- Yu, H.-C.; Shih, Y.-A.; Law, K.-M.; Hsieh, K.-Y.; Cheng, Y.-C.; Ho, H.-C.; Lin, Z.-A.; Hsu, W.-C.; Fan, Y.-C. Enhancing distractor generation for multiple-choice questions with retrieval augmented pretraining and knowledge graph integration. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, Bangkok, Thailand, 11–16 August 2024; pp. 11019–11029. [Google Scholar] [CrossRef] [Scilit]
- Cavusoglu, D.; Sen, S.; Sert, U. DisGeM: Distractor Generation for Multiple Choice Questions with Span Masking. In Proceedings of the EMNLP 2024—2024 Conference on Empirical Methods in Natural Language Processing, Findings of EMNLP 2024, Association for Computational Linguistics (ACL), Miami, FL, USA, 12–16 November 2024; pp. 9714–9732. [Google Scholar] [CrossRef] [Scilit]
- Pan, L.; Lei, W.; Chua, T.-S.; Kan, M.-Y. Recent advances in neural question generation. arXiv 2019, arXiv:1905.08949. [Google Scholar] [CrossRef] [Scilit]
- Guo, S.; Liao, L.; Li, C.; Chua, T.-S. A survey on neural question generation: Methods, applications, and prospects. In Proceedings of the 33rd International Joint Conference on Artificial Intelligence (IJCAI-24), Jeju, Republic of Korea, 3–9 August 2024; pp. 8038–8047. [Google Scholar] [CrossRef] [Scilit]
- Mulla, N.; Gharpure, P. Automatic question generation: A review of methodologies, datasets, evaluation metrics, and applications. Prog. Artif. Intell. 2023, 12, 1–32. [Google Scholar] [CrossRef] [Scilit]
- Alhazmi, E.; Sheng, Q.Z.; Zhang, W.E.; Zaib, M.; Alhazmi, A. Distractor generation in multiple-choice tasks: A survey of methods, datasets, and evaluation. In Proceedings of the EMNLP 2024—2024 Conference on Empirical Methods in Natural Language Processing, Miami, FL, USA, 12–16 November 2024; pp. 14437–14458. [Google Scholar] [CrossRef] [Scilit]
- Lin, R.; Xu, D.; Wang, H.; Chen, Z.; Wang, Y.; Liu, S. DGRC: An Effective Fine-tuning Framework for Distractor Generation in Chinese Multi-choice Reading Comprehension. In Proceedings of the 2024 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Kuching, Malaysia, 6–10 October 2024; IEEE: New York, NY, USA, 2024; pp. 815–820. [Google Scholar] [CrossRef] [Scilit]
- Mucciaccia, S.S.; Paixão, T.M.; Mutz, F.; De Souza, A.F.; Badue, C.S.; Oliveira-santos, T. Automatic Multiple-Choice Question Generation and Evaluation Systems Based on LLM: A Study Case With University Resolutions. In Proceedings of the 31st International Conference on Computational Linguistics, Abu Dhabi, United Arab Emirates, 19–24 January 2025; pp. 2246–2260. Available online: https://aclanthology.org/2025.coling-main.154/ (accessed on 12 October 2025).
- Paiva, J.C.; Figueira, Á.; Leal, J.P. Bibliometric Analysis of Automated Assessment in Programming Education: A Deeper Insight into Feedback. Electronics 2023, 12, 2254. [Google Scholar] [CrossRef] [Scilit]
- Dong, X.; Li, Z.; Su, H.; Xue, J.; Dang, X. Transformer-enhanced hierarchical encoding with multi-decoder for diversified MCQ distractor generation. Artif. Intell. Rev. 2025, 58, 229. [Google Scholar] [CrossRef] [Scilit]
- Ait, O.A.; Mohib, Z.; Adrdour, M. Natural Language Processing in Higher Education Institutions: A Bibliometric Analysis Using Scopus Database. In Proceedings of the International Conference on Artificial Intelligence in Education Technology, Barcelona, Spain, 29–31 July 2024; pp. 99–120. [Google Scholar] [CrossRef] [Scilit]
- Shaik, T.; Tao, X.; Li, Y.; Dann, C.; McDonald, J.; Redmond, P.; Galligan, L. A review of the trends and challenges in adopting natural language processing methods for education feedback analysis. IEEE Access 2022, 10, 56720–56739. [Google Scholar] [CrossRef] [Scilit]
- Awalurahman, H.W.; Aji, R.F.; Budi, I. Transformer and Large Language Models for Automatic Multiple-Choice Question Generation: A Systematic Literature Review. IEEE Access 2025, 13, 127100–127112. [Google Scholar] [CrossRef] [Scilit]
- Gavira Durón, N.; Jiménez-Preciado, A.L. Exploring the role of AI in higher education: A natural language processing analysis of emerging trends and discourses. TQM J. 2025. ahead-of-print. [Google Scholar] [CrossRef] [Scilit]
- Al Shuraiqi, S.; Aal Abdulsalam, A.; Masters, K.; Zidoum, H.; AlZaabi, A. Automatic generation of medical case-based multiple-choice questions (MCQs): A review of methodologies, applications, evaluation, and future directions. Big Data Cogn. Comput. 2024, 8, 139. [Google Scholar] [CrossRef] [Scilit]
- Xue, H.; Liu, W. Bibliometric Analysis of Natural Language Processing Technology in Education: Hot Topics, Frontier Evolution, and Future Prospects. SAGE Open 2025, 15, 21582440251319891. [Google Scholar] [CrossRef] [Scilit]
- Paladines, J.; Ramirez, J. A systematic literature review of intelligent tutoring systems with dialogue in natural language. IEEE Access 2020, 8, 164246–164267. [Google Scholar] [CrossRef] [Scilit]
- Mousavinasab, E.; Zarifsanaiey, N.; Kalhori, S.R.N.; Rakhshan, M.; Keikha, L.; Saeedi, M.G. Intelligent tutoring systems: A systematic review of characteristics, applications, and evaluation methods. Interact. Learn. Environ. 2021, 29, 142–163. [Google Scholar] [CrossRef] [Scilit]
- Ng, S.-L.; Ho, C.-C. Generative AI in Education: Mapping the Research Landscape Through Bibliometric Analysis. Information 2025, 16, 657. [Google Scholar] [CrossRef] [Scilit]
- Guo, S.; Zheng, Y.; Zhai, X. Artificial intelligence in education research during 2013– 2023: A review based on bibliometric analysis. Educ. Inf. Technol. 2024, 29, 16387–16409. [Google Scholar] [CrossRef] [Scilit]
- Ch, D.R.; Saha, S.K. Generation of multiple-choice questions from textbook contents of school-level subjects. IEEE Trans. Learn. Technol. 2022, 16, 40–52. [Google Scholar] [CrossRef] [Scilit]
- Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 2021, 372, 71. [Google Scholar] [CrossRef] [Scilit]
- Marles, E.; Gomez-Luna, E.; Guerrero, J.; Vasquez, J. Innovative Bibliometric Methodology: A New Big Data-Based Framework for Scientific Research. Energies 2025, 18, 2437. [Google Scholar] [CrossRef] [Scilit]
- Mukherjee, D.; Lim, W.M.; Kumar, S.; Donthu, N. Guidelines for advancing theory and practice through bibliometric research. J. Bus. Res. 2022, 148, 101–115. [Google Scholar] [CrossRef] [Scilit]
- Donthu, N.; Kumar, S.; Mukherjee, D.; Pandey, N.; Lim, W.M. How to conduct a bibliometric analysis: An overview and guidelines. J. Bus. Res. 2021, 133, 285–296. [Google Scholar] [CrossRef] [Scilit]
- Tian, K.; Zhu, Z.; Mbachu, J.; Ghanbaripour, A.; Moorhead, M. Artificial intelligence in risk management within the realm of construction projects: A bibliometric analysis and systematic literature review. J. Innov. Knowl. 2025, 10, 100711. [Google Scholar] [CrossRef] [Scilit]
- Klarin, A. How to conduct a bibliometric content analysis: Guidelines and contributions of content co-occurrence or co-word literature reviews. Int. J. Consum. Stud. 2024, 48, e13031. [Google Scholar] [CrossRef] [Scilit]
- Mitroulia, M.; Armakolas, S.; Krotký, J. A bibliometric analysis on research trends of distance education using scopus database. J. Educ. Technol. Comput. Sci. 2023, 34, 153–164. [Google Scholar] [CrossRef] [Scilit]
- Lachheb, A.; Leung, J.; Abramenka-Lachheb, V.; Sankaranarayanan, R. AI in higher education: A bibliometric analysis, synthesis, and a critique of research. Internet High. Educ. 2025, 67, 101021. [Google Scholar] [CrossRef] [Scilit]
- Alduais, A.; Yassin, A.A.; Allegretta, S. Computational linguistics: A scientometric review. In Quality and Quantity; Springer: Dordrecht, The Netherlands, 2025; Volume 59. [Google Scholar] [CrossRef] [Scilit]
- Hirsch, J. An Index to Quantify An Individual’s Scientific Research Output. Proc. Natl. Acad. Sci. USA 2005, 102, 16569–16572. [Google Scholar] [CrossRef] [Scilit]
- Cobo, M.J.; López-Herrera, A.G.; Herrera-Viedma, E.; Herrera, F. An approach for detecting, quantifying, and visualizing the evolution of a research field: A practical application to the Fuzzy Sets Theory field. J. Informetr. 2011, 5, 146–166. [Google Scholar] [CrossRef] [Scilit]
- Mingers, J.; Leydesdorff, L. A Review of Theory and Practice in Scientometrics. Eur. J. Oper. Res. 2015, 246, 1–19. [Google Scholar] [CrossRef] [Scilit]
- Kessler, M.M. Bibliographic coupling between scientific papers. Am. Doc. 1963, 14, 10–25. [Google Scholar] [CrossRef] [Scilit]
- Zupic, I.; Čater, T. Bibliometric Methods in Management and Organization. Organ. Res. Methods 2014, 18, 429–472. [Google Scholar] [CrossRef] [Scilit]
- Crane, D. Social Structure in a Group of Scientists: A Test of the “Invisible College” Hypothesis. Am. Sociol. Rev. 1969, 34, 335–352. [Google Scholar] [CrossRef] [Scilit]
- Sigala, M.; Kumar, S.; Donthu, N.; Sureka, R.; Joshi, Y. A bibliometric overview of the Journal of Hospitality and Tourism Management: Research contributions and influence. J. Hosp. Tour. Manag. 2021, 47, 273–288. [Google Scholar] [CrossRef] [Scilit]
- Chytis, E.; Eriotis, N.; Mitroulia, M. ESG in Business Research: A Bibliometric Analysis. J. Risk Financ. Manag. 2024, 17, 460. [Google Scholar] [CrossRef] [Scilit]
- Samara, H.H.; Qudah, H.A.; Mohsin, H.J.; Abualhijad, S.; Hani, L.Y.B.; Al Rahamneh, S.; AlQudah, M.Z. Artificial intelligence and machine learning in corporate governance: A bibliometric analysis. Hum. Syst. Manag. 2025, 44, 349–375. [Google Scholar] [CrossRef] [Scilit]
- Kavitha, K.; Joshith, V.P.; Rajeev, N.P.; Asha, S. Artificial Intelligence in Higher Education: A Bibliometric Approach. Başlık 2024, 13, 1121–1137. [Google Scholar] [CrossRef] [Scilit]
- Lampropoulos, G. Intelligent Virtual Reality and Augmented Reality Technologies: An Overview. Futur. Internet 2025, 17, 58. [Google Scholar] [CrossRef] [Scilit]
- Mitroulia, M.; Chytis, E.; Kitsantas, T.; Skordoulis, M.; Kalantonis, P. ESG Strategy and Tax Avoidance: Insights from a Meta-Regression Analysis. J. Risk Financ. Manag. 2025, 18, 503. [Google Scholar] [CrossRef] [Scilit]
- Mitroulia, M.; Chytis, E.; Kitsantas, T.; Mitroulias, D. ESG Reported Activities and Corporate Tax Planning: A Meta-Analysis. In Proceedings of the 14th International Conference of the Financial Engineering and Banking Society (FEBS), Montpellier, France, 11–13 May 2025; Available online: https://febs2025.eventsadmin.com/Home/Welcome (accessed on 7 October 2025).
- Durán-Sánchez, A.; Río-Rama, M.d.l.C.D.; Álvarez-García, J.; García-Vélez, D.F. Mapping of scientific coverage on education for Entrepreneurship in Higher Education. J. Enterprising Communities People Places Glob. Econ. 2019, 13, 84–104. [Google Scholar] [CrossRef] [Scilit]
- Ragazou, K.; Passas, I.; Garefalakis, A.; Galariotis, E.; Zopounidis, C. Big Data Analytics Applications in Information Management Driving Operational Efficiencies and Decision-Making: Mapping the Field of Knowledge with Bibliometric Analysis Using R. Big Data Cogn. Comput. 2023, 7, 13. [Google Scholar] [CrossRef] [Scilit]
- Soliman, M.; Fatnassi, T.; Elgammal, I.; Figueiredo, R. Exploring the Major Trends and Emerging Themes of Artificial Intelligence in the Scientific Leading Journals amidst the COVID-19 Era. Big Data Cogn. Comput. 2023, 7, 12. [Google Scholar] [CrossRef] [Scilit]
- Gusenbauer, M. Search where you will find most: Comparing the disciplinary coverage of 56 bibliographic databases. Scientometrics 2022, 127, 2683–2745. [Google Scholar] [CrossRef] [Scilit]
- Comerio, N.; Strozzi, F. Tourism and its economic impact: A literature review using bibliometric tools. Tour. Econ. 2019, 25, 109–131. [Google Scholar] [CrossRef] [Scilit]
- Clermont, M.; Dyckhoff, H. Coverage of business administration literature in Google Scholar: Analysis and comparison with Econbiz, Scopus and Web of Science. Bibliometr. Forsch. 2012, 1, 54. [Google Scholar] [CrossRef] [Scilit]
- Joshi, A. Comparison between Scopus & ISI Web of Science. J. Glob. Values ISSN 2016, 7, 976–9447. [Google Scholar]
- Nobanee, H.; Dilshad, M.N.; Al Dhanhani, M.; Al Neyadi, M.; Al Qubaisi, S.; Al Shamsi, S. Big data applications the banking sector: A bibliometric analysis approach. Sage Open 2021, 11, 21582440211067230. [Google Scholar] [CrossRef] [Scilit]
- Fan, L.; Li, L.; Ma, Z.; Lee, S.; Yu, H.; Hemphill, L. A Bibliometric Review of Large Language Models Research from 2017 to 2023. ACM Trans. Intell. Syst. Technol. 2024, 15, 91. [Google Scholar] [CrossRef] [Scilit]
- Hadi, M.U.; Al-Tashi, Q.; Shah, A.; Qureshi, R.; Muneer, A.; Irfan, M.; Zafar, A.; Shaikh, M.; Akhtar, N.; Wu, J.; et al. Large Language Models: A Comprehensive Survey of its Applications, Challenges, Limitations, and Future Prospects. Authorea Prepr. 2023, 1, 1–26. [Google Scholar] [CrossRef]
- Nguyen, T.N.; Truong, H.T. Trends and emerging themes in the effects of generative artificial intelligence in education: A systematic review. Eurasia J. Math. Sci. Technol. Educ. 2025, 21, em2613. [Google Scholar] [CrossRef] [Scilit]
- Fu, Y.; Weng, Z.; Wang, J. Examining AI Use in Educational Contexts: A Scoping Meta-Review and Bibliometric Analysis. Int. J. Artif. Intell. Educ. 2024, 35, 1388–1444. [Google Scholar] [CrossRef] [Scilit]
- Pessin, V.Z.; Yamane, L.H.; Siman, R.R. Smart bibliometrics: An integrated method of science mapping and bibliometric analysis. Scientometrics 2022, 127, 3695–3718. [Google Scholar] [CrossRef] [Scilit]
- Sureka, R.; Donthu, N.; Kumar, S. Three Decades of the Journal of Teaching in International Business: A Bibliometric Overview. J. Teach. Int. Bus. 2020, 31, 259–285. [Google Scholar] [CrossRef] [Scilit]
- Kasneci, E.; Seßler, K.; Küchemann, S.; Bannert, M.; Dementieva, D.; Fischer, F.; Gasser, U.; Groh, G.; Günnemann, S.; Hüllermeier, E. ChatGPT for good? On opportunities and challenges of large language models for education. Learn. Individ. Differ. 2023, 103, 102274. [Google Scholar] [CrossRef] [Scilit]
- Frandsen, T. Evolution of modularity literature: A 25-year bibliometric analysis. Int. J. Oper. Prod. Manag. 2017, 37, 703–747. [Google Scholar] [CrossRef] [Scilit]
- Seyedghorban, Z.; Matanda, M.J.; LaPlaca, P. Advancing theory and knowledge in the business-to-business branding literature. J. Bus. Res. 2016, 69, 2664–2677. [Google Scholar] [CrossRef] [Scilit]
- Ferreira, B.; Diz, A.; Silva, P.; Sousa, L.; Pinho, L.; Fonseca, C.; Lopes, M. Bibliometric Analysis of the Informal Caregiver’s Scientific Production. J. Pers. Med. 2022, 12, 61. [Google Scholar] [CrossRef] [Scilit]
- Meyer, J.G.; Urbanowicz, R.J.; Martin, P.C.N.; O’Connor, K.; Li, R.; Peng, P.-C.; Bright, T.J.; Tatonetti, N.; Won, K.J.; Gonzalez-Hernandez, G.; et al. ChatGPT and large language models in academia: Opportunities and challenges. BioData Min. 2023, 16, 20. [Google Scholar] [CrossRef] [Scilit]
- Guizani, S.; Mazhar, T.; Shahzad, T.; Ahmad, W.; Bibi, A.; Hamam, H. A systematic literature review to implement large language model in higher education: Issues and solutions. Discov. Educ. 2025, 4, 35. [Google Scholar] [CrossRef] [Scilit]
- Hassan, A.; Eid, M. A Systematic Review of Automatic Neural Question Generation. J. ACS Adv. Comput. Sci. 2025, 16. [Google Scholar] [CrossRef] [Scilit]
- Bornmann, L.; Daniel, H. What do citation counts measure? A review of studies on citing behavior. J. Doc. 2008, 64, 45–80. [Google Scholar] [CrossRef] [Scilit]
- Dwivedi, Y.K.; Kshetri, N.; Hughes, L.; Slade, E.L.; Jeyaraj, A.; Kar, A.K.; Baabdullah, A.M.; Koohang, A.; Raghavan, V.; Ahuja, M.; et al. Opinion Paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. Int. J. Inf. Manage. 2023, 71, 102642. [Google Scholar] [CrossRef] [Scilit]
- Gilson, A.; Safraneck, C.; Huang, T.; Socrates, V.; Chi, L.; Taylor, R.; Chartash, D. How Does ChatGPT Perform on the United States Medical Licensing Examination? The Implications of Large Language Models for Medical Education and Knowledge Assessment. JMIR Med. Educ. 2023, 9, e45312. [Google Scholar] [CrossRef] [Scilit] [PubMed]
- Rudolph, J.; Tan, S.S.S.; Tan, S.S.S. ChatGPT: Bullshit spewer or the end of traditional assessments in higher education? J. Appl. Learn. Teach. 2023, 6, 342–363. [Google Scholar] [CrossRef] [Scilit]
- Rudolph, J.; Tan, S.; Tan, S. War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education. J. Appl. Learn. Teach. 2023, 6, 364–389. [Google Scholar] [CrossRef] [Scilit]
- Jeon, J.; Lee, S. Large language models in education: A focus on the complementary relationship between human teachers and ChatGPT. Educ. Inf. Technol. 2023, 28, 15873–15892. [Google Scholar] [CrossRef] [Scilit]
- Crawford, J.; Cowling, M.; Allen, K.-A. Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI). J. Univ. Teach. Learn. Pract. 2023, 20, 1–19. [Google Scholar] [CrossRef] [Scilit]
- Javaid, M.; Haleema, A.; Singh, R.P. ChatGPT for healthcare services: An emerging stage for an innovative perspective. BenchCouncil Trans. Benchmarks, Stand. Eval. 2023, 3, 100105. [Google Scholar] [CrossRef] [Scilit]
- Nikolic, S.; Daniel, S.; Haque, R.; Belkina, M.; Hassan, G.M.; Grundy, S.; Lyden, S.; Neal, P.; Sandison, C. ChatGPT versus engineering education assessment: A multidisciplinary and multi-institutional benchmarking and analysis of this generative artificial intelligence tool to investigate assessment integrity. Eur. J. Eng. Educ. 2023, 48, 559–614. [Google Scholar] [CrossRef] [Scilit]
- Barrett, A.; Pack, A. Not quite eye to AI: Student and teacher perspectives on the use of generative artificial intelligence in the writing process. Int. J. Educ. Technol. High. Educ. 2023, 20, 59. [Google Scholar] [CrossRef] [Scilit]
- Sajja, R.; Sermet, Y.; Cikmaz, M.; Cwiertny, D.; Demir, I. Artificial intelligence-enabled intelligent assistant for personalized and adaptive learning in higher education. Information 2024, 15, 596. [Google Scholar] [CrossRef] [Scilit]
- Hwang, K.; Challagundla, S.; Alomair, M.; Chen, L.K.; Choa, F.-S. Towards AI-assisted multiple choice question generation and quality evaluation at scale: Aligning with Bloom’s Taxonomy. In Proceedings of the Artificial Intelligence in Education, Recife, Brazil, 8–12 July 2024. [Google Scholar] [CrossRef] [Scilit]
- Ling, J.; Afzaal, M. Automatic question-answer pairs generation using pre-trained large language models in higher education. Comput. Educ. Artif. Intell. 2024, 6, 100252. [Google Scholar] [CrossRef] [Scilit]
- Workum, J.; Volkers, B.; Van de Sande, D.; Arora, S.; Goeijenbier, M.; Gommers, D.; van Genderen, M. Comparative evaluation and performance of large language models on expert level critical care questions: A benchmark study. Crit. Care 2025, 29, 72. [Google Scholar] [CrossRef] [Scilit]
- Azzi, A.; Erdős, F.; Németh, R.; Varadarajan, V.; Afrifa, S. Comparative analysis of NLP-driven MCQ generators from text sources. Comput. Educ. Artif. Intell. 2025, 9, 100440. [Google Scholar] [CrossRef] [Scilit]
- Wu, H.; Zerner, T.; Lee, D.; Court-Kowalski, S.; Devitt, P.; Palmer, E. GPT-4 versus human authors in clinically complex MCQ creation: A blinded analysis of item quality. Med. Teach. 2025, 47, 1–14. [Google Scholar] [CrossRef] [Scilit]
- Roumeliotis, K.I.; Tselikas, N.D.; Nasiopoulos, D.K. LLMs and NLP Models in Cryptocurrency Sentiment Analysis: A Comparative Classification Study. Big Data Cogn. Comput. 2024, 8, 63. [Google Scholar] [CrossRef] [Scilit]
- Hamid, R.; Brohi, S. A Review of Large Language Models in Healthcare: Taxonomy, Threats, Vulnerabilities, and Framework. Big Data Cogn. Comput. 2024, 8, 161. [Google Scholar] [CrossRef] [Scilit]
- Zhan, X.; Xu, Y.; Abdi, N.; Collenette, J.; Abu-Salma, R.; Sarkadi, S. Banal Deception Human-AI Ecosystems: A Study of People’s Perceptions of LLM-generated Deceptive Behaviour. arXiv 2024, arXiv:2406.08386. [Google Scholar] [CrossRef] [Scilit]
- Caccavale, F.; Gargalo, C.L.; Gernaey, K.V.; Krühne, U. Towards Education 4.0: The role of Large Language Models as virtual tutors in chemical engineering. Educ. Chem. Eng. 2024, 49, 1–11. [Google Scholar] [CrossRef] [Scilit]
- Mutanga, M.B.; Msane, J.; Mndaweni, T.N.; Hlongwane, B.B.; Ngcobo, N.Z. Exploring the Impact of LLM Prompting on Students’ Learning. Trends High. Educ. 2025, 4, 31. [Google Scholar] [CrossRef] [Scilit]
- Venter, J.; Coetzee, S.A.; Schmulian, A. Exploring the use of artificial intelligence (AI) in the delivery of effective feedback. Assess. Eval. High. Educ. 2025, 50, 516–536. [Google Scholar] [CrossRef] [Scilit]
- Calderon, R.; Herrera, F. And Plato met ChatGPT: An ethical reflection on the use of chatbots in scientific research writing, with a particular focus on the social sciences. Humanit. Soc. Sci. Commun. 2025, 12, 713. [Google Scholar] [CrossRef] [Scilit]
- Grundmann, S.; Al Owayyed, M.; Bruijnes, M.; Vroonhof, E.; Brinkman, W.-P. Lilobot: A Cognitive Conversational Agent to Train Counsellors at Children’s Helplines. J. Med. Syst. 2025, 49, 5. [Google Scholar] [CrossRef] [Scilit]
- Isiaku, L.; Kwala, A.; Sambo, K.; Isiaku, H. Academic Evolution in the Age of ChatGPT: An In-depth Qualitative Exploration of its Influence on Research, Learning, and Ethics in Higher Education. J. Univ. Teach. Learn. Pract. 2024, 21, 67–91. [Google Scholar] [CrossRef] [Scilit]
- Medina-Romero, M.Á.; Jinchuña Huallpa, J.; Flores-Arocutipa, J.; Panduro, W.; Huete, L.; Flores Limo, F.; Herrera, E.; Callacna, R.; Ariza Flores, V.; Quispe, I.; et al. Exploring the ethical considerations of using Chat GPT in university education. Period. Eng. Nat. Sci. 2023, 11, 105–115. [Google Scholar] [CrossRef] [Scilit]
- Peykani, P.; Ramezanlou, F.; Tanasescu, C.; Ghanidel, S. Large Language Models: A Structured Taxonomy and Review of Challenges, Limitations, Solutions, and Future Directions. Appl. Sci. 2025, 15, 8103. [Google Scholar] [CrossRef] [Scilit]
- Fuligni, C.; Figaredo, D.D.; Stoyanovich, J. “Would You Want an AI Tutor?” Understanding Stakeholder Perceptions of LLM-based Chatbots in the Classroom. arXiv 2025, arXiv:2503.02885. [Google Scholar]
- Wang, J.; Huang, C.; Yan, S.; Xie, W.; He, D. When young scholars cooperate with LLMs in academic tasks: The influence of individual differences and task complexities. Int. J. Hum.–Comput. Interact. 2025, 41, 4624–4639. [Google Scholar] [CrossRef] [Scilit]
- Sáez-Velasco, S.; Alaguero-Rodríguez, M.; Rodríguez-Cano, S.; Delgado-Benito, V. Students’ Attitudes Towards AI and How They Perceive the Effectiveness of AI in Designing Video Games. Sustainability 2025, 17, 3096. [Google Scholar] [CrossRef] [Scilit]
- Idris, M.D.; Feng, X.; Dyo, V. Revolutionizing higher education: Unleashing the potential of large language models for strategic transformation. IEEE Access 2024, 12, 67738–67757. [Google Scholar] [CrossRef] [Scilit]
- Liu, L. AI and the Crisis of Legitimacy in Higher Education. Am. J. STEM Educ. 2025, 4, 16–26. [Google Scholar]
- Semeler, A.; Pinto, A.; Koltay, T.; Dias, T.; Oliveira, A.; González, J.; Rozados, H.B.F. Algorithmic literacy: Generative artificial intelligence technologies for data librarians. EAI Endorsed Trans. Res. Artic. Scalable Inf. Syst. 2024, 11. [Google Scholar] [CrossRef] [Scilit]
- Kajiwara, Y.; Kawabata, K. AI literacy for ethical use of chatbot: Will students accept AI ethics? Comput. Educ. Artif. Intell. 2024, 6, 100251. [Google Scholar] [CrossRef] [Scilit]
- Nguyen, T.A.; Kharitonov, E.; Copet, J.; Adi, Y.; Hsu, W.-N.; Elkahky, A.; Tomasello, P.; Algayres, R.; Sagot, B.; Mohamed, A.; et al. Generative Spoken Dialogue Language Modeling. Trans. Assoc. Comput. Linguist. 2023, 11, 250–266. [Google Scholar] [CrossRef] [Scilit]
- Hwang, Y.; Kang, D.; Bak, J. Dialogue response coherency evaluation with feature sensitive negative sample using multi list-wise ranking loss. Eng. Appl. Artif. Intell. 2025, 150, 110609. [Google Scholar] [CrossRef] [Scilit]
- Demirkol, M.; Malkoç, N. Assessing the Intellectual Structure of the Evolving Knowledge Base on ChatGPT in the Field of Education and Health. Educ. Process Int. J. 2023, 12, 36–64. [Google Scholar] [CrossRef] [Scilit]
- Bozkurt, A. Unleashing the Potential of Generative AI, Conversational Agents and Chatbots in Educational Praxis: A Systematic Review and Bibliometric Analysis of GenAI in Education. Open Prax. 2023, 15, 261–270. [Google Scholar] [CrossRef] [Scilit]
- Ismail, F.; Tan, E.; Rudolph, J.; Crawford, J.; Tan, S. Artificial intelligence in higher education. A protocol paper for a systematic literature review. J. Appl. Learn. Teach. 2023, 6, 56–63. [Google Scholar] [CrossRef] [Scilit]
- Zhai, X.; Chu, X.; Chai, C.S.; Jong, M.S.Y.; Istenic, A.; Spector, M.; Liu, J.-B.; Yuan, J.; Li, Y. A Review of Artificial Intelligence (AI) in Education from 2010 to 2020. Complexity 2021, 2021, 8812542. [Google Scholar] [CrossRef] [Scilit]
- Tahiru, F. AI in education: A systematic literature review. J. Cases Inf. Technol. 2021, 23, 1–20. [Google Scholar] [CrossRef] [Scilit]
- Zhang, K.; Aslan, A.B. AI technologies for education: Recent research & future directions. Comput. Educ. Artif. Intell. 2021, 2, 100025. [Google Scholar] [CrossRef] [Scilit]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.



