Collaborative AI-Integrated Model for Reviewing Educational Literature ^†

Abstract

The increasing complexity of networked research demands approaches that combine rigor, efficiency, and collaboration. In this context, artificial intelligence (AI) emerges as a strategic ally in the analysis and organization of scientific literature, facilitating the construction of a robust state-of-the-art framework to support decisions. The present study focuses on evaluating a model for the use of AI that facilitates collaborative literature review by integrating AI tools. The present study employed a descriptive, non-experimental, cross-sectional design. Participants (N = 10) completed a purpose-built questionnaire comprising twenty-five indicators on specific aspects of the model’s use. The participants indicated a high level of knowledge regarding ICT use (M = 8.3; SD = 1.25). The results showed that the System Usability Scale for the tools demonstrated variability; Google Drive scored the highest (M = 77.75; SD = 11.45), while Rayyan.AI scored the lowest (M = 66.00; SD = 20.69). While the findings indicated that AI enhances the efficiency of documentary research and the development of ethical and digital competencies, the participants expressed a need for further training in AI tools to optimize the usability of those integrated into the model. The proposed model CAIM-REL proves to be replicable and holds potential for collaborative research.

1. Introduction

The term “Artificial Intelligence” (AI) has become ubiquitous in a relatively brief period, permeating not only academic and professional discourse but also economic and social spheres. It is a broad concept encompassing not only diverse technologies and applications but also various machine learning approaches, ranging from simple algorithms and applications to advanced systems such as machine learning and neural networks [1]. Thus, AI goes beyond being just a tool to identify itself as a system with “abilities to solve specific problems and tasks” [2] (p. 17) in various fields in which its ability to synthesize and analyze, recommend and propose, organize and evaluate, generate and create, or even, “learn and provide advice based on digital data” [3] (p. 2) is put into practice.

The incorporation of artificial intelligence systems in education is substantially reshaping both teaching, learning, and assessment [4] and innovation and research [5], posing new challenges to either policy makers, managers, teachers, students or researchers. Recent studies [6,7] highlight the rapid growth of AI research in education, especially from 2020 onwards. Such studies emphasize the transformative potential of AI, as well as the need for ethical considerations, addressing key issues such as machine learning, adaptive learning, and the use of generative AI technologies such as ChatGPT [7].

Particularly, in educational research, the use of artificial intelligence has grown exponentially not only in terms of research “on” AI but also “with” AI, generating important contributions, reshaping practices as well as optimizing study times and procedures. This fact is essentially justified because it allows processing large volumes of data quickly and accurately, as according to [8], Jordan and Mitchell, machine learning algorithms can identify complex patterns and hidden correlations in massive datasets, thus enabling discoveries that would be practically impossible with traditional statistical methods. As a result, automating repetitive tasks using AI frees up time and resources, which [9] highlights as improving the performance of professionals. In the field of research, AI can also formulate new scientific hypotheses. In this regard, [10] Kitano launches the “Nobel Turing Challenge” project, which seeks to develop AI systems capable of autonomously making scientific discoveries, highlighting AI as an analysis system and an active knowledge-generating agent.

Especially in the context of documentary or bibliographic research, the researcher needs to contextualize their object of study and base their hypotheses on the knowledge generated. [11] Gil stresses the importance of analyzing and systematizing the available expertise to understand the state of the art on the topic to be developed, since it requires a critical exercise in the selection of information gathering, interpretation, and articulation of relevant sources. [12] Lakatos and Marconi complement this perspective by showing that reference research can also establish the conceptual and methodological bases for a study, supporting the justification of the problem, the formulation of the objectives, and the delimitation of the theoretical approach that guides the interpretation of the findings. This dialog with the scientific literature, in particular the literature review that summarizes the ideas of various authors, functions as an intellectual argument that situates the research within an academic debate [13]. In this scenario, AI represents a strong ally for 21st-century science, and its ethical and responsible use promises to transform the way scientific research is conducted.

Similarly, recent studies refer to how AI increases research productivity and the quality of output. [14] De la Torre-López et al. show that, in the field of systematic review and literature searching, AI systems reduce the time spent on tasks such as article filtering, selection of relevant studies, and extraction of key information. In this framework, AI has revolutionized academic research by streamlining processes and improving consistency while minimizing human error. For [15] TechTitute, technologies such as deep learning, predictive models, and data mining algorithms enable researchers to process large volumes of information, find complex patterns, and make more informed decisions. This capability is especially useful in fields where the volume of data is massive and the speed of processing can directly affect the validity of the results, i.e., AI can be a catalyst for innovation in scientific research, contributing to faster, more accessible, and rigorous science.

Systematic reviews of scientific literature, as a method to identify, evaluate and interpret the scientific work published on the object of study, have been seen as a necessity from the amount of information that currently exists, so it aims to find gaps and needs of a particular field of educational research [16]. So the application of generative AI allows the researcher to optimize the processes of systematic literature reviews mainly because of its document search, analysis, and synthesis properties [17]. [18] Bolaños et al. provide a comprehensive overview of 21 AI tools employed in the semi-automatic generation of literature reviews, with a particular focus on the screening and extraction phases.

Nevertheless, the Pipeline learning model of active learning, inspired by the systematic review method of Carbajal-Degante et al. [19], recognizes the Researcher-In-The-Loop to enhance systematic reviews with the use of AI. Mainly, in the screening processes, proposing tools such as Rayyan.ai [20], a platform that supports the process of analysis and evaluation of references, as well as Elicit as a platform to automate search and review tasks, finally the use of ChatGPT in conjunction with SciBERT is mentioned, which is a natural language model for scientific texts. In turn, [21] Adel y Alani proposed the Systematic Research Processing Framework (SRPF), conceived as a hybrid workflow model for systematic reviews that integrates AI tools (such as ChatGPT and Elicit) with rigorous human oversight throughout the literature review process. Its aim is to reduce time and workload while maintaining standards of rigor and transparency, as well as addressing issues such as hallucinations and variability in accuracy. To this end, AI is used for tasks such as searching, preliminary screening, data extraction, and drafting summaries. However, the responsibility for critical decisions and interpretation remains with the researchers. Likewise, Zohery [22] proposes a model for the proper use of ChatGPT for writing scientific articles.

This development benefits academics by streamlining and freeing time-consuming tasks. In the field of collaborative research, AI plays a pivotal role in the effective management of large and diverse data sets, leading to significant cost savings in coding textual content and facilitating access to advanced resources and analyses [23]. It also helps to coordinate work among teams, making it easier for people to review each other’s work and combine results [24].

However, challenges must be acknowledged, such as the widespread use of AI, which can be interpreted as a loss of cognitive benefits for researchers. Furthermore, there is malpractice associated with academic plagiarism. These challenges are associated with authorship of manuscripts, so it is vital that the tools used for writing, searching or analyzing published data are detailed [25].

The aim of this study is to evaluate a model for the utilization of artificial intelligence, which facilitates collaborative literature reviews. This evaluation will be conducted by analyzing the influential internal and external factors perceived by researchers. It is evident that the design of the model is articulated as a methodological process of piloting the proposal, as a preliminary step to the evaluation. So, in this article, we do not examine isolated AI applications but present and evaluate the Collaborative AI-Integrated Model for Reviewing Educational Literature (CAIM-REL). This model frames AI-assisted reviewing as an end-to-end workflow that links all stages of a systematic review, formulating research questions, defining search strategies, screening, coding and synthesizing evidence, while enabling coordinated work among geographically distributed teams. Rather than treating tools such as Consensus, Rayyan.ai, Mendeley, MAXQDA or VOSviewer 1.6.18 as independent solutions, CAIM-REL integrates them into a coherent methodological structure aligned with PRISMA procedures. Although AI has been increasingly used to support specific tasks in systematic reviews, empirically evaluated, fully integrated and collaboration-oriented models remain scarce in educational research. CAIM-REL addresses this gap by offering a replicable framework that combines mixed AI tools with structured human collaboration. The present study goes beyond describing the model: it provides an evidence-based evaluation of CAIM-REL, examining perceived usability, strengths and limitations, and its potential to enhance the rigor and scalability of collaborative literature reviews.

2. Materials and Methods

The study employs a concurrent mixed-methods design (qualitative and quantitative), since an instrument was developed to evaluate a model of the use of AI in documentary research processes and to evaluate the experience of applying a systematic literature review model supported by AI. This made it possible to collect and analyze both quantitative and qualitative data [26] from the perspective of the network members, to configure an integrative view of the perception of the experience. According to Pole [27], the use of both quantitative and qualitative data is beneficial to the research, enhancing its strengths and compensating for the limitations of each method.

2.1. Context and Participants

The study is part of the RedTICPRaxis, an initiative that brings together a group of researchers to exchange experiences in the use of ICT in external practices or practicum, to generate a community of knowledge through the development of research projects that in turn promote the production of scientific knowledge. This network is led by members of the association Asociación para el Desarrollo del Prácticum y de las Prácticas Externas: Red de Prácticum (REPPE), a non-profit organization, whose spirit is to bring together researchers interested in analyzing the object of study is the phenomenon of the practicum or external practices, which in some cases is also called professional practices in some parts of Latin America.

RedTICPRaxis arose during the XV International Symposium on Practicum and External Practices entitled “Present and future challenges” which was held in Poio (Spain) in July 2019. This symposium is held every two years and between each edition a specific research project of collaborative networking is raised. Since its inception, there have been three major projects that have brought together researchers from all over Ibero-America. In the first biennium [28], the focus was on experimenting with a model of a knowledge community on good practices in the use of ICT in the practicum through video annotations. In the second biennium [29], we analyzed the technological diaries of internships used by both students and their tutors. In the third biennium, period 2023–2025, the contributions that artificial intelligence can make to this training experience are the object of study, specifically in the network where a total of 19 members participate. The number of members participating in the network had a significant impact on the sample, resulting in a limited number of participants. Ten members of the network from 8 universities in 5 Latin American countries (Argentina, Spain, Mexico, Peru, Portugal) participated in this study (

Table 1. The demographic and professional characteristics of the study participants.

ID	Country	University	Gender	Age	Teaching Experience	Research Area
1	Argentina	National University of Rosario	Woman	45	23	Mathematics Education
2	Spain	University of Granada	Woman	46	23	Teacher training
3	Spain	CEU Cardenal Spinola	Woman	55	30	Education
4	Spain	University of Vigo	Woman	56	30	ICT, Practicum
5	Spain	University of Jaén	Woman	33	3	ICT, Teacher training
6	Mexico	University of Guadalajara	Woman	51	24	Education, ICT, Finance
7	Mexico	University of Guadalajara	Woman	47	23	Education, ICT
8	Mexico	University of Guadalajara	Man	48	21	ICT
9	Peru	Catholic University Santo Toribio de Mogrovejo	Woman	47	27	Education, ICT
10	Portugal	ESE Instituto Politecnico de Porto	Women	60	13	ICT, Innovation

). Therefore, this is a purposive, non-probabilistic sampling strategy, in which the informants are deliberately selected because they meet the specific criterion of having experienced the model, which is relevant to the research. Of these, 90% were women and 10% men, aged between 33 and 60 years (M = 48.7, SD = 6.97).

Regarding the teaching experience of the participants, all were university research professors, with a trajectory ranging from 3 to 30 years (M = 21.7, ST = 7.75). Concerning their area of research, most confirmed education or educational technology. Despite the diversity of the sample, it is crucial to acknowledge the limited number of participants and the purposive sampling method was used to select the participants.

2.2. Model Design and Piloting

The design of the Collaborative AI-Integrated Model for Reviewing Educational Literature (CAIM-REL) model involves the collaborative work of a research group. After the formation of the team, it is necessary to pose research questions that will support the search for references for the systematic literature review. These will serve as a compass in the subsequent stages of the search for references [30]. Once the main question has been established, it is necessary to establish the search rules such as the terms, the inclusion or exclusion criteria and the databases which will form part of the review sample. It is also recommended to follow the PRISMA model throughout the process. When the initial sample of documents found is available, it is recommended to extract the metadata in a Research Information Systems (.RIS) file, to be imported by a set of reference manager applications; in our case it was essential to concentrate the works in the Mendeley manager and its subsequent import into the rayyan.ai tool used for the screening and labeling of the articles. Collaborative work was fundamental for the development of the tasks of classification and categorization of bibliographic information. Once the information was analyzed, the data were imported into a spreadsheet to be processed in qualitative analysis software such as MAXQDA and processed in VOSviewer for the construction and visualization of bibliometric networks. The scheme of this process can be seen in Figure 1 taken from the work of González et al. [31].

2.3. Instrument and Data Analysis

The questionnaire was designed ad hoc to find out the participants’ assessment of the artificial intelligence model used in the collaborative literature review. It consisted of a total of 25 items, divided into four sections (see Appendix A).

The first section focuses on collecting the sociodemographic data of the participants, such as country, university, gender, age, teaching experience and main area of research. The second section is structured in five items on previous experience and training in AI tools, with a response scale from 1 to 10.

The third section is based on the System Usability Scale (SUS) [32], translated into Spanish [33], which evaluates the effectiveness of the tool, the efficiency related to the effort and resources needed to meet its objectives, and the user’s satisfaction with the usage experience. This scale consists of ten Likert-type items with 5 response values (where 1 meant “strongly disagree”, 2 “disagree”, 3 “neither agree nor disagree”, 4 “agree” and 5 “strongly agree”). The scale score is obtained by applying a mathematical formula to convert the responses of the 10 items to a single, quantitative value [34]. The value is obtained by subtracting 1 from the score of each odd question (1, 3, 5, 7, and 9) and 5 from each even question (numbers 2, 4, 6, 8, and 10), subsequently, the values of the ten questions are summed and multiplied by 2.5 to convert the score to a scale from 0 to 100. The SUS has been utilized for the four tools (Consensus, Rayyan.AI, Mendeley, and Drive) employed in the collaborative model CAIM-REL. Cronbach’s alpha was used to assess the reliability of the scale, yielding an acceptable result of 0.70 in Consensus, a good result of 0.78 in Rayyan.AI, a good result of 0.81 in Mendeley, and a high result of 0.91 in Drive.

The fourth section is composed of five open-ended questions related to the identification of strengths, opportunities, difficulties and threats facing the AI model and the training received. The questionnaires were administered and completed online using a Google form, in which the privacy notice and informed consent were inserted, ensuring the anonymous and confidential nature of the responses.

Descriptive analyses of the scores of the items in the second and third sections of the questionnaire were performed using Microsoft Excel. The analysis of the qualitative information was performed with the MAXQDA 2024 program, using a deductive–inductive categorization model that integrated pre-established categories with open coding of the data.

The deductive coding process began with four categories: Strengths, Opportunities, Weaknesses, and Threats, and the study objectives. Subsequently, open coding was performed on the set of responses, allowing for the inductive generation of emergent codes. The validity of the coding system was reinforced through review by four experts who evaluated the relevance, clarity, and coherence of the proposed categories and codes to ensure inter-rater reliability. A percent agreement of 80% was achieved in the process, indicating a high level of reliability. Minor discrepancies were resolved, and the coding scheme was refined. The process concluded when theoretical saturation was reached, meaning that reviewing new units of analysis no longer generated additional codes.

3. Results

The evaluation of the design by the participants, as measured by the responses to the questionnaire is seen in Figure 2 which shows the mean of the responses regarding mastery of tools and training received on AI tools. The participants in the literature review model acknowledged having a high mastery (knowledge and use) of ICT (M = 8.3; SD = 1.25), although lower mastery was observed when considering specifically each of the AI tools.

In the case of mastery (knowledge and use) of AI tools (Figure 2), participants perceived a higher mastery of those oriented to education (M = 6.9; SD = 1.85) compared to those focused on research (M = 6.5; SD = 1.90). Very similarly, in the assessment of the training received on AI tools, it was found that the one aimed at education (M = 6.9; SD = 1.73) was superior to the one received for research purposes (M = 6.5; SD = 1.84).

Regarding the results of the System Usability Scale, Figure 3 shows the total mean score of the various tools used in the collaborative model for literature reviews. The tools Google Drive (M = 77.75; SD = 11.45) and Consensus (M = 70.05; SD = 16.49) show a higher level of usability than Rayyan.AI (M = 66.00; SD = 20.69) and Mendeley (M = 66.75; SD = 13.13). While the scores for the four tools demonstrated an adequate level of usability, participants considered Rayyan.AI to be the least usable. This could be related to unfamiliarity with the tool or the need for greater use, as highlighted in the qualitative comments.

The qualitative results were organized using a SWOT analysis (Strengths, Weaknesses, Opportunities, Threats) of the model and its relationship with the training of researchers.

Table 2. Encoding percentage the SWOT (Strengths, Weaknesses, Opportunities, Threats) analysis.

Code/Subcode	Percentages	Code/Subcode	Percentages
Weaknesses	8.85%	Threats	8.85%
Weaknesses > Technological	0.00%	Threats > Restriction of access to tools	1.77%
Weaknesses > Technological > Access restrictions	1.77%	Threats > Learning to use tools	0.88%
Weaknesses > Technological > Technical–Connectivity issues	0.88%	Threats > Reliability of results/biases	3.54%
Weaknesses > Skilles	0.00%	Threats > Constructive critical attitude	0.88%
Weaknesses > Skilles > Lack of knowledge	1.77%	Threats > Absence of threat	1.77%
Weaknesses > Skilles > Insufficient practice	0.88%	Threats > Adaptation	0.88%
Weaknesses > Skilles > Technical unawareness	2.65%
Weaknesses > Interpretation of results	0.88%
Weaknesses > Underutilization	0.88%
Strengths	8.85%	Opportunities	8.85%
Strengths > Guidance	0.88%	Opportunities > Ethical use	0.88%
Strengths > Efficiency in data management	3.54%	Opportunities > Automation and efficiency	1.77%
Strengths > Collaborative work	2.65%	Opportunities > Personalization	0.88%
Strengths > Process automation	1.77%	Opportunities > Data processing	5.31%
Strengths > Optimization	3.54%	Opportunities > Socialization	2.65%
		Opportunities > AI as assistants	0.88%

presents the percentages for each of the codes.

Within the strengths, five codes associated with this category were found (Figure 4); highlighting the model’s data management efficiency and task optimization as one of the advantages of implementing AI tools. Among the comments we found the following:

“Efficiency for the systematization of information, i.e., filter the articles most related to the object of study, or if it is for a literature review for those that meet the inclusion and exclusion criteria.”

(ID7, pos. 6, 2025)

Thus, the use of tools such as www.Rayyan.ai consolidated the analysis of the references, and above all helped to speed up the categorization work.

Another strength is the promotion of collaborative work as pointed out by participant ID10 “which facilitates collaboration among the members of the NETWORK and the systematization of information” (ID10, pos. 4, 2025).

The areas of opportunity of the model were coded by six codes (Figure 4), highlighting data analysis as one of the most frequent; this aspect is considered an innovation in the research processes. Among the most significant comments is “They allow analyzing and presenting data in a scientific way different from the traditional one”(ID2, pos. 5, 2025).

“Automation of the research, analysis and data generation process allows researchers to focus on data interpretation. Processing large volumes of information minimizes human error, facilitates decision-making, and speeds up research. Personalization, synchronization, interdisciplinary and international collaboration”.

(ID6, Pos. 5, 2025)

Another area of opportunity is socialization as mentioned by participant 3, “I perceive professional socialization and the invitation to be more sophisticated in what we do and generate as areas of opportunity (ID3, pos. 5, 2025)”. The use of tools encourages collaboration and shares new ways of sharing research results.

Among the difficulties, the following were found to be important codes: Skill Development, due to lack of practice, lack of knowledge of the tools and lack of technical knowledge. This is interrelated with the perceived threats, including technological dependence and the difficulty of adapting to digital environments, which evidences the need for continuous training. These limitations are reflected in the participants’ testimonies, who note “the need to receive training in order to make appropriate use of the tools for collaborative work and to use them efficiently” (ID7, pos 2–3, 2025), as well as that “it is necessary to have prior knowledge in order to learn how to use them” (ID9, pos 2, 2025).

Also, the one that has to do with the technological part, such as the fact that the tools require connectivity and access to the platforms since they require subscriptions. There is also reticence about the veracity in the interpretation of the results and the underutilization of the tools. This is reflected in comments such as the following:

“Need to receive training to make proper use of the tools for collaborative work and efficient use of the tools”.

(ID7, pos. 2, 2025)

Finally, the perceived threats are dependence, reliability of the results, lack of a constructive critical attitude for the use of tools, adaptation, and some also stated that they have not detected any threats.

Some comments were the following: “It is important to become accustomed to it and to rely on it.” (ID1, pos.3., 2025); “I do not know of threats but I do see as an imperative need to be a constructive critical professional, that is, who knows how to take and at the same time discern, all in the sense of strengthening and feedback”(ID3, pos. 3, 2025). The comparative graph of the SWOT analysis can be seen in Figure 4.

Another important aspect that was detected is the opinion on the training received and required to continue applying the tools and the model itself. In this regard, the participants mentioned that it was necessary to carry out the implementation of the model and that they require specialized training to make optimal use of the existing tools. The co-occurrence map by proximity of codes shows (Figure 5) the relationship between training with the need for training, that such training is practical and applied to concrete situations, that such training is related to the perception of various difficulties in the use of AI, such as lack of technical knowledge, insufficient practice and lack of knowledge of ways to apply AI for research.

4. Discussion and Conclusions

The analysis of the results obtained from the application to ten participating researchers has revealed that the design of the pilot model for collaborative review of scientific literature, created by the international network REDTICPraxis and based on AI systems, has influenced in three main areas: efficiency, the quality of the process and interaction between researchers. In terms of efficiency, the participants highlighted the speed of systematization and categorization of information offered by Rayyan.ai. They also found that the automation of routine tasks, such as article filtering, metadata extraction and category organization, significantly reduced the time required to build a solid literature base, as researchers focused their efforts on data analysis and interpretation and on building robust theoretical foundations.

Likewise, the model has facilitated a systematization of the review process. This is facilitated by the PRISMA model and the integration of software such as Mendeley, Google Drive and Consensus. The combination of these tools promoted methodological triangulation that enriched the quality of the analysis and minimized human error. In fact, an improvement in the quality of the process was observed, with effects on consistency and reliability in the selection of relevant studies, but also in the interaction between researchers coming from different countries and locations. Therefore, it should be noted that the tools facilitated communication, data exchange and co-construction of knowledge in real time, while promoting a spirit of active, reflexive, and international scientific community.

Regarding the usability of the tools, the results indicate their relevance for the model of collaborative reviews with artificial intelligence. It should be noted that, given the increasing availability of AI tools nowadays, it is a challenge for researchers to select the most effective ones for conducting literature reviews [18,35]. In this study, Consensus, Rayyan.ia, Mendeley, and Google Drive, all free options, have been highlighted as effective and efficient, and participants are satisfied with their use in the proposed model. It should be noted that the participants considered Rayyan.AI to be the least usable option, due to a lack of familiarity with the tool and the need for greater use. As ref. [18], Bolaños et al., indicates, the current generation of tools, when used effectively, can be very powerful. However, they may exhibit deficiencies in terms of usability and ease of use, which limits their adoption in the research community.

The integration of AI into collaborative systematic literature reviews could enhance the efficiency and methodological quality of the scientific collaboration process and interaction between researchers. This experience underscores the role of AI systems in contemporary educational research. Because of the above, the model represents good practice. However, the participants placed significant emphasis on the necessity of receiving technical training. The results indicate that a paucity of training was a significant challenge (Weakness) and that training was regarded as the principal factor contributing to difficulties in usage (Figure 5). In order to ensure effective AI training, it is essential to facilitate ongoing professional development for educators and researchers. This will enable them to adapt to new technologies. Finally, it should be noted that the participants indicate the importance of adhering to rigorous ethical principles. It is imperative that policies mandate rigorous data protection measures to ensure the security of personal information utilized in AI training.

However, despite the benefits, difficulties and threats were also detected, namely the lack of technical training in AI tools, as well as the dependence on platforms with paid versions. Another potential threat is the possibility of hallucinations and biases in AI systems. These challenges can be addressed by adopting a critical and reflective approach to AI-generated results, thereby ensuring scientific rigor.

As [36] Burger et al. said: “as with other technologies, it is probable that AI will not entirely replace human work but will integrate with it, generating new and more complex forms of human–machine interaction” (p. 40). The Collaborative AI-Integrated Model for Reviewing Educational Literature (CAIM-REL) is an example of such forms.

The study’s main limitation was the small sample size of 10 participants (teachers and researchers from the RedTICPRaxis network), despite the inclusion of expert teachers and researchers with substantial experience. Moreover, the preponderance of female participants and the inclusion of data from five countries constitute substantial limitations. In subsequent studies, the questionnaire will be administered before and following the utilization of the model to evaluate the perceived usability and the participants’ ratings of the model with greater precision. Furthermore, the model will be employed with larger and more diverse samples to strengthen external validity.