Educational Roles and Scenarios for Large Language Models: An Ethnographic Research Study of Artificial Intelligence

Abstract

This paper reviews the theoretical background and potential applications of Large Language Models (LLMs) in educational processes and academic research. Utilizing a novel digital ethnographic approach, we engaged in iterative research with OpenAI’s ChatGPT-4 and Google’s Gemini Ultra—two advanced commercial LLMs. The methodology treated LLMs as research participants, emphasizing the AI-guided perspectives and their envisioned roles in educational settings. Our findings identified the potential LLM roles in educational and research processes and we discussed the AI challenges, which included potential biases in decision-making and AI as a potential source of discrimination and conflict of interest. In addition to practical implications, we used the qualitative research results to advise on the relevant topics for future research.

1. Introduction

Artificial intelligence (AI) is defined differently by professional and academic communities. AI can be viewed as a replication of human cognitive functions through the use of technology, particularly computer systems [1]. AI systems can perform tasks previously restricted to intelligent organic beings [2]. Traditionally, separate AI systems were used to replicate the separate capabilities of living organisms, such as expert reasoning, recognizing speech and processing language, viewing and responding to the physical objects in the environment, etc. [3]. Large Language Models (LLMs) represent a novel form of AI, combining multiple functionalities of previously separate AI systems. LLMs can collect and analyze data from the environment using a multi-modal approach (i.e., collecting text inputs, images, videos, and other inputs). Their data analysis capabilities focus on producing natural language and communicating the outputs in a human-like form, using text output or human-sounding synthesized speech. Examples of such applications include chatbots, AI assistants, translation, and other popular services. Language models can detect, translate, forecast, or create text or other material because they employ transformer models and are trained on enormous datasets [4].

LLMs and other information and communication technologies (ICTs) have become widely accepted in education. They are commonly applied to check for plagiarism and research integrity, provide students with advice on enrollment and support, or provide other information to different stakeholders of educational institutions. LLM systems also provide different functionalities to online learning systems, such as learning analytics, individualized learning plans, automatic transcription of faculty lectures, etc. [5]. Those functionalities and applications are usually called AI in education (AIEd).

There are multiple research areas, as demonstrated by the bibliometric research [6]. They include student evaluation, prediction of critical performance indicators (including student performance, drop-out rates, developmental tracks, etc.), AI assistant tools (in descriptive terms and related to the type of assistance provided), and tools/approaches for managing student learning processes. Our theoretical overview (see Section 2) loosely follows these specialized fields of AI applications in education.

Although AI-powered tools and models have a significant potential to improve education processes and their performance, there is a pertinent risk of creating some forms of harm and discrimination, including issues related to security and privacy, loss of human decision-making, and contribution to an overall belief the AI can “work instead of us” [7]. This impression is based on a widely held belief that LLMs can independently write acceptable academic texts. However, research shows that the accuracy and integrity of using these models in scientific writing are currently unknown [8]. Weaknesses of LLMs included referring to obsolete or non-existent publications or simply making up facts, i.e., “hallucinating”. AI-generated answers present inaccurate or misleading data as reality [9]. This is due to the inability of most LLMs to perform real-time checks of their output, which is based on a vast amount of training data, and the ability to predict relationships among words, sentences, and paragraphs, rather than the ability of critical reasoning [10].

Based on the general characteristics of AI and its application in the educational domain, our research objectives were the following:

• To establish a general framework for the evaluation of LLMs in education, especially in the context of their current benefits and challenges and the extant literature, treating LLMs by using qualitative and ethnographic methods;
• To open a new direction for the role of AI in educational research by providing an AI-guided perspective concerning its own role in the educational process.

Since LLMs are expected to evolve quickly toward more comprehensive forms of Artificial General Intelligence (AGI) [10], innovative research methodologies based on structured conversations with different AI forms should be tested. When AI reaches the general intelligence level (i.e., as the current AI systems approach AGI knowledge and competencies), qualitative research with AI as a research participant, equal to human actors, will become a reality. Although such systems do not currently exist and cannot be predicted, exploratory qualitative research involving AI responses might be an appropriate step for analyzing the current state of AI technology and its applications in specific domains. Such an approach can also predict future research methods appropriate for social science research as AI becomes closer to the predicted AGI levels of cognitive competence.

Therefore, this study opens a potentially new methodological approach, which might be extremely useful in future social science research scenarios. In addition, we identify the opportunities for AI in education (AIEd) applications and relevant future directions.

2. Theoretical Background: Artificial Intelligence in Education (AIEd)

AI-powered adaptive learning systems work to alter curricula and pedagogical approaches dynamically, dependent on the real-time assessments of students’ performance and engagement [11]. These systems are referred to as Intelligent Tutoring Systems (ITSs) and have the potential to simulate one-on-one instruction, thus providing a “personal touch” to teaching and learning processes, regardless of the student enrolment size [12]. In addition to personalized learning experiences, AI can help students prepare for exams. LLMs seem to perform better than specialized AI tools, specifically trained to serve as discipline-specific virtual experts [13]. AI systems make updating educational materials possible in real time to align them with the latest labor market requirements needed to solve real-life problems and improve student employability [14].

LLMs, such as OpenAI’s GPTs and Google’s Gemini, are built on transformer architectures with the following key components: (a) a self-attention mechanism, which helps the model to evaluate the human language patterns in user inputs; (b) positional encoding, which helps position the user input into the wider context; and (c) feed-forward neural networks [15]. Training LLMs involves processing datasets that contain diverse and comprehensive language examples. The training process can be categorized into two stages: (a) pre-training, where the model learns to predict the next word in a sentence and (b) the finetuning of specific tasks [1,3].

LLMs offer numerous applications in the educational domain: (1) the already mentioned, ITSs, providing a personal touch to teaching and learning processes [16]; (2) AI-driven assessment tools providing immediate and personalized feedback to students [17]; and (3) the creation and enhancement of educational content, ensuring it is up-to-date [18].

The potential of AI to enhance learning experiences, provide personalized education, and streamline administrative tasks has been widely acknowledged. Several studies have confirmed that learning is a social endeavor, with interaction and teamwork as key components (see, e.g., [19]). However, supervised and encouraged online communication is required [20]. AI in education can enhance collaborative learning by encouraging adaptive group formation based on educational models, promoting virtual engagement, and summarizing discussions that a tutor may use [12]. Intelligent virtual reality and tutoring systems, game-based learning environments, and authentic virtual reality are used to guide and engage students. For instance, in virtual labs, virtual agents can take on the roles of peers, teachers, or facilitators [18].

AI technology can be included in learning activities to continuously assess student achievement as an alternative to stop-and-test methods. Moreover, these technologies can accurately forecast the likelihood of a student failing a course or an assignment [21]. According to Luckin et al. [16], there are three types of AI software solutions for education: personal tutors, intelligent collaborative learning support, and intelligent virtual reality. Moreover, Baker and Smith [22] analyzed AI-based technologies from three different angles, i.e., learners, teachers, and systems.

The integration of AI in education has been a subject of different theoretical perspectives [22], usually focusing on the benefits of AIEd [6,23], including the use of AI to focus on students’ cognition and meta-cognition [24], provide novel approaches to educational evaluation based on student activities [25], improve learners’ autonomy [26], and offer other forms of individualized learner support [27].

Ouyang et al. [17], in their systematic review (see also [5]), highlighted four basic purposes of AI application in higher education: predicting student performance; recommending teaching and learning resources; automatizing student assessments; and improving the learning experience. Salas-Pilco and Yang [28] also analyzed the use of AI in higher education in Latin America. They concluded that the most common uses of AI in higher education are related to the predictive modeling and analysis of educational content and providing personalized assistance in teaching and learning.

While the potential benefits of LLMs in education are substantial, there are also some ethical challenges: (a) fairness and equity in AI-driven educational tools [29]; (b) privacy and data security [6]; and (c) dependence on AI tools and reduction in critical thinking and problem-solving skills among students [24]. All these challenges can lead to potential conflicts of interests involving using student data collected by the AI tools for legitimate purposes, which could still be (mis)used for analytical or commercial purposes. In addition, decisions based on AI-enabled tools and algorithms could lead to harm or discrimination due to biases in the AI training data or AI algorithms.

3. SWOT Analysis

In order to provide a theoretical starting point for digital ethnographic research, we summarized the extant theoretical findings using the conventional analysis of Strengths, Weaknesses, Opportunities, and Threats of artificial intelligence in education (SWOT analysis of AIEd)—a simple and extremely popular method of assessing an organizational environment and its characteristics (see Table 1). Although this method is inadequate for theory building [30], its popularity and wealth of applications make it an indispensable tool in analyzing strategic developments and decision-making [31].

Table 1. SWOT analysis of AIEd.

Strengths AI tools that aid in data and literature analysis can be used to accelerate and streamline academic research. Embedding AI-based functionalities within electronic learning platforms helps optimize the learning experience, ultimately enhancing knowledge levels and improving grades. AI helps with administrative tasks like scheduling and grading, giving educators more time to research or teach. AI escalates pedagogical practices that enhance students’ learning, e.g., simulation- and project-based teaching.

Weaknesses AI systems can also exacerbate bias, which might already be present in their training data, further supporting unethical decisions and recommendations. The high costs of AI technology could potentially exclude smaller academic institutions with fewer resources. Investing in training educators and researchers to learn AI can be expensive.

Opportunities AI can bring different research fields together. Breaking down language barriers and promoting cross-cultural understanding makes research collaboration easier. When curricula are enhanced with AI, students can use their theoretical knowledge practically and meaningfully.

Threats Data security and privacy issues are particularly important since the heavy use of AI in education poses significant risks related to data breaches with unauthorized access. AI could replace administrative staff and some faculty, as it possesses a high potential for process automation. Consequently, the existing staff might resist, trying to protect their jobs. Some researchers and students may gain access to most AI resources at well-funded universities, while underfunded universities could have only limited support. Using AI as a tool for content creation or to inform research could also raise concerns about whether the existing higher education ecosystem is needed.

The SWOT analysis of AIEd aligned well with our exploratory research objectives since we did not seek to build novel theoretical foundations but rather to identify the relevant dimensions of AI’s influence on educational practices. These can be approximated to strengths and weaknesses, while the potential future AIEd applications might be explored by analyzing environmental opportunities and threats. Thus, the SWOT analysis of AIEd could be interpreted as a structured framework for analyzing the key issues and future research tasks of AIEd [32]. In addition, concerning the emergent use of LLMs in qualitative research, our approach fits well with future research, which will probably need to address the AGI levels of cognitive competence, currently predicted as a likely outcome of AI development.

Nevertheless, there are weaknesses and limitations to using the SWOT analysis as a foundation for further research, especially concerning its formulation based on the author’s subjective assessment of the literature. Other methods, more grounded in the literature, should be used as the AIEd literature matures. However, since the SWOT matrix was used as a starting point for discussions with AI actors, this simplistic approach was deemed adequate by the authors.

4. Materials and Methods

We employed an iterative approach to prompting the LLMs and developing the qualitative research material to identify the theoretical determinants pertinent to our issue. This method is robust in identifying how AI and LLMs influence educational interactions, procedures, and results. Since the technology side of ICT-assisted learning was well covered by the extant literature [33], it was important to understand the social function of emerging ICTs (such as AI) and their role in complex educational processes. Namely, AI algorithms are not isolated tools but are rather embedded into larger socioeconomic environments [34]. With the ever-increasing role of ICTs and AI in education, they can become the co-creators of educational tactics, interactions, and human stakeholders.

Using an iterative approach in leading the conversations with the LLMs, we tried to use the emergent capabilities for which they were not specifically trained. Although LLMs could not be evaluated as Artificial General Intelligence (AGI), it was hypothesized that some of their capacity for generalization and adaptation resulted from the patterns they picked up from large training datasets [35]. Our approach’s adaptability and iterative character allowed for the accommodation of unexpected emergent patterns and the creation of new theories in response to the uncertainties and dynamics of the AI field [36].

In addition, the extant literature does not seem to recognize LLMs or other AI tools as participants in qualitative research, equal to human ones. Nevertheless, Argyle et al. [37] and De Seta et al. [38] started to discuss the role of AI actors in qualitative research by introducing the idea of “algorithmic fidelity”, which describes the level to which an LLM might be able to mimic an actual human research participant [37]. The initial results seemed to support the potential role of LLMs in generating responses comparable to the human way of expression, mirroring actual human attitudes and behaviors. There are still some critical issues in using LLMs as research participants, as they might engage in “hyper-accuracy distortion”, where an AI tool is too keen on producing precise answers but does not regard the research context.

On the other hand, De Seta et al. [38] referred to including digital (AI) actors in ethnographic research, as they can generate text, images, and other ethnographic materials, which could be used within traditional qualitative research. Both studies positioned LLMs as dynamic actors, able to mimic and influence the extant sociocultural practices and processes. Adopting the notions of “algorithmic fidelity” and “synthetic ethnography”, we used LLMs as active research participants, able to change the socioeconomic and cultural contexts of the AIEd rather than as simple tools, mirroring the extant socioeconomic and cultural situation(s).

Our approach was inspired by digital ethnography [39], as treating conversations with LLMs as an ethnographic field site could reveal implicit understandings, biases, and potential co-creative roles that these models could have in educational processes. Consequently, we engaged in open-ended, iterative discussions with OpenAI ChatGPT-4 and Google Gemini Ultimate 1.0, the two most advanced LLMs now available to the public. All prompts and LLM responses can be found in the SM to this manuscript, located in the Supporting Materials. In a different context, we followed the approach used by Eysenbach [40], who used unedited ChatGPT conversation transcripts to describe an LLM’s capabilities applicable to medical education and provide a new research perspective on treating LLM output as an expert interview to be analyzed with conventional qualitative methods.

In qualitative studies, there are inherent limitations and biases in using AI actors, such as LLMs, as quasi-participants. Firstly, there is an immersion into the studied socioeconomic context, the communities, and their social practices, which is the case with the traditional ethnographic approach. Secondly, AI actors cannot communicate their lived experiences as human research participants; they only mimic human responses based on vast training datasets. Still, an AI tool can serve as a “mediated interlocutor”, with responses that can simulate authentic human experiences, as suggested by the previously discussed concept of “algorithmic fidelity”. While AI tools (still) do not have capabilities for genuine critical and reflective thinking, we believe that even simulated experiences are valid for performing innovative “synthetic ethnography” since AI actors discuss their perspectives concerning their participation and roles in educational and research processes. In addition, our approach provides a critical perspective on treating AI actors as potentially equal to human research participants, which might become a critical element of ethnographic research as AI continues its development toward the full AGI level.

In the first step of our analysis, informed by the previously presented SWOT analysis, we let the LLMs define the initial research questions. Since meaningful conversations with LLMs are facilitated by carefully constructing prompts, setting the context and researchers’ expectations, and guiding the LLM focus, we constructed the prompts for this step using the LLM developers’ recommendations [41,42]. We consulted the LLMs and iteratively refined a set of prompts focused on the critical evaluation of the baseline SWOT analysis and ethnographic perspective of LLMs and summarized the findings (see Supplementary Material File S1).

In the second step of data collection, the emerging research questions were further discussed with ChatGPT and Gemini to identify patterns and generate insights based on the iterative approach, i.e., the continuous (re)evaluation of their own output produced in the previous stages of the LLM conversations. The LLMs’ outputs were treated as textual artifacts for standard qualitative content analysis to offer insights into how LLMs perceive AI’s instructional roles and scenarios. The conversations with the LLMs were held over three weeks in April 2024, shortly after the public release of Google Gemini 1.0 Ultimate. As already mentioned, all the research materials are fully disclosed and available in the Supporting Materials. We used qualitative content analysis to identify the wider categories in the LLM output. We selected the most illustrative quotes in the manuscript and present them in the italic typeface.

To illustrate the previously described approach, we also prepared a flowchart, presented in Figure 1.

Figure 1. An illustrative overview of the research methodology.

5. Results

5.1. Identification of Ethnographic Research Questions (Preliminary Ethnographic Study)

As described in the previous section, we employed an iteratively developed set of LLM prompts to define the relevant research questions. In guided discussions with two leading LLMs, we obtained critical evaluations of the baseline SWOT analysis from both AI systems using the prompts presented in Supplementary Material File S1. While their reasoning can be reviewed by reading the chat transcripts (see Supplementary Material File S2), we have highlighted parts of the conversation relevant to identifying research questions for this study and future research in the field.

In the first interview, the AI system (labeled as LLM1) produced a straightforward but rather generic evaluation of our initial SWOT analysis, touching upon the need to fully address the social dynamics of the human–AI relationship, equity and access to AI technologies, and AI implications for the field. In Table 2, we present the quotes related to the extension of the previously presented SWOT analysis if digital ethnography were to be successfully used.

Table 2. Extensions of the SWOT analysis (according to quotes from LLM1).

Cultural and Social Dynamics: The analysis might not fully consider how cultural and social dynamics influence the adoption and use of AI in different educational contexts. Ethnographic research could uncover how societal values, educational philosophies, and institutional cultures shape the integration of AI technologies, revealing variances in acceptance and resistance across different communities.	Human–AI Interaction: Ethnographic studies can provide detailed insights into the everyday practices of human–AI interaction, highlighting how students, educators, and researchers engage with, interpret, and negotiate the outputs of AI systems. This includes examining the trust in AI, developing new forms of literacy to interpret AI-generated content, and reconfiguring roles and authority in educational settings.	Equity and Access: While the SWOT analysis touches on the digital divide, ethnographic research could delve deeper into how disparities in access to AI technologies affect educational opportunities and outcomes for marginalized and underrepresented groups. It can explore the lived experiences of these disparities, providing a more nuanced understanding of the implications for social equity and justice.
Implications for Pedagogy and Knowledge Production: Ethnographically examining the role of AI in pedagogy and knowledge production can reveal how AI technologies are transforming teaching methods, student engagement, and the construction of knowledge. This includes investigating how LLMs influence the development of critical thinking, creativity, and collaboration skills, and how they are reshaping disciplinary boundaries and research methodologies.

Note. Table contents are directly quoted from the LLM output.

The response from the second AI system showed emergent capabilities, which is in line with the previous theoretical discussions [43]. This AI system took a more critical viewpoint toward the role of AI in education and even suggested that our initial analysis was “technically correct, contextually shallow”, as it “lacks the ‘on the ground’ understanding of how these play out in the lived experiences of students and educators”. Suggested extensions to our initial SWOT analysis are presented in Table 3.

Table 3. Extensions of the SWOT analysis (according to quotes from LLM2).

Cultural impact: How does AI promote or hinder a culture of collaboration in research? Does its use reshape what “knowledge” is valued within education?

User experience: Beyond accuracy, do students find AI-generated explanations genuinely helpful? Do researchers trust the summaries enough to replace deep reading?

Human dimensions: Adoption: Who decides to use AI, why, and under what pressure? Impact on values: Does it conflict with how educators or researchers view their mission? Non-users: Who isn’t using AI and why?

Note. Table contents are directly quoted from the LLM output.

This AI system has continued to frame the tasks for future ethnographic research of the educational roles and scenarios for AI in education (see Supplementary Material File S2), using the wide categories of (a) adoption and user perception; (b) impact on practices; (c) values and power dynamics; and (d) focus on non-users. While the entire interaction with the system can be reviewed in Supplementary Material File S2, LLM2 suggested the research questions in Table 4.

Table 4. Relevant research questions for the main ethnographic study (according to quotes from LLM2).

Adoption and User Perception: Who within the educational system (administrators, teachers of specific subjects, research departments) are driving the adoption of AI tools? What are their motivations? What anxieties do educators and research faculty have about their roles changing due to AI use? Do they feel replaced or see it as a positive co-creation opportunity? How does AI use differ depending on a student’s socioeconomic background or the institution’s funding level?

Impact on Practices: How does the availability of AI-generated content alter how educators design assignments and evaluate student work? Does AI-enhanced efficiency in literature review lead to different kinds of research questions being asked, or merely a faster production of similar studies? How do students actually interact with AI tutors or “personalized learning”? Do they fall into the trap of over-reliance and passive learning?

Values and Power Dynamics: Does emphasizing efficiency and automation, often driven by AI adoption, conflict with traditional educational values like slow, critical thinking? Do researchers come to place an unhealthy amount of trust in AI outputs? How do they balance AI-generated insights with their own expertise? Does reliance on AI-powered platforms (often created by large tech companies) give those companies influence over the direction of research and what is considered “knowledge”?

Focus on Non-Users: Why do some educators actively choose not to use AI tools? Is it due to ethical beliefs, lack of training, or other reasons? How are underfunded schools or those in areas with poor internet access disadvantaged compared to their AI-equipped counterparts? Do students who do not utilize “personalized learning” platforms fall behind academically, and how does this impact social equity?

Note. Table contents are directly quoted from the LLM output.

Since the SWOT analysis of AIEd was used as the output for the two LLMs in the preliminary ethnographic study, there was a direct relationship between the AIEd SWOT categories and the emergent research questions (see Table 4). Both the perspective provided by the SWOT analysis of AIEd and the emergent perspective of LLM2 fit well with the theoretical notion of the sociocultural theory developed by Lev Vygotsky in the early years of the 20th century. Following Vygotsky’s theoretical perspective of learning [44,45], AI can be considered as a tool that shapes social and cultural interactions, as it depends on social interactions and cultural artifacts. Those interactions among the stakeholders of academic teaching and learning (students/learners, teachers/faculty, and other actors) still play a key role in acquiring new skills and knowledge, while now, they are not only mediated by social interactions and culture but also by new technologies.

Using the LLM2 suggestions, we decided to focus the subsequent LLM interviews on the four suggested dimensions for our digital ethnographic study and perform the LLM interviews by prompting the AI systems with the previously presented research questions (see full transcript in Supplementary Material File S3).

5.2. Results of the (Main) Digital Ethnographic Study

Discussions with the selected LLMs (reproduced in Supplementary Material File S3) were held in February 2024, with the two AI systems’ most powerful, subscription-only versions. Traditional content analysis was used to analyze the transcripts of the LLM interviews, with five common themes identified. We further present the LLM viewpoints related to their own “perception” of the applicable educational roles and scenarios in each of the identified categories, along with the relevant interview quotes:

• Adoption and user perception: Stakeholders of AI applications include educators, researchers, administrators, and students. The need for efficiency, individualized instruction, opportunities for effective research, and skill development drives them. According to LLMs, their motives vary greatly, with administrators focusing on operational efficiency and educators on the pedagogical advancement. As pointed out by LLM1, “The adoption of AI tools within the educational system is influenced by various stakeholders, each with their motivations”. LLM2 provided a more complex and comprehensive view of how educators might perceive LLMs in educational settings: “How educators might perceive me:

  ○

  A valuable resource: I could be a powerful tool for research, streamlining literature reviews, and aiding data analysis for both educators and students. I can also help with lesson planning, content creation, and provide personalized support to students.

  ○

  A potential disruptor: The potential to outsource some tasks to LLMs can both enhance education and raise concerns about academic integrity, student skill development, and changing roles in academia.

  ○

  An ethical question: My use raises discussions about originality, plagiarism, the potential for biases within my training data, and the need for responsible guidelines as AI integration increases.

  ○

  Collaborative tool: I’m best viewed as an assistant rather than a replacement for educators. The most effective applications of LLMs will involve human oversight and critical thinking skills”.

• Threats and concerns about AI in education: Threats and concerns about AI in education should be viewed in the context of its potential benefits, which were previously discussed in this paper. Probably the most important fear among many educators concerns being replaced by AI. While we can be optimistic about the capabilities of AI, the debate becomes more complex due to skills gaps, ethical dilemmas rooted in privacy concerns, and the risks of depersonalizing educational processes due to AI integration. Relevant interview quotes for this content dimension include the impact on evaluation practices raised by an LLM1: “Educators are redesigning assignments to focus more on critical thinking, creativity, and originality”. LLM2 analyzed perceptions of human stakeholders in the education and research sectors, as presented in Table 5.
  Table 5. Perceptions of human stakeholders in the academic sector (LLM2 quotes).

  Common anxieties: Fear of replacement: The biggest fear is of AI fully replacing human teachers and researchers. This stems from the misconception that AI can replicate the complex aspects of teaching, like empathy, critical thinking, and the ability to build relationships. Deskilling and devaluation: Concern exists about AI automating certain tasks that educators see as central to their profession (like assessment or lesson design). This could lead to a sense of losing control and skill decline. Diminished role of creativity: Some educators believe originality and creativity are inherently human and worry that over-reliance on AI-generated content will stifle student development in these areas. Algorithmic bias and lack of transparency: AI models are trained on data that might contain hidden biases. There’s a fear that uncritical use could perpetuate biases and raise ethical issues around fairness in grading or student recommendations. Student over-reliance: Educators fear students will become too dependent on AI, losing motivation to develop problem-solving and independent research skills.

  Positive co-creation outlook: Offloading repetitive tasks: Many educators see AI as a way to automate mundane tasks (grading simple assignments, attendance taking) allowing for more focus on higher-level teaching and mentoring. Enhanced personalization: AI-powered adaptive learning offers the potential for individualized feedback and pathways, something difficult to achieve with traditional methods for large classes. Opening new research avenues: For researchers, AI can open entirely new ways of analyzing data, revealing patterns and connections previously undetectable. Assisting with accessibility: AI tools can help students with disabilities access educational materials and participate more fully in the learning process.

  Note. Table contents are directly quoted from the LLM output.

Additional research questions were answered as the following:

• Impact on educational practices: When considering the effects of AI in educational practice, the issue of tyranny of efficiency could be raised. The focus on efficiency might undermine educational values such as critical thinking and individual ethics. At the same time, AI holds great promise in allowing the personalization of learning pathways and increasing student engagement. With more and more educational content being created by AI, there is an increased need to generally transform assignment evaluation and grading. More emphasis should be placed on integrity and ethics in writing assignments, student engagement, creativity, and critical thinking above rote learning.
• Research efficiency vs. human values: In terms of balancing the outcomes of AI applications in education, there is a delicate balance between achieving research productivity and preserving human values. AI can potentially expedite and extend the reach of literature reviews, but legitimate concerns about placing quantity over quality and research integrity exist. AI technology can facilitate interdisciplinary research and enable researchers to ask more holistic questions. Nonetheless, concerns have been raised about relying excessively on AI outputs and processes. The tension between the depth of critical engagement and the speed of information processing is part of a much larger discussion over what role education should play in the digital age. In this context, LLM1 warned of a possibly more general role of AI responsibility that would risk educational quality: “There is concern that an over-reliance on AI could lead to a one-size-fits-all approach to education”. Simultaneously, LLM2 pointed out that the potential biases and ethical issues are not a consequence of AI per se, but rather a consequence of the existing societal biases and issues: “It is crucial to remember that AI itself isn’t inherently biased. It’s the biases within the data used to train AI, along with unequal access, that create the disparity in how it is used”.
• Reasons for non-usage: Concerns over the impact of technology companies on research directions and the concept of knowledge were raised by the dependence on AI-powered platforms. Ethical concerns also play a role in some stakeholders resisting adopting AI systems in environments. For institutions lacking internet access expertise in AI technologies and funding for implementation, there is a risk of widening the AI divide, i.e., the lack of AI access (comparable to the classical notion of the “Digital Divide”): “Students from lower socioeconomic backgrounds or those in underfunded institutions may face significant barriers to accessing AI technologies” (LLM1). In addition, LLM2 further emphasized the social consequences of technological development by raising the following points:

  ○

  “Reinforced educational divide: The potential for AI to help bridge educational gaps is undermined if access isn’t equitable.

  ○

  Loss of potential: Talented students from underfunded backgrounds are hindered in reaching their full potential and contributing to the innovation economy of the future.

  ○

  Social impacts: The AI-driven economy will rely on AI-literate workers. Unequal access to AI education contributes to wider societal divides and lack of diversity in the tech sector”.

6. Discussion and Practical Implications

6.1. Discussion of Qualitative Research Results

The identified categories of AI user perception and adoption viewed vs. the ethical and practical concerns once again advised that the integration of AI into educational practices should carefully consider interactions between educational values and technological development. The common theme within all categories, emerging from the LLM interview transcripts, highlighted the need to balance technological advancements with preserving human connection and critical thinking in the learning process [46].

The LLMs highlighted oversimplifications and gaps in our theoretical understanding of AI’s educational impact. Although some of the extant literature [47] underscored the promise of AI concerning learning, adaptive curricula, and operational efficiency, there were also plentiful comments on implications, potential biases, and the risk of increasing the existing educational inequalities [48].

However, some novel insights can be drawn from the LLM interviews and the analysis of their content. Those include four implications for AI and the LLMs’ educational roles and scenarios:

• Routine tasks can be automated, freeing up more time for meaningful interactions with students, as already indicated by the literature on the nature of learning in the age of rapid AI development [49,50]. This also creates possibilities for innovation, such as AI-driven adaptive learning platforms and novel AIEd use cases.
• Too much reliance on automation using AI might diminish faculty precision, nuance, and creativity in creating educational materials and focusing on their specializations. Educational materials should emphasize students’ analytical thinking [51] as grading becomes oriented toward the assessment of reasoning and critical interpretation of facts [52]. Faculty also needs to engage in fact-checking continuously and transparently communicate with students about the role of AI in creating content.
• The contemporary curricula should also include AI literacy [53] to ensure that faculty and students can use AI tools efficiently and critically and follow technological developments.
• As a collaborative partner [54,55], AI positions teachers and students as thinkers in the educational process rather than just users of AI.

6.2. Practical Implications for Educational Practice Innovation

AI can help create more immersive and engaging classrooms by using simulations and virtual labs, which support practical learning experiences in STEM subjects. This could be especially useful in educational environments with low resource availability. AI-powered tutoring systems also provide real-time assistance outside the classroom, allowing students to receive immediate feedback and help using online learning systems and tools.

AI-based learning may bring a revolutionary change in educational personalization and assessment by evaluating each learner’s needs and individually changing the learning materials and evaluation approaches. Education becomes more inclusive by catering to different learning styles and paces in personalized learning paths [56]. This method can help students with special needs or those living in areas with less accessible education services, thus contributing to the SDG4 goal of ensuring quality education for all. All those processes can be scaled massively, on the level of regional and national education systems, due to the very low costs of AI systems, compared to the alternatives, involving hiring and training hundreds or thousands of new educational staff. Multiple case studies confirm the potential of AI in education systems across developing countries [57].

AI will undoubtedly improve other aspects of efficiency in educational systems by providing grading automation, offering targeted feedback, and adjusting educational resources on a mass scale based on identifying individual learners’ weaknesses. Our analysis also underscores the importance of AI for raising administrative efficiency within educational institutions, including scheduling, enrollment, and record-keeping. Multilateral institutions, such as the World Bank and OECD, have cited their assessments of the potential of AI to support efficiency not only in public education but also across the entire public sector of developing countries [58,59].

However, the new challenges regarding equity require policy solutions and measures to guarantee that more institutions have access to useful AI tools. The potential for biases and discrimination also requires new approaches to ensuring the ethical application of AI technologies.

This study concludes that AI tools and resources are not yet at a stage where teachers can best utilize them without appropriate professional development programs to support them. For AI in education to become successful, different stakeholders, including educators, administrators, technology developers, and policymakers, need to cooperate and address the identified issues of adoption and user perception, ethics and skills gaps, research efficiency vs. human values, and AI non-usage.

6.3. Limitations and Biases of Current Research and Future Research Directions

First and foremost, there is a need for additional research in the educational domain on how AI can redefine personalized learning and tailor educational materials and approaches to individual student needs. There is a critical need to assess how these AI capabilities can be employed massively to solve the critical issues of global educational inequalities. The efficiency opportunities offered by AI should be used to assist in removing the SDG4 gaps and provide support to understaffed regional and national education systems and learners in need. Educational research on these issues is critical and highly needed if AI is to assist the world in achieving the SDGs.

Future efforts should also focus on developing AI models trained with high diversity-related data in their training datasets. This approach can help reduce issues concerning potential gender, race, and socioeconomic discrimination. Future research should also look into transparent methods for auditing AI and correcting biased systems.

For increased trust and security, there is a need for additional research on mechanisms to make AI models in education more transparent. AI models should be able to describe how they make decisions and how different variables influence their results. This could be achieved by implementing the principles of the explainable AI (XAI) concept [60].

Given the critical importance of ensuring AI-generated results are reproducible and reliable, more research is needed on standardizing AI model training, testing, and validation protocols. Applicable standardization procedures should also be developed for ethical and governance frameworks addressing data privacy, consent, and responsible AI use.

7. Conclusions

This paper employed an ethnographic methodology to explore the uses and functions LLMs might fulfill in academic and educational roles. We offered a discussion of the AI and LLM landscape through interviews with two of the most advanced (and commercially available) LLMs. The findings described the positive and negative aspects of AI implementation in higher education and academic research by using the cognitive perspectives of the LLMs. They highlighted the significant potential of AI to increase research productivity, enhance personalized learning experiences, and reduce administrative overhead. Nevertheless, there were reservations about the implications of ethical biases and concerns, potentially diminishing the aptitude for critical thinking due to dependence on AI.

This study is methodologically innovative since it used traditional qualitative research methodology to analyze the transcripts of conversations with the two leading LLMs, which served as interviews with field experts. In this context, the emergent capabilities of LLMs were used to guide analysis through the iterative approach. Thus, we identified topics for further research, including AI’s impact on educational practices, values, and power dynamics and focusing on non-users of AI.

Further research is needed to validate and extend the findings presented in this study. In addition, there is a critical need to discuss the ethical aspect of AI and LLM applications in education and academic research, focusing on assessing issues with AI-generated content, AI bias, and the digital (AI) divide for non-users.