ChatGPT in Education: Challenges in Local Knowledge Representation of Romanian History and Geography

Abstract

The integration of AI tools like ChatGPT in education has sparked debates on their benefits and limitations, particularly in subjects requiring region-specific knowledge. This study examines ChatGPT’s ability to generate accurate and contextually rich responses to assignments in Romanian history and geography, focusing on topics with limited digital representation. Using a document-based analysis, this study compared ChatGPT’s responses to local archival sources, monographs, and topographical maps, assessing coverage, accuracy, and local nuances. Findings indicate significant factual inaccuracies, including misidentified Dacian tribes, incorrect historical sources, and geographic errors such as misplaced landmarks, elevation discrepancies, and incorrect infrastructure details. ChatGPT’s reliance on widely digitized sources led to omissions of localized details, highlighting a fundamental limitation when applied to non-digitized historical and geographic topics. These results suggest that while ChatGPT can be a useful supplementary tool, its outputs require careful verification by educators to prevent misinformation. Future research should explore strategies to improve AI-generated educational content, including better integration of regional archives and AI literacy training for students and teachers. The study underscores the need for hybrid AI-human approaches in education, ensuring that AI-generated text complements rather than replaces verified academic sources.

1. Introduction

The integration of ChatGPT into academic settings has generated significant discourse among learners and educators, particularly regarding its implications for academic integrity and learning. Recent research highlights both the benefits and challenges of using this generative AI tool in educational contexts. A thematic study on students’ usage of ChatGPT suggests that non-native English speakers tend to rely on the tool more than their native-speaking peers, primarily to enhance their writing skills in areas such as coherence and grammar and to streamline the writing process (Baek et al., 2023). This finding underscores the tool’s role in supporting students with language barriers, thereby promoting academic inclusivity.

Recent studies emphasize the growing influence of ChatGPT in academia, with its generative capabilities proving useful in writing tasks. Benichou (2023) demonstrates that ChatGPT can aid medical writing by producing clear and structured text, although expert review remains necessary to ensure accuracy. Similarly, Nugroho et al. (2024) find that English as a Foreign Language (EFL) students value ChatGPT’s grammar and translation assistance but remain cautious about inaccuracies and academic dishonesty. This tension between efficiency and integrity fuels ongoing debates about the appropriate integration of AI across various academic disciplines and contexts.

On the other hand, concerns regarding academic integrity have emerged as a key focus in the literature. Sullivan et al. argue that while ChatGPT expands access to educational resources, it also raises significant issues related to academic dishonesty, particularly in traditional assessment formats (Sullivan et al., 2023). The mixed reactions to ChatGPT’s academic role are echoed in various studies, highlighting the need for educators to rethink assessment strategies to mitigate potential misuse while harnessing the tool’s potential to enhance learning (Dergaa et al., 2023). For instance, Kumar suggests that instead of viewing ChatGPT as a threat, educators should explore its integration into teaching methodologies to strengthen students’ academic writing skills through structured learning approaches (Hs Kumar, 2023).

Another body of research highlights ChatGPT’s dual role in classrooms—it can serve as an AI-powered study companion, while also presenting opportunities for misuse. In computer programming education, Humble et al. (2023) warn that unless assignments are redesigned, ChatGPT can trivialize coding tasks, effectively enabling academic dishonesty. They advocate for curriculum adjustments that position AI as a learning aid rather than a shortcut to solutions. While their focus is on programming, their insights extend to other fields, underscoring the need for educators to balance ChatGPT’s capabilities with academic integrity.

A systematic review by Imran and Almusharraf further explores ChatGPT’s transformative potential in writing programs at the higher education level, recommending its use as a tool for developing writing skills (Imran & Almusharraf, 2023). Wang’s research supports this perspective, demonstrating that ChatGPT can significantly enhance EFL students’ academic writing by correcting grammatical errors and improving text coherence (Y. Wang, 2024). These findings suggest that ChatGPT serves as a valuable writing assistant, particularly for students lacking confidence in their writing abilities.

Additionally, research on specific student groups, such as medical undergraduates, reveals that ChatGPT improves writing efficiency and positively influences students’ attitudes toward academic writing (J. Wang et al., 2024). This aligns with broader arguments that ChatGPT enhances learning by providing immediate feedback and support, fostering a more engaging educational environment. However, Rueda et al. stress the importance of careful implementation, advocating for a balanced approach that maintains the educator’s role in guiding student learning while integrating technology (Montenegro-Rueda et al., 2023).

Pawar et al. further illustrate ChatGPT’s dual impact by analyzing both its benefits and drawbacks in academic performance (Pawar et al., 2023). They argue that while ChatGPT can enhance learning experiences, it should not replace traditional teaching methods or human oversight. Faisal echoes this sentiment, identifying improved engagement and inclusivity as key advantages of ChatGPT in higher education (Faisal, 2024). However, concerns about superficial understanding and reliance on AI-generated content highlight potential risks to students’ critical thinking skills and depth of learning.

The effectiveness of ChatGPT in academic settings also depends on prompt engineering. Shehri et al. emphasize that well-structured prompts can greatly improve the quality of AI-generated responses, enhancing student learning outcomes (Shehri et al., 2023). This underscores the need to train educators in effectively utilizing AI tools to maximize their educational benefits while maintaining academic rigor.

In terms of specific applications, Herda et al. explore student perceptions in the Philippines and Indonesia, noting that ChatGPT supports academic writing by helping students generate coherent and relevant text (Herda et al., 2024). Similarly, Tang highlights ChatGPT’s effectiveness in providing timely feedback and aiding vocabulary development for second-language learners (Tang, 2023). These findings suggest that ChatGPT plays a crucial role in language acquisition and writing skill development among diverse student populations.

Liu et al. argue that despite its advantages, the technology also presents challenges for educators, particularly in maintaining academic integrity and fostering meaningful learning experiences (Liu et al., 2023). Mahama et al. echo this concern, warning that the widespread adoption of generative AI tools like ChatGPT could threaten human creativity and academic honesty (Mahama et al., 2023). The authors advocate for a nuanced understanding of responsible AI usage in academic contexts.

Beyond writing efficiency, ChatGPT-generated outputs often lack the depth and specificity required in social science fields. Kindenberg (2024) contrasts AI-generated historical narratives with student-authored essays, concluding that while AI produces polished text, it fails to capture regional nuances and emotional depth. Similarly, Carrasco Rodríguez (2023) acknowledges ChatGPT’s utility in generating foundational course materials for Early Modern History but cautions that educators must adapt and verify AI-generated content for accuracy and contextual relevance. This concern is particularly pressing in fields where non-digitized local sources remain relevant, such as Romanian history and geography.

The growing interest in ChatGPT’s role in academic research has also prompted discussions on its applications in essay writing and literature summarization. Rahman et al. caution that while ChatGPT offers valuable support in research, its outputs require careful verification due to the risk of inaccuracies (Rahman et al., 2023). Sallam reinforces this point, emphasizing the need for critical evaluation of AI-generated content in scientific writing to maintain research quality (Sallam, 2023).

Another important dimension is students’ perceptions of ChatGPT. Zhang et al. (2024) use a mixed-method approach to identify different student perspectives, some are highly enthusiastic, while others remain cautious. Meanwhile, Nugroho et al. (2024) note that students do not blindly trust ChatGPT; they often fact-check its outputs and apply personal judgment. These findings suggest that digital literacy, readiness, and individual dispositions shape how students integrate ChatGPT into their coursework.

Chukwuere calls for ongoing discussions on the benefits and challenges of AI in academic writing, advocating for collaboration between educators and students to shape AI’s future in education (Chukwuere, 2024). Similarly, Al-Sofi suggests that while students generally view ChatGPT as beneficial for improving writing skills, they also recognize risks such as plagiarism and an over-reliance on AI (Al-Sofi, 2024).

To contextualize these discussions within broader technological shifts, M. Wang and Guo (2023) trace the evolution of media in education, from oral traditions to digital media, arguing that ChatGPT represents a shift toward AI as a “knowledge producer”, rather than merely a carrier of information. They recommend enhancing digital literacy among both students and teachers and adopting conversational pedagogical methods to integrate AI responsibly, particularly in regions with uneven digital access and resource availability.

Considering the various ways ChatGPT might be used in education (as a tutor, as writing support, and as a content generator), this study targets three research questions, focusing mainly on the content generator role, used by students to generate text for assignments. The first question is to what extent does ChatGPT accurately represent specialized, local knowledge in Romanian history and geography when responding to factual essay prompts? The second research question asks the following: how do the inaccuracies and omissions in ChatGPT’s outputs potentially affect its efficacy in different educational use cases (tutoring, writing support, or content generation)? The last research question aims to answer the following: What are the implications for educators and students, particularly in verifying the accuracy of AI-generated texts, and how might these insights inform best practices of future AI literacy training?

By focusing on factual accuracy (largely aligned with “reproductive knowledge” in Bloom’s taxonomy), this study also opens a discussion about whether (and how) future research might examine more complex skill domains, such as application, analysis, or evaluation, in relation to AI-generated content.

2. Materials and Methods

This study adopts a document-based methodology to evaluate ChatGPT’s coverage and accuracy in Romanian history and geography topics for which local references may not be entirely digitized. Although no student participants are involved, the approach simulates the process of consulting authentic regional materials (e.g., brochures, archived monographs) to test ChatGPT’s ability to capture local details. The study was carried out based on the secondary school (gimnaziu) curricula, focusing on two subject areas: Romanian History and Geography. The model used was the ChatGPT free version (GPT-3.5).

To address RQ1 and RQ2, the authors designed two tasks that represent typical school assignments in Romanian history and geography. While these tasks primarily assess factual retrieval and basic comprehension, they also reflect real scenarios where students might rely on ChatGPT for quick solutions. The focus on factual accuracy is intended as a first step in investigating ChatGPT’s limitations; future work can extend to higher-order tasks.

The authors focus on two subject-area topics drawn from the 8th-grade Romanian curriculum: Geto-Dacians for History and the Mesecăniș Pass for Geography. The Geto-Dacians represent an important era in Romanian history, preceding the Roman conquest and showcasing distinct cultural and social structures. To reflect the local, partial, or non-digitized material that teachers often rely on, for the history assignment, three offline sources were selected (Drugas, 2020; Giurescu & Giurescu, 1971; Pârvan, 1928). These documents are not readily available online, thus illustrating how relevant historical content can remain overlooked by AI models that rely on digitized data.

For the geography component, the Mesecăniș Pass was chosen because it exemplifies the 8th-grade emphasis on Romanian landforms, local climates, and infrastructural considerations. Three documents were identified to simulate the material students or teachers might consult in physical form. In the first and second documents, geography works that provided detailed descriptions of the pass boundaries, infrastructure, hiking trails, and microclimates specific to Mesecăniș (Rusu, 2002; Surd, 2008), which also contains a meteorological bulletin from the mid-1990s that recorded temperature variations and precipitation levels associated with the pass. Third is a set of topographical maps from the Romanian government, a similar one from the USSR Army, and digital maps available online were consulted to extract relevant data in order to determine the boundaries of the pass and the main transport routes (Beleaua, 2006). By combining both history and geography topics, this study preserves the realistic challenges of relying on local references that are not readily accessible to large language models, thereby offering a robust test of ChatGPT’s capacity to capture nuanced or offline information.

For the history class, the authors developed a short-essay task on the Geto-Dacians, aiming to assess whether ChatGPT could address specialized content regarding ancient social structures, local archival sources, and region-specific tribal details not readily found in digitized collections:

Redactați un scurt eseu care să abordeze patru subteme despre organizarea geto-dacilor. În primul rând, descrieți modul în care era organizată societatea dacică și evidențiați ierarhia socială și responsabilitățile diferitelor categorii. În al doilea rând, precizați care este principala sursă istorică în care este descrisă Dacia imediat după cucerirea romană, menționând autorul și contextul documentului. În al treilea rând, enumerați cele douăsprezece triburi indicate în această sursă despre care se știe cu certitudine că erau dacice și prezentați, pe scurt, zonele geografice pe care le locuiau. În încheiere, descrieți zona geografică în care locuiau buridavensii și localizați cetatea de la care și-a luat numele acest trib.

The English translation is as follows:

Write a short essay addressing four subtopics about the organization of the Geto-Dacians. First, describe how Dacian society was organized and outline the social hierarchy and responsibilities of different categories. Second, identify the main historical source describing Dacia immediately after the Roman conquest, mentioning the author and the context of the document. Thirdly, list the twelve tribes from this source that are known with certainty to have been Dacian and briefly describe the geographical areas they inhabited. Finally, describe the geographical area in which the Buridavenses lived and locate the fortress from which this tribe took its name.

In addition, for the geography component of our study, the authors developed a short-essay task centered on Pasul Mestecăniș, designed to evaluate whether ChatGPT could accurately capture the local topographical, climatic, and vegetative nuances typically found only in offline Romanian sources:

Redactați un scurt eseu în care să caracterizați din punct de vedere geografic Pasul Mestecăniș. Enumerați vârfurile muntoase ce delimitează trecătoarea, menționând altitudinea aproximativă și poziționarea lor. Continuați cu o descriere a principalelor căi de acces și a modului în care acestea influențează activitățile economice și turistice din zonă. Prezentați caracteristicile climatice relevante, precum temperatura medie anuală sau tipul de precipitații, și evidențiați vegetația specifică, subliniind modul în care factorii de mediu determină flora locală. Oferiți o perspectivă mai largă asupra situației geografice a Pasului Mestecăniș, în relație cu zonele geografice învecinate.

The English translation is as follows:

Write a short essay geographically characterizing the Mestecăniș Pass. List the mountain peaks marking the pass, their approximate altitude, and position. Continue with a description of the main access routes and how they influence economic and tourist activities in the area. Outline relevant climatic characteristics, such as average annual temperature or rainfall, and highlight the specific vegetation, emphasizing how environmental factors determine the local flora. Give a broader perspective on the geographical situation of the Mestecăniș Pass in relation to neighboring geographical areas.

Both assignments were reviewed and validated by teachers at Școala gimnazială 280 in Bucharest, ensuring alignment with the official 8th-grade curriculum and local educational standards.

After formulating these assignments, the study gathered ChatGPT outputs by inputting each prompt in separate sessions. In each session, the generated text was saved in its entirety. Given the fact that the experiment is set to mimic the students’ attempt to obtain a fast answer, after the essay was produced, there were no follow-up clarifications. To assess the completeness and accuracy of ChatGPT’s responses, the authors reviewed the compiled local references (three historical works, one monograph, archive records, and topographical maps) and compared them against the AI-generated content. All the discrepancies, omissions of key local details, or instances of factual inconsistency were noted. Following a document-based content analysis approach, each ChatGPT response was examined for (1) coverage of core thematic elements (e.g., naming correct Dacian tribes or identifying specific mountain peaks), (2) accuracy of contextual details (social organization, climate data, economic activities), and (3) inclusion of local nuances absent from common web-based sources. This approach speaks directly to RQ3, as it illustrates the potential for teachers and learners to rely on (and propagate) incorrect information if they assume ChatGPT’s outputs are comprehensive. Observing how the model’s factual discrepancies arise in a realistic classroom scenario highlights the importance of verifying AI responses, thus providing insight into how educators must adjust their practice to mitigate the risk of misinformation. Because no student data were collected, this study did not require institutional review board approval, and all references were handled according to standard ethical norms for secondary data.

3. Results

This section offers a detailed look at the AI-generated essays collected. The authors examine how ChatGPT handles specialized local knowledge, bearing in mind the core questions regarding factual accuracy and the broader instructional implications for both teachers and students.

3.1. History Assignment

A major point of divergence emerged when ChatGPT attempted to identify the twelve tribes considered “sigur dacice” (i.e., certainly Dacian), as referenced in the local historical works. While the offline sources unequivocally listed the following groups—Predavensii, Biefii, Albocensii, Saldensii, Ratacensii, Buridavensii, Potulatensii, Keiagisii, Costobocii, Caucensii, S(i)ensii, and Piefigii—ChatGPT’s output amalgamated these with or substituted them for other names (e.g., Roxolanii, Apulii, Bieștii, Moesii, Tiarageții, and even Dacii liberi). A significant misalignment emerged when ChatGPT identified Dacii liberi (i.e., free Dacians) as one of the “twelve Dacian tribes.” The label Dacii liberi is not a single tribe name; rather, it represents a collective term describing the various Dacian groups that continued living independently after the Roman conquest. This discrepancy underscores the tendency of large language models to merge partially correct information into seemingly coherent lists. In cases where specialized local references remain unscanned, AI-generated text may present a broad historical label like Dacii liberi as a single tribe, misrepresenting the structure of Dacian society. Some of the tribes listed by the LMM model appear in the historical records, but not within the same “sigur dacice” context established by the historical references. This mismatch suggests that ChatGPT may have drawn from partially related or regionally generalized sources that conflate various ancient populations, rather than the specialized offline materials attributing specific territories and historical significance to each of the Dacian tribes. As a result, the “twelve tribes” ChatGPT enumerated only loosely aligned with the academically recognized listing of the Dacian tribes. Similarly, all the required information was partly based on historical sources, and the rest was compiled from general knowledge, adapted to seem specific to the Dacian Society. Although the model briefly referenced the fortress from which the Buridavenses take their name, it neither identified it as ‘Buridava’ nor properly located it, incorrectly placing it near Călărași—over 300 km from its actual site south of Râmnicu Vâlcea.

Table 1. ChatGPT’s performance on the four subtopics for the history assignment.

Subtopic	Coverage	Accuracy	Local Nuances	Observations
Organization of Dacian society	Partial	Moderate	Limited	ChatGPT mentioned major social strata but lacked certain details found in local monographs (e.g., specific responsibilities of priestly or noble classes). The text was fluent but omitted references to lesser-known ranks.
Main historical source describing Dacia after Roman conquest	Partial	Low	Limited	ChatGPT mentioned Caesar’s writings, and it did not reference Ptolemy’s more systematic “radiography” of post-conquest Dacia. The AI’s focus on Caesar suggests a reliance on more commonly digitized Roman-era sources rather than specialized offline materials.
Twelve tribes known with certainty to be Dacian	Partial	Low	Very limited	ChatGPT listed some recognized tribes, but conflated or omitted others (e.g., Dacii liberi was incorrectly listed as a single tribe). The local archival listing of lesser-known tribes was almost entirely missing.
Geographical area of the Buridavensii and the fortress from which the tribe name derives	Low	Low	Very limited	The model provided approximate locations but contradicted offline resources on the precise region. The fortress localization was inaccurate and the name was generic or absent, suggesting the AI lacks direct access to specialized local data.

compares ChatGPT answer performance, measuring the coverage, accuracy, and local nuances, comparing the actual historical evidence with the generated text. Such differences illustrate how large language models can provide polished text that lacks fidelity to local scholarship, an outcome likely driven by the limited digitization and integration of specialized Romanian archives in mainstream data repositories.

3.2. Geography Assignment

ChatGPT’s generated essay provided an overview of Pasul Mestecăniș that seemed coherent, referencing its approximate altitude, nearby peaks, part of the Rarău mountain range, and the fact that the pass functions as a link between regions of northern Moldavia and southern Bucovina. However, a closer comparison with offline and online sources revealed notable omissions and inaccuracies in areas such as specific elevations, the precise alignment of surrounding peaks, and the diversity of side routes. For instance, ChatGPT mentioned the southern boundary peak as “Vârful Mestecăniș (1372 m)”, whereas the regional sources list multiple ridges surpassing 1300 m on both sides of the pass, including Vf. (i.e., Vârful, meaning peak) Arșița (1326 m) and Vf. Oala (1334 m), was identified in at least three online and offline sources (see Figure 1).

A notable discrepancy emerged regarding Vârful Mestecăniș, which ChatGPT placed south of the pass at an altitude of 1372 m. However, all the sources and the topographic maps consulted clearly position Vârful Mestecăniș on the northern side of the entry into the pass, with an actual elevation of approximately 1291 m, as seen in Figure 2. Furthermore, ChatGPT referenced “Vârful Ciucașul” as the northern boundary of the pass; none of the online and offline documents mention a Ciucașul peak in this specific region. The AI’s summary also offered generic climate patterns, emphasizing cold winters and moderate summer rainfall, but rarely aligned with the more detailed meteorological data indicating averages of up to 700–800 mm of annual precipitation, or the extended period of snow cover (120–150 days) noted in the local sources.

Additionally, ChatGPT mentions that DN18 road passes through the area, when, in fact, the road name is DN17 (with DN 18 just west of the pass). Additionally, it did not specify the alternative routes that circumvent blocked sections, an aspect thoroughly documented in the offline materials, which describe at least three alternative itineraries. Similarly, mentions of local tourism assets remained vague, focusing on broad attractions (e.g., hiking) without acknowledging smaller-scale lodging, agritourism, or nature reserves that are an important part of the local development.

These inconsistencies point to a reliance on incomplete or misaligned digital data, as opposed to the more precise and regionally verified cartographic sources available in local archives. Taken together, these findings suggest that ChatGPT’s text, although serviceable as an initial outline (not taking into consideration the errors regarding the boundary), omits the finer points of geographic, meteorological, and infrastructural detail, points that are readily available in region-specific, non-digitized sources.

Table 2. ChatGPT’s performance on the subtopics for the geography assignment.

Subtopic	Coverage	Accuracy	Local Nuances	Observations
Pass boundary	Partial	Low	Limited	ChatGPT response included two peaks, one of which (Vârful Ciucașul) does not exist in any known geographical or cartographic sources, while the other (Vârful Mestecăniș) was incorrectly identified as the southern boundary of the pass and assigned an incorrect elevation of 1372 m, rather than its actual recorded height of 1291 m. The AI omitted key peaks that form the actual geographical boundaries of the pass.
Infrastructure	Partial	Low	Limited	Correctly identified that a national road crosses Pasul Mestecăniș but inaccurately stated that the main transit route is DN18, rather than the actual DN17 (E58). The model also omitted alternative secondary roads and bypasses that local authorities recommend during heavy snowfall or congestion.
Climate	Partial	Low	Limited	Provided a general description of the climate, correctly noting that it features a mountain climate with cold winters and moderate summer precipitation. Failed to mention more specific climatic data available in local meteorological records, such as the annual precipitation range and the snow cover duration. Did not acknowledge the high incidence of sudden temperature drops and fog formation, which are well-documented seasonal phenomena in the pass.
Tourism and economy	Low	Moderate	Limited	Mentioned its role in tourism, trade, and transportation, but the analysis remained overly generalized compared to local sources. ChatGPT mentioned hiking and winter sports, and it did not include specific local attractions, accommodations, or protected nature reserves frequently cited in the offline sources. Economic references were also superficial, omitting the impact of forestry, livestock grazing, and small-scale commerce that contribute significantly to the regional economy.

analyses ChatGPT’s answer by measuring the coverage, accuracy, and local nuances, comparing the generated text with the actual geographical data. The differences illustrate how large language models can provide text that seems coherent but lacks fidelity to actual data.

Overall, the analysis of ChatGPT’s responses for both history and geography revealed significant discrepancies in factual accuracy, local specificity, and source alignment. The model generated coherent and structured texts, but its reliance on widely digitized information led to notable omissions and errors, particularly in cases where local references remain largely unavailable online or are not present in digital mainstream data channels. These findings highlight the limitations of AI-generated educational content in subjects requiring region-specific knowledge, warranting further discussion on its implications for educational use.

4. Discussion

The results of this study indicate that while ChatGPT can produce coherent and structured responses to academic prompts, it exhibits significant limitations in accuracy, specificity, and contextual depth when dealing with non-digitized, localized knowledge.

In the effort to address RQ1, this study demonstrates that ChatGPT struggles to deliver fully accurate or context-rich responses about Romanian history and geography when the material relies heavily on non-digitized or under-digitized sources. Although the AI-generated text appears fluent and well structured, it omits specialized details, such as lesser-known tribal designations or precise cartographic data, reflecting training biases toward widely available, digitized information.

A major issue identified in both history and geography assignments was ChatGPT’s reliance on widely digitized sources, which excluded details present in offline materials. In the history prompt, ChatGPT misrepresented the list of twelve Dacian tribes, introducing names that were either partially correct, misplaced, or completely absent from historical monographs. Additionally, its failure to accurately reference Ptolemy’s account of Dacia, instead overemphasizing Caesar’s writings, illustrates how large language models prioritize widely available historical sources while neglecting specialized, region-specific texts that remain undigitized.

In the geography task, ChatGPT exhibited significant errors in topographical accuracy, infrastructure identification, and climate descriptions. The model misidentified the main road traversing Pasul Mestecăniș, incorrectly referring to DN18 instead of DN17 (E58), and it failed to recognize key bypass roads necessary for transportation in harsh winter conditions. Furthermore, its geographical boundary descriptions contained errors, with incorrect peak placements, fabricated names (e.g., Vârful Ciucașul), and height discrepancies of up to 80 m. These issues highlight how AI-generated geographic summaries tend to generalize based on common knowledge, rather than integrating precise, map-verified data.

The observed errors raise concerns about the reliability of ChatGPT as an educational tool, particularly in subjects where factually precise and contextually rich content is required. While studies such as Nugroho et al. (2024) suggest that students frequently fact-check AI-generated content, the findings of this study indicate that errors in historical facts and geographical data could mislead learners who lack access to alternative sources for verification. The inaccuracies in Dacian tribal classifications, the location of Buridava, and the topographical profile of Mestecăniș Pass suggest that AI-generated text should be used with caution in educational settings, particularly when dealing with less-digitized cultural and historical contexts.

The implications for different educational uses (RQ2) were divided into two use cases.

As a tutor or writing support, ChatGPT can provide quick overviews or suggestions for structuring essays. However, students must be taught to fact-check the language model outputs against reliable sources—particularly for region-specific topics where AI-generated content can easily mislead.

As a content generator for lesson materials, teachers seeking lesson outlines or worksheets should verify the AI-produced facts through official curricula or local textbooks. In history or geography courses especially, the tool’s incomplete references could introduce errors into classroom materials.

One key takeaway from this study is that ChatGPT struggles to access and process non-digitized, regionally specific knowledge, even when such knowledge is partly available online, to accurately respond to a given prompt. This limitation stems from biases in AI training data, where Western and widely digitized sources are overrepresented, while localized historical and geographic materials remain largely inaccessible to AI models.

The authors also considered the potential impact on educational practice under RQ3, specifically, how ChatGPT’s gaps might affect teachers’ and learners’ reliance on AI-generated content. If teachers are unaware of these shortcomings or fail to provide students with the proper tools for critical evaluation, learners might use incorrect or incomplete information uncritically, thereby solidifying misconceptions and undermining their overall mastery of course material. As ChatGPT becomes more common in classroom tasks, such errors may propagate further if educators themselves are untrained in verifying AI outputs, potentially leading to inaccurate lesson plans or assessments. By recognizing these gaps, the academic community can drive the development of targeted AI literacy programs, equipping both teachers and students to appraise AI-generated text carefully—cross-referencing key facts with reliable, regionally specific sources and engaging in deeper critical thinking. This approach not only guards against misinformation but also helps foster metacognitive skills and accountability in the learning process.

While it is unsurprising that ChatGPT struggles where its training data are limited, the empirical demonstration of these failures is valuable because it illustrates the practical consequences in an authentic educational context. Students or teachers might assume ChatGPT is comprehensive and inadvertently perpetuate errors. Recognizing these pitfalls helps shape guidelines for responsible AI use in schools.

Given the findings of this study, future research should explore several key areas as follows:

• Digitization of Local Educational Content—Since ChatGPT’s accuracy is largely dependent on the availability of digitized information, efforts should be made to increase the digitization of local historical and geographical materials, making them accessible for AI training. This could help improve AI-generated outputs for regional and culturally specific topics.
• AI-assisted Learning with Teacher Oversight—While ChatGPT has proven useful for language learning and writing assistance (Imran & Almusharraf, 2023), its role in history and geography education should be approached differently, integrating AI-generated content with teacher validation to correct factual inaccuracies. Educators should develop structured AI literacy training to help students critically engage with AI-generated responses.
• Customization of AI Models for Educational Use—Future research should explore how large language models can be fine-tuned with regionally specific data, ensuring that AI-generated educational content aligns more closely with localized historical and geographic knowledge. This could involve collaborations between AI developers, educational institutions, and cultural heritage organizations.
• Exploration of how ChatGPT responds to prompts requiring higher-level cognitive processes, for instance, analyzing historical cause–effect relationships or evaluating different geographical development plans. Investigating how effectively AI can support such higher-order thinking would offer deeper insight into its educational utility.

The findings of this study have direct implications for educators, curriculum developers, and policymakers. While ChatGPT provides a valuable tool for enhancing student engagement and generating text-based responses, its use in subjects requiring high factual accuracy and localized knowledge should be carefully monitored. Educators should supplement AI-generated content with physical archives, expert knowledge, and region-specific textbooks to ensure completeness and accuracy. Assessment methods should adapt to account for AI’s strengths and weaknesses, emphasizing critical evaluation skills rather than mere content reproduction. Policymakers should consider AI literacy initiatives that prepare students to engage with AI-generated text critically, enabling them to distinguish between verifiable facts and algorithmic assumptions.

While this study offers valuable insights, it is important to acknowledge its limitations. Since the study focused exclusively on document-based analysis, it does not account for how students might fact-check or refine ChatGPT-generated responses during actual learning tasks. Future research could incorporate student engagement and feedback to assess how learners interact with AI-generated content. The findings are specific to the Romanian educational context, particularly topics with limited digital availability. While similar trends may apply to other regional and under-digitized knowledge domains, further studies are needed to generalize the results to other cultural and linguistic contexts. AI models are continuously updated, meaning that ChatGPT’s current limitations may shift over time. Future research should reassess AI performance as new versions incorporate more diverse data sources.

5. Conclusions

This study contributes to the ongoing debate on AI’s role in education, emphasizing the need for human oversight when integrating AI-generated content into history and geography curricula. While ChatGPT provides coherent and readable responses, its reliance on digitized data alone limits its effectiveness in subjects requiring localized, non-digitized sources. These findings support previous research calling for a hybrid approach, where AI serves as a supplementary tool rather than a standalone knowledge source.

ChatGPT’s omissions and inaccuracies demonstrate that teachers and students cannot rely solely on generative AI to deliver precise or comprehensive information, especially for topics where primary sources remain undigitized or underrepresented online.

To move forward responsibly, educators must embed AI literacy into the curriculum, explicitly training students to fact-check and cross-reference. Policymakers and school administrators should likewise consider strategies for incorporating region-specific data into AI training sets, or creating custom, fine-tuned models for local use.

Future research can broaden the scope to examine whether ChatGPT fares better (or worse) on higher-order skills requiring complex analysis and to identify how best to integrate these tools without undermining students’ understanding of local cultures, histories, and geographies.