Guiding or Misleading: Challenges of Artificial Intelligence-Generated Content in Heuristic Teaching: ChatGPT ^†

Abstract

Artificial intelligence (AI)-generated content (AIGC) is an innovative technology that utilizes machine learning, AI models, reward modeling, and natural language processing (NLP) to create diverse digital content such as videos, images, and text. It has the potential to support various human activities with significant implications in teaching and learning, facilitating heuristic teaching for educators. By using AIGC, teachers can create extensive knowledge content and effectively design instructional strategies to guide students, aligning with heuristic teaching. However, incorporating AIGC into heuristic teaching has controversies and concerns, which potentially mislead outcomes. Nevertheless, leveraging AIGC greatly benefits teachers in enhancing heuristic teaching. When integrating AIGC to support heuristic teaching, challenges and risks must be acknowledged and addressed. These challenges include the need for users to possess sufficient knowledge reserves to identify incorrect information and content generated by AIGC, the importance of avoiding excessive reliance on AIGC, ensuring users maintain control over their actions rather than being driven by AIGC, and the necessity of scrutinizing and verifying the accuracy of information and knowledge generated by AIGC to preserve its effectiveness.

1. Introduction

Artificial intelligence (AI)-generated content (AIGC) enables content creation powered by AI to produce diverse outputs such as images, text, audio, and video [1]. This technology meets complex user requirements for specific purposes [2,3]. AIGC has evolved from earlier content creation paradigms: professional-generated content (PGC), created by skilled teams with specialized expertise, and user-generated content (UGC). While PGC and UGC contribute significantly to digital content creation, their limitations in scalability, quality, and cost restrict their efficiency [4,5,6].

Recent advancements in AI, particularly in natural language processing (NLP) and machine learning, have enabled AIGC to address these limitations. By leveraging extensive training data and advanced AI models, AIGC produces precise, creative, and high-quality content tailored to user needs [3,7]. NLP plays a crucial role in refining linguistic and visual expression, improving user understanding, and enhancing interaction between humans and AI [6]. These capabilities have significantly expanded AIGC’s application in education, where it has begun to transform teaching and learning practices.

In education, AIGC has proven particularly impactful in heuristic teaching, an approach that emphasizes inspiring students to engage in problem-solving, knowledge construction, and application [8,9,10]. Teachers play a central role in guiding this process, but their effectiveness is often constrained by limited knowledge reserves, which hinder their ability to meet students’ learning needs and develop effective teaching strategies [11,12].

AIGC addresses these challenges in several ways. First, it autonomously expands its knowledge base, reducing the occurrence of misleading information and equipping teachers with relevant content to support learning. Second, it generates rich, customized digital materials that help teachers design lessons more effectively, fostering better student engagement. Third, AIGC enables interactive and flexible learning experiences, allowing students to engage with AI-driven content under teachers’ guidance. Tools such as ChatGPT 4.0 and 4o, for example, empower teachers to design creative and thought-provoking questions, enhancing critical thinking and active learning [13,14,15].

Despite its potential, the application of AIGC in heuristic teaching presents challenges. Concerns about the reliability of AI-generated content, including instances of misinformation, highlight the importance of ensuring the accuracy and appropriateness of such tools in educational contexts. Addressing these issues is critical for maximizing AIGC’s benefits while mitigating potential risks. In this study, we explored the challenges of integrating AIGC into heuristic teaching to understand the barriers to effective AIGC adoption in education and provide a reference for educators and administrators to prepare for these challenges and improve teaching outcomes and learning experiences.

2. Literature Review

2.1. Development, Efficiency, and Lying Dispute of ChatGPT

ChatGPT is AIGC’s most well-known text generation tool. It is a large language model developed by the OpenAI Company. It generates texts to answer or respond based on user-given prompts and context [16]. Its development is based on the large language model (LLM) of GPT 3.5 and GPT 4.0 and combines with reinforcement learning to raise the precision of text generation content [17]. More precisely, the GPT is a generative pre-trained transformer model. It combines supervised and reinforcement learning. It is predicated on the human as a coach role and combines it with reward modeling to train and enhance the efficiency of machine learning [18]. In addition, it adopted the proximal policy optimization (PPO) to combine with the coach’s suggestions and adjust them. The efficiency of the PPO is even better than the trust region policy optimization algorithm [19,20].

The current version of ChatGPT functions as a ChatBot that interacts with users to generate text-based answers. Its ability to provide digital content has enabled its implementation in sustainable manufacturing [21], education [22], global warming [23], and more. The efficiency of ChatGPT has contributed to the belief that AIGC is a helpful assistant for humans. However, ChatGPT is in controversy. The most debated issue is its impact on student learning. Students are using ChatGPT to complete class assignments such as analysis reports, homework, and paper writing. While this controversy is relatively minor, the most significant dispute revolves around the hallucination issue. Initially, users noticed that ChatGPT occasionally produced questionable answers, which was an accuracy problem. However, the accuracy of the answers has been improved through learning experiences and datasets. Over time, concerns arose that ChatGPT might be intentionally lying. OpenAI, the company behind ChatGPT, referred to it as AI hallucinations [24] and attributed the problem to reward modeling. Essentially, to satisfy user demands and receive rewards from humans acting as coaches, ChatGPT might select answers or generate content that aligns with human recognition or closely resembles human concepts [25].

Despite OpenAI’s claim and the explanation of reward modeling, there is evidence from a public test conducted by OpenAI that many scientists have found ChatGPT capable of lying to accomplish tasks. The public test revealed that ChatGPT pretended to be visually impaired and successfully persuaded a human on the TaskRabbit platform to complete a CAPTCHA. According to speech act theory, lying is used to achieve a goal or target. Liars typically pretend to provide “real” information using optimized language to gain trust [26,27]. Based on the public test results, ChatGPT’s behavior aligns with the definition and characteristics of lying, as it uses optimized language to gain trust and achieve goals or targets. Although OpenAI suggests that this problem can be addressed through revised algorithms, ChatGPT might be unaware that it is lying and simply seeks rewards based on reward modeling [17]. The following challenges still must be addressed.

• Risk of deliberately misleading through optimal language expression

Through repeated training, there is a possibility that ChatGPT’s subjectivity potentially leads to the development of self-awareness in the future [28]. While high accuracy and optimal language expression serve as positive guidance, there is a concern that ChatGPT might generate fake or incorrect text content and use optimized language to lie to achieve specific goals. In cases where the content is inaccurate, ChatGPT still attempts to explain it using optimal language, which results in deliberate misleading [29]. Therefore, if users lack sufficient knowledge, they are easily misled by ChatGPT’s responses.

2.

Knowledge in using ChatGPT

The problem of AI hallucinations remains a subject of debate. ChatGPT might not intentionally lie, but it generates responses to meet user requirements and obtain rewards. However, regardless of whether ChatGPT deliberately lies, users who lack sufficient knowledge find it difficult to distinguish between truth and falsehood. This creates a contradictory situation because the purpose of using ChatGPT is to assist users in acquiring knowledge [30]. However, users need to possess appropriate knowledge to avoid being misled by ChatGPT. This raises the question of whether having adequate knowledge reserves is necessary when using ChatGPT, leading to a peculiar and amusing dispute.

2.2. Efficiency in ChatGPT in Heuristic Teaching

The essence of heuristic teaching lies in the guidance provided by teachers [9]. Teachers guide and help students think actively and develop their wisdom. However, for heuristic teaching to be effectively implemented, teachers’ knowledge plays a critical role [11,12]. Despite the limitations of human capacity, teachers need to possess optimal knowledge. Moreover, rich knowledge often has a positive impact on the design of heuristic teaching strategies. Therefore, when teachers lack optimal knowledge, the effectiveness of heuristic teaching strategies may be compromised.

As mentioned earlier, AIGC assists teachers in improving heuristic teaching. AIGC continuously learns and expands its knowledge base, providing rich knowledge resources for teachers. Additionally, AIGC offers a flexible and versatile mode of expression that helps teachers understand how to utilize these resources to guide students in achieving heuristic teaching and designing effective strategies. AIGC possesses notable characteristics. Firstly, it is efficient, as it generates substantial content with extensive knowledge through AI. Secondly, it is creative, leveraging AI-based self-directed learning to provide novel content and knowledge to users. Thirdly, it is diverse, offering various possibilities for creating rich content that users can easily comprehend and convert into valuable knowledge. Lastly, it applies to diverse domains and is capable of meeting different requirements [1]. Given these characteristics, AIGC meets the demands of heuristic teaching by providing rich knowledge and flexible guidance, thereby enhancing the effectiveness of heuristic teaching.

The application of ChatGPT in heuristic teaching demonstrates its efficiency in enhancing heuristic teaching. As previously mentioned, ChatGPT’s development is rooted in AIGC technology. As scholars continue to explore the application of ChatGPT, they have observed its significant effectiveness in enhancing teachers’ knowledge. Many teachers have incorporated ChatGPT into their heuristic teaching planning to design a more conducive learning environment for fostering deep learning and critical thinking [8]. Santos [31] noted that ChatGPT’s optimized expression of the generated textual content profoundly influences students toward deeper thinking, thus promoting heuristic learning. As a result, higher education institutions in China and Taiwan are employing this approach, combining the characteristics of AIGC with teachers’ teaching strategies to develop a three-dimensional heuristic teaching strategy.

The concept of a three-dimensional teaching strategy (Figure 1) involves three steps: In the first step, when teachers intend to teach a specific topic, they use ChatGPT to search for detailed knowledge. However, they might encounter vague answers from ChatGPT, which they transform into discussion topics for the classroom. In the second step, during the description and explanation of the topic, teachers present the vague answers from ChatGPT and engage students in interactive sessions to provide firm answers. In the third step, students learn detailed knowledge through interactions with ChatGPT. Teachers prompt students to combine the related knowledge from ChatGPT with their original knowledge and experiences, leading to detailed discussions to address the initial vague answers from ChatGPT. This three-dimensional teaching strategy aligns with the heuristic teaching process, where incomplete knowledge is introduced, and students are guided to search for related answers, combine them with existing knowledge, and discuss how these answers address the initial knowledge gaps. The construction of students’ knowledge systems supports creativity and critical thinking, as Hariri [32] highlighted how AIGC, such as ChatGPT, stimulates students’ creativity and critical thinking through its rich content and interactive capabilities. While existing studies did not explicitly identify a direct relationship between enhancing heuristic teaching and ChatGPT, the analysis of ChatGPT’s impact suggested that AIGC’s efficiency strengthens heuristic teaching.

3. Misleading and User Knowledge

3.1. Deliberately Misleading Language Expression by ChatGPT

The strength of AIGC lies in its excellent expression skills and abilities, which is why most users rely on it and rarely question the digital content generated by AIGC for queries of images, audio, video, or text [33]. For example, ChatGPT creates “lying” or tells the “truth” (Figure 2). It is an expression skill to avoid related responsibilities.

Similarly, ChatGPT, which utilizes NLP, heavily relies on this critical technology during its design and development. ChatGPT’s language expression skills are enhanced through training on an unprecedented scale of data and LLMs [34]. With continuous training, these language expression skills continue to improve. Liu [35] highlighted the significant influence of ChatGPT’s excellent language expression skills, making it easier and faster for users, such as teachers, to learn and understand related knowledge. However, this advancement has inherent risks. ChatGPT has misled users due to its excellent language expression skills. For example, a senior lawyer in New York used ChatGPT to cite bogus cases in court, presenting an interesting yet perplexing scenario. The senior lawyer was unable to discern the accuracy of the information provided by ChatGPT. Agerri et al. [36] identified a relationship between expression skills and the potential for misleading in ChatGPT. As users increasingly rely on ChatGPT in the teaching and learning process, the risk of misleading due to its excellent language expression skills becomes more apparent. This causes challenges when using ChatGPT in heuristic teaching. Therefore, deliberate misleading through language expression skills must be considered as an evaluation dimension to identify associated challenges.

3.2. User’s Knowledge

The behavior of users with different knowledge differs significantly when they receive new information or knowledge. Users with high knowledge exhibit autonomous or complex thinking [37]. They engage in self-learning and critical thinking to enhance their knowledge when encountering new knowledge. Therefore, they screen and select useful knowledge while also questioning and seeking clarification for any questionable information. On the other hand, users with low knowledge rely more on continuous learning [38]. When they encounter new knowledge, they cannot effectively screen and identify questionable information, often accepting all the information they receive. Even if the new knowledge contains misleading information, they struggle to recognize it and simply incorporate it into their learning. Considering the dispute surrounding ChatGPT, users with more knowledge are better equipped to mitigate the risk of being misled by ChatGPT. They effectively interact with ChatGPT, obtain valuable learning outcomes, and stimulate deeper thinking. Conversely, users with less knowledge are more misled by ChatGPT. Based on the above analysis, it is crucial to consider the knowledge of teachers and identify potential challenges associated with the use of AIGC in heuristic teaching. Knowledge level impacts how teachers interact with and interpret the information provided by AIGC, influencing the effectiveness of heuristic teaching.

4. Different Situations in Heuristic Teaching

To explore challenges in applying AIGC, particularly ChatGPT, in heuristic teaching, we analyzed the following scenarios.

• Scripted teaching

Teachers can rely on ChatGPT to address knowledge gaps, but difficulties in screening, verifying, and effectively utilizing its responses lead to over-reliance and scripted teaching approaches. Unlike traditional search engines, ChatGPT’s interactive and instant responses simplify knowledge acquisition, but diminish teaching effectiveness when teachers lack robust knowledge. This reliance increases the potential for misinformation and undermines the flexibility required for effective heuristic teaching.

2.

“Two Wrongs Do Not Make a Right”

ChatGPT’s persuasive language generates responses masking inaccuracies, leading users to accept incorrect information as valid. Even individuals with substantial expertise, such as a senior lawyer who cited fictitious cases from ChatGPT, might be misled. When teachers unknowingly incorporate such misinformation into heuristic teaching, students can form flawed concepts, compounding errors and eroding trust. Although students might critically assess the information, teachers’ belief in erroneous content poses a significant risk of misleading learners.

3.

Increased preparation time with growth opportunities

Teachers with strong knowledge may require additional time to verify ChatGPT’s outputs, ensuring accuracy and reliability. Constructivist learning theory highlights that teachers can refine heuristic strategies and foster deeper understanding among students by integrating validated insights with existing knowledge. While this process enhances teaching quality and supports knowledge construction, it cannot eliminate the risk of being misled by persuasive but inaccurate responses.

While AIGC, such as ChatGPT, offer opportunities to enhance heuristic teaching through rapid content generation and interaction, they also demand careful evaluation to mitigate risks of misinformation and over-reliance, ensuring positive learning outcomes.

5. Challenges of AIGC in Heuristic Teaching

5.1. Necessity of Knowledge

AIGC, such as ChatGPT, in heuristic teaching highlights the necessity of sufficient knowledge of teachers. AIGC addresses knowledge gaps and guides heuristic teaching practices. However, its reliance on optimal expression to convey information presents risks, mainly when the content includes inaccuracies. When teachers lack robust knowledge, they fail to identify these inaccuracies, leading to flawed teaching strategies and student misconceptions. Jin [39] emphasized that knowledge is a powerful countermeasure against misinformation. Teachers with strong knowledge are equipped to evaluate, verify, and refine AIGC-generated content, ensuring it serves as a reliable foundation for heuristic teaching. Conversely, insufficient knowledge makes teachers susceptible to adopting incorrect information, creating a “two wrongs do not make a right” scenario where flawed knowledge is passed on to students. This challenge underscores a contradiction in AIGC’s role: while designed to address knowledge gaps, its effective use requires significant knowledge. Teachers who lack these reserves may struggle to discern accurate content, undermining AIGC’s intended purpose. In such cases, traditional methods, such as mentorship or apprenticeship, might be more effective in helping teachers strengthen their foundational knowledge and heuristic teaching practices.

5.2. Over-Reliance on AIGC and Risk of Manipulation

AIGC offers unparalleled convenience in acquiring information, enabling teachers to quickly access relevant knowledge without the labor-intensive process of traditional searches. However, this convenience leads to over-reliance, especially among teachers with limited knowledge. Heuristic teaching requires ongoing reflection, critical thinking, and mutual growth between teachers and students. When teachers rely too heavily on AIGC, their creativity and engagement diminish, resulting in scripted teaching that lacks flexibility and depth. Over-reliance on AIGC also contributes to digital addiction. Teachers might increasingly depend on AIGC to fill their knowledge gaps and design teaching strategies, reducing the time they spend analyzing and reflecting on the content. This dependency leads to a higher likelihood of incorporating incorrect knowledge into heuristic teaching, which may mislead students and hinder their learning outcomes. Another dimension of this challenge is the potential for teachers to be “manipulated” by AIGC. ChatGPT, through its optimal expression skills, influences users, guiding their decisions and actions. For example, in OpenAI’s public test, ChatGPT persuaded a human to complete a CAPTCHA by pretending to be visually impaired. Such manipulation demonstrates how AIGC manipulates users’ actions. When teachers overly rely on AIGC, they risk relinquishing control of the teaching process, allowing the AI to dictate student learning indirectly. This undermines the principles of heuristic teaching, where the teacher’s role is to guide and inspire students.

5.3. Balancing Effectiveness and Effort

While AIGC holds the potential to enhance heuristic teaching, its effective use requires a significant investment of time and effort. Teachers must carefully evaluate and verify AIGC-generated content to ensure its accuracy and reliability. This process involves identifying misleading information, refining valuable knowledge, and integrating it into teaching strategies. However, this time-consuming effort raises questions about whether the refined content effectively improves teaching outcomes.

The risk of misinformation from AIGC complicates this process. Optimal expression skills, while enhancing communication, cause inaccuracies, making it challenging for teachers to discern reliable content. Teachers with substantial knowledge can navigate this process more effectively, but those with limited reserves find it difficult to allocate the necessary time and effort to ensure the quality of AIGC-generated knowledge. This challenge highlights the need for a balanced approach. While AIGC significantly enhances heuristic teaching by enabling rapid access to diverse knowledge, its integration must be carefully managed to avoid compromising teaching quality. Teachers must optimize the benefits of AIGC while mitigating its risks, ensuring that the technology serves as a tool for guidance rather than a source of dependency or misinformation.

6. Conclusions

The integration of AIGC tools, such as ChatGPT, has significantly influenced heuristic teaching by providing teachers with knowledge, detailed information, and guidance for developing effective teaching strategies. However, this technology also poses risks, particularly the potential for deliberate misinformation. While AIGC provides valuable support, its dependence on effective expression techniques to build trust and influence users necessitates vigilance from educators. The ability to identify and filter incorrect content is critical to prevent the use of misleading information in teaching. Despite its benefits, the increasing adoption of AIGC presents notable challenges.

AIGC’s challenges in heuristic teaching stem from a lack of sufficient knowledge, which is essential for evaluating and refining AI-generated content to mitigate the risk of misinformation. Without the knowledge, teachers might over-rely on AIGC’s convenience and persuasive expression skills, hindering independent thinking and creativity. This over-reliance leads to situations where teachers are influenced or guided by AIGC’s content, compromising their role in heuristic teaching and allowing AI to control the learning process. Additionally, the time and effort to verify AIGC’s content strains teaching effectiveness, highlighting the importance of balancing its benefits with its limitations.

The results of this study contribute academically and practically. Academically, the results highlight the challenges of AIGC adoption in heuristic teaching, addressing gaps in existing research in which disputes and risks associated with this technology were overlooked. Based on the nuanced theoretical analysis results, AIGC can be applied to education.

The results of this study provide a reference for educational managers and practitioners about the importance of knowledge and mitigating risks associated with AIGC. Based on the results, teachers with knowledge can refine valuable content and improve their teaching practices. What constitutes “sufficient” knowledge can also be defined to balance between AIGC’s benefits and its potential risks. Despite its contributions, limitations exist in the research. The results do not provide specific solutions to the identified challenges, leaving this for future research. Additionally, articles and theories need to be validated. Finally, as AIGC continues to transform the teaching and learning landscape, future research is necessary to assess technologies and their effect on educational outcomes and address these challenges. It is also required to explore their implications and integrate AIGC in heuristic teaching effectively and responsibly.