Skip to Content
MDPIMDPI

Advanced

Education SciencesEducation Sciences
PDF
  • Systematic Review
  • Open Access

20 March 2026

23 Pages

Beyond Language Gains: A Meta-Analysis of Non-Linguistic Outcomes of ChatGPT-Integrated English Instruction in South Korea

Department of English Education, Jeonju University, Jeonju-si 55069, Republic of Korea
Educ. Sci.2026, 16(3), 481;https://doi.org/10.3390/educsci16030481
This article belongs to the Special Issue Innovative Teaching Strategies to Promote Language and Literacy Development: Multimodal and Multidisciplinary Perspectives
Version Notes
Order Reprints

Abstract

This meta-analysis investigates the outcomes associated with ChatGPT-integrated English instruction on non-linguistic outcomes in South Korea. Following the PRISMA 2020 guidelines, 22 experimental studies (k = 26, N = 1303) were synthesized using a random-effects model, which revealed a significant medium overall effect size (g = 0.55). Subgroup analyses showed medium-to-large effects in Affective (g = 0.67) and AI Literacy (g = 0.59) domains, but a small effect on Cognitive/Metacognitive outcomes (g = 0.17). Moderator analyses (e.g., educational level, duration) yielded no significant differences, suggesting a meaningful overall trend across contexts. However, the descriptive disparity between affective gains and cognitive growth indicates an ‘Affective–Cognitive Gap.’ Findings suggest that while ChatGPT is associated with a reduction in psychological barriers—partly by reducing mental effort—it may lead to ‘cognitive offloading’ without intentional pedagogical scaffolding. The study concludes that mere tool adoption is insufficient. To prevent a ‘plateau effect’ after the initial novelty wears off, instruction must evolve from passive use to ‘agentic engagement’ through structured metacognitive routines. These results emphasize the necessity of teacher-mediated integration to repurpose AI-afforded efficiency toward higher-order evaluative and critical thinking tasks.

1. Introduction

The integration of Artificial Intelligence (AI) into education is no longer a futuristic concept but a rapidly unfolding reality, fundamentally reshaping how languages are taught and learned. This digital paradigm shift has been accelerated by significant progress in Natural Language Processing (NLP) and the emergence of Large Language Models (LLMs), which are now exerting a profound influence across various sectors, including education (H.-S. Kim, 2025; Liu et al., 2025). Among these innovations, OpenAI’s ChatGPT has emerged as a disruptive force. Unlike earlier retrieval-based chatbots or traditional machine translators, ChatGPT utilizes advanced generative AI to produce human-like dialogue and contextually appropriate feedback (Kang, 2025). These capabilities provide rich input and interactive affordances, creating personalized learning environments that were previously difficult to implement at scale (Kohnke et al., 2023; Y. Wang, 2025).
South Korea represents a unique and critical testbed for examining outcomes associated with such AI integration. While the nation boasts a highly digitalized infrastructure and rapid adoption of educational technology, the context of English as a Foreign Language (EFL) instruction remains characterized by high-stakes testing and grammar-oriented approaches. Historically, this environment has limited opportunities for productive practice, frequently making speaking and writing anxiety-inducing tasks for Korean learners (Kang, 2025; Song & Kim, 2025). This juxtaposition of advanced technology and traditional pedagogical constraints makes the Korean context an ideal environment to examine whether GenAI can dismantle long-standing psychological barriers to communication.
Recent scholarship has begun to provide systematic answers regarding the cognitive efficacy of ChatGPT in this specific context. A recent meta-analysis by J.-Y. Lee (2025) synthesized 12 experimental studies conducted in Korea and confirmed that ChatGPT-assisted instruction showed a medium-to-large improvement (g = 0.69) in English writing proficiency. His findings demonstrated consistent linguistic gains across both secondary and tertiary levels, suggesting that ChatGPT is arguably a robust tool for enhancing language skills.
However, while J.-Y. Lee (2025) provided compelling evidence for linguistic gains, the non-linguistic dimension—equally critical in high-anxiety EFL environments—remains synthetically unexplored. Empirical studies suggest that ChatGPT may reduce psychological barriers, such as anxiety, while enhancing confidence and willingness to communicate (Kang, 2025; S. Kim & Lee, 2023). Theoretically, these non-linguistic benefits align with the reduction in the ‘Affective Filter’ (Krashen, 1982) and the promotion of autonomy, as AI provides a non-judgmental, private space for practice. Yet, unlike the linguistic domain synthesized by J.-Y. Lee (2025), it remains unclear how large and robust these effects on affective and behavioral outcomes are across the accumulated evidence.
Furthermore, existing syntheses in the field often suffer from ‘language bias’ by relying predominantly on English-medium publications. This exclusion risks over-looking a significant body of indigenous empirical evidence published in domestic journals, which often capture the nuanced realities of local implementation (Amano et al., 2016; Morrison et al., 2012). To ensure a comprehensive synthesis, this study targeted both English- and Korean-language publications available in domestic academic databases. By mitigating language bias, we aim to capture the full spectrum of educational outcomes—ranging from highly controlled experiments to eco-logical classroom implementations—within the South Korean context.
Therefore, the field has reached a critical juncture where a comprehensive meta-analysis on non-linguistic outcomes is required. Building upon the linguistic evidence established by J.-Y. Lee (2025), this study specifically focuses on synthesizing outcomes linked to ChatGPT-based English learning on non-linguistic variables (e.g., anxiety, motivation, critical thinking, and AI literacy). By doing so, this study aims to offer universal insights into how GenAI can transcend linguistic drills to support the psychological and autonomous capacity of language learners, providing evidence-based implications for future education models globally.

2. Literature Review

The rapid proliferation of large language models (LLMs), particularly ChatGPT, has catalyzed an increasing number of experimental studies exploring its impact across educational contexts. To synthesize this burgeoning body of evidence, recent meta-analyses have quantified the effects of ChatGPT and related generative AI (GenAI) tools on various student outcomes, particularly within the field of English language education.

2.1. International Meta-Analyses

Recent meta-analyses focusing on GenAI applications demonstrate varied, but generally positive, effects on student learning, often categorized by linguistic and non-linguistic outcomes.

2.1.1. Linguistic Outcomes

Meta-analyses consistently suggest that GenAI interventions significantly improve students’ academic performance and language proficiency. Deng et al. (2025) conducted a systematic review and meta-analysis of experimental studies and found that ChatGPT interventions generally improve academic performance with a significant large effect (g = 0.712). Importantly, the highest observed effect size on academic performance was found in instruction related to arts and humanities (g = 1.045), a category that predominantly encompasses language education studies. J. Wang and Fan (2025), synthesizing 51 studies, also reported a large positive impact on learning performance overall (g = 0.867).
A specific focus on language skills highlights efficacy across different modalities. Guan et al. (2024), examining GenAI in informal digital English learning (IDLE), found a small overall effect on English proficiency (g = 0.447), but observed a medium effect on standardized English tests (g = 0.626) and a large effect on speaking ability (g = 0.794). Similarly, a meta-analysis on AI effectiveness in English speaking and listening skills by Jantakoon et al. (2025) reported a substantial overall effect (SMD = 0.981), with a statistically significant large effect specifically for speaking proficiency (SMD = 1.033). However, they cautioned that the effect on listening skills (SMD = 0.714) was positive but did not reach statistical significance, possibly due to limited statistical power.

2.1.2. Non-Linguistic Outcomes

Beyond measurable linguistic performance, recent meta-analyses have consistently highlighted the robust impact of ChatGPT on learners’ affective and motivational states. Deng et al. (2025) found that ChatGPT interventions significantly promoted these states with a large effect size (g = 0.881). Similarly, Xia et al. (2025), focusing exclusively on university students, reported an even higher effect size specifically on motivation (g = 1.20).
In terms of student engagement, findings indicate multidimensional benefits. Heung and Chiu (2025) determined that learning activities supported by ChatGPT resulted in a medium effect size for overall student engagement (g = 0.55), characterized by medium effects on behavioral (g = 0.454) and emotional engagement (g = 0.512), alongside a medium-to-large effect on cognitive engagement (g = 0.593). These patterns were corroborated in the higher education context by Xia et al. (2025), who noted comparable medium effects for cognitive (g = 0.59), behavioral (g = 0.52), and emotional engagement (g = 0.64), while further identifying a high effect on agency engagement (g = 0.87).
Regarding cognitive processing, the literature presents a nuanced picture of higher-order thinking and mental effort. Deng et al. (2025) reported a significant large effect on higher-order thinking propensities (g = 0.703), whereas J. Wang and Fan (2025) observed a more moderate positive impact in this domain (g = 0.457). Furthermore, Deng et al. (2025) found that ChatGPT significantly reduced mental effort (g = −0.675), suggesting that AI-integrated learning tasks may become less cognitively demanding for students.

2.2. Domestic Meta-Analyses

Meta-analytic reviews focusing on AI implementation within the Korean EFL context reveal clear trends in both linguistic and non-linguistic outcomes, largely shaped by studies conducted before the mass adoption of ChatGPT.

2.2.1. Pre-ChatGPT AI Chatbot Meta-Analyses

Prior meta-analyses on AI chatbots have consistently demonstrated substantial benefits for linguistic competence. J.-Y. Lee and Hwang (2022) reported an overall effect size surpassing the medium threshold (d = 0.672), with particularly strong results observed in writing (d = 0.802) and listening (d = 0.765) skills. Similarly, Seo and Kim (2023) found the overall linguistic effect to be greater than medium (g = 0.656), highlighting large effects specifically for vocabulary (g = 1.047) and speaking (g = 0.873).
In the affective domain, AI chatbots have demonstrated even stronger positive effects. J.-Y. Lee and Hwang (2022) found that affective outcomes showed an overall effect size greater than medium (d = 0.725), with sub-domains such as motivation (d = 0.947), anxiety reduction (d = 0.861), and confidence (d = 0.815) approaching large effect sizes. This pattern was corroborated by Seo and Kim (2023), who also reported an overall affective effect greater than medium (g = 0.724). They notably observed that the effect size was larger in the affective domain compared to the linguistic domain, with motivation showing a large effect (g = 0.896), while interest, confidence, and belief showed medium effects.
Regarding moderating variables, both studies concurred that efficacy tended to be larger at lower school levels. However, findings regarding intervention duration diverged. J.-Y. Lee and Hwang (2022) reported better results for shorter durations (8 weeks or less), whereas Seo and Kim (2023) found stronger results for longer durations (11 weeks or more).

2.2.2. ChatGPT-Specific Domestic Meta-Analyses

Among the identified literature, the sole meta-analytic study specifically targeting ChatGPT utilization within Korean EFL classrooms is that of J.-Y. Lee (2025). Regarding linguistic outcomes, J.-Y. Lee (2025) reported that ChatGPT-supported instruction resulted in a medium-to-large improvement in English writing proficiency (g = 0.69). Furthermore, the study determined that this positive effect was statistically consistent across educational levels, benefiting both secondary and tertiary learners without significant disparity. It is important to note, however, that J.-Y. Lee’s (2025) analysis was confined exclusively to writing performance and did not extend to a synthesis of affective or non-linguistic domains.

2.3. Research Gap: Need for Non-Linguistic Focus in Domestic ChatGPT Research

While international meta-analyses confirm that GenAI significantly impacts non-linguistic outcomes such as motivation, engagement, and critical thinking (Deng et al., 2025; Heung & Chiu, 2025; Xia et al., 2025), and domestic AI chatbot studies (J.-Y. Lee & Hwang, 2022; Seo & Kim, 2023) have previously demonstrated large positive effects on the affective domain in Korea, there is a critical absence of meta-analytic research specifically on the non-linguistic effects of ChatGPT in the Korean context. J.-Y. Lee (2025), the meta-analysis focusing solely on ChatGPT nationally, limited its scope exclusively to linguistic (writing) proficiency, explicitly necessitating an expansion of the evidence base. Therefore, future domestic research focusing on analyzing the effectiveness of ChatGPT regarding non-linguistic factors (such as motivation, confidence, critical thinking, and AI literacy) is required to provide a comprehensive understanding of GenAI’s impact on Korean EFL learners.

3. Methods

3.1. Literature Search and Selection Criteria

To ensure a comprehensive and unbiased retrieval of relevant studies, a systematic literature search was conducted in November 2025. The search utilized the Research Information Sharing Service (RISS), which is the primary academic database in South Korea. RISS serves as a centralized meta-search engine that aggregates data from major domestic repositories, including the Korea Citation Index (KCI), Kyobo Scholar, Korean Studies Information Services System (KISS), and DBpia.
The search strategy employed a combination of keywords related to the target technology and the subject matter in both English and Korean. The specific search string was: ‘(‘ChatGPT’ OR ‘chaetGPT’ [Korean transliteration] OR ‘chaetjipiti’ [Korean transliteration]) AND (‘English’ OR ‘L2’ OR ‘ESL’ OR ‘EFL’ OR ‘yeongeo’ [Korean for English])’.
The initial search yielded a total of 439 records. To select studies suitable for the meta-analysis, the following eligibility criteria were applied:
  • Intervention: Experimental studies investigating the effects of ChatGPT-integrated English instruction.
  • Participants: Studies involving South Korean students at any educational level.
  • Outcomes: Studies reporting quantitative measures of non-linguistic (non-academic) outcomes (e.g., anxiety, motivation, AI literacy).
  • Data Availability: Studies providing sufficient statistical data (e.g., sample size, mean, standard deviation, t-value, p-value) necessary for calculating effect sizes.
A multi-stage screening process was conducted to refine the selection (see Figure 1). First, after removing duplicates, the titles, abstracts, and keywords of the retrieved records were screened. Studies irrelevant to the research topic—such as those focusing solely on AI-based machine translation accuracy or textual analysis of AI-generated texts—were excluded.
Figure 1. PRISMA flowchart of the screening process.
Second, a full-text review was performed on the remaining articles. At this stage, qualitative research, case studies, and quantitative studies lacking essential statistical indicators were excluded.
Finally, to distinguish the present study from prior syntheses focusing on linguistic gains (e.g., J.-Y. Lee, 2025), studies that reported only academic outcomes (e.g., writing scores or L2 proficiency test achievements) without measuring non-linguistic variables were excluded. Through this rigorous selection process, following the PRISMA 2020 guidelines (Page et al., 2021), a total of 22 independent studies (containing 26 effect sizes) were ultimately selected for the final meta-analysis.

3.2. Coding Scheme and Moderator Variables

To systematically analyze the characteristics and outcomes of the included studies, a comprehensive coding scheme was developed. The coding process focused on three primary dimensions: (a) study characteristics and methodological features, (b) intervention details (pedagogical roles), and (c) outcome variables. The specific coding criteria are summarized in Table 1.
Table 1. Coding scheme.
First, regarding study characteristics, the publication ‘Year’ was coded to examine whether the rapid evolution of generative AI models influences non-linguistic outcomes. This variable tracks the technological progression from the release of ChatGPT-3.5 (late 2022) and GPT-4 (March 2023) to the more advanced GPT-4o (May 2024) and the recently updated GPT-5.1 (November 2025), which boasts enhanced reasoning and multimodal capabilities.
Experimental Design was categorized based on Brown’s (1988) framework. Since no true experimental studies (random assignment with control groups) were identified, studies were classified as either (1) quasi-experimental (control group without random assignment) or (2) pre-experimental (one-group pretest-posttest design). Educational Level was dichotomized into (1) secondary school and (2) university (tertiary education), as no studies conducted at the primary school level met the inclusion criteria.
Treatment Duration was coded in two formats to allow for sensitive moderator analysis. Weeks were recorded as continuous numerical data. Additionally, a categorical variable (Treatment Period) was established with a cut-off of 8 weeks (half of a standard semester), grouping studies into (1) 8 weeks or less and (2) 9 weeks or more. For studies that stated the duration as ‘one semester’ without specifying the exact weeks (e.g., S. Lee & Kang, 2025; Rha et al., 2025), a duration of 15 weeks was imputed, reflecting the standard academic calendar in South Korean universities. Number of Participants was recorded as the total sample size (experimental plus control groups).
Second, the Pedagogical Role of ChatGPT was coded to investigate how different usage patterns affect learner outcomes. Drawing on recent frameworks (Hong, 2023; Kohnke et al., 2023; Lo, 2023), the interventions were categorized into six types: (1) idea generation & resource search, (2) drafting assistance, (3) feedback & correction, (4) interactive partner, (5) tutoring & explanation, and (6) others. Studies employing multiple roles were strictly coded to allow for multiple-response analysis.
Third, the Dependent Variables were classified into three theoretical domains based on the non-academic outcomes reported:
  • Affective Domain: Encompassing variables such as motivation, anxiety, confidence, and self-efficacy, grounded in established theories (Bandura, 1997; Dörnyei, 2005).
  • Metacognitive & Cognitive Domain: Including self-regulated learning, strategies, and critical thinking (Oxford, 1990; Zimmerman, 2002).
  • AI & Digital Literacy: Covering emerging competencies related to AI usage and digital fluency (Hwang & Lee, 2025; Ng et al., 2021; L. Zhao et al., 2022).
Variables such as region and weekly instruction hours were initially considered but excluded due to missing data in more than half of the studies. Similarly, the distinction between regular and extra-curricular programs was omitted, as all included studies were conducted within regular curricular contexts.

3.3. Statistical Analysis

The statistical analysis was conducted using a rigorous multi-stage procedure. First, all quantitative data extracted from the primary studies, including sample sizes, means, and standard deviations, were organized using Microsoft Excel. Subsequently, the meta-analysis was performed using RStudio (Posit Software PBC, 2025) and the metafor package (Viechtbauer, 2010).

3.3.1. Effect Size Calculation

To standardize the findings across studies with varying scales, the standardized mean difference (SMD) was calculated. Specifically, Hedges’ g was selected as the effect size metric instead of Cohen’s d. While both metrics are similar, Cohen’s d tends to overestimate effect sizes in studies with small sample sizes. Hedges’ g applies a correction factor (J) to adjust for this bias, providing a more conservative and accurate estimate, which is particularly critical in educational research where sample sizes often vary significantly (Hedges & Olkin, 1985).
For studies employing a pre-experimental design (one-group pretest-posttest), the calculation of effect sizes requires the correlation coefficient (r) between pre- and post-test scores. Since the majority of the included studies did not report this correlation, a conservative value of r = 0.50 was imputed, following standard practices in social science meta-analyses (Papadopoulos, 2025; White et al., 2019). When a single study reported multiple outcomes, they were aggregated into a single effect size for the overall analysis to strictly adhere to the independence assumption (Borenstein et al., 2021). However, for the subgroup analyses by dependent variable domains, the specific outcomes were analyzed separately to capture domain-specific effects (Cooper, 2017).

3.3.2. Model Selection and Heterogeneity

Heterogeneity across studies was assessed using Cochran’s Q-statistic and the I2 index. The Q-statistic tests the null hypothesis that all studies share a common effect size (homogeneity), while the I2 index quantifies the percentage of variation across studies that is due to heterogeneity rather than chance. In the field of applied linguistics and educational research, extreme heterogeneity is often considered an inherent characteristic rather than a statistical anomaly, reflecting the complex interplay of diverse pedagogical designs, learner proficiencies, and classroom contexts (Lorenc et al., 2016; H. Zhao & Aryadoust, 2024).
Given this diversity, it was conceptually inappropriate to assume a single underlying effect size. Therefore, a random-effects model was adopted from the outset to explicitly account for between-study variance. As argued by Plonsky and Oswald (2014), the random-effects model allows for unconditional inferences that generalize beyond the specific studies included in this analysis, thereby providing more ecologically valid conclusions. To further address the extreme heterogeneity and provide a transparent assessment of the dispersion of effects, 95% prediction intervals (PI) were calculated alongside confidence intervals (CI). While CI indicates the precision of the mean effect, PI provides a realistic range within which the effect of a new study in a similar context is likely to fall, thereby acknowledging the potential variability in individual future implementations.

3.3.3. Publication Bias and Sensitivity Analyses

To ensure the integrity and robustness of the meta-analytic findings, a comprehensive assessment of potential biases and result stability was conducted. First, to assess potential publication bias, four complementary methods were employed: (1) Funnel plots (Sterne et al., 2001) for visual inspection of asymmetry, (2) Egger’s regression test (Egger et al., 1997) for statistical quantification of bias, (3) the Trim-and-fill method (Duval & Tweedie, 2000) for sensitivity analysis regarding missing studies, and (4) Rosenthal’s Fail-Safe N (Rosenthal, 1979) to determine the robustness of the significant results against hypothetical missing studies.
Beyond assessing publication bias, a sensitivity analysis was conducted to examine the internal stability of the findings. This procedure involved a leave-one-out analysis, which iteratively removed one study at a time and recalculated the pooled effect size to determine whether any single influential study disproportionately skewed the overall results (Borenstein et al., 2021). Furthermore, to address the potential impact of the imputed pre-post correlation (r) used for pre-experimental designs, an additional robustness check was performed by varying the correlation value at 0.30, 0.50, and 0.70. Finally, Baujat plots (Baujat et al., 2002) were inspected to identify potential outliers contributing significantly to heterogeneity. Studies identified as outliers were scrutinized to decide on their inclusion or exclusion, thereby validating the stability of the meta-analytic conclusions.

3.3.4. Moderator Analysis

To investigate the sources of heterogeneity, moderator analyses were conducted. For categorical moderators (e.g., educational level), the Q-test for subgroup differences (Q_b) was used. For continuous moderators (e.g., treatment duration in weeks), meta-regression was performed. Notably, to visually examine the relationship between continuous moderators and effect sizes, bubble plots were generated. In these plots, the size of each bubble corresponds to the weight of the study, allowing for an intuitive assessment of how variables such as intervention duration influence the magnitude of the outcomes (Borenstein et al., 2021).

3.3.5. Interpretation Benchmarks

To interpret the magnitude of the effect sizes, the traditional criteria proposed by Cohen (1988) were adopted as the primary framework: small (g = 0.20), medium (g = 0.50), and large (g = 0.80). While recent scholarship in applied linguistics has suggested field-specific benchmarks (e.g., Plonsky & Oswald, 2014) that often propose different thresholds for practical significance, this study adheres to Cohen’s established standards to ensure comparability with the broader body of educational meta-analyses. Nevertheless, these benchmarks were applied with contextual caution, acknowledging that the magnitude of effects should be understood within the specific nature of L2 interventions.

4. Results

4.1. Descriptive Statistics of Included Studies

A total of 22 independent studies met the inclusion criteria for this meta-analysis. The key characteristics of these studies, including publication year, experimental design, educational level, and intervention details, are presented in Appendix A.

4.1.1. Publication Trend and Context

The distribution of publication years illustrates the nascent and rapidly evolving nature of this field. All included studies were published between 2023 and 2025, with a sharp upward trajectory: only 2 studies (9.1%) were published in 2023, followed by 8 studies (36.4%) in 2024, and a surge to 12 studies (54.5%) in 2025. This chronological concentration reflects the immediate academic response to the proliferation of generative AI tools. Regarding the educational context, the vast majority of research was conducted in tertiary education (k = 21, 95.5%), with only one study (S. Kim & Lee, 2023) addressing the secondary school level.

4.1.2. Methodological Characteristics

In terms of experimental design, pre-experimental designs (one-group pretest-posttest) were predominant, accounting for 19 studies (86.4%). Only three studies (Kang, 2025; S. Kim & Lee, 2023; P. Park & Kim, 2024) employed a quasi-experimental design with a control group. The number of participants in individual studies ranged from 20 to 225, with a total sample size of 1303 learners across all studies.

4.1.3. Intervention and Outcomes

The duration of the interventions varied from 4 to 15 weeks. Notably, a significant proportion of studies (k = 12) were conducted over a 15-week period, corresponding to a typical academic semester in South Korean universities. Regarding the pedagogical role of ChatGPT, most studies employed the tool for multiple purposes simultaneously (e.g., as a feedback provider, interactive partner, and idea generator) rather than isolating a single function. Finally, regarding the non-linguistic outcomes, the studies yielded a total of 26 effect sizes (some studies reported outcomes across multiple domains). The Affective Domain was the most frequently investigated area (k = 15), followed by AI & Digital Literacy (k = 6) and the Cognitive/Metacognitive Domain (k = 5). This distribution indicates that early research on ChatGPT in Korea has primarily focused on its potential to alleviate psychological barriers and foster positive learning attitudes.

4.2. Overall Effect of ChatGPT on Non-Linguistic Outcomes

To examine the overall efficacy of ChatGPT-integrated instruction on non-linguistic outcomes, a random-effects meta-analysis was conducted. The analysis synthesized 26 effect sizes (k = 26) derived from the 22 included studies, involving a total of 1303 participants. Table 2 summarizes the overall mean effect size and heterogeneity statistics, while Figure 2 visually presents the forest plot containing the effect sizes and 95% confidence intervals (CIs) for each study.
Table 2. Overall effect size and heterogeneity statistics.
Figure 2. Forest plot of the overall effect of ChatGPT on non-linguistic outcomes.
As presented in Table 2, the weighted mean effect size (Hedges’ g) for the non-linguistic outcomes was 0.55 (SE = 0.09, 95% CI [0.38, 0.73]). The test for the overall effect was statistically significant (z = 6.22, p < 0.001). According to Cohen’s (1988) traditional benchmarks, this represents a medium-to-large effect. Furthermore, when interpreted within the specific context of L2 research benchmarks proposed by Plonsky and Oswald (2014), a g of 0.55 indicates a substantial and practically meaningful positive trend, suggesting that ChatGPT interventions are associated with fostering positive affective and cognitive engagement among Korean EFL learners.
However, the 95% prediction interval (PI), which accounts for the extreme heterogeneity (I2 = 94.19%), ranged from −0.20 to 1.30. The fact that the PI includes zero suggests that while the average effect is positive, the outcomes associated with ChatGPT-integrated instruction is highly variable across different settings. In some specific instructional contexts, the intervention may be linked to negligible or even slightly negative results, highlighting that the tool’s associated outcomes are not universal but strictly dependent on instructional design and teacher-mediated orchestration.
As visualized in the forest plot (Figure 2), extreme variability was observed across the individual studies. The effect sizes ranged widely, from a slight negative effect in P. Park and Kim (2024) (g = −0.10) to an exceptionally large positive effect in Noh (2025) (g = 2.55). The heterogeneity statistics corroborated this visual dispersion, revealing a highly significant Q-value (Q(25) = 348.72, p < 0.001) and an extremely high I2 index of 94.19%. This indicates that approximately 94% of the variance in observed effects reflects real differences in study characteristics rather than sampling error. Such high heterogeneity necessitates further investigation into potential moderators (Borenstein et al., 2021; Card, 2012), which are explored in the subsequent sections.

4.3. Subgroup Analysis by Outcome Domains

To investigate specific non-linguistic dimensions associated with ChatGPT-integrated instruction, a subgroup analysis was performed across three domains: (1) Affective, (2) Cognitive/Metacognitive, and (3) AI/Digital Literacy. Table 3 details the descriptive and inferential statistics for each domain.
Table 3. Subgroup analysis by outcome domains.
Contrary to initial expectations, the test for subgroup differences (Q_b) indicated that the variation in effect sizes across the three domains was not statistically significant (Q_b(2) = 3.13, p = 0.209). This suggests that, from a statistical standpoint, ChatGPT-integrated instruction tends to be associated with positive non-linguistic outcomes regardless of the specific domain.
However, a descriptive examination reveals a notable contrast in magnitudes: the Affective Domain showed a medium-to-large effect (g = 0.67), whereas the Cognitive/Metacognitive Domain (k = 5) was linked to a comparatively small effect (g = 0.17). While this tentative pattern does not constitute a statistically demonstrated differential—potentially due to the limited power of the small subgroup size—it warrants qualitative attention in the subsequent discussion.
Heterogeneity remained substantial within the Affective (I2 = 94.08%) and AI/Digital Literacy (I2 = 75.34%) domains. The high within-group variability likely contributed to the non-significant Q_b result, masking the differences between the domains. This indicates that factors other than the outcome domain itself—such as instructional design or learner characteristics—may be stronger predictors of efficacy.

4.4. Moderator Analysis

To identify potential sources of heterogeneity and examine whether the magnitude of the non-linguistic outcomes varied across different study contexts, moderator analyses were conducted on five categorical variables: (1) publication year, (2) experimental design, (3) educational level, (4) treatment period, and (5) pedagogical roles. Table 4 presents the detailed results of these analyses.
Table 4. Moderator analysis of categorical variables.
The analyses revealed that none of the examined moderators showed statistically significant differences in effect sizes (p > 0.05), suggesting that the positive impact of ChatGPT on non-linguistic outcomes is relatively substantial across diverse experimental conditions.
First, regarding Publication Year, although the mean effect size descriptively increased from 2023 (g = 0.44) and 2024 (g = 0.43) to 2025 (g = 0.64), this trend did not reach statistical significance (Q_b(2) = 0.99, p = 0.609). This implies that the observed outcomes associated with ChatGPT-integrated instruction have remained relatively stable despite the rapid evolution of the technology.
Second, concerning methodological and demographic factors, no significant differences were found based on Experimental Design (Q_b(1) = 0.06, p = 0.803) or Educational Level (Q_b(1) = 0.10, p = 0.755). While the Secondary level (g = 0.70) showed a nominally larger effect size than the University level (g = 0.55), this comparison requires cautious interpretation given the substantial imbalance in the number of studies (k = 2 for secondary vs. k = 24 for university).
Third, in terms of Treatment Period, interventions lasting 8 weeks or less (g = 0.72) exhibited a larger mean effect size compared to those lasting 9 weeks or more (g = 0.52). However, this difference was not statistically significant (Q_b(1) = 0.41, p = 0.524), indicating that ChatGPT integration is associated with meaningful non-linguistic benefits regardless of whether the intervention is intensive (short-term) or extensive (semester-long).
Finally, the analysis of Pedagogical Roles (Roles 1–6) indicated that the specific manner in which ChatGPT was utilized did not significantly moderate the outcomes (all p > 0.05). Descriptively, studies utilizing ChatGPT for Idea Generation & Resource Search (Role 1, g = 0.71) and Others (Role 6, g = 0.71) reported the highest effect sizes, while Drafting Assistance (Role 2, g = 0.48) and Tutoring (Role 5, g = 0.63) also showed medium-to-large effects. This suggests that ChatGPT serves as a versatile tool, effectively supporting learners’ affective and cognitive needs across various instructional functions.
Despite investigating these five moderators, the residual heterogeneity within subgroups remained high (mostly I2 > 90%). This persistent heterogeneity suggests that other unobserved factors—such as teacher feedback quality, specific prompt engineering strategies, or individual learner differences—may be driving the variation in results.

4.5. Meta-Regression Analysis: Intervention Duration

Given the substantial heterogeneity observed in the overall analysis (I2 = 94.19%), a meta-regression analysis was conducted to examine whether the duration of the intervention (measured in weeks) could explain the variation in effect sizes. Table 5 presents the coefficients and statistical significance of the model, and Figure 3 illustrates the relationship visually.
Table 5. Meta-regression analysis of intervention duration.
Figure 3. Bubble plot illustrating the relationship between intervention duration and effect sizes.
The meta-regression analysis revealed that intervention duration was not a significant moderator of the overall effect (b = −0.008, p = 0.712). This indicates that, within the observed range of 4 to 15 weeks, the length of the intervention did not linearly predict the magnitude of non-linguistic outcomes. However, the restricted range of durations and the relatively small number of studies (k = 22) may have limited the statistical power to detect a longitudinal trend.
This trend is visually corroborated by the bubble plot (Figure 3), where the size of each bubble corresponds to the weight of the individual study. The regression line is almost horizontal, demonstrating that the magnitude of the effect remains relatively stable across varying timeframes, ranging from short-term interventions (e.g., 4 weeks) to semester-long courses (e.g., 15 weeks). This finding suggests that the positive impact of ChatGPT on affective and cognitive domains is robust and can manifest even within relatively short instructional periods.

4.6. Analysis of Publication Bias and Sensitivity

To ensure the trustworthiness of the meta-analytic findings, a rigorous assessment of publication bias and sensitivity was conducted. Table 6 summarizes the statistical results of the publication bias tests.
Table 6. Statistics for publication bias.

4.6.1. Assessment of Publication Bias

First, the funnel plot (Figure 4) was inspected visually. The plot displayed some asymmetry, with a cluster of studies distributed towards the right side of the mean effect. To quantify this asymmetry statistically, Egger’s regression test was performed. The results indicated a significant asymmetry (z = 2.65, p = 0.008), suggesting the potential presence of small-study effects or publication bias where studies with larger positive effects were more likely to be published.
Figure 4. Funnel plot of standard error by Hedges’ g.
However, complementary analyses suggest that the overall findings remain robust despite this asymmetry. The Trim-and-Fill method (Duval & Tweedie, 2000), which imputes hypothetical missing studies to correct for funnel plot asymmetry, identified zero missing studies on the left side (kmissing = 0). Consequently, the adjusted effect size remained identical to the observed random-effects estimate. Furthermore, Rosenthal’s Fail-Safe N yielded a value of 3778, which far exceeds the recommended tolerance level of 140 (5k + 10, where k = 26). This implies that nearly 3800 unpublished studies with null results would be required to overturn the significant finding of this meta-analysis. Collectively, while some asymmetry exists—likely exacerbated by the high heterogeneity (I2 > 90%) among studies—the core finding of a positive ChatGPT effect is unlikely to be an artifact of publication bias.

4.6.2. Sensitivity Analysis

To verify the stability of the results against outliers, a leave-one-out sensitivity analysis was conducted. This procedure involved iteratively removing one study at a time and recalculating the pooled effect size. The analysis revealed that the overall effect size fluctuated between 0.49 and 0.58, consistently maintaining a statistically significant medium effect (p < 0.001) regardless of any single study’s exclusion.
Notably, the study by Noh (2025) was identified as the most influential single study. Excluding Noh (2025) resulted in the largest decrease in the overall effect size (from g = 0.55 to g = 0.49). This finding was corroborated by the Baujat plot (Figure 5), which visually identifies studies contributing excessively to the overall heterogeneity and influence. In Figure 5, the study labeled ‘1’ (Noh, 2025) is located in the top-right quadrant, indicating it contributes disproportionately to the overall heterogeneity (I2) and the summary result.
Figure 5. Baujat plot identifying influential studies contributing to heterogeneity.
Despite identifying Noh (2025) as a significant outlier, its exclusion did not alter the direction or statistical significance of the meta-analytic conclusion. The effect size remained close to 0.50, reinforcing that the positive impact of ChatGPT on non-linguistic outcomes is a robust phenomenon not driven solely by a single outlier.
Finally, an additional sensitivity analysis was conducted to evaluate the impact of the imputed pre-post correlation (r) on the overall effect size, as the majority of included studies did not report this value. By varying the assumed correlation from 0.30 to 0.70, the stability of the pooled estimate against potential variations in study variance was assessed. As presented in Table 7, the pooled Hedges’ g remained remarkably consistent across all scenarios, with values ranging from 0.54 to 0.56. In every case, the overall effect remained statistically significant (p < 0.001) and within the medium-effect range. These results confirm that the baseline assumption of r = 0.50 was a conservative and robust choice that did not disproportionately influence the meta-analytic conclusions.
Table 7. Sensitivity analysis of pre-post correlation (r).

5. Discussion

The primary objective of this meta-analysis was to synthesize the outcomes associated with ChatGPT-integrated English instruction on non-linguistic outcomes within the South Korean educational context. By aggregating 26 effect sizes from 22 independent studies (N = 1303), the analysis identified an overall random-effects estimate of 0.55 (95% CI [0.38, 0.73]). This result indicates that ChatGPT-integrated instruction is associated with statistically significant and practically meaningful positive trends in learners’ non-linguistic outcomes.
However, a defining characteristic of the findings is the extreme heterogeneity (I2 = 94.19%), which is further reflected in the wide 95% prediction interval (PI: −0.20 to 1.30). While the pooled average (g = 0.55) indicates a substantial overall potential for ChatGPT, the fact that the PI includes zero suggests that the positive outcomes linked to the intervention are not guaranteed in every classroom. This dispersion is unsurprising given the diverse ecological implementations in L2 research. Consequently, the field is characterized by a strong positive trend alongside a heightened sensitivity to how the technology is integrated.

5.1. The Affective–Cognitive Gap: A Tentative Pattern

The patterns across the 22 included studies reveal a potential ‘Affective–Cognitive Gap.’ While the Q_b test was non-significant, the descriptive disparity suggests a more immediate positive trend in the affective realm than in higher-order cognitive processing. This pattern, although tentative, serves as an important cautionary signal regarding ‘cognitive offloading’.
Outcomes observed in the Affective Domain are consistent with the premise that LLM-based tools lower psychological barriers by offering immediate, low-stakes interaction and supportive feedback. In the Korean EFL context, where fear of negative evaluation often constrains participation, ChatGPT is associated with the role of a private interlocutor, linked to gains in confidence and willingness to communicate. This psychological boost appears to be a crucial companion to linguistic growth. When compared with J.-Y. Lee (2025), who identified a medium-to-large improvement in English writing proficiency within the same South Korean context, the proximity of affective findings in this study suggests that ChatGPT’s efficacy is multi-dimensional.
However, the empirical patterns across the 22 included studies reveal a notable ‘Affective–Cognitive Gap.’ While these individual studies did not formally quantify ‘cognitive load’ or ‘mental effort’ using validated scales, their quantitative results and qualitative findings provide strong indirect evidence for this phenomenon. For instance, Noh (2025) explicitly reported that ChatGPT reduced the burden of sentence generation and lexical search, which likely catalyzed the exceptionally large gain in learner confidence. Yet, the marginal effect sizes in the cognitive domain reported in several domestic studies, such as Im and Cha (2024) and Cha et al. (2023), suggest that when AI handles the linguistic ‘heavy lifting,’ learners may bypass the essential cognitive struggle required for cognitive/metacognitive development.
This gap can be further elucidated through the concept of ‘cognitive offloading.’ N.-Y. Kim and Kim (2025) warn that excessive reliance on ChatGPT is linked to a state where students depend on technology for their thinking processes, associated with a potential decrease in their critical thinking abilities. Similarly, J. Kim (2024) observed that while AI facilitates immediate task completion, Korean EFL learners often struggle to internalize metacognitive writing strategies, failing to develop as competent and independent writers. These observations are corroborated by the qualitative concerns regarding ‘dependency’ raised by J.-H. Lee (2025) and H. Park (2024), who noted that over-reliance could hinder the development of a learner’s own creative thinking and writing skills.
When synthesized with Deng et al. (2025)’s finding that ChatGPT significantly reduces learners’ mental effort, a clear pattern emerges. The tool primarily facilitates ‘doing more’ by offloading cognitive tasks, allowing learners to feel more confident and supported. However, as evidenced by the small cognitive effect size in this meta-analysis, the reduction in mental effort effectively bypasses the ‘desirable difficulties’ required for deeper cognitive engagement. Without intentional scaffolding to re-introduce these challenges, ChatGPT remains a tool that makes learners ‘feel better’ about their performance without necessarily making them ‘think deeper’ about the learning process itself.

5.2. Interpreting Non-Significant Moderators Amidst High Heterogeneity

Despite the extreme heterogeneity, the pooling of results remains defensible and necessary to identify the ‘central tendency’ of outcomes associated with the use of ChatGPT in the South Korean EFL landscape. In applied linguistics meta-analyses, extreme variance often reflects the complex reality of classroom-based research rather than a failure of the statistical model. The use of a random-effects model, explicitly accounting for between-study variance, results in an ecologically valid synthesis that acknowledges such diversity. The lack of significant categorical moderators suggests that the observed variance may be driven by unmeasured qualitative factors, such as the ‘instructional depth’ of the AI integration.
Notably, the lack of significant difference between educational levels is consistent with a growing body of domestic evidence suggesting the tool’s pervasive association. Within the South Korean context, J.-Y. Lee (2025) reported consistent positive trends across secondary and tertiary levels (p = 0.655). This is further supported by Rha and Baek (2025) and Rha et al. (2025), who found that college students’ satisfaction and confidence in using ChatGPT remained relatively uniform regardless of demographic variables such as gender or academic major. These findings, paired with Heung and Chiu’s (2025) international meta-analysis, suggest that the positive associations with ChatGPT are robust and pervasive, transcending specific learner demographics or institutional settings.
Furthermore, the ‘pedagogical role’ coding may be too coarse to capture the nuances of implementation. The qualitative synthesis of the 22 studies reveals that the quality of mediation is a more critical determinant than the nominal role. For instance, P. Park and Kim (2024) demonstrated that the synergy of ‘ChatGPT-assisted teacher feedback’ was linked to distinct motivational and performance trends compared to traditional or AI-only feedback. Similarly, the persistent residual heterogeneity within subgroups (I2 > 90%) suggests that unmeasured factors—such as the explicitness of prompt engineering instruction and the level of learner proficiency—are likely driving the variation in results. Thus, the outcomes of GenAI integration may be more closely linked to the ‘instructional depth’ and ‘teacher involvement’ than on the broad categorical variables typically analyzed in meta-analytic models.

5.3. Duration and the Novelty Effect: A Theoretical Hypothesis

The meta-regression indicated no significant relationship between intervention duration and effect size (b = −0.008, p = 0.712), with an essentially flat slope. While this suggests that non-linguistic benefits can emerge even in relatively brief implementations, this non-significant result must be interpreted with caution. Given the relatively small number of studies (k = 22) and the restricted duration range (4–15 weeks), the analysis may have lacked sufficient statistical power to detect a subtle longitudinal trend.
Nevertheless, this lack of duration-based variation is consistent with recent domestic findings; for instance, J.-Y. Lee (2025) reported that intervention duration did not significantly moderate the linguistic gains of ChatGPT in Korean classrooms (p = 0.552). One potential explanation for this lack of a linear duration effect is the ‘novelty effect’. Theoretically, the initial excitement of interacting with GenAI may be associated with an early peak in affective engagement, which then plateaus as the tool becomes integrated into the classroom routine.
This hypothesis finds tentative support in broader AI-mediated learning trends. Guan et al. (2024) observed that in informal digital English learning (IDLE), short-term interventions were associated with a significantly larger effect size (g = 0.672) compared to long-term interventions (g = 0.334). In the South Korean context, this pattern is descriptively illustrated by the studies of Rha et al. (2025) and Rha and Baek (2025). Despite a 15-week (full semester) implementation, these studies found no statistically significant improvement in students’ satisfaction or confidence between the pre- and post-tests.
Such evidence supports that early positive changes may be linked to rapid changes in perceived support—driven by the novelty of the AI tool—rather than sustained cognitive or affective growth. Once this novelty wears off, learners may reach a ‘plateau effect.’ Therefore, supporting these outcomes over the long term may be associated with evolving pedagogy—such as progressively sophisticated prompting and critical evaluation routines. For non-linguistic outcomes, the ‘quality of orchestration over time’ appears to be more critical than mere ‘time on tool,’ a hypothesis that warrants further investigation through longitudinal research spanning beyond a single semester.

5.4. Pedagogical Implications

The findings of this meta-analysis offer concrete implications for EFL practitioners, emphasizing that the value of ChatGPT depends not on its mere presence, but on its strategic and teacher-mediated integration. While the results provide evidence for outcomes associated with ChatGPT, they also suggest the potential importance of a pedagogical shift to move beyond the initial ‘novelty effect’.

5.4.1. Using ChatGPT for Psychological Safety and Failure-Friendly Environments

ChatGPT should be leveraged for ‘affective support’ to create a low-anxiety learning environment. In the South Korean EFL context, where fear of negative evaluation often inhibits participation, ChatGPT serves as a private, non-judgmental interlocutor. This provides an environment where learners may experiment with language in a ‘failure-friendly’ space before presenting their work to a human audience. This approach is strongly supported by the medium-to-large affective gains (g = 0.67) observed in this study. Specifically, the tool’s ability to ‘reduce the burden’ of initial drafting—as seen in Noh’s (2025) study where self-efficacy surged (g = 2.55)—suggests that AI integration is associated with the reduction in psychological barriers that traditionally hinder verbal and written production. By addressing these psychological safety needs, educators can better support the ‘agentic engagement’ required for long-term growth.

5.4.2. Bridging the ‘Affective–Cognitive Gap’ Through Desirable Difficulties

Instructors should consider proactively addressing the observed Affective–Cognitive Gap (g = 0.17 for cognitive outcomes) through structured teacher-led mediation. The significant reduction in mental effort (g = −0.675) reported in Deng et al. (2025) warns of a ‘cognitive offloading’ effect, where learners may bypass the essential mental struggle required for deep learning. To counteract this, instruction should transition toward a ‘ChatGPT-assisted teacher feedback’ model, which has been associated with more effective for motivation and performance than AI-only interaction. ‘Prompt-Reflect-Verify’ routine can be proposed to re-introduce ‘desirable difficulties’ into the learning process:
Strategic Planning: Learners must plan prompts aligned with specific task rubrics rather than using generic commands (Zimmerman, 2002).
Evaluative Comparison: Students should compare AI-generated output against their original drafts, identifying specific linguistic or structural changes (P. Park & Kim, 2024).
Justification of Choice: Learners must justify why they accepted or rejected specific AI suggestions, fostering metacognitive awareness (Xia et al., 2025).
Documentation: Keeping a ‘Revision Log’ to track decision-making ensures that learners remain the primary agents of their writing, transforming them into independent practitioners rather than passive users (J. Kim, 2024).

5.4.3. Cultivating Multi-Dimensional AI Literacy as a Meta-Skill

Fostering multi-dimensional AI literacy appears to be beneficial as a fundamental meta-skill. The findings indicate that structured integration is associated with significant enhancements in digital competencies (g = 0.59). However, this literacy must extend beyond technical operation to include critical and ethical competence. Following the evidence from N.-Y. Kim et al. (2025), instructors are encouraged to consider tasks that require students to verify AI-generated ‘hallucinations’ or analyze the socio-cultural biases inherent in LLM outputs. Choi and Choe (2024) further emphasize that explicit instruction on AI principles helps students move beyond superficial usage. By embedding these evaluative practices, educators can repurpose the efficiency afforded by AI toward higher-order critical thinking tasks, potentially supporting ChatGPT’s role as a catalyst for intellectual maturity rather than a mere crutch.

5.5. Limitations and Future Directions

Several limitations constrain the interpretation of the results. First, the evidence base remains heavily concentrated in tertiary education. Even among the most recent wave of studies included in this synthesis (e.g., Kang, 2025; J.-H. Lee, 2025; H. Park, 2024), the focus is almost exclusively on university-level EFL learners. This demographic imbalance severely limits the generalizability of the findings to primary or secondary contexts, where cognitive maturity and digital literacy levels differ significantly.
Second, a critical methodological limitation is the dominance of pre-experimental designs, which account for 86.4% of the analyzed studies. Such designs are inherently vulnerable to threats to internal validity, such as maturation and history effects. Consequently, the results should be interpreted as associative rather than causal. Future research must prioritize rigorous quasi-experimental designs or randomized controlled trials (RCTs) to isolate the genuine effect of GenAI from other pedagogical variables.
Third, as non-linguistic outcomes in the current pool are predominantly measured via self-reports, there is a risk of ‘social desirability bias’ or ‘novelty-driven satisfaction.’ Future studies should triangulate these perceptions with behavioral indicators, such as revision logs, interaction traces, or prompt history, to clarify whether observed positive trends reflect enduring metacognitive capacities or merely short-term psychological boosts.
Fourth, the ‘plateau effect’ observed in long-term implementations (e.g., Rha et al., 2025) suggests a need for longitudinal research that tracks learner trajectories beyond a single semester. Future syntheses should aim to separate ‘tool effects’ from ‘pedagogical design effects’ by investigating how different levels of teacher mediation—such as the ‘ChatGPT-assisted teacher feedback’ model proposed by P. Park and Kim (2024)—influence the sustainability of non-linguistic gains. Such efforts will be vital in shifting the research agenda from ‘whether AI works’ to ‘how it can be sustained’ as a meaningful educational catalyst.
Finally, a conceptual limitation arises from construct aggregation within broad outcome domains. While grouping variables into domains such as ‘Affective’ or ‘Cognitive’ allows for an assessment of the central tendency, these pooled estimates combine conceptually distinct constructs—such as anxiety, motivation, and self-efficacy—that were measured using heterogeneous instruments. Consequently, the high heterogeneity observed (I2 = 94.19%) may partly reflect the diversity of these underlying psychological constructs rather than a uniform effect of the intervention. This aggregation may affect the granularity of interpretability, and future research is encouraged to conduct construct-specific syntheses as the evidence base expands.

6. Conclusions

In sum, this meta-analysis provides convergent evidence that ChatGPT-integrated English instruction in South Korea is associated with a meaningful positive trend in learners’ non-linguistic outcomes (g = 0.55), with particularly strong gains in affective factors and AI/digital literacy. Nonetheless, the very high heterogeneity and the comparatively weaker effects on cognitive/metacognitive domains indicate that benefits are not uniform.
The findings suggest that while ChatGPT is linked to lower barriers to entry for language practice, it does not guarantee deeper learning on its own. Consequently, the focus of future research and practice must shift from mere adoption to instructional design—specifically, scaffolding that fosters reflective, self-regulated, and critical use of AI. Ultimately, achieving durable educational value depends not on the tool itself, but on how effectively educators can orchestrate its integration to foster environments where learners not only feel better but think deeper.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Data Availability Statement

The original contributions presented in this study are included in the article. Further inquiries can be directed to the corresponding author.

Acknowledgments

During the preparation of this manuscript, the author used ChatGPT-5.1 solely for language editing purposes.

Conflicts of Interest

The author declares no conflicts of interest.

Appendix A. Coding Results

Table A1. Study Characteristics and Individual Effect Sizes.

References

  1. Amano, T., González-Varo, J. P., & Sutherland, W. J. (2016). Languages are still a major barrier to global science. PLoS Biology, 14(12), e2000933. [Google Scholar] [CrossRef] [Scilit]
  2. Bandura, A. (1997). Self-efficacy: The exercise of control. W. H. Freeman. [Google Scholar]
  3. Baujat, B., Mahé, C., Pignon, J. P., & Hill, C. (2002). A graphical method for exploring heterogeneity in meta-analysis: Application to a meta-analysis of 65 trials. Statistics in Medicine, 21(18), 2641–2652. [Google Scholar] [CrossRef] [Scilit]
  4. Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021). Introduction to meta-analysis (2nd ed.). Wiley. [Google Scholar]
  5. Brown, J. D. (1988). Understanding research in second language learning: A teacher’s guide to statistics and research design. Cambridge University Press. [Google Scholar]
  6. Card, N. A. (2012). Applied meta-analysis for social science research. The Guilford Press. [Google Scholar]
  7. Cha, M., Im, H., & Lee, Y. (2023). The effects of English major college students’ self-regulated learning and grit. Culture and Convergence, 45(9), 1325–1334. [Google Scholar] [CrossRef] [Scilit]
  8. Choe, Y. (2025). Effects of ChatGPT-generated feedback on genre-specific writing competence perception. Modern English Education, 26, 197–212. [Google Scholar] [CrossRef] [Scilit]
  9. Choi, S., & Choe, Y. (2024). Exploring impacts of AI-based English instruction on Korean middle school EFL students’ AI literacy and perceptions. Multimedia-Assisted Language Learning, 27(1), 96–130. [Google Scholar] [CrossRef]
  10. Cohen, J. (1988). Statistical power for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates. [Google Scholar]
  11. Cooper, H. (2017). Research synthesis and meta-analysis: A step-by-step approach (5th ed.). Sage Publications. [Google Scholar]
  12. Deng, R., Jiang, M., Yu, X., Lu, Y., & Liu, S. (2025). Does ChatGPT enhance student learning? A systematic review and meta-analysis of experimental studies. Computer & Education, 227, 105224. [Google Scholar] [CrossRef] [Scilit]
  13. Dörnyei, Z. (2005). The psychology of the language learner: Individual differences in second language acquisition. Routledge. [Google Scholar]
  14. Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455–463. [Google Scholar] [CrossRef] [Scilit]
  15. Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ, 315(7109), 629–634. [Google Scholar] [CrossRef] [Scilit]
  16. Guan, L., Li, S., & Gu, M. M. (2024). AI in informal digital English learning: A meta-analysis of its effectiveness on proficiency, motivation, and self-regulation. Computers and Education: Artificial Intelligence, 7, 100323. [Google Scholar] [CrossRef] [Scilit]
  17. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Academic Press. [Google Scholar]
  18. Heung, Y. M. E., & Chiu, T. K. F. (2025). How ChatGPT impacts student engagement from a systematic review and met-analysis study. Computers and Education: Artificial Intelligence, 8, 100361. [Google Scholar] [CrossRef] [Scilit]
  19. Hong, W. C. H. (2023). The impact of ChatGPT on foreign language teaching and learning: Opportunities in education and research. Journal of Educational Technology and Innovation, 5(1), 37–45. [Google Scholar] [CrossRef] [Scilit]
  20. Hwang, Y., & Lee, J.-Y. (2025). Exploring the effects of MyGPTs-based chatbot design on pre-service English teachers’ prompt literacy competence. Studies in English Education, 30(2), 209–234. [Google Scholar] [CrossRef] [Scilit]
  21. Im, H., & Cha, M. (2024). The effects of online reading strategies on English writing strategies: The mediating effects of ChatGPT. Culture and Convergence, 46(1), 529–537. [Google Scholar]
  22. Jantakoon, T., Jantakun, T., Jantakun, K., Pongpanich, W., Pasmala, R., Wannapiroon, P., & Nilsook, P. (2025). The effectiveness of artificial intelligence in English instruction for speaking and listening skills: A meta-analysis. Contemporary Educational Technology, 17(4), ep596. [Google Scholar] [CrossRef] [Scilit]
  23. Kang, E. Y. (2025). Enhancing L2 learners’ outcomes and oral proficiency through AI chatbot interaction. Korean Journal of English Language and Linguistics, 25, 1299–1314. [Google Scholar] [CrossRef] [Scilit]
  24. Kim, H.-S. (2024). A comparative study of students’ perceptions on ChatGPT and machine translators. Convergence Studies in English Language & Literature, 9(3), 339–368. [Google Scholar] [CrossRef] [Scilit]
  25. Kim, H.-S. (2025). AI-supported language learning for developing English listening and speaking skills and ChatGPT literacy. Korean Journal of English Language and Linguistics, 25, 1353–1377. [Google Scholar] [CrossRef] [Scilit]
  26. Kim, J. (2024). Cumulative impact of L1 and AI tools on English writing strategies among EFL learners. English21, 37(1), 119–140. [Google Scholar] [CrossRef]
  27. Kim, N.-Y., Cha, Y., & Kim, H.-S. (2025). Exploring ChatGPT literacy in EFL reading education. Multimedia-Assisted Language Learning, 28(1), 39–60. [Google Scholar] [CrossRef]
  28. Kim, N.-Y., & Kim, H.-S. (2024). An empirical study on the utilization of the Large Language Model, in English education. International Journal of Contents, 20(3), 48–61. [Google Scholar] [CrossRef] [Scilit]
  29. Kim, N.-Y., & Kim, H.-S. (2025). The impact of AI-based English classes on students’ multimodal literacy, critical thinking, and creative thinking. Convergence Studies in English Language & Literature, 10(2), 57–85. [Google Scholar] [CrossRef] [Scilit]
  30. Kim, S., & Lee, J. (2023). The effect of voice-enabled ChatGPT on speaking proficiency, affective and metacognitive domains of Korean EFL learners. Journal of Research in Curriculum & Instruction, 27(4), 344–359. [Google Scholar] [CrossRef]
  31. Kohnke, L., Moorhouse, B. L., & Zou, D. (2023). ChatGPT for language teaching and learning. RELC Journal, 54(2), 537–550. [Google Scholar] [CrossRef] [Scilit]
  32. Krashen, S. D. (1982). Principles and practice in second language acquisition. Pergamon Press. [Google Scholar]
  33. Lee, J.-H. (2025). ChatGPT-assisted writing: Learners’ perceptions of AI feedback and influencing factors. Korean Journal of English Language and Linguistics, 25, 493–515. [Google Scholar] [CrossRef] [Scilit]
  34. Lee, J.-Y. (2025). A meta-analysis of ChatGPT’s effects on English writing in Korean classrooms. English Teaching, 80(5), 175–193. [Google Scholar] [CrossRef] [Scilit]
  35. Lee, J.-Y., & Hwang, Y. (2022). A meta-analysis of the effects of using AI chatbot in Korean EFL education. Studies in English Language & Literature, 48(1), 213–243. [Google Scholar] [CrossRef]
  36. Lee, S., & Kang, S. (2025). Self-directed learning of English writing using learning contract and ChatGPT. Studies in Linguistics, 74, 249–275. [Google Scholar] [CrossRef] [Scilit]
  37. Lee, S.-Y., & Lee, Y.-J. (2025). The effects of TED-Ed and ChatGPT-based English listening classes on EFL university students’ listening achievement and listening self-efficacy. The Journal of Linguistic Science, 114, 295–316. [Google Scholar] [CrossRef] [Scilit]
  38. Liu, G. L., Darvin, R., & Ma, C. (2025). Exploring AI-mediated informal digital learning of English (AI-IDLE): A mixed investigation of Chinese EFL learners’ AI adoption and experiences. Computer Assisted Language Learning, 38(7), 1632–1660. [Google Scholar] [CrossRef] [Scilit]
  39. Lo, C. K. (2023). What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13(4), 410. [Google Scholar] [CrossRef] [Scilit]
  40. Lorenc, Y., Felix, L., Petticrew, M., Melendez-Torres, G. J., Thomas, J., Thomas, S., O’Mara-Eves, A., & Richardson, M. (2016). Meta-analysis, complexity, and heterogeneity: A qualitative interview study of researchers’ methodological values and practices. Systematic Review, 5, 192. [Google Scholar] [CrossRef] [Scilit]
  41. Morrison, A., Polisena, J., Husereau, D., Moulton, K., Clark, M., Fiander, M., Mierzwinski-Urban, M., Clifford, T., Hutton, B., & Rabb, D. (2012). The effect of English-language restriction on systematic review-based meta-analyses: A systematic review of empirical studies. International Journal of Technology Assessment in Health Care, 28(2), 138–144. [Google Scholar] [CrossRef] [Scilit] [PubMed]
  42. Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., & Qiao, M. S. (2021). Conceptualizing AI literacy: An exploratory review. Computers and Education: Artificial Intelligence, 2, 100041. [Google Scholar] [CrossRef] [Scilit]
  43. Noh, Y. (2025). A study on the use of ChatGPT in a college English composition course. The New Studies of English Language & Literature, 91, 183–208. [Google Scholar] [CrossRef] [Scilit]
  44. Oxford, R. K. (1990). Language learning strategies: What every teacher should know. Newbury House. [Google Scholar]
  45. Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., … Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, n71. [Google Scholar] [CrossRef] [Scilit]
  46. Papadopoulos, V. (2025). On the appropriateness of fixed correlation assumptions in repeated-measures meta-analysis: A Monte Carlo assessment. Stats, 8(3), 72. [Google Scholar] [CrossRef] [Scilit]
  47. Park, H. (2024). ChatGPT-integrated English writing: Writing strategies and perceptions. The Linguistic Association of Korea Journal, 32(3), 85–103. [Google Scholar] [CrossRef]
  48. Park, P., & Kim, T.-W. (2024). The effects of ChatGPT-assisted teacher feedback on Korean college students’ English writing. The Journal of Yeolin Education, 32(6), 43–65. [Google Scholar] [CrossRef] [Scilit]
  49. Plonsky, L., & Oswald, F. L. (2014). How big is “big”? Interpreting effect sizes in L2 research. Language Learning, 64(4), 878–912. [Google Scholar] [CrossRef] [Scilit]
  50. Posit Software PBC. (2025). RStudio (Version 2025.09.2+418). Available online: https://posit.co/ (accessed on 1 October 2025).
  51. Rha, K.-H., & Baek, J.-Y. (2025). Examining changes in Korean college students’ perceptions toward ChatGPT use in their English class. International Journal of Internet, Broadcasting and Communication, 17(1), 346–351. [Google Scholar] [CrossRef]
  52. Rha, K.-H., Baek, J.-Y., & Huh, S. (2025). Exploring Korean college students’ voices on utilizing ChatGPT in their English speaking. Studies in Linguistics, 75, 61–88. [Google Scholar] [CrossRef] [Scilit]
  53. Rosenthal, R. (1979). The “file drawer problem” and tolerance for null results. Psychological Bulletin, 86(3), 638–641. [Google Scholar] [CrossRef]
  54. Seo, S., & Kim, J.-R. (2023). A meta-analysis on the effects of using AI chatbots in Korean English education. Foreign Languages Education, 30(3), 25–42. [Google Scholar] [CrossRef] [Scilit]
  55. Shi, J., Shim, J., & Ryu, T. (2024). The effect of ChatGPT’s grammar and reformulation feedback on EFL college students’ writing performance. Journal of AI Humanities, 18, 49–74. [Google Scholar] [CrossRef] [Scilit]
  56. Song, E., & Kim, H.-S. (2025). Investigating the effects of ChatGPT-assisted English speaking practice based on student engagement levels. Journal of Research in Curriculum & Instruction, 29(5), 298–313. [Google Scholar] [CrossRef]
  57. Sterne, J. A. C., Egger, M., & Smith, G. D. (2001). Investigating and dealing with publication and other biases in meta-analysis. BMJ, 323(7304), 101–105. [Google Scholar] [CrossRef] [Scilit]
  58. Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1–48. [Google Scholar] [CrossRef] [Scilit]
  59. Wang, J., & Fan, W. (2025). The effect of ChatGPT on students’ learning performance, learning perception, and higher-order thinking: Insights from a meta-analysis. Humanities & Social Sciences Communications, 12, 621. [Google Scholar] [CrossRef] [Scilit]
  60. Wang, Y. (2025). A study on the efficacy of ChatGPT-4 in enhancing students’ English communication skills. Sage Open, 15(1). [Google Scholar] [CrossRef] [Scilit]
  61. White, C. A., Uttl, B., & Holder, M. D. (2019). Meta-analyses of positive psychology interventions: The effects are much smaller than previously reported. PLoS ONE, 14(5), e0216588. [Google Scholar] [CrossRef] [Scilit]
  62. Xia, Q., Li, W., Yang, Y., Weng, X., & Chiu, T. K. F. (2025). A systematic review and meta-analysis of the effectiveness of Generative Artificial Intelligence (GenAI) on students’ motivation and engagement. Computers and Education: Artificial Intelligence, 9, 100455. [Google Scholar] [CrossRef] [Scilit]
  63. Zhao, H., & Aryadoust, V. (2024). A meta-analysis of the reliability of second language reading comprehension assessment tools. Studies in Second Language Acquisition, 47(1), 388–416. [Google Scholar] [CrossRef] [Scilit]
  64. Zhao, L., Wu, X., & Luo, H. (2022). Developing AI literacy for primary and middle school teachers in China: Based on a structural equation modeling analysis. Sustainability, 14(21), 14549. [Google Scholar] [CrossRef] [Scilit]
  65. Zimmerman, B. J. (2002). Becoming a self-regulated learner: An overview. Theory Into Practice, 41(2), 64–70. [Google Scholar] [CrossRef] [Scilit]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Article Metrics

Citations

Article Access Statistics

Journal Statistics
Multiple requests from the same IP address are counted as one view.
The Intersection Between Moodle, Active Methodologies, and Artificial Intelligence in Higher Education: A Narrative Review and Thematic Analysis
Previous
What Science Fairs Reveal About STEM Learning
Next