Can Generative Artificial Intelligence Effectively Enhance Students’ Mathematics Learning Outcomes?—A Meta-Analysis of Empirical Studies from 2023 to 2025

Abstract

Generative artificial intelligence (GenAI) shows transformative potential in mathematics education. However, empirical findings remain inconsistent, and a systematic synthesis of its effects across distinct engagement dimensions is lacking. This preregistered meta-analysis (INPLASY2025110051) systematically reviewed 22 empirical studies (46 independent samples, N = 5232) published between 2023 and 2025. The results indicated that GenAI has a moderate positive impact on students’ mathematics learning outcomes (g = 0.534). Moderation analysis further revealed that the level of GenAI integration in teaching, sample size, and learning content are the primary factors influencing this effect. The study found that the effect was most pronounced under the creative transformation (CT) integration mode, was significant when applied to geometry learning, and was stronger in studies with small samples or small class sizes; collaborative learning approaches also significantly enhance these mathematics learning outcomes. By contrast, educational stage and intervention duration did not show significant moderating effects. The GRADE assessment indicated that while the overall evidence is supportive, the certainty of evidence is stronger for cognitive outcomes than for non-cognitive domains. The findings also offer a reference for future research on constructing a human–machine collaborative learning environment.

1. Introduction

Mathematics, as a fundamental discipline, occupies a central position in the cultivation of innovative talent. Its role as the foundation for advances in science, engineering and digital technologies has been widely recognised. However, the subject’s abstract nature and rigorous logic often present substantial challenges for students, giving rise to problems such as learning anxiety and reduced motivation (Passolunghi et al., 2020). The information age has arrived. Consequently, contemporary mathematics education can no longer focus solely on knowledge transmission. Instead, it must cultivate students’ critical thinking, creativity, and self-regulated learning. The emergence of GenAI has provided a new route for addressing these challenges and for realising that transformation of educational aims. GenAI tools generate text, step-by-step explanations, mathematical proofs and visual content, and they have transformed the potential for creating personalised, interactive mathematics learning environments (Kasneci et al., 2023; Wardat et al., 2023). This gives them transformative potential in mathematics education, supporting students’ active knowledge construction and providing scaffolding in collaborative learning. However, empirical evidence regarding the effectiveness of GenAI in mathematics education has been contradictory and inconsistent. Some studies have affirmed its positive role in providing immediate feedback and personalised tutoring, which has enhanced students’ problem-solving abilities and learning motivation (Walkington, 2025). Conversely, other studies have reported that excessive reliance on GenAI can reduce students’ capacity for deep thinking and may negatively affect the long-term development of mathematical skills (Kim et al., 2024). Moreover, some empirical results have shown no significant effects. Therefore, this study adopts a meta-analysis to investigate systematically the impact of generative artificial intelligence on students’ mathematics learning outcomes and to explore potential moderators, aiming to provide practical guidance for teaching practitioners and other professionals in mathematics education.

2. Literature Review

2.1. GenAI Applications in Mathematics Education

GenAI, a branch of artificial intelligence that creates new content, offers wide application prospects in mathematics education. By enabling natural-language interaction and content generation, it provides tools for personalised learning and complex problem solving, and its application scenarios span K-12 through higher education (Wardat et al., 2023). Research indicates that GenAI promotes cognitive skills through several mechanisms. For example, it can provide instant feedback, construct conceptual frameworks and deliver multimodal visualisations to help students grasp mathematical content (Qu et al., 2025). In higher education, undergraduates who use GenAI to support mathematical proof work deepen their conceptual understanding with personalised feedback and animated demonstrations (De Simone et al., 2025). In middle school, GenAI reduces students’ cognitive load when learning number theory and algebra, thereby improving task-completion efficiency (Polydoros et al., 2025). In primary school, it helps to make abstract concepts more concrete and thus facilitates mastery of basic knowledge. GenAI also exerts a positive influence on non-cognitive skills. Studies indicate that by offering personalised learning pathways and contextualised tasks, GenAI improves students’ learning emotions and motivation. For example, after using GenAI, primary pupils reported reduced mathematics anxiety and increased classroom participation (Yoon et al., 2024). X. Wang and Wei (2025) likewise found that it effectively lowered students’ mathematics anxiety, enhanced self-efficacy and increased academic engagement. For teachers, GenAI assists in lesson design through iterative interaction, thereby reducing their burden of preparation (Yanar & Ergene, 2025). It can also simulate students in conversational practice, providing pre-service teachers with opportunities for teaching rehearsal (L. Zhou, 2025). However, several studies have identified potential risks and challenges. For example, an experiment found that undergraduates who used GenAI in calculus tests scored lower than those in a traditional teaching group (Sánchez-Ruiz et al., 2023). Scholars argue that when GenAI supplies complete solutions, students may engage less in questioning and reflection, which undermines the development of critical thinking (Ali et al., 2024). Although immediate feedback can reduce cognitive load, it may also constrain deep thinking and the transfer of concepts to complex problems (Bastani et al., 2024). Moreover, GenAI can produce incorrect or incomplete explanations, causing systematic misunderstandings (Marzano, 2025), and prolonged use has been associated with reduced learning confidence and increased technological anxiety (Al-Smadi, 2023).

2.2. Meta-Analysis Evidence of the Impact of Artificial Intelligence on Students’ Learning

GenAI in education has attracted considerable attention, and several meta-analyses have examined its overall impact. Existing research indicates that GenAI positively influences academic performance, with most effect sizes ranging from moderate to large. For instance, Gu and Yan (2025) reported a moderate effect on academic achievement (g = 0.683) and found that teacher support significantly enhanced this effect. J. Wang and Fan (2025) documented a large effect on academic performance (g = 0.867), alongside moderate effects on learning perceptions and higher-order thinking. Notably, studies employing a cognitive objective framework suggest that the impact of GenAI varies across skill levels. Qu et al. (2025) demonstrated a stronger effect on lower-order cognitive skills (g = 0.926) than on higher-order cognitive skills (g = 0.640). Since learning is shaped by multidimensional factors, recent work has also begun to assess effects on both cognitive and non-cognitive skills simultaneously (Xia et al., 2025). However, most existing comprehensive syntheses span multiple disciplines and lack a specific focus on mathematics—a field characterised by its distinct structure of thinking and instructional logic.

In mathematics education, earlier meta-analyses have examined conventional artificial intelligence technologies, such as intelligent tutoring systems and adaptive learning systems, and reported small to moderate benefits for mathematics learning. For example, Yi et al. (2025) analysed 21 studies from 2000 to 2023 and showed that artificial intelligence produced a small positive effect on K-12 students’ mathematics performance (g = 0.343), with the effect moderated by learning content and grade level. S. Hwang (2022) likewise reported a small facilitating effect of AI on primary-school mathematics learning (g = 0.351). These findings provide a foundation for understanding technology-assisted learning. However, the novel features of GenAI, notably generativity and strong interactivity, suggest that its pedagogical integration and impact mechanisms may differ. Therefore, a focused, comprehensive evaluation of generative AI’s effects within the mathematics discipline is required, together with further exploration of the specific conditions and boundaries under which those effects arise.

We specifically examine three categories of moderating variables: (1) GenAI application features, the level of GenAI integration, and learning mode (e.g., independent or collaborative); (2) intervention settings, intervention duration and sample size; and (3) educational context, educational stage and learning content. By integrating existing evidence, we provide a focused, comprehensive review to clarify the role of GenAI in mathematics teaching. We incorporate both cognitive and non-cognitive outcomes within a single analytical framework to analyse systematically how the level of technology integration influences effects. We anticipate that this work will supply more detailed evidence for future research and offer practical insights for educators designing GenAI-supported mathematics learning environments. On this basis, the study explores the following three questions through meta-analysis.

Q1: What is the overall effect of GenAI on students’ mathematics learning outcomes, distinguishing cognitive from non-cognitive skills and examining differences between higher- and lower-order cognitive skills?

Q2: To what extent is this relationship influenced by key moderating variables, including intervention duration, the level of GenAI integration, educational stage, learning content, and sample size?

3. Methods

3.1. Literature Search

This study strictly adhered to the PRISMA guidelines to ensure the methodological process was systematic and transparent. The literature screening procedure is shown in Figure 1 and comprised four consecutive stages: (1) identifying relevant records through systematic database searches; (2) conducting a preliminary screen of titles and abstracts; (3) performing full-text evaluation of potentially eligible studies; and (4) including studies that met the prespecified criteria (Page et al., 2021). Searches were carried out in major academic databases, including Web of Science, EBSCO, CNKI and Google Scholar. The search strategy centred on three concepts: generative artificial intelligence, the mathematics discipline and learning outcomes. Search terms were adjusted to match each database’s syntax and vocabulary features to balance recall and precision. The detailed search strategy appears in Appendix A 

Table A1. Literature Search Strategy.

Component	Description
Databases/Platforms	Web of Science, EBSCO (e.g., ERIC, APA PsycINFO), CNKI, Google Scholar
Time Frame	1 January 2023–30 September 2025
Search Strategy	Boolean queries were constructed by combining terms from four core conceptual groups using the AND operator:
	1. Technology: (“generative AI” OR “generative artificial intelligence” OR ChatGPT OR “GenAI” OR “large language model” OR “AI-powered” OR “AI-driven”)
	2. Subject: (math OR mathematics OR algebra OR geometry OR calculus OR statistics OR “problem-solving”)
	3. Outcome: (learn OR performance OR achievement OR outcomes OR anxiety OR attitudes OR motivation OR “computational thinking” OR skill)
	4. Population: (student OR pupil OR learner OR “elementary school” OR “primary school” OR “middle school” OR “high school” OR “undergraduate” OR “higher education”)
	The specific syntax and field codes were adapted for each database.
Additional Searches	Manual screening of reference lists and citation tracking for included studies.

. To maximise coverage, we also performed supplementary searches by tracing reference lists of included studies and following reviews in related fields.

The inclusion and exclusion criteria are presented in

Table 1. Literature Screening Criteria.

Screening Stage	Inclusion Criteria	Exclusion Criteria	Literature Count
Initial Screening after	1. Records identified from Databases (Web of Science, EBSCO, CNKI), Google Scholar, Other methods	1. Repetitive literature was removed (n = 369)	Initial: 2104 After: 1658
		2. Records removed for other reasons (n = 77)
Titles/Abstract Screening	1. Records assessed for relevance to the research topic	Records excluded as not related to the research topic (n = 1489)	Initial: 1658 After: 169
Full-Text Eligibility Assessment	1. Experimental/quasi-experimental studies employing GenAI as the intervention, with a control group using traditional instructional methods	1. Non-experimental/quasi-experimental design (n = 124)	Initial: 169 After: 22
	2. Provided complete effect size data or data calculable for effect sizes (e.g., means, standard deviations, sample sizes).	2. Incomplete data for effect size calculation (n = 22)
	3. MERSQI score ≥ 10.5 points	3. MERSQI score below 10.5 (n = 1)

. Given that key applications of generative artificial intelligence emerged only at the end of 2022, many relevant empirical studies in education began to appear after 2023. This study restricts the retrieval period to studies published between 1 January 2023 and 30 September 2025. The remainder of this section outlines the detailed implementation of the research methods.

A total of 22 articles were ultimately included. To ensure their quality, we evaluated the studies using the Medical Education Research Study Quality Instrument (MERSQI). This instrument has good reliability and validity for assessing the quality of quantitative research in educational settings. It comprises 10 items across six key domains. The specific scoring rules are shown in Appendix A 

Table A2. MERSQI Quality Assessment Scores.

No.	Author(s) & Year	1	2	3	4	5	6	7	8	9	10	Total
1	Febriantoro et al. (2024)	2	0.5	0.5	2	1	0	2	1	2	1	12
2	Polydoros et al. (2025)	2	0.5	0.5	2	1	0	2	1	2	1	12
3	Sánchez-Ruiz et al. (2023)	2	0.5	0.5	1.5	1	0	2	1	1	1	10.5
4	Wahba et al. (2024)	2	0.5	0.5	2	2	0	2	1	2	1	13
5	Yavich (2025)	3	0.5	0.5	1.5	2	0	2	1	2	1	13.5
6	X. Wang and Wei (2025)	2	0.5	0.5	1.5	2	0	2	1	1	1	11.5
7	Xing et al. (2025)	2	0.5	0.5	2	2	0	2	1	2	1	13
8	Kadhim and Fares (2025)	2	0.5	0.5	2	1	0	2	1	2	1	12
9	Noviyana et al. (2025)	2	0.5	0.5	2	2	0	2	1	2	1	13
10	Luo et al. (2024)	2	0.5	0.5	2	2	0	2	1	1	1	12
11	Dasari et al. (2024)	2	0.5	0.5	2	2	0	2	1	2	1	13
12	Karaman and Göksu (2024)	2	0.5	0.5	2	2	0	2	1	2	1	13
13	Utami et al. (2024)	2	0.5	0.5	2	2	0	2	1	2	1	13
14	Xuan et al. (2025)	2	0.5	0.5	2	2	0	2	1	2	1	13
15	Nakavachara et al. (2025)	2	0.5	0.5	2	2	0	2	1	2	1	13
16	Liao (2024)	2	0.5	0.5	2	2	0	2	1	2	1	13
17	X. C. Liu and Zhang (2025)	2	0.5	0.5	2	2	0	2	1	2	1	13
18	Fardian et al. (2025)	2	1	0.5	2	2	0	2	1	2	1	13.5
19	Adelegan (2023)	2	0.5	0.5	2	1	0	2	1	2	1	12
20	J. Liu et al. (2025)	2	0.5	0.5	2	2	0	2	1	2	1	13
21	Alvarez (2024)	2	0.5	0.5	1.5	1	0	2	1	1	1	10.5
22	R. Zhou et al. (2025)	2	0.5	0.5	2	2	0	2	1	2	1	13

.

3.2. Data Encoding

Learning effectiveness serves as a critical metric for assessing educational quality and is conceptualised as a composite construct shaped by the synergistic influence of both cognitive and non-cognitive domains (Poynton, 2015; B. Wu et al., 2023). Guided by established assessment frameworks (e.g., Bloom’s taxonomy for cognitive objectives and the taxonomy for affective domains), this study operationalizes students’ learning outcomes along two dimensions: cognitive skills and non-cognitive skills.

Cognitive skills encompass the intellectual processes engaged in mathematics learning. Following the revised Bloom’s Taxonomy (Adams, 2015), they are categorised into lower-order skills (remembering, understanding, applying) and higher-order skills (analysing, evaluating, creating). Non-cognitive skills refer to the emotional, motivational, and self-regulatory factors that, while not directly involving specific knowledge acquisition, substantially influence the learning process and its outcomes. These include mathematics anxiety, self-efficacy, learning motivation, and metacognitive strategies (Marzano, 2025).

This classification reflects a basic understanding of the complexity of learning mathematics. Learning outcomes are shaped not only by cognitive processes but also by Non-cognitive factors. Moreover, Non-cognitive skills such as emotion and motivation indirectly influenced final learning outcomes by affecting cognitive engagement and the use of strategies (Kumar et al., 2025). This perspective is consistent with existing meta-analyses, which indicate that a comprehensive assessment of the impact of educational technology must consider both Cognitive and Non-cognitive dimensions (Xia et al., 2025).

Accordingly, the coding framework categorises learning outcomes into Cognitive and non-cognitive skills. Cognitive skills are further subdivided into higher- and lower-order skills, a structure consistent with Xia et al. (2025).

Based on established meta-analysis practices (e.g., Gu & Yan, 2025; J. Wu et al., 2025), a systematic classification of moderating variables, and preliminary pilot coding, this study constructed a systematic coding framework. Two coders independently implemented the coding process, adhering to a clearly defined, comprehensive and transparent protocol (Pigott & Polanin, 2020). The coding scheme comprised two main components. The first recorded basic document information, including document ID, authors, publication year and country/region. The second comprised feature coding to extract key variables such as intervention duration, educational stage, GenAI integration level, sample size, effect size, learning content and research methods.

Initially, intervention duration was classified, following J. Wu et al. (2025), as short-term (≤1 semester) and long-term (>1 semester). Because GenAI technologies emerged relatively recently, there are few long-term intervention studies lasting more than one semester. Most existing studies examine interventions of 1 to 8 weeks. Consequently, this study adopts the same dichotomy as Stephenson (2022), classifying interventions as short-term (≤1 month) or long-term (>1 month).

Educational stages are categorised into three distinct groups: primary, secondary, and tertiary education (Bartolini et al., 2025). Based on Piaget’s cognitive-development theory, students at different stages exhibit fundamental differences. Primary school students, typically in the concrete operational stage, rely on concrete and imaginative thinking. Secondary school students, transitioning into or within the formal operational stage, develop logical thinking and problem-solving abilities. In contrast, university students, having fully entered this stage, apply professional abstract reasoning within specific disciplines (Gray, 1975). Consequently, these three stages also differ markedly in their knowledge systems and learning objectives. Primary education emphasises foundational concepts and basic skills, secondary education focuses on systematic knowledge and logical structuring, while tertiary education prioritises professional modelling, theoretical proof, and higher-order application (Ghazi et al., 2016). This three-stage categorisation effectively reduces heterogeneity arising from cognitive and contextual disparities, clarifies the functional patterns of GenAI across different educational settings, and thereby enhances the interpretability and practical utility of the meta-analysis results.

The integration of GenAI in teaching is frequently analysed using the PIC–RAT model (Borup et al., 2022). This model establishes a two-dimensional framework: the vertical axis (PIC) describes the relationship between students and technology and comprises three modes—passive, interactive and creative—while the horizontal axis (RAT) indicates the technology’s impact on teachers’ methods and comprises three levels—replacement, enhancement and transformation. So, the degree of GenAI integration in teaching is classified into nine collaboration modes: creative substitution, creative enhancement, creative transformation, interactive substitution, interactive enhancement, interactive transformation, passive substitution, passive enhancement, and passive transformation (Borup et al., 2022).

Using this framework, the extent of GenAI integration in classroom teaching and its educational implications can be systematically evaluated. In the passive mode, students primarily receive information and technology functions as a presentation tool, providing limited support for active knowledge construction or social interaction; consequently, its potential to foster deep learning is theoretically low (Srivastava et al., 2024). The interactive mode foregrounds dialogue and feedback, with technology acting as a conversational partner or scaffold within sociocultural theory. By furnishing timely support within the zone of proximal development, it is theoretically more conducive to conceptual understanding and the collaborative construction of social knowledge (Ramos et al., 2025). The creative mode requires students to employ technology to generate new content or solutions and thus most directly embodies the active, exploratory knowledge construction that constructivism advocates; theoretically, it offers the greatest potential to stimulate higher-order thinking and motivation for deep learning (Srivastava et al., 2024). Existing research shows that higher-level integration, such as Creative Transformation, enhances students’ agency and deep learning outcomes (Kimmons et al., 2022). However, the literature has not systematically examined PIC–RAT as a moderating variable, which this paper investigates further.

An analysis of the 22 included studies showed that GenAI’s collaborative role with teachers and students was primarily manifested as creative transformation and as interactive/passive enhancement. Notably, the interactive-enhancement and passive-enhancement modes frequently co-exist within the same instructional setting. This pattern may arise from the inherently multi-step structure and hierarchical objectives of mathematics teaching. Classrooms involve exploratory phases that require students to engage interactively and construct understanding, alongside instructional and practice phases that focus on knowledge transmission and skill consolidation. At different stages of instruction, teachers therefore adjust GenAI’s functional role according to specific goals, producing a dynamic teaching landscape in which multiple integration modes coexist (Shrestha & Yi, 2025).

Based on an analysis of the instructional content in the included literature, the learning material was classified into four mathematical domains: Number & Algebra, Geometry, Statistics, and Integration. This categorization follows the widely adopted content framework of TIMSS (Trends in International Mathematics and Science Study), which organises mathematics learning into key areas including Number, Algebra, Geometry, and Data & Probability. In the present meta-analysis, the domains of Number and Algebra were merged into a combined “Number & Algebra” category. This consolidation reflects their shared emphasis on symbolic reasoning and procedural thinking, as well as the frequent overlap of these topics in instructional settings. The Statistics category corresponds to content involving data analysis, probability, and statistical inference within the broader “Data & Probability” domain. Finally, Integration refers to studies that explicitly address two or more of the core domains listed above (Yi et al., 2025).

Sample size was dichotomized as large (>100 participants) or small (≤100 participants) based on the conventional cutoff proposed by Bernard et al. (2014). Learning mode was categorised into two types: Independent Learning and Collaborative Learning. Independent Learning describes contexts in which students use GenAI individually and independently to acquire knowledge or complete learning tasks (K. Wang & Guo, 2025). In contrast, Collaborative Learning refers to group-based settings where students interact with one another while using GenAI as a shared tool or mediator to co-construct understanding and solve problems (K. Wang & Guo, 2025). The detailed coding criteria for all variables are presented in

Table 2. Specific Coding Criteria.

Dimension	Category	Description	References
Cognitive or Non-cognitive	Cognitive Skills	Higher-order cognitive skills; Lower-order cognitive skills	(Xia et al., 2025)
	Non-cognitive Skills	Affective, motivational, and related abilities	(Xia et al., 2025)
Intervention Duration	Short-term	≤1 month	(Stephenson, 2022)
	Long-term	>1 month	(Stephenson, 2022)
Sample Size	Small	≤100 participants	(Bernard et al., 2014)
	Large	>100 participants	(Bernard et al., 2014)
Education Level	Primary School	Primary school students	(Bartolini et al., 2025)
	Secondary School	Junior or senior high school students	(Bartolini et al., 2025)
	Tertiary	University students	(Bartolini et al., 2025)
Learning Content	Number & Algebra	Number or Algebra	(Yi et al., 2025)
	Geometry	Geometry	(Yi et al., 2025)
	Statistics	Data and Chance	(Yi et al., 2025)
	Integration	involves two or more of the above core fields	(Yi et al., 2025)
Degree of GenAI Integration	CT	Creative Transformation	(Borup et al., 2022)
	IPA	Interactive or Passive Augmentation (A combined category for interventions fitting either or both modes)	(Borup et al., 2022)
Learning mode	Independent Learning	Self-directed use of GenAI for learning	(K. Wang & Guo, 2025)
	Collaborative Learning	Group-based interaction with GenAI for learning	(K. Wang & Guo, 2025)

.

After the coders completed their independent work, they resolved discrepancies and reached consensus through discussion. A total of 46 effect sizes were coded, involving N = 5232 participants. Inter-coder consistency was measured by Cohen’s Kappa coefficient, yielding κ = 0.906. According to the criteria of Landis and Koch (1977), this value indicates that the level of agreement met the required standard and that the coding results were highly reliable. See Appendix A 

Table A3. Document Coding List.

No.	Author(s) & Year	Country	Duration	Stage	Content	Integration	Size	Participant	H/L	C/NC	Learning Mode
1	Febriantoro et al. (2024)	Indonesia	Long	Primary School	Geometry	CT	Small	60	Low	Cognitive	Collaborative Learning
2	Polydoros et al. (2025)	Greece	N/A	Primary School	Geometry	IPA	Large	436	Low	Cognitive	Independent Learning
3	Sánchez-Ruiz et al. (2023)	Spain	Long	Tertiary	Integration	IPA	Large	245	Low	Cognitive	Independent Learning
4	Sánchez-Ruiz et al. (2023, a)	Spain	Long	Tertiary	Integration	IPA	Large	246	Low	Cognitive	Independent Learning
5	Sánchez-Ruiz et al. (2023, b)	Spain	Long	Tertiary	Integration	IPA	Large	241	Low	Cognitive	Independent Learning
6	Sánchez-Ruiz et al. (2023, c)	Spain	Long	Tertiary	Integration	IPA	Large	235	Low	Cognitive	Independent Learning
7	Sánchez-Ruiz et al. (2023, d)	Spain	Long	Tertiary	Integration	IPA	Large	238	Low	Cognitive	Independent Learning
8	Sánchez-Ruiz et al. (2023, e)	Spain	Long	Tertiary	Integration	IPA	Large	240	Low	Cognitive	Independent Learning
9	Sánchez-Ruiz et al. (2023, f)	Spain	Long	Tertiary	Integration	IPA	Large	245	Low	Cognitive	Independent Learning
10	Wahba et al. (2024)	Jordan	Short	Tertiary	Statistics	CT	Small	56	High	Cognitive	Independent Learning
11	Yavich (2025)	Israel	Long	Secondary School	Number & Algebra	IPA	Small	50	High	Cognitive	Collaborative Learning
12	X. Wang and Wei (2025)	China	Short	Primary School	Integration	IPA	Large	105	N/A	Non-cognitive	Independent Learning
13	Xing et al. (2025)	USA	Short	Secondary School	Number & Algebra	CT	Large	212	Low	Cognitive	N/A
14	Kadhim and Fares (2025)	Iraq	Long	Secondary School	Integration	IPA	Small	78	High	Cognitive	N/A
15	Noviyana et al. (2025)	Indonesia	N/A	Tertiary	Integration	IPA	Small	60	High	Cognitive	Independent Learning
16	Luo et al. (2024)	China	Long	Tertiary	Integration	IPA	Large	117	N/A	Non-cognitive	Collaborative Learning
17	Luo et al. (2024, a)	China	Long	Primary School	Integration	IPA	Large	117	N/A	Non-cognitive	Independent Learning
18	Dasari et al. (2024)	Indonesia	N/A	Primary School	Statistics	IPA	Small	20	Low	Cognitive	Independent Learning
19	Dasari et al. (2024, a)	Indonesia	N/A	Primary School	Statistics	IPA	Small	20	Low	Cognitive	Independent Learning
20	Karaman and Göksu (2024)	Turkey	Long	Tertiary	Geometry	IPA	Small	39	Low	Cognitive	Independent Learning
21	Utami et al. (2024)	Indonesia	Long	Primary School	Geometry	CT	Small	51	Low	Cognitive	Collaborative Learning
22	Utami et al. (2024, a)	Indonesia	Long	Tertiary	Geometry	CT	Small	51	Low	Cognitive	Independent Learning
23	Utami et al. (2024, b)	Indonesia	Long	Tertiary	Geometry	CT	Small	51	High	Cognitive	Independent Learning
24	Xuan et al. (2025)	Vietnam	Short	Tertiary	Number & Algebra	IPA	Small	60	Low	Cognitive	Collaborative Learning
25	Xuan et al. (2025, a)	Vietnam	Short	Primary School	Number & Algebra	IPA	Small	60	Low	Cognitive	Collaborative Learning
26	Nakavachara et al. (2025)	Thailand	Short	Secondary School	Statistics	IPA	Large	242	High	Cognitive	Independent Learning
27	Liao (2024)	China	Long	Primary School	Integration	IPA	Large	115	Low	Cognitive	N/A
28	Liao (2024, a)	China	Long	Secondary School	Integration	IPA	Large	115	N/A	Non-cognitive	Independent Learning
29	Liao (2024, b)	China	Long	Secondary School	Integration	IPA	Large	115	N/A	Non-cognitive	Independent Learning
30	Liao (2024, c)	China	Long	Secondary School	Integration	IPA	Large	115	Low	Cognitive	N/A
31	Liao (2024, d)	China	Long	Secondary School	Integration	IPA	Large	115	Low	Cognitive	N/A
32	Liao (2024, e)	China	Long	Secondary School	Comprehensive	IPA	Large	115	N/A	Non-cognitive	N/A
33	Liao (2024, f)	China	Long	Secondary School	Comprehensive	IPA	Large	115	Low	Cognitive	N/A
34	Liao (2024, g)	China	Long	Secondary School	Comprehensive	IPA	Large	115	N/A	Non-cognitive	N/A
35	X. C. Liu and Zhang (2025)	China	Long	Secondary School	Comprehensive	IPA	Large	115	N/A	Non-cognitive	N/A
36	Fardian et al. (2025)	Indonesia	N/A	Secondary School	Geometry	IPA	Small	205	High	Cognitive	N/A
37	Fardian et al. (2025, a)	Indonesia	N/A	Secondary School	Geometry	IPA	Small	22	High	Cognitive	N/A
38	Adelegan (2023)	USA	Short	Secondary School	Number & Algebra	IPA	Small	18	Low	Cognitive	Independent Learning
39	Adelegan (2023, a)	Nigeria	Short	Secondary School	Number & Algebra	IPA	Small	28	Low	Cognitive	Independent Learning
40	Adelegan (2023, b)	Finland	Short	Secondary School	Number & Algebra	IPA	Small	28	Low	Cognitive	Independent Learning
41	Adelegan (2023, c)	USA	Short	Secondary School	Number & Algebra	IPA	Small	62	Low	Cognitive	Independent Learning
42	Adelegan (2023, d)	Nigeria	Short	Secondary School	Number & Algebra	IPA	Small	62	Low	Cognitive	Independent Learning
43	Adelegan (2023, e)	Finland	Short	Secondary School	Number & Algebra	IPA	Small	44	Low	Cognitive	Independent Learning
44	Z. Liu et al. (2025)	China	Short	Primary School	Number & Algebra	IPA	Large	104	Low	Cognitive	Independent Learning
45	Alvarez (2024)	Philippines	Short	Tertiary	Number & Algebra	IPA	Small	20	Low	Cognitive	Independent Learning
46	R. Zhou et al. (2025)	China	N/A	Tertiary	Statistics	IPA	Small	29	Low	Cognitive	Independent Learning

Note: “N/A” indicates that the information is not applicable or was not specified in the original study. H/L: Higher-order/Lower-order cognitive outcome. C/NC: Cognitive/Non-cognitive outcome.

for detailed coding items and Appendix A 

Table A4. Research Characteristics and Effect Size Distributions.

Characteristic	Category	Number of Effect Sizes (n)	% of Effect Sizes
Research Design	Quantitative research	8	17.39%
	Mixed-methods research	38	82.61%
Region	Asia	31	67.39%
	Europe	10	21.74%
	North America	3	6.52%
	Other (MENA, Africa)	2	4.34%
Grade Level	Primary School	10	21.74%
	Secondary School	20	43.48%
	Tertiary	16	34.78%
Mathematics Content	Number & Algebra	12	26.09%
	Geometry	8	17.39%
	Statistics	5	10.87%
	Integration	21	45.65%
Outcome Type	Cognitive Skills	38	82.61%
	Lower-order Cognitive	30	65.22%
	Higher-order Cognitive	8	17.39%
	Non-cognitive Skills	8	17.39%
Intervention Duration	Short-term (≤1 month)	14	30.43%
	Long-term (>1 month)	25	54.35%
	Not specified	7	15.22%
Integration Degree	CT	6	13.04%
	IPA	40	86.96%
Sample Size	Large	23	50.00%
	Small	23	50.00%
Learning mode	Independent Learning	29	63.04%
	Collaborative Learning	6	13.04%
	Not specified	11	23.92%

for Research Characteristics and Effect Size Distributions.

3.3. Data Analysis

All data analysis and modelling were carried out in R (version 4.3.0) using the metafor package (version 4.0.0). The meta-analysis incorporated 46 independent effect sizes drawn from 22 studies. Hedges’g was computed for each effect size from reported means, standard deviations, sample sizes, or test statistics. Heterogeneity among studies was then assessed using the Q-test and the I² statistic, with the Q-test considered significant at p < 0.10, and I² values interpreted as follows: 25% low, 50% moderate, and 75% high heterogeneity (Borenstein et al., 2017). A random-effects model was selected for pooled analysis when heterogeneity was significant; otherwise, a fixed-effects model was considered. The Q-test was significant (p < 0.001) and I² equalled 48.94%, indicating moderate heterogeneity; consequently, a random-effects model was employed for the subsequent analyses.

To account for differences in true effects across studies and to address the potential multilevel data structure (that is, multiple effect sizes within the same study), we fitted a three-level model. The model indicated that level-2 (within-study) variance accounted for 0% of the total variance while level-3 (between-study) variance accounted for 84.18%, implying that heterogeneity principally arose from differences among studies (Assink & Wibbelink, 2016). The results of model comparisons are reported in

Table 3. Model Fit Comparison Results.

	DF	AIC	BIC	AICc	logLik	LRT	p-Value	QE
Full	3	94.98	100.4	95.57	−44.49			90.91
Reduced	2	92.98	96.6	93.27	−44.49	0	1	90.91

. The two-level model showed lower AIC and BIC values, and the likelihood ratio test was not significant (p = 1.000); hence, a two-level random-effects model was deemed more appropriate. We therefore treated the 46 effect sizes as independent observations. When only a very small number of studies contribute multiple effect sizes, this approach has a negligible impact on meta-analysis results (Van Den Noortgate et al., 2015).

We adopted the restricted maximum likelihood (REML) estimator as the primary method for the random-effects model. This choice rested on methodological considerations: when the number of studies is small or when subgroups contain few observations, the DerSimonian–Laird (DL) estimator may yield unstable and biased estimates of the heterogeneity variance (Veroniki et al., 2016). REML provides more accurate estimates under finite-sample conditions.

To assess robustness, we performed a sensitivity analysis using three estimators, as shown in

Table 4. Sensitivity Analysis of Effect Sizes Across Different Estimators.

	Estimate	SE	CL	Tau2	ST	I2	H2	Q	P
DL	0.535	0.098	[0.343, 0.728]	0.2142	0.463	50.5	2.02	90.91	0.001
REML	0.534	0.097	[0.345, 0.723]	0.2013	0.449	48.9	1.96	90.91	0.001
SJ	0.547	0.117	[0.318, 0.776]	0.3908	0.391	65.1	2.86	90.91	0.001

. The point estimates of the overall effect size from the three random-effects models were highly consistent (g = 0.535–0.547) and all reached statistical significance. The SJ estimator produced a wider confidence interval and a larger heterogeneity estimate (τ² = 0.391), reflecting its more conservative weighting of between-study variation; nevertheless, the conclusion that GenAI yields a moderate positive effect on mathematics learning remained unchanged.

These results indicate that the principal findings were insensitive to the choice of estimator and thus robust. For balance between small-sample accuracy and broad methodological acceptance, we therefore report the results based on REML.

3.4. Experimental Results

Publication bias arises when the selection of studies for publication is influenced by the direction or strength of their results, often due to preferences of journal editors, reviewers, and researchers. To assess publication bias in the included studies, this study employed multiple methods: visual inspection of funnel plots, the fail-safe number test, Egger’s linear regression test, and trim-and-fill method.

A funnel plot was drawn with the effect size (Hedges’g) on the x-axis and the reciprocal of its standard error on the y-axis (Figure 2). Under the random-effects model, the scatter was roughly symmetric and resembled a typical inverted funnel, which preliminarily suggests no obvious publication bias. Egger’s linear regression test was used to quantify funnel-plot asymmetry. The regression intercept did not deviate significantly from zero (t = 1.68, p = 0.107), so there was no statistical support for the presence of publication bias. The fail-safe number was Nfs = 3030, which far exceeded Rosenthal’s (1979) robustness threshold (5k + 10 = 120, k = 22). This implies that over 3000 unpublished null-effect studies would need to be added to render the pooled effect size non-significant, providing evidence for the robustness of the meta-analysis findings.

To further assess potential publication bias, we applied the Duval and Tweedie (2000) trim-and-fill method. After imputing three theoretically missing small-effect studies on the left side of the funnel plot, the distribution became symmetrical. Although the adjusted pooled effect size decreased slightly relative to the original estimate, it remained within the range of a statistically significant moderate positive effect. This indicates that, even when potential missing studies were taken into account, the positive effect of GenAI on mathematics learning remained robust and the core conclusion did not change.

It should be noted, however, that publication bias remains an inherent methodological concern in any meta-analysis. Although our statistical tests did not indicate significant asymmetry, the possibility of unpublished or small-effect studies cannot be entirely ruled out—a consideration particularly relevant given the emerging nature of research on GenAI in mathematics education.

On the basis of these analyses and considerations, we concluded that publication bias did not pose a substantial threat to the study’s main findings.

4. Results

This study systematically evaluated the impact of GenAI on students’ mathematics learning outcomes through a meta-analysis, encompassing both cognitive and non-cognitive dimensions. It also examined the moderating effects of intervention duration, educational stage (K-12 versus university), learning content area (algebra, geometry, etc.), sample size, and the degree of GenAI integration in instruction. To enhance transparency, the distribution of effect sizes from each independent study is displayed in a forest plot (see Figure A1 in Appendix B).

4.1. GenAI Exerts a Moderate Positive Impact on Students’ Mathematics Learning Outcomes

Based on the 22 included studies, 46 effect sizes and 5232 participants, we conducted a meta-analysis, as shown in

Table 5. Meta-Analytic Results of GenAI’s Effects on Mathematics Learning Outcomes: Overall Effects, Cognitive/Non-Cognitive Skills.

Outcome Variables	n	g	95% CI	Q	P
Overall	46	0.534 ***	[0.345, 0.723]
Cognitive	38	0.596 ***	[0.367, 0.824]	1.355	0.2443
Non-cognitive	8	0.299	[−0.003, 0.601]

Note. *** p < 0.001.

. The overall effect size of GenAI on mathematics learning outcomes was g = 0.534 (p < 0.001), which, according to Cohen (2009)’s criteria, falls in the medium-effect range.

To examine how GenAI differentially affected ability types, the study performed a subgroup analysis using Bloom’s taxonomy of educational objectives. The analysis showed that GenAI produced a clear and significant enhancement of cognitive skills, yielding an overall effect size of g = 0.596 (p < 0.001), a moderately large effect. Further breakdown indicated a particularly strong effect on higher-order cognitive skills (g = 0.718, p < 0.001), while lower-order cognitive skills also benefited consistently (g = 0.569, p < 0.001), as shown in

Table 6. Meta-Analytic Results of Gen AI’s Effects on Mathematics Learning Outcomes: Higher/Lower-Order Cognitive Skills.

Outcome Variables	n	g	95% CI	Q	P
Cognitive-high	8	0.718 ***	[0.344, 1.092]	1.735	0.42
Cognitive-low	30	0.569 ***	[0.298, 0.840]

Note. *** p < 0.001.

. By contrast, GenAI’s overall impact on non-cognitive skills was smaller and did not reach statistical significance (g = 0.299, p = 0.052); the confidence interval included the null value, indicating that current evidence was insufficient to draw a firm conclusion.

Using the GRADE framework to appraise the robustness of the evidence, we found that while the point estimates suggest a positive impact, key limitations temper the confidence in these conclusions. For overall effects and cognitive skills, moderate to substantial heterogeneity across studies indicates that the effects are context-dependent and not uniform. For non-cognitive outcomes, the very small number of available studies results in imprecise effect estimates, as reflected in wide confidence intervals that include the null value. Consequently, this latter finding in particular should be viewed as preliminary, and all conclusions are amenable to change with future research.

4.2. Regulatory Effect Analysis

To examine how moderating variables affect students’ mathematics learning outcomes, this study performed a subgroup analysis of intervention duration, grade level, teaching content, degree of GenAI integration in instruction, and sample size. The specific results are presented in

Table 7. Subgroup Analysis Results of Moderator Variables.

Variables			n	g	95% CI	Q	P
intervention settings	Intervention Duration	Long	25	0.376 ***	[0.172, 0.579]	3.330	0.1892
		Short	14	0.735 ***	[0.468, 1.002]
		N/A	7	0.672	[−0.466, 1.810]
	Sample Size	Small	23	0.832 ***	[0.470, 1.193]	7.501	0.0062
		Large	23	0.336 ***	[0.151, 0.522]
educational context	Learning Content	Integration	21	0.256 **	[0.081, 0.431]	10.750	0.0131
		Geometry	8	0.906 **	[0.366, 1.446]
		Number & Algebra	12	0.784 ***	[0.469, 1.098]
		Statistics	5	0.775	[−0.742, 2.293]
	Grade Level	Primary School	10	0.754 **	[0.196, 1.313]	3.811	0.1487
		Secondary School	20	0.313 **	[0.105, 0.520]
		Tertiary	16	0.667 ***	[0.285, 1.049]
GenAI application features	Integration Degree	CT	6	1.164 ***	[0.656, 1.673]	6.624	0.0101
		IPA	40	0.443 ***	[0.252, 0.634]
	Learning Mode	Independent Learning	29	0.592 ***	[0.328, 0.856]	7.372	0.0251
		Collaborative Learning	6	1.008 ***	[0.522, 1.494]

Note. ** p < 0.01, *** p < 0.001. “N/A” indicates that the information is not applicable or was not specified in the original study.

.

The analysis indicates that interventions of different durations positively affect students’ mathematics learning outcomes, although the between-group difference did not reach statistical significance (p = 0.189). Specifically, short-term interventions (≤1 month) yielded a large effect size (g = 0.735, p < 0.001), while long-term interventions (>1 month) also produced a significant, though smaller, positive effect (g = 0.376, p < 0.001).

The moderating effect of teaching content on effect size was statistically significant (p = 0.013). Geometric content produced the largest effect (g = 0.906, p = 0.001), which lies in the large-effect range. Number & Algebra content followed (g = 0.784, p < 0.001), corresponding to a medium-to-large effect. Comprehensive content also showed a significant positive impact (g = 0.256, p = 0.004), amounting to a small-to-medium effect. By contrast, the Statistics category had a relatively high point estimate (g = 0.775) but an extremely wide confidence interval and did not reach statistical significance (p = 0.317). Consequently, this result was unstable and should be interpreted with caution.

When grade level was included as a moderator, the between-group difference was not statistically significant (p = 0.149). Generative AI had a significant positive effect on mathematics learning for primary school students (g = 0.754, p < 0.01), secondary school students (g = 0.313, p < 0.01), and tertiary education students (g = 0.667, p < 0.001). All effect sizes fell within the small-to-large range, and no significant difference was observed among the three educational stages.

Analysis using the PIC–RAT framework indicates significant between-group differences in how integration level affects learning outcomes (p = 0.010). The creative-transformation integration mode produced a very large positive effect (g = 1.164, p < 0.001), while the interaction/passive-enhancement mode yielded a small-to-medium effect (g = 0.443, p < 0.001). The creative-transformation mode was significantly superior to the interaction/passive-enhancement mode.

The analysis also revealed significant differences in effect sizes among different sample-size groups (p = 0.006). Small-sample studies showed a relatively large effect size (g = 0.832, p < 0.001), indicating a medium-to-large effect. In contrast, large-sample studies showed a relatively small effect size (g = 0.336, p < 0.001), suggesting a small-to-medium effect.

The analysis revealed a statistically significant difference in effect sizes between the two learning modes (p = 0.025). Both modes independently demonstrated statistically significant positive effects. Specifically, studies employing an independent learning mode yielded a moderate effect size (g = 0.592, p < 0.001). In contrast, studies adopting a collaborative learning mode showed a significantly larger effect, falling within the large range (g = 1.008, p < 0.001).

A notable consideration is the varying evidential basis across moderator subgroups. Several findings with the largest effect sizes, including those for the Creative Transformation integration mode, Geometry content, and Collaborative Learning are derived from a limited number of independent studies. This necessitates a cautious interpretation of these estimates, as their precision and stability are correspondingly lower.

5. Discussion

5.1. Responses to the First Research Question

In response to the first research question regarding the effect of GenAI on students’ mathematics learning outcomes, this study quantified its impact on both cognitive and non-cognitive skills.

This study found that the enhancing effect of GenAI on mathematical cognitive skills (g = 0.596) exceeded the results reported in previous meta-analyses of general educational AI (e.g., Yi et al., 2025; S. Hwang, 2022). This enhancement may arise from fundamental differences in interaction paradigms between GenAI and earlier AI. Traditional intelligent tutoring systems mostly provide rule-based feedback, whereas GenAI externalises the problem-solving process through chain-of-thought reasoning and natural language dialogue, thereby offering students procedural cognitive scaffolds that can reduce extraneous cognitive load (Sweller, 2011), allowing students to focus their mental resources on deeper conceptual understanding. This capability aligns precisely with the constructivist view that knowledge is actively constructed through social interaction and meaning negotiation (Sánchez Muñoz et al., 2025). Consequently, GenAI can serve as a dynamic cognitive partner in mathematical inquiry, assisting students to shift their focus from obtaining answers to constructing mathematical reasoning itself (Walkington, 2025). Ultimately, the realisation of these benefits depends on how instructional designs and student interactions strategically leverage GenAI as a cognitive tool.

A detailed analysis of cognitive skills reveals a pedagogically important trend: GenAI shows a numerically greater facilitative effect on higher-order cognitive skills (analysis, evaluation, creation; g = 0.718) than on lower-order skills (memory, comprehension, application; g = 0.569). Although the between-group difference did not reach statistical significance, possibly because of the limited number of relevant studies, this pattern suggests that GenAI’s potential is particularly pronounced in supporting tasks that require deep processing, strategic thinking and creative output. For example, in activities such as mathematical proof (Yoon et al., 2024) or problem posing (Walkington et al., 2025), GenAI can act as a thinking collaborator that supports analytical verification and generative reasoning rather than merely serving as an aid for fact recall or procedural practice. Despite the current non-significant difference, the trend provides a preliminary basis for designing GenAI integration models that emphasise the development of higher-order thinking.

The effect size for cognitive skills in this study (g = 0.596) is slightly lower than that reported for multidisciplinary applications of GenAI (Gu & Yan, 2025). This difference highlights the adaptation challenges between the current capabilities of GenAI and the demands of mathematical rigour. Mathematics learning requires logical exactitude, precision in symbol manipulation and a tightly structured body of knowledge. Research has shown that GenAI can produce plausible but incorrect arguments or minor calculation errors when handling mathematical content (Mustapha et al., 2024; Yoon et al., 2024), and these flaws may impede the development of students’ rigorous mathematical thinking. Therefore, to realise deep integration of GenAI into mathematics education, it is urgent to develop subject-specific tools that offer greater transparency in reasoning, reliable symbolic calculation and structured cognitive scaffolding (Hetmanenko & Khoruzha, 2025).

In contrast, the impact of GenAI on non-cognitive skills (g = 0.299, p = 0.052) was marginally significant, with the effect size approaching the conventional threshold and being numerically lower than reports from other fields (e.g., Xia et al., 2025). This finding may reflect the complexity and persistence of emotions related to mathematics learning. Math anxiety and self-efficacy are often intertwined with long-standing belief systems and deep-seated situational factors (Sammallahti et al., 2023). Although evidence indicates that GenAI can reduce cognitive load in specific tasks by means of step-by-step problem decomposition and immediate feedback (Cosentino et al., 2025), short-term, problem-solving-oriented interactions typically struggle to address the diverse, entrenched causes of math anxiety, such as fear of negative evaluation, performance concerns under high time pressure, and an aversion to highly abstract concepts. Therefore, GenAI system design should evolve from an “efficient problem-solving assistant” to a “companion throughout the learning process.”; Systems must incorporate finer-grained multimodal emotion recognition, analysis of learning engagement, and adaptive motivational frameworks (Barno & Phelps, 2025), thereby moving beyond mere task support to offer personalised and empathetic emotional and motivational scaffolding.

5.2. Responses to the Second Research Question

In response to the second research question, the focus is on variables that moderate effectiveness.

Both short-term (≤1 month, g = 0.735) and long-term (>1 month, g = 0.376) interventions produced positive effects on mathematics learning outcomes, with the short-term intervention showing a larger effect size. However, the difference between the groups did not reach statistical significance (p = 0.189). This finding aligns with previous meta-analyses of educational technology (Ma et al., 2014; Al-Smadi, 2023). The absence of a statistically significant advantage for longer interventions, coupled with the numerically larger point estimate for short-term studies, warrants critical examination. We propose that this pattern serves as a diagnostic mirror reflecting the predominant level of GenAI integration in current practice. The initial, robust effect in short-term interventions can be theoretically linked to the “novelty effect” and heightened “situational interest” (L. Zhou, 2025), which boost engagement and cognitive investment when a new technology is introduced. However, the attenuated effect in longer-term implementations suggests a potential pitfall of superficial integration. This approach often remains at the level of “enhancement/substitution,” primarily relying on technology to reduce short-term cognitive load and provide immediate feedback. If GenAI is used primarily for cognitive offloading (e.g., providing answers) or repetitive practice without fostering deeper cognitive partnership, its initial benefits are susceptible to the limitations outlined by Klar (2025) and may plateau or decline. This can lead to “technology dependency” (J. Liu et al., 2025), where students’ intrinsic motivation and development of metacognitive skills and self-efficacy are undermined, explaining the more modest effect over time.

Therefore, mere length of time is not the key factor. What truly matters is whether progressive instructional scaffolding is designed during technology integration and whether it can support students’ transformation from tool users to cognitive partners (Wulff & Kubsch, 2025). Future research is necessary to further uncover the mediating mechanisms and boundary conditions of GenAI’s impact on learning outcomes across different time spans. Particular attention should be paid to the dynamic interactions among technical proficiency, instructional design adaptability, and students’ self-regulation abilities.

The findings of this study indicate that GenAI positively influences mathematics learning outcomes across different educational stages (primary: g = 0.754; secondary: g= 0.313; tertiary: g = 0.667). However, the differences in effect sizes between stages were not statistically significant (p > 0.05). The higher effect size observed in primary education can be attributed to the strong alignment between GenAI’s capacity for concrete, multimodal generation and students’ cognitive need for concrete operational thinking. At this stage, mathematics learning focuses on building foundational concepts, and GenAI’s strengths in contextualization and visualisation support this process (Walkington et al., 2025; Pando & Leon, 2025). At the tertiary level, the relatively higher effect size reflects learners’ ability to engage in what can be described as “critical collaboration” with GenAI. University students generally possess more advanced formal operational thinking and metacognitive skills. They are able to use GenAI as a “thinking partner” to explore complex problems and verify reasoning (Yoon et al., 2024) while self-regulating their interaction with the technology (J. Liu et al., 2025). This allows for extended thinking through critical dialogue. In contrast, the lower effect size in secondary education coincides with a key period for developing internalised abstract logical reasoning. If GenAI is used primarily as a tool for obtaining answers, this conflicts with the goal of fostering deep reasoning (Song et al., 2024; Zhuang, 2025), potentially limiting GenAI’s positive impact. Overall, these patterns highlight how the effectiveness of GenAI is closely tied to learners’ cognitive characteristics, disciplinary tasks, and how the tool is used at each educational stage.

This study found that GenAI exerts a positive effect across mathematical content domains, but effect sizes differ significantly (p = 0.013). Geometry (g = 0.906), Number & Algebra (g = 0.784) and Statistics yielded a large but non-significant point estimate (g = 0.775, p = 0.317), whereas the effect for comprehensive content was relatively small (g = 0.256). This disparity may arise from differing degrees of alignment between GenAI’s core capabilities and the principal cognitive tasks in each domain. The relatively large effect in geometry learning might reflect GenAI’s multimodal generation, which supplies intuitive visual support. For example, in the study by Segal and Klemer (2025), teachers used GenAI to design dynamic geometry exploration tasks, suggesting that it can act as a visual mediator between abstract properties and intuitive representations and so promote spatial reasoning. The advantage observed in Number & Algebra learning may stem from a strong compatibility between GenAI’s chain-of-thought reasoning and procedural symbolic operations. Effect sizes in fields such as statistics tend to be relatively large, possibly because the demands for data processing, algorithm implementation and knowledge retrieval in these areas align naturally with the text- and code-generation strengths of GenAI. By contrast, comprehensive content learning typically requires deep concept integration, cross-domain transfer and higher-order problem solving. The smaller effect sizes observed in this domain may not reflect GenAI’s unsuitability (Manzke et al., 2025); rather, instructional design often remains limited to retrieval or practice when employing GenAI, and fails to integrate it as a cognitive partner that supports cross-domain inquiry and creative work (Wulff & Kubsch, 2025).

A further analysis suggests that the comparable effect sizes in Geometry and Number & Algebra indicate a practical ceiling for visual-spatial advantages, which likely stems from current technological limitations in complex reasoning (Oh, 2025). The non-significant effect in the field of Statistics may be due to the limited number of studies (k = 5), which inherently challenges the stability and generalizability of the observed effect.

The extent to which generative AI is integrated into teaching is a key factor modulating its effects on mathematics learning. This study found a significant difference in effect sizes between Creative Transformation (CT) and Interactive/Passive Augmentation (IPA) (p = 0.0101). Specifically, CT produced a very large effect (g = 1.164), whereas IPA produced a small-to-moderate effect (g = 0.443). When interpreting these results, it should be noted that the small number of studies in the CT category (k = 6) may reduce the stability of the estimate. Thus, the observed large effect size for CT should be interpreted with caution, pending replication in future studies with larger samples. From the perspective of the PIC–RAT integration model, this difference arises from how well each mode aligns with distinct goal levels in mathematics learning. CT ,alters the learning process through open-ended, inquiry-based tasks that commonly involve cycles of mathematical modelling, conjecture and proof (Cevikbas & Kaiser, 2021), and therefore requires students to undertake multi-step reasoning and strategy selection. For example, Polydoros et al. (2025) asked students to use ChatGPT to explore practical applications of symmetric figures, design items independently and verify their geometric properties (note: the specific version of ChatGPT was not stated in the study). This task promotes abstract reasoning and concept construction. GenAI functions as a thinking collaborator and exploration partner, and its capacity to generate diverse solution paths and perform complex reasoning (Yu et al., 2025) aligns closely with higher-order thinking activities. By contrast, IPA primarily concentrates on optimising practice and feedback within existing frameworks, and its empirical scenarios largely involve using GenAI for structured skills training (G.-J. Hwang & Tu, 2021). For example, in the study by X. Wang and Wei (2025), students used Kimi Chat for interactive exercises on geometric theorems: the tool posed questions, students answered, and when errors occurred, they received standard problem-solving steps. Although this approach improved procedural fluency, it essentially reinforced established knowledge and struggled to support deep conceptual understanding and knowledge transfer. Therefore, the notable difference in effect sizes confirmed the central conclusion that “the depth of integration determines the height of utility”. If GenAI is used merely as a tool for providing answers or practice problems (IPA), its effectiveness in enhancing learning is limited. To maximise its educational potential, instructional design must shift towards the CT model and construct a learning environment that guides students through a complete mathematical inquiry process: posing questions, modelling, reasoning, validating and reflecting. This transformation will change GenAI’s role from an answer provider to a thinking stimulator and cognitive partner.

The sample size was a significant moderator affecting the instructional effectiveness of GenAI (p = 0.0062). This study found that small-sample studies (n ≤ 100) exhibited a relatively large effect size (g = 0.832), whereas large-sample studies displayed a smaller effect size (g = 0.336). Two explanations should be considered concurrently. First, small-sample studies may have produced stronger effects because of more refined designs, closer guidance and a higher degree of personalisation (Walkington et al., 2025). Second, small-sample bias must be regarded as an important alternative explanation: small-scale studies may have overestimated effects owing to methodological flexibility or publication bias, which cautions against uncritical interpretation and generalisation. By contrast, large-scale studies commonly adopt standardised procedures, and their greater generalisability may come at the cost of reduced personalised interactions. The key challenge, therefore, lies in developing a scalable framework that systematically integrates personalised support (Barno & Phelps, 2025) to bridge the gap between scale and depth. Therefore, while pedagogical explanations are plausible, the potential for small-sample bias remains a primary alternative explanation that tempers the confidence with which the overall effect magnitude, and particularly the strong effects from small-scale studies, can be interpreted and generalised to large-scale educational contexts.

Our findings are consistent with the meta-analytic results reported by K. Wang and Guo (2025). This finding highlights the contextual adaptability of Generative AI (GenAI) in educational settings. In independent mathematics learning, GenAI functions as a personal tutor, providing step-by-step guidance and personalised feedback for exercises such as algebraic operations or geometric proofs. In collaborative learning scenarios, it acts as a facilitator, assisting groups in organising problem-solving approaches, generating visual discussion materials, and co-constructing solutions (Ye et al., 2025). This role flexibility enables both instructional modes to be effective. Furthermore, the core pedagogical value of GenAI—derived from generating contextualised content and providing immediate feedback—is inherently independent of the learning activity’s organisational format (Chen & Hou, 2024). This explains why both independent and collaborative modes exhibited comparable effectiveness in supporting mathematics learning.

Our study corroborates the findings of K. Wang and Guo (2025), confirming that both individual and collaborative learning can effectively enhance mathematics learning outcomes, with collaborative learning demonstrating a significantly stronger effect. This difference may stem from the inherent alignment between the rigorous logic of mathematics and GenAI’s capacity to function as a “dialogic partner.” In individual learning, GenAI acts as a personal tutor, whose core value lies in stimulating and sustaining continuous cognitive dialogue. For example, when learning about quadratic functions, students can ask GenAI to generate varied word problems and then question, verify, and refine the solutions and graphs it provides—a process that deepens conceptual understanding (Yoon et al., 2024; Zhuang, 2025). The larger effect size observed in collaborative learning suggests that when GenAI mediated dialogue is embedded in social interaction, a powerful synergistic effect emerges. Here, GenAI transforms from a “personal tutor” into a “collaborative cognitive tool” for the group. For instance, while exploring geometric proofs, team members can jointly propose conjectures to GenAI, ask it to generate proof strategies or counterexamples, and then discuss, critique, and synthesise the AI’s output (Walkington et al., 2025; Segal & Klemer, 2025). This collective negotiation of meaning and co-construction of knowledge significantly amplifies learning benefits (Song et al., 2024).

Furthermore, the effectiveness of GenAI is deeply dependent on disciplinary context. Its impact in mathematics education hinges on how well it integrates into specific subject practices—such as assisting teachers in designing inquiry-based tasks (Bernardi et al., 2025), developing students’ disciplinary language proficiency (Pando & Leon, 2025), or supporting the teaching of specialised content like fractal geometry (Sureda et al., 2025). Thus, GenAI is not merely a generic tool but a collaborative resource capable of embedding itself into mathematical thinking processes and fostering both logical and socially mediated cognitive development.

5.3. Practical Implications

This meta-analysis demonstrates that generative artificial intelligence (GenAI) has a moderate positive effect on mathematics learning, while multiple factors moderate its effectiveness. In particular, the mode of collaboration (individual vs. collaborative), depth of integration, learning content domain and research scale exert significant moderating effects.

For teachers, the core task is to design interventions wisely. Instructional design should strategically integrate collaborative activities where GenAI serves as a group partner. Tasks that use GenAI for exploration and creation should be prioritised over those intended only for practice. Support strategies should be tailored to the characteristics of the teaching content: when teaching structured knowledge, GenAI can provide clear steps or diagrams; when guiding students through comprehensive projects, emphasis should be placed on designing guidance, assessment and integration links.

For researchers, the immediate priority is to strengthen the robustness and depth of evidence. Future studies with larger samples and long-term follow-ups are needed to validate the effects and to conduct in depth analyses of the specific cognitive processes through which GenAI influences students’ learning. It is necessary to investigate how to implement creative transformation effectively across different teaching contents and how to design technology-intervention models that support comprehensive learning.

5.4. Limitations and Future Research

First, a key finding that warrants careful consideration is that several of the largest effect sizes in this study rest on a relatively limited evidence base. For instance, the pronounced benefits associated with the Creative Transformation integration mode, Geometry learning, and Collaborative Learning are each derived from a modest number of independent studies (k ≤ 8). Thus, these high-effect findings should be interpreted as promising yet preliminary conclusions whose robustness urgently requires verification through future large-scale research.

This limitation is further accentuated by the potential risk of small-sample bias, as indicated by the significant moderating effect of sample size. The effect sizes observed in smaller-scale studies may be influenced by methodological factors or implementation intensity. Consequently, caution is warranted when generalising these findings, particularly from studies with small samples to broader, real-world educational contexts.

In summary, a primary direction for future research is to prioritise large-scale, rigorous replication studies targeting these high-potential areas. Only such research can establish a more stable and reliable evidence base for the scalable application of GenAI in education.

6. Conclusions

This meta-analysis synthesises empirical evidence to examine the effects of GenAI on students’ mathematics learning outcomes and the contextual factors that moderate these effects. Findings indicate that GenAI exerts a moderately positive overall impact on mathematics learning, though its effects vary across outcome types. The strongest impact is observed in cognitive domains—particularly higher-order thinking—while effects on non-cognitive outcomes, though positive, are not statistically significant. Moderator analyses reveal several nuanced patterns: deeper integration of GenAI through creative transformation yields substantially larger gains than superficial interactive enhancement; benefits are more pronounced in geometry and algebra than in other content domains; and collaborative small-group use proves more effective than individual application. Caution is advised in interpreting results from small-sample studies, which may inflate effect estimates. These findings contribute to theory in educational technology and learning engagement, and offer actionable guidance for educators, educational technology developers, and policymakers advancing personalised and engaging learning ecosystems.