In this section, we present the results of both the quantitative and qualitative analyses.
3.1. Quantitative Results
Table 2 presents descriptive statistics for students in the control and treatment sections. Overall, the two groups appear reasonably comparable in terms of background characteristics.
We first summarize the results of the psychometric analyses conducted to compute the various constructs of interest from the survey data items. These analyses consisted of exploratory factor analysis using the “fa()” function in the “semTools” R package ver. 0.5-7 (
Jorgensen et al., 2025) and calculation of the McDonald’s
coefficient omega of internal reliability (
McDonald, 1999;
Flora, 2020) computed using the “cfa()” function in the “lavaan” R package ver. 0.6-19 (
Rosseel, 2012). The factor analysis of the mindset items showed a good fit for the mindset scale to the data, with the root mean square of the residuals (
RMSR) = 0.003/0.002 and
Omega = 0.69/0.73 for pre-survey/post-survey. On the other hand, the SRL items did not fit the original three-factor structure, with the items of the Seeking and Learning Information subscale not loading on a single factor as hypothesized. The two-factor structure provided an acceptable fit, with the Managing Environment and Behavior (
SRL-1) and the Maladaptive Regulatory Behaviors (
SRL-2) subscales forming two separate factors (presurvey
RMSR = 0.057 and
Omega = 0.84 & post-survey
RMSR = 0.063 and 0.83). These two subscales did not load on a common factor (pre-survey factor Pearson correlation coefficient
r = 0.032 & post-survey
r = −0.068) and were analyzed separately. The math, gender, and racial identity scales demonstrated an acceptable fit to the data, as indicated by the fit statistics (RSMR and Omega coefficient) in
Table 3. Math identity was composed of three subscales: competency, recognition, and interest, and they were kept separate in the analysis. Gender identity had two separate subscales: centrality and reflection. For racial identity, the scale for Black students—MIBI—had three subscales (centrality, private regard, and public regard), whereas the scale for non-Black students—MEIM—had two subscales (exploration and commitment). To facilitate calculating a common racial identity variable for both groups, the MIBI subscales were combined into one composite, and similarly, the MEIM subscales were combined into one composite.
Table 4 presents pre-, post-, and difference scores for math mindset and self-regulated learning (SRL) across the control (CTRL) and treatment (TRT) groups. Both cohorts showed marginal, statistically indistinguishable declines in mindset (CTRL = −0.20; TRT = −0.18): the control mean reduced from 4.47 (
SD = 0.90) to 4.27 (
SD = 0.96), whereas the treatment mean moved from 4.42 (
SD = 0.98) to 4.25 (
SD = 1.04). SRL results were similar. For SRL-1, which captures management of environment and behavior, CTRL edged up from 3.43 (
SD = 0.67) to 3.45 (
SD = 0.63) for a +0.03 gain (
SD = 0.53), while TRT rose from 3.41 (
SD = 0.66) to 3.45 (
SD = 0.66) for a +0.04 gain (
SD = 0.50). Conversely, SRL-2, where higher scores denote fewer maladaptive strategies, decreased slightly in both groups (CTRL = −0.16; TRT = −0.21), suggesting possible challenges in curbing ineffective learning habits despite the intervention.
Correlation testing, as shown in
Figure 2, mapped how the intervention’s psychological and behavioral shifts were intertwined with Research Questions 1 and 2, revealing a network of modest yet meaningful associations. Change scores on the two self-regulation dimensions rose and fell together (SRL-1 Diff × SRL-2 Diff:
r = 0.239,
p < 0.001), an alignment that grew stronger among students who received explicit learning-strategy instruction (TRT:
r = 0.269 vs. CTRL:
r = 0.204), implying the instructional scaffold promoted parallel refinement of adaptive habits and reduction of maladaptive ones. Concurrently, mindset shifts displayed small but positive ties to shifts on both SRL facets (Mindset Diff × SRL-1 Diff:
r = 0.132; Mindset Diff × SRL-2 Diff:
r = 0.209), with the association between mindset shifts and SRL-1 shifts being statistically significant only among the treatment cohort whereas the association with SRL-2 shifts was statistically significant across both control and treatment groups. This suggests that as students adjusted their regulatory repertoire, they also nudged their beliefs toward a growth orientation. Performance gains, though weaker, trended in a similar direction: score change was positively correlated with mindset shifts (
r = 0.134) and SRL-1 shifts (
r = 0.073), inching higher and being statistically significant only in the treatment group (0.151 and 0.146). A stronger correlation was observed between score change and SRL-2 shifts (
r = 0.172), but it was only significant among control students (
r = 0.244). A deeper Pearson analysis confirmed that starting dispositions carried weight such that initial and post-semester scores were moderately, but significantly, linked for mindset (
r = 0.51), SRL-1 (
r = 0.69), and SRL-2 (
r = 0.47) yet it also showed that post-semester mindset bore only low insignificant ties to early environment and behaviors regulation (
r = −0.03) and early maladaptive regulation (
r = 0.33). Collectively, these results suggest that while early traits shape future outcomes, targeted strategy instruction can promote gradual, coordinated improvements in both mindset and self-regulation.
Results from the path models are presented in
Table 5. Treatment assignment was not significantly associated with post-test GM (a
1: β = −0.046,
p = 0.599), SRL1 (a
2: β = −0.006,
p = 0.908), or SRL2 (a
2’: β = −0.082,
p = 0.178). Likewise, the post-test mediators were not significant predictors of post-test performance: GM (b
1: β = 0.887,
p = 0.366), SRL1 (b
2: β = 0.367,
p = 0.832), or SRL2 (b
2’: β = 2.308,
p = 0.102).
By contrast, the direct effect of treatment on post-test performance remained significant after adjusting for the mediators (c′: β = −3.830, p = 0.016), suggesting that treatment was directly associated with lower performance outcomes.
Table 6 shows the decomposition of the total effect. None of the indirect effects through GM (a
1 × b
1), SRL1 (a
2 × b
2), or SRL2 (a
2’ × b
2’) were statistically significant, and the sum of all indirect effects was small (β = −0.232, 95% CI [−0.984, 0.143]). The total effect of treatment on performance was significant (β = −4.061, 95% CI [−6.945, −0.947]), with only 5.7% of this effect accounted for by the mediators.
Overall, the mediation models provide little evidence that GM or self-regulated learning served as pathways through which the treatment influenced performance. Instead, the treatment effect appears to operate primarily through a direct path.
To test whether the treatment effects varied by student identity measures, we estimated the moderation models 4–6, including interaction terms between treatment (role) and math, gender, and racial identity variables. Wald χ
2 tests of the joint null hypothesis that all interaction coefficients equal zero were not significant in any model (all
p > 0.05). This indicates that the treatment effects did not significantly differ across levels of students’ math, gender, or racial identity measures. In other words, there was no evidence that identity moderated the effects of treatment on GM, SRL, or subsequent performance. Including interaction terms did not improve model fit compared to models with only main effects.
Table 7 below summarizes the results of the models with only main effects.
Consistent with the earlier path analyses (
Table 5 and
Table 6), the results in
Table 7 indicate that treatment assignment was not significantly related to post-test GM (β = −0.046,
p > 0.10), SRL1 (β = −0.006,
p > 0.10), or SRL2 (β = −0.082,
p > 0.10), providing little evidence that the learning-strategy instruction directly improved students’ GM or self-regulated learning. This addresses
RQ1 and
RQ2, suggesting that the intervention did not produce measurable gains in these psychosocial and learning-strategy constructs.
In contrast, several baseline covariates strongly predicted post-test outcomes. For example, prior mindset was positively associated with the post-test mindset (β = 0.464, p < 0.001), while (β = 0.656, p < 0.001) and (β = 0.459, p < 0.001) were the strongest predictors of their respective post-test measures. This underscores the stability of these constructs over time.
Turning to
RQ3, the performance model shows that treatment had a significant negative direct effect on post-content-test performance (β = −3.830,
p < 0.05), even after accounting for post-test mindset and SRL measures. None of the mediators significantly predicted post-test performance, consistent with the indirect effects reported in
Table 6. Instead, post-test performance was more strongly explained by academic background (e.g., pre-test score, β = 0.118,
p < 0.05), course enrollment (e.g., Calculus I: β = −18.865,
p < 0.001), and overall course grade (e.g., C grade: β = −18.173,
p < 0.001). Socio-demographic covariates such as PELL eligibility (β = 6.322,
p < 0.05) also emerged as significant predictors.
In all models, the low proportion of variance explained by the model (adjusted R2) suggests that additional factors other than those included in the models might be driving changes in GM, SRL, and performance.
Overall, the combined results from
Table 5,
Table 6 and
Table 7 show that while the intervention did not foster measurable improvements in GM or SRL, treatment had a direct negative association with post-content-test performance. Performance outcomes were shaped more by baseline academic standing, pre-course readiness, instructional context (e.g., Algebra vs. Calculus course), and demographic factors.
3.2. Qualitative Results
In this section, we report the results of the qualitative analysis. First, we summarize the focus group participation in
Table 8. A sample of the focus group questions and corresponding codes is given in
Table 9; please refer to
Appendix G for the full list of focus group questions.
Students were asked how their course differed from previous courses to determine how they responded to the discussion boards more organically, but if students in the treatment condition did not report out about the discussion boards, they were asked about them directly to help determine students’ perceptions of the treatment. In a calculus focus group, five students reported they did not have discussion boards when they did, in another calculus focus group two students did not mention the discussion boards, although they had them. The remaining focus groups and interviews remembered doing the discussion boards, but four reported that doing these boards did not change their behaviors. A student in Calculus II said, “I know a lot of the students didn’t actually look at that. They just kinda glanced over it and answered the questions, and then moved on.”
A student in Algebra II was able to give a more detailed description of the treatment activities:
Most of the discussion boards just ranged on ways to better your study habits, so we just watched a bunch of videos on different ways you could do so, … like reaching out on a textbook or peer tutoring or finding your [own private] tutor, and … what steps you would take to get [one]. I wanna say the discussion boards for me weren’t [very] helpful because I already kinda knew those techniques, but I guess for people that don’t know. I guess that would be helpful for them…
Of the four that said they changed their behavior as a result of the discussion boards, they all referenced their study habits. A student in Algebra explained:
Yeah, I feel like I changed my behavior at the beginning of the semester. … I would see … my grades … weren’t what I needed them to be. So, I feel like I kinda … took that into consideration when he said … studying it after class [is helpful]. … [F]or my class, there [were] office hours… right after our class, so I would go like right after class. … [A]nd it would help me because … we would talk about previous classes or … previous [topics], then we would talk about what we learned … in this class. So, it helped me [be] much better.
Themes
Several key themes emerged that directly address the research questions and provide deeper insight into the quantitative findings. These themes include Math Mindset, Self-Regulated Learning, and the professor’s impact.
To assess students’ math mindsets, they were asked a series of reflective questions. First, they were prompted to consider their general experiences with math before and during college: “To what degree do you feel your math abilities have changed over time?” This question was then repeated in the context of their current math course. Students were also asked: “Do you think your math abilities can change over time?” followed by a related question: “Have you always thought this way?”
In response, 34 students indicated that their math abilities had grown or could grow, with one student responding, “kind of.” Six students specifically noted that their math skills had improved. Notably, no students stated that their abilities could not change. These findings suggest that, across both control and treatment groups and across various courses, students generally held a GM.
A student in the control group of an Algebra II course shared:
I think just like how it can improve, it can also decrease, so it’s something I always try to stay on top of. Like during the summer, I am going to those channels I usually use during the school year. I’m watching … trying to get a little bit ahead of reviewing stuff that …we may go over the next year, so when we get to those subjects, I’m kinda familiar with it.
(codes: math GM, math preparation)
Similarly, students in the treatment condition of an Algebra course said:
Yes. I don’t think that anyone … is just incapable of … − not capable of like being good at math. I feel like … it’s just something that you have to put your mind to. … Not everybody is really good at math, so it’s kinda like you have to … strive to do better in math.
(codes: math ability, math GM)
“Yeah, I think anything you put effort in and practice will be better even if you feel like you’re trash at it. Like repetition and just being yourself grace will help in any subject and stuff.”.
(codes: math GM)
In an Algebra II course under the treatment condition, another student reflected on their progress:
I definitely think it’s grown. I was able to do more studying this year than I did before … I was just … not studying and thinking that I could just get it just like that when I wasn’t able to. And so now, when it comes to more formulas for … this specific math, … you have to study because you’re not gonna be able to get it at all unless you take your time to actually do it and not … give up on yourself either.
(codes: math GM, math preparation)
However, not all students reported recent improvements. Some shared that their math abilities had declined, often citing the challenges of college-level coursework or the lasting effects of the COVID-19 pandemic
For example, a student in the control condition of an Algebra class stated:
… I was actually always advanced in math. I was actually really good at math. I was great at math. I was always ahead in math, but then after COVID, I got so behind. My brain was just all messed up. It got really bad, and I’m still trying to get myself there. So yeah, it’s just bad.
(codes: COVID impact, math ability)
Another student stated:
“I’d probably say decrease all things considered. A lot more of things you have to remember now. The whole basis of it has not really changed at all but it’s just … is more stuff stacked on top of each other and a bunch of rules that coincide.
(codes: math ability)
Students were asked a series of questions to determine whether they engaged in self-regulated learning, and responses indicated that such behaviors were present across all courses and conditions. When asked, “When are you most engaged in math?”, students identified a range of activities that helped them stay focused and involved. Seven students reported being most engaged when working on practice problems, while six said they were most engaged during group work. Four students found homework to be the most engaging, and individual responses included using internet videos, the Pearson online platform provided with the course, and reviewing personal notes. These varied responses highlight the diverse strategies students use to take ownership of their learning.
When asked about their ideal way of learning a new math concept, the majority of students—23 in total—indicated a preference for learning outside of class. Among these, 12 students favored doing practice problems, five preferred using internet videos, and another five relied on reviewing their class notes. One student reported group work as their preferred method. Additionally, two students expressed a preference for receiving study notes directly from the professor.
A student in the control group of a Calculus course described their approach:
“Yeah. Usually just watching [YouTube] videos, paraphrasing those videos, paraphrasing notes as well. Doing some example problems really helps”.
(codes: math learning strategies)
When asked, “What types of tasks did you do to help you learn math?”, students across all focus groups reported using self-regulated resources—both those provided by the course and external tools like internet videos.
For example, a student in the control group of a College Algebra course shared:
“I would say the Help Me Solve on Pearson, that saved me a lot of going through YouTube videos and looking up formulas and how to complete formulas. So I would say the Help Me Solve because it was straight to the point and it tells you how to get it done in the correct way”.
(codes: Pearson experience, math learning strategies)
Another student in the same course mentioned using Photomath:
Sometimes I use the Photomath to get steps of something, ‘cause I don’t like the steps Pearson gives me, so I would use [Photomath] to see how they did it. Usually … I get a better understanding with how they did it. ….
(codes: Pearson experience, math learning strategies)
Students in the treatment group also referenced external resources:
Yeah, Khan Academy definitely has … good videos … teaching … different methods because everybody doesn’t teach math the same, so I wouldn’t say there’s a specific way to teach math. It’s like whatever you understand, you’re probably gonna teach. So if you don’t understand it, I’m sure [there’re] … other methods to like learn the same exact part of the curriculum.
(codes: math learning strategies)
“I would Google practice problems on the topic and like go through and do all of them just to make sure that I had it down.”.
(codes: math learning strategies)
Notably, many students indicated that their perceived math ability was closely tied to their professor’s teaching ability or instructional style. This suggests that some students may not have felt a strong sense of personal agency over their own learning.
As one student in the treatment group of a Calculus course explained:
I think your instructor can 100 percent influence your abilities, because I think you could be good at math but if you have a bad instructor, it will make you think you’re bad at it, so you might … turn away from it and—yeah….
(codes: negative professor impact, math ability)
A student in the control group of an Algebra course echoed this sentiment:
[W]hen it comes down to the professor because I’ve had great math teachers where they teach, and I automatically understand what’s happening. And then I get to college and it’s like I’m dealing with people who feel like they don’t have the time to spare to help you or help you understand what’s being given. And it’s frustrating ….
(code: positive professor impact, negative professor impact)
Another student in the same course emphasized how both the pace and approach of the professor affected their experience:
I feel like in general I do pretty well in math. Though kind of like the previous questions, it really kinda depends on how good or bad the teacher is, how fast they go, how willing they are to slow down or reteach. Like I said, it—math, it isn’t really like a subject where you’re either good at it or you’re not. It’s really just whether or not you’re given a good space or ways to figure it out….
(codes: math ability, professor impact, math GM)
Similarly, a student in the control group of an advanced Algebra course attributed their confidence in math directly to their professor:
I say I have high ability. I think that—the only reason is because of the professor. Like I said, I think the professor is very important. She’ll help you understand it and get through it or not. … The professor will help you understand it and get through it and just use your time more wisely. I feel like if you just get lost in math, you’ll just spend hours on it and then just give up at the end of the day.
(codes: math ability, professor impact, math GM)
Despite placing significant importance on their professors for success, few students reported taking advantage of office hours or seeking help outside of class. One student in the treatment group of an advanced Calculus course reflected:
That would also be one thing I need to be cognizant of, because I’ve been told multiple times that it’s better to just—if you don’t understand something, just go to your professor in office hours, but like me, many students still don’t do that and it’s kinda just—it just seems awkward in a way, but yeah—I’m not sure how to explain it. I just know I don’t really do the things that I know I’ve been told will—that have been recommended that I do.
(code: resource utilization)