Problem-Based Learning (PBL) and Student Interest in STEM Careers: The Roles of Motivation and Ability Beliefs

Abstract

Amid growing concerns about the future of the U.S. economy and workforce, educators and policymakers alike have increasingly emphasized the need to expand the number of students interested in, qualified for and actually pursuing careers in science, technology, engineering and mathematics (STEM). The current study draws on survey responses from a sample of 3852 high school students at inclusive STEM schools across the U.S. to investigate how project- and problem-based learning (PBL) may work to address this need. Multivariate regression results indicate that student ratings of PBL are associated with interest in pursuing a career in STEM, as well as with intrinsic motivation for science and students’ ability beliefs for both science and math. Further, mediation analysis using Hayes’ (2014) MEDIATE macro suggests that science intrinsic motivation and ability beliefs mediate the relationship between perceived PBL experiences and student interest in a future STEM career (IFSC). Our results highlight the important potential of PBL for increasing student STEM attitudes and interest in future STEM careers. As one of the only large-scale quantitative analyses of its kind, this study provides critical information for educators, school administrators and policymakers as they continue to seek effective ways of encouraging students to pursue STEM careers.

1. Introduction

In recent years, the subjects of science, technology, engineering and mathematics (STEM) have been at the forefront of U.S. education due to growing concerns that American students are unprepared for the 21st century workforce and a global economy [1,2]. Improvements in STEM education have also been cited as “critical to building a just and inclusive society,” as STEM participation and achievement are particularly low in women and minorities [3] (p. vii). Attracting, retaining and better educating larger and broader populations of students necessitates a focus not only on improving achievement in STEM but also on building interest in the disciplines. As such, policymakers, researchers and educators alike are implementing and examining potential strategies for engaging students and furthering their intentions to pursue STEM coursework and careers.

The current study examines the use of problem- and project-based learning in inclusive STEM high schools across the country as one such effort. Both problem- and project-based learning have significant histories in education: problem-based learning was first utilized in medical schools over 40 years ago [4] and project-based learning has been a cornerstone of institutions such as the Buck Institute for over 25 years [5]. More recently, educators in the K-12 sector have adopted a variety of approaches based in problem- and project-based learning—often collectively referred to as “PBL”, which we will use here—as a method for increasing student engagement, especially in regard to STEM. Yet little empirical evidence exists on whether PBL approaches work to increase student interest in pursuing STEM careers, much less studies that explore the mechanisms through which any increase might occur. Here, we draw on Social Cognitive Career Theory (SCCT) [6], which stresses experiential factors (such as learning experiences—in this case, PBL) and social-cognitive constructs (such as self-efficacy and outcome expectations) in people’s career-related interests, choices and performance, while also taking into account individual and contextual factors. Using survey data from a large sample of students at 17 inclusive STEM high schools, we examine how PBL relates to student interest in STEM careers and the role of intrinsic characteristics such as ability beliefs and motivation in that relationship.

1.1. The Importance of Increasing Interest in STEM Careers for All Students

Much of the focus on STEM education has centered on increasing student achievement in science and math, with the assumption that higher achievement will ensure that students are “ready” to compete [2,7,8,9]. This is at least in part motivated by the U.S.’s underwhelming performance in STEM subjects as compared to much of the world: on the most recent PISA, U.S. students performed below the Organization for Economic Co-operation and Development (OECD) average in math (ranking 31st out of the 35 OECD countries) and near average in science (ranking 19th out of 35) [10].

While achievement in STEM disciplines is certainly important, it is not in and of itself the indicator of success. First, achievement is predicated on the notion that students are actually taking STEM courses, when in fact this is not always the case. A recent report by the U.S. Department of Education Office for Civil Rights [11] states that only 78% of high schools offer Algebra II, 48% offer calculus, 72% offer chemistry and only 60% offer physics; the numbers are even lower when looking specifically at schools with the highest populations of black and Latino students—of these schools, only 71% offer Algebra II, 33% offer calculus, 65% offer chemistry and 48% offer physics. These disparities in type and caliber of STEM course offerings mean that only some students have the opportunity to enroll, let alone to achieve at high levels. And even when courses are offered, exposure to math and science in high school is less influential on underrepresented minority students’ (vs. white students’) intent to pursue STEM [12]. Additionally, although women are on par with men in terms of access and achievement in advanced high school math and science courses, they remain underrepresented in STEM majors [13,14]. Even high performing students do not necessarily persist in STEM: Lowell, Zalzman, Bernstein and Henderson [15] found a sharp decrease in STEM retention from the 1970s to mid-2000s for students with the highest academic success, suggesting that something other than achievement influenced students to stay in, or to stay out of, STEM fields. Thus, it seems that while high school course-taking and achievement are well-deserving of attention, they are only two factors in ensuring that students actually enter STEM industries.

Another predictor of STEM persistence is student interest in STEM. “Interest” as a motivational construct refers to the state of being engaged or being likely to reengage with particular object, event, or idea [16]. In a 2010 study, Maltese and Tai [17] found that, compared to enrollment in STEM courses or achievement, high school students’ interest in science and math was more strongly associated with pursuing a STEM major. Other research shows that women and minorities are less likely to show interest in or pursue STEM careers even if they have the aptitude to do so [7,18]. These findings suggest that interest may account at least in part for differences in career paths for women and underrepresented minorities. As such, finding ways to increase students’ interest in STEM may be key to building a stronger and broader STEM workforce. This stance was echoed by the President’s Council of Advisors on Science and Technology [7], which concluded that proficiency in STEM is not enough to ensure that students continue on to STEM majors and careers. Instead, the report identifies the need for both rigorous K-12 education curricula and new initiatives that inspire students to learn STEM and in turn, motivate them to pursue STEM careers.

1.2. Inclusive STEM High Schools

In response to these calls, educators, policymakers and even industry leaders (e.g., Google, Code.org, Oracle) have begun large-scale initiatives to increase student STEM interest, involvement and achievement with the goal of building the next generation of STEM professionals. One such initiative, which has taken recent priority in the federal government’s investment in improving K-12 STEM education [19], is the development of inclusive STEM high schools. Unlike traditional math and science-focused high schools with selective enrollment and rigorous pre-entrance academic requirements [20], inclusive STEM high schools have open admissions and often a specific focus on increasing minority interest and participation in STEM [21,22]. These schools are a relatively new development in education, gaining in popularity and visibility only since the turn of the century. Prior research suggests that inclusive STEM schools operate differently and have different student goals than comprehensive high schools [22,23,24]. For example, many of these schools place a strong focus on the development of 21st century skills and engaging students in real-world applications of STEM, rather than solely providing additional STEM courses [24,25,26]. Inclusive STEM schools tend to place importance on interdisciplinary learning, with some restructuring the normal school schedule to accommodate team-teaching and others infusing interdisciplinary learning into each subject [25]. Additionally, these schools are often focused on preparing the students they serve, many of whom at some schools will be first-generation college-goers, for postsecondary options, offering dual enrollment options for students to receive college credit or even take courses at local colleges and universities [26].

While early research on the impact of such schools on achievement is mixed and inconclusive [24,27,28,29,30,31], attending inclusive STEM high schools has been associated with student interest in and pursuit of STEM fields in college and careers. Compared to students at traditional high schools, those who attend inclusive STEM schools have been found to participate in more extracurricular STEM activities and have more interest in and positive attitudes toward STEM subjects [30,32]. Some studies also show that students at inclusive STEM schools are more likely to take early college STEM courses, have aspirations for postsecondary education, declare a STEM major and be interested in one or more STEM careers [30,33]. As such, inclusive STEM high schools may be important mechanisms for growing and sustaining student interest and persistence in STEM.

One limitation of the current research literature, however, is a lack of understanding of specifically how inclusive STEM high schools influence student attitudes and learning (see Means et al. [30], described below, for an exception). Much of the current literature takes an econometric approach to understanding the “innovation” of STEM schools, where attending versus not attending is the main variable of interest [34]. Less attention has been paid to which strategies and interactions are employed in these schools, such as particular pedagogical techniques and learning experiences and how these specific components drive student outcomes. Indeed, Lavertu and Gnagey [29] found that students from two specific STEM schools in their study drove the negative effects on math and science achievement, while students at a third STEM school actually performed better in science compared to students in traditional schools. In that case, some unobserved characteristic hindered a clear understanding of not just if but how these schools influenced student outcomes. One exception is the recent work by Means and colleagues [30], who found that quality student-teacher relationships, real-world STEM experiences and quality science and math instruction are all associated with higher student achievement, suggesting that these components might influence student achievement more than others. While these initial results are critical for practitioners and policymakers to understand as they look for the best approaches to boosting students’ STEM attitudes and achievement, further work is needed to fully understand which components of inclusive STEM schools are critical for student outcomes beyond achievement.

1.3. The PBL Experience

One common learning experience in inclusive STEM schools is problem- or project-based learning (PBL), terms used for a set of instructional strategies that empower learners to conduct research, integrate theory and practice and apply knowledge [4]. In an investigation of what defines inclusive STEM schools, LaForce et al. [25] identified PBL as one of eight Essential Elements, a finding supported by the work of others in the field [22,26] (The eight Essential Elements [25] include six core Elements (four instructional, two non-instructional) and two supporting Elements. Instructional Elements focus on pedagogical strategies and academic goals for students; non-instructional Elements represent STEM schools’ focus on the school culture and commitments to establishing and maintaining relationships with their broader communities; supporting Elements including strategies and external factors that enable implementation of the core Elements. The core Elements are: rigorous learning (instructional); personalization of learning (instructional); career, technology and life skills (instructional); problem-based learning (instructional); school community and belonging (non-instructional); and external community (non-instructional). The supporting Elements are staff foundations and essential factors.) While no singular definition of PBL and what it looks like in K-12 education practice exists, as noted above, for the purposes of this study we use “PBL” as a broad term encompassing many approaches employed under the term (including both problem- and project-based). In our own work (in development), we observed three “types” of PBL: (1) Short-term PBL, where teachers use tangible problem-solving as a strategy for day-to-day instruction; (2) Projects, where students complete a series of discrete tasks resulting in an end product but which is not rooted in a tangible problem; and (3) Problem-solving projects, where students complete a series of discrete tasks and form a product that is rooted in a problem to be solved. In the literature [4,35], PBL is commonly described as the third type identified here: problem-solving projects. While we consider all three approaches to be distinct, they often work toward a similar set of student outcomes, from increasing subject area content knowledge to developing effective problem-solving strategies and metacognitive strategies, such as self-directed learning, goal setting and collaboration [36,37]. In our data collection and analyses, we focus on the third type of PBL (problem-solving projects) for operational uniformity.

Despite its roots in higher-level medical education, over the last several decades PBL has spread to the K-12 education environment as educators look for better ways to engage students in the classroom and ultimately increase student achievement [37,38,39]. PBL’s constructivist nature was originally designed to replace traditional lecture-based teaching methods in medicine to actively engage students in self-directed and interdisciplinary learning through real-world problem solving [40]. Research on K-12 education has suggested that PBL increases students’ engagement and enjoyment of STEM subjects in particular, especially for girls and underrepresented minorities [41,42,43,44]. Further, studies have shown that PBL can improve students’ creativity [45], critical thinking and problem-solving skills [46,47], reflective thinking [48], communication and collaboration skills [45,48,49,50] and ability to self-direct learning [45,46,51]. Teachers also report that they incorporate more 21st century skills into their teaching and assessment when using PBL [52]. As such, using PBL may be an effective way to not only engage students in STEM learning but also provide them with the necessary foundational skills to pursue STEM careers. According to the OECD [53], American students have strong skills in low cognitively demanding tasks but lack higher cognitively demanding abilities that enable them to successfully apply math and science skills to real world applications and vice versa—starting with a real-world problem and understanding which skills, knowledge and processes are necessary to solve it. As PBL strategies often emphasize real world problem solving and 21st century skills, many inclusive STEM schools look to PBL as one way to provide opportunities for students to make connections between what they learn in the classroom and the real-world application of such knowledge. These tenets of PBL led to the development of Hypotheses 1, 2 and 3 (described below) which posit that PBL will be related to students’ interest in a future STEM career, science and math intrinsic motivation and beliefs about their science and math abilities.

1.4. Students’ Beliefs about Ability, Interest and Intrinsic Motivation in STEM

It will be helpful before going further to clearly define the theoretical constructs on which we draw for the current study, namely, interest, intrinsic motivation, ability beliefs and attitudes. In the literature, interest may refer to situational interest, a more temporary state induced by particular features of an environment or activity, or to individual interest—“a relatively stable evaluative orientation towards certain domains” [54] (p. 114). As noted above, in the current study we focus on the latter, with the domain being future STEM education and career paths. While interest influences the direction in which a person may act, intrinsic motivation is what causes him to act and to maintain that action [55]. It refers to behavior that is driven by a person’s finding it inherently interesting, enjoyable, or fun. Intrinsic motivation is activity-specific, meaning that one might be intrinsically motivated to do one thing but not another; additionally, it has been found to result in high-quality learning but may be impacted negatively or positively by parent and/or teacher practices [56].

Ability beliefs, which include concepts such as confidence, self-efficacy and self-concept, “are defined as the individual’s perception of his or her current competence at a given activity” [57] (pp. 70). Beliefs about abilities play large parts in various motivation theories (such as expectancy-value theory [57], social cognitive theory [58], attribution theory [59] and self-determination theory [60]) and similarly are measured in different ways, depending on the researchers’ interest in particular aspects and specificity of perceived ability [57]. We focus in this study on student’s confidence in their abilities in math and science classes. Generally, beliefs about ability have been found to be positively related to meaningful cognitive engagement in tasks and involvement in academic work as well as effort, persistence and academic achievement [61,62]. More specific findings related to academic beliefs about ability, specifically self-efficacy and self-concept, in STEM are discussed at greater length below. Attitudes represent a person’s stance, disposition, or opinion toward people, objects, events, situations, or in this case, disciplines [63,64]. The construct of attitudes is separate from interest, in that one can have a positive attitude toward something without being interested in it [55]. Attitudes towards a subject consist of a number of sub-constructs and studies have used a variety in their measures of attitudes towards science, including perceptions of teachers, anxiety towards science, value of science, self-esteem in science and motivation towards science, among others [65]. Here we refer to students’ general attitudes towards school, as well as their attitudes towards STEM subjects—in this case meaning their ability beliefs and intrinsic motivation—more specifically.

Given the demonstrated potential of student interest for driving persistence in STEM, as well as the current efforts underway to help students develop and maintain that interest, we draw on the Social Cognitive Career Theory (SCCT) [66] as the framework for the current study. Grounded in Bandura’s Social Cognitive Theory [67], SCCT emphasizes both the experiential factors (e.g., specific learning experiences) and social-cognitive constructs (e.g., self-efficacy, outcome expectations) in explaining a person’s career-related interest, choice and performance. SCCT also recognizes the importance of individual and contextual factors, such as gender, race and socioeconomic climate. More specifically, SCCT suggests that individual and contextual factors influence the learning experiences an individual has, which then directly influence that person’s confidence in their abilities and outcome expectations. These in turn influence his or her career interests, goals, choices and performance.

Research related to students’ ability beliefs in STEM has primarily focused on self-efficacy in STEM classes and to a lesser extent, students’ self-concept (while these two constructs are related, measures of self-concept are often more general than those of self-efficacy, which typically reside at task level [68]). Prior work in STEM education suggests that when students have confidence in their abilities in STEM classes as well as positive attitudes toward STEM, they are more likely to be interested in and ultimately pursue and persist in STEM-related courses and careers e.g., [12,69,70,71,72]. Bandura [73] found that students who reported higher math self-efficacy also reported greater intrinsic motivation, a finding reciprocated across STEM disciplines [64,74]. In turn, such high levels of ability beliefs and motivation are associated with increased enrollment in STEM courses [75,76], increased performance in such courses [77], increased likelihood of taking STEM classes and/or declaring a STEM major [12,72] and increased interest in pursuing a STEM career e.g., [12,66,69,74]. For example, Tai and colleagues [72] found that students who were more interested in STEM and that had higher STEM self-efficacy were more likely to pursue STEM courses in college. Additionally, using large-scale nationally-representative data from the Education Longitudinal Study of 2012, Wang [12] found that students’ math self-efficacy and attitudes in high school were strongly associated with pursuing a STEM major in college. Another study of middle school students found that those who viewed themselves as “smart” or “good” in a subject (i.e., had high self-concept and self-efficacy) were more inclined to express positive attitudes towards their ability and interest to pursue STEM careers [69]. Thus, in working to grow the number of students pursuing STEM education and careers, it is important to understand what strategies may be used to increase student beliefs in their ability and in turn, interest, in STEM.

The research suggests that PBL may be one such approach. Some studies have shown that students who engage in STEM PBLs have more positive attitudes toward and interest in pursuing STEM [41,44,78]. Several have found that engagement in PBL is associated with increased self-efficacy and confidence in STEM disciplines e.g., [41,42], which, in turn, could lead to more positive attitudes toward STEM and continued pursuit in STEM fields. For example, Massa and colleagues [43] found that PBLs not only improved students’ conceptual knowledge but also their intrinsic motivation and self-efficacy. This finding is support by Cerezo [42], who showed that PBLs were specifically effective for enhancing middle school girls’ STEM self-efficacy. Dominguez and Jamie [48] also found that students were less likely to drop out of STEM courses if they engaged in PBL activities, while Berk and colleagues [78] showed that students who engaged in PBL had more positive attitudes toward STEM and were more interested in pursuing STEM-related careers. As such, PBL may be a critical learning experience in the pathway towards STEM majors and careers by increasing students’ ability beliefs and intrinsic motivation for STEM subjects. This notion is tested (as described below) by Hypotheses 4 and 5.

1.5. The Current Study

Given this strong body of evidence on the importance of ability beliefs and motivation in students’ continued interest in and pursuit of STEM courses and careers, and the potential of PBL as a key learning experience that influences these social-cognitive factors, the present study seeks to investigate how PBL learning experiences in the inclusive STEM school context can enhance such foundational dispositions that lead students to persist in STEM fields. Our focus on PBL also arises from the frequency with which inclusive STEM school leaders in this study described PBL as an important component of their schools [25] and from the recent rise in attention to K-12 implementations of PBL in the literature [42,79,80,81]. Based on the Social Cognitive Career Theory [66] and research supporting the positive influence of PBL on students’ ability beliefs, intrinsic motivation and ultimately their career interests [50,78], our main research question is: Are positive student experiences with PBL related to students’ STEM ability beliefs and intrinsic motivation and in turn, their interest to pursue a career in STEM? (Hypothesis 6).

2. Materials and Methods

This study draws on a subset of data collected as part of a larger four-year project (NSF #1238552) to develop a clear understanding of inclusive STEM schools across the United States and how the various strategies and methods they employ relate to student outcomes.

2.1. Procedure

Over the course of the project, we worked with 20 inclusive STEM schools from seven states; members of these schools were invited to participate in school leader interviews, teacher and student interviews and questionnaires and classroom observations. Schools were selected in a two-stage process. First, we identified seven states with established STEM networks to connect with; we worked closely with these networks to identify schools to reach out to for participation, based on several a priori criteria: (1) STEM identity, meaning that STEM is a priority at the school; (2) Independent school structure, meaning that STEM activities occur for all students and STEM is not a program or a “school-within-a-school”; (3) Maturity, meaning that schools were in at least their second year of implementation (two schools in their first year of implementation were included, based on recommendations from state network leaders); (4) Inclusive environment, meaning that schools did not base enrollment on prior achievement or STEM ability; and (5) Collaborative nature, meaning that we sought schools that were willing and eager to participate in a large research study. Additionally, we asked the STEM network leaders to provide us with a group of potential schools that were representative of the STEM school landscape in their state. We then reached out to the schools’ leaders directly to invite them to participate. In a few cases, the initial school leaders contacted did not respond or declined to participate and other schools (again suggested by network leaders) were invited to fill those slots.

This study draws on student questionnaire data from the final year of data collection and includes 17 of the original 20 schools (due to two schools dropping out of the study after year one and an additional school not having a large enough response rate to be included). Teachers at these 17 schools administered an online student questionnaire between February and May 2015, during various class times determined by a main point-of-contact for the study and the teachers in each school. Prior to survey administration, human subject approval was obtained for all data collection and students were sent home with parental consent forms that included an option to “opt-out” of participation in the surveys. Students whose parents did not opt out were invited to take the survey and upon entering the online questionnaire, shown a screen asking for their assent; students who did not assent were immediately exited from the survey. The questionnaire took approximately 25 min for students to complete and addressed a range of school experiences and attitudes.

2.2. Participants

The sample drew from 3852 9th–12th-grade students at 17 public inclusive STEM high schools. Sample schools were located in Ohio (N = 4 schools), Washington (N = 4), Texas (N = 3), California (N = 2), Tennessee (N = 2), New York (N = 1) and North Carolina (N = 1). Approximately 43% of the sample identified as White, 28% as Latino/Hispanic, 10% as Black, 8% as Asian, 8% as Mixed Race and 3% as another race (American Indian or Alaskan Native, Hawaiian or Pacific Islander, Middle Eastern, or Prefer Not to Answer). Fifty-one percent of the sample identified as female. Thirty-four percent of the sample was in 9th grade, 28% in 10th grade, 24% in 11th grade and 14% in 12th grade.

2.3. Measures

2.3.1. Perceptions of PBL

Perceptions of PBL, a 12-item PBL scale, was derived from Munshi, El Zayat and Dolmans [82] and adapted for a high school population based on descriptions of PBL from STEM school leaders [25] and previous literature articulating high quality PBL [4].

The scale captured the frequency with which students reported experiencing various aspects of PBL at their school, with each item measured on a 6-point Likert scale anchored by never and always. For example, students were asked to rate the frequency with which they engage in PBL activities that are “interesting and relevant to their lives.” The scale was introduced to students with the text: “Now we’d like to ask you some questions about problem- or project-based learning (PBL), or problem-solving projects at your school. These are extended learning experiences that focus on real-world problem-solving.”

Table 1. Student Perceptions of PBL Scale.

Item	How Often Is the Following True?
1	PBL projects get students to discuss ideas in class.
2	PBL projects do a good job of getting students to do research to look for background information.
3	PBL projects draw from multiple courses or subjects.
4	PBL projects are interesting and fun.
5	PBL projects are relevant to students’ daily lives.
6	PBL projects give students a chance to think about future careers.
7	PBL projects help students to better understand current events and/or environmental issues.
8	PBL projects draw on things students have learned previously.
9	PBL projects require students to apply knowledge learned in the classroom to a real-life event.
10	PBL projects are central to the curriculum.
11	PBL projects require a thorough process of inquiry, knowledge building and resolution.
12	PBL projects are more student-led than teacher-led.

Note: Response options included 1 = never, 2 = rarely; 3 = occasionally; 4 = frequently; 5 = very frequently; 6 = always.

shows the scale items used to measure the Perceptions of PBL construct. We conducted exploratory factor analyses (principal axis factoring) to investigate potential latent constructs. EFA analyses using an oblique promax rotation support a one-factor solution, with all 12 items loading at 0.4 or above on one factor with a Kaiser [83] eigenvalue over 1. This factor accounted for 70% of the variance. A second factor emerged in scree plots [84] but did not obtain an eigenvalue of 1 and accounted for only 5% of the variance. Recently, it has been argued that the Kaiser method and scree test may be insufficient for obtaining unbiased estimates of underlying components in principal axis factoring, or EFA [85]. Many researchers now prefer a Horn’s parallel analysis [86], which employs a Monte Carlo method to estimate differences between simulated (expected) and observed eigenvalues. Given the size of the PBL scale (12 items), we also conducted a parallel analysis using SAS [87] with 1000 iterations. Observed and simulated eigenvalues are presented in

Table 2. Observed and Simulated Eigenvalues for Perceptions of PBL Rating.

Factor	Actual	Simulated
1	8.133	1.072
2	0.744	1.054
3	0.555	1.040
4	0.424	1.027
5	0.354	1.015

. Observed eigenvalues that exceed simulated eigenvalues are determined to be valid [88]. The parallel analysis supports a one-factor solution. Thus, a one-factor scale (α = 0.96) average for PBL rating was used in all analyses.

2.3.2. Interest in a Future STEM Career (IFSTEMC)

The overall scale average derived from a 4-item scale regarding students’ interest in pursuing STEM in college and career. This scale was developed for the current study. Each item was measured on a 6-point Likert scale anchored by strongly disagree and strongly agree and the scale achieved high reliability (α = 0.97). Interest in a future STEM career consisted of the following four items: (a) I see myself pursuing a career in STEM; (b) I expect to take a lot of STEM courses in college; (c) A career in STEM sounds exciting to me; and (d) If I had to pick a college major right now, it would be in a STEM field.

2.3.3. Ability Beliefs

The overall scales average derived from 4-item scales describing students’ confidence in their science/math skills and abilities. Science and math ability belief items were adapted from the Student Self-Report of Academic Self-Efficacy [89,90]. Each item was measured on a 6-point Likert scale anchored by strongly disagree and strongly agree and both the science and math scales achieved high internal consistency (α = 0.92).

2.3.3.1. Science Ability Beliefs

Science ability beliefs consisted of the following items: (a) I have the skills and ability to learn about science; (b) I’m better at science than most of the other kids at my school; (c) I am very good at science; and (d) I can figure out how to do the most difficult science problems if I try.

2.3.3.2. Math Ability Beliefs

Math ability beliefs consisted of the following items: (a) I have the skills and ability to learn about math; (b) I’m better at math than most of the other kids at my school; (c) I am very good at math; and (d) I can figure out how to do the most difficult math problems if I try.

2.3.4. Intrinsic Motivation

Intrinsic motivation items (for science and math) were developed by the authors for a pilot study of STEM schools in Ohio (NSF #1008569). Each item used in the current study was measured on a 6-point Likert scale anchored by strongly disagree and strongly agree and achieved high reliability (α = 0.96).

2.3.4.1. Science Intrinsic Motivation

Science items were derived in part from the Attitude toward Science in School Assessment [91]. Science intrinsic motivation consisted of the following four items: (a) I find science very interesting; (b) I enjoy science investigations; (c) I want to learn more science; and (d) Learning about science is fun.

2.3.4.2. Math Intrinsic Motivation

Math intrinsic motivation consisted of the following four items: (a) I find math very interesting; (b) I enjoy solving math problem; (c) I want to learn more math; and (d) Learning about math is fun.

It should be noted that our survey measures also addressed intrinsic motivation and ability beliefs for both engineering and technology. However, these measures exhibited less consistency. One likely reason for this is that far fewer students reported having experience with engineering and technology courses and activities. Thus, we opted to use only the math and science scales for these analyses.

Due to the potential for collinearity between ability beliefs and intrinsic motivation constructs (as well as their prominence in mediation models), we conducted factor analyses for (1) Science ability beliefs and science intrinsic motivation and (2) Math ability beliefs and math intrinsic motivation. We first conducted traditional EFA analyses in SAS using an oblique promax rotation for the 8 items across the science ability beliefs (4 items) and science intrinsic motivation (4 items) scales. Scree plots and factor loadings supported a two-factor solution, with ability beliefs items and intrinsic motivation items loading at 0.7 or above on their respective scales. However, while the eigenvalue for factor 1 = 5.68, a second eigenvalue failed to reach 1, at 0.84. To examine further, we conducted Horn’s [86] parallel analysis to examine convergence with our initial EFA. This analysis supported a 2-factor solution, with actual eigenvalue estimates (Factor 1 = 5.89, Factor 2 = 1.05) exceeding simulated estimates (Factor 1 = 1.05, Factor 2 = 1.03).

Next, we replicated these analyses for math ability beliefs and intrinsic motivation items. Once again, while factor loadings and scree plots supported a two-factor solution in EFA analysis (items loading on their respective scales at 0.7 or above), only a Factor 1 eigenvalue (5.73) exceeded 1 (Factor 2 = 0.72). Horn’s parallel analysis showed slight support for a one-factor solution vs. a two-factor solution. Actual eigenvalue exceeds simulated eigenvalue for Factor 1 (Actual = 5.93, Simulated = 1.05), while actual eigenvalue is quite close yet does not exceed the simulated eigenvalue for Factor 2 (Actual = 0.93, Simulated = 1.03).

As these analyses did not give us sufficient confidence in two-factor solutions for both science and math scales, we conducted confirmatory factor analyses using PROC CALIS in SAS [92]. For both science and math, a two-factor solution loading 4 ability beliefs items on one factor and 4 intrinsic motivation items on a second factor was moderately supported. One-factor solutions did not show sufficient fit. Interpretation of CFA fit indices does not follow strict rules [93], however, it is widely acknowledged that chi-square estimates (which evaluate a “badness of model fit”) are too sensitive to sample size and thus may not serve as a valid indicator of true model fit. Common rules of thumb include striving for a root mean square error of approximation (RMSEA) < 0.08 [94], as well as non-normed fit index (NNFI) [95], comparative fit index (CFI) [96] and adjusted goodness-of-fit index (AGFI) [93] all > 0.90. CFA findings indicate moderate support for two-factor solutions for both science and math items. These are summarized in

Table 3. Fit estimates for science and math self-efficacy and intrinsic motivation scales.

Discipline	Model	EFA	Parallel Analysis	CFA Chi-Square	RMSEA	CFI	NNFI	AGFI
Science
	Two Factor (AB & IM as unique constructs)	Moderately supported	Supported	p < 0.001 (Supported)	0.085 (Not supported)	0.987 (Supported)	0.987 (Supported)	0.933 (Supported)
	One Factor (AB & IM as one construct)	Moderately supported	Not supported	p < 0.001 (Supported)	0.307 (Not supported)	0.807 (Not supported)	0.715 (Not supported)	0.321 (Not supported)
Math
	Two Factor (AB & IM as unique constructs)	Moderately supported	Not supported	p < 0.001 (Supported)	0.093 (Not supported)	0.984 (Supported)	0.973 (Supported)	0.919 (Supported)
	One Factor (AB & IM as one construct)	Moderately supported	Supported	p < 0.001 (Supported)	0.295 (Not supported)	0.818 (Not supported)	0.732 (Not supported)	0.355 (Not supported)

.

2.3.5. Additional Covariates

2.3.5.1. General Intrinsic Motivation for Schoolwork

Developed for the current study, items were measured on a 6-point Likert scale anchored by strongly disagree and strongly agree; the scale achieved high reliability (α = 0.93). This general intrinsic motivation scale serves as a covariate to assess the unique contribution of intrinsic motivation for STEM subjects and reduce bias emerging from students who are generally positive about school and their work. In other words, to gauge a truer measure of pure science and math intrinsic motivation, we have isolated these constructs from general intrinsic motivation for all schoolwork. General intrinsic motivation consisted of the following three items: (a) In general, I find my schoolwork to be interesting; (b) I enjoy my schoolwork; and (c) My schoolwork is fun.

2.3.5.2. General Ability Beliefs for Schoolwork

Items for general ability beliefs for schoolwork were derived from Student Self-Report of Academic Self-Efficacy [89] and the New General Self-Efficacy Scale [97]. Like intrinsic motivation above, this general ability beliefs scale serves as a covariate to assess the unique contribution of ability beliefs for STEM subjects and reduce bias emerging from students who are generally confident in their academic abilities. The overall scale average derived from a 7-item scale. Each item was measured on a 6-point Likert scale anchored by strongly disagree and strongly agree and the scale achieved high reliability (α = 0.92). General ability beliefs for schoolwork consisted of the following seven items: (a) I have the skills and ability to do my schoolwork; (b) When faced with difficult tasks, I know that I will accomplish them; (c) I will be able to achieve most of the goals I have set for myself; (d) I can do the hardest homework if I try; (e) I can learn the things taught in school; (f) There isn’t any schoolwork that’s too hard for me; and (g) If I can’t do a task the first time, I know I’ll eventually get it if I keep trying.

2.3.5.3. Attitudes toward School

Developed for the current study, the overall scale average derived from a 5-item scale describing students’ attitudes toward attending their respective school. Each item was measured on a 6-point Likert scale anchored by strongly disagree and strongly agree and the scale achieved high reliability (α = 0.94). Attitudes toward school consisted of the following five items: (a) I care about this school; (b) I like being a part of this school; (c) If I could go back and choose any high school to attend, I would choose to go to this school again; (d) I enjoy being a student at this school; (e) Going to school here is fun.

It should also be noted that while we did not cognitively pretest attitude measures for students in this particular study, they were primarily derived from established measures as noted above. In addition, several of these scales were pretested with elementary and middle school samples as part of a prior NSF study of Ohio STEM schools (#1008569).

2.3.5.4. School, Grade, Race/Ethnicity, Gender

Binary variables were created for each school to use as the school fixed effects in the analyses. We tested several schools as the reference (intercept) category to identify any unusual trends but findings held consistent and thus we used a school with a respective sample size similar to the median. Binary (dummy) variables were also created for grade (9–12), race/ethnicity (Asian, Black, Hispanic/Latino, Other, White) and gender (male, female). White male 9th grade students served as a reference point.

2.4. Analysis

2.4.1. Multivariate Regression Analyses

We conducted a series of multivariate regressions to examine the association between student ratings of PBL, their STEM attitudes (ability beliefs and intrinsic motivation) and IFSTEMC, controlling for students’ race/ethnicity, gender and grade, as well as school fixed effects. Figure 1 describes our initial hypotheses, tested in the regression analyses described below.

First, we examined the association between student ratings of PBL and IFSTEMC to establish whether or not such an association existed, even after controlling for students’ math and science intrinsic motivation and ability beliefs. We hypothesized that positive student perceptions of PBL at their school would predict student IFSTEMC (Hypothesis 1).

Second, we examined the association between student ratings of PBL and students’ math and science intrinsic motivation and ability beliefs. Based on prior research suggesting engagement in PBL may increase students’ self-perceptions of these constructs in STEM subjects [41,42,43,69], we hypothesized that higher PBL ratings would be positively correlated with science intrinsic motivation and ability beliefs (Hypothesis 2) and math intrinsic motivation and ability beliefs (Hypothesis 3).

We also examined the associations between math and science intrinsic motivation and ability beliefs and students’ interest in pursuing a career in STEM. We hypothesized that science intrinsic motivation and ability beliefs (Hypothesis 4) and math intrinsic motivation and ability beliefs (Hypothesis 5) would be positively associated with IFSTEMC, given prior work suggesting math and science intrinsic motivation and ability beliefs influence students’ pursuit of those fields in the future [12,69,72].

2.4.2. Mediation Analyses

Following this series of regressions, we conducted a more rigorous test of the mediating effect of math and science intrinsic motivation and self-efficacy on the relationship between ratings of PBL and IFSTEMC. Researchers have traditionally used normal theory approach, or the Sobel [98] test, to identify mediation/indirect effects, typically within structural equation modeling (SEM) path analysis programs [99]. However, the normal theory approach is conservative and thus has low power [100], which may be insufficient for identifying small to medium effects. In addition, normal theory approach assumes normality of the sampling distribution, an assumption that is often violated in empirical studies [101]. Preacher and Hayes [102] suggest that mediators can be rigorously verified through demonstration of statistically significant reduction in correlation using bootstrapping. Bootstrapping is a non-parametric procedure that tests an indirect effect based on resampling with replacement (e.g., we used 5000 resamples). Across these samples, a confidence interval is generated. If the interval (upper and lower bound) does not cross zero, the indirect effect is different from zero [99]. Thus, many researchers now suggest that a bootstrapping approach is superior because it does not suffer from the power and distribution drawbacks of the normal theory approach [103].

Thus, we hypothesized that science intrinsic motivation and ability beliefs would mediate the effect of PBL on IFSTEMC (Hypothesis 6) and that math intrinsic motivation and ability beliefs would also mediate the effect of PBL on IFSTEMC (Hypothesis 7). Figure 2 shows the addition of these mediation predictions.

We used Hayes’ (2012) MEDIATE macro program for SPSS to calculate the total effect and the direct and indirect effects of PBL using bootstrapped samples [102]. The bootstrap procedure utilizes repeated sampling with replacement (suggested minimum of 1000; we used 5000 samples) to compute the point estimate of the indirect (mediation) effect. The analysis provides confidence intervals around point estimates [104]. Demographic covariates (i.e., gender, race/ethnicity and grade) and school fixed effects were also included in models. The MEDIATE strategy is one of the most robust analyses for testing mediation as it allows for multiple mediators to be examined simultaneously and provides both direct and indirect effects for each mediation path while controlling for all other mediation paths [105]. Further, this strategy reduces the potential for parameter bias due to omitted variables that often plague separate simple mediation models as well as offers insight into the relative magnitudes of the indirect effects of all mediators [102].

3. Results

3.1. PBL and Interest in a Future STEM Career

Results from initial regression analyses supported Hypothesis 1, indicating that students’ ratings of PBL significantly predicted their interest in pursuing a STEM career (p < 0.001). This association remained significant even after controlling for math and science intrinsic motivation and ability beliefs (p = 0.007, see

Table 4. Multivariate Regression Effects; Saturated Model.

Hypothesis	Independent Variable	Dependent Variable	Coefficient	p-Value	Semi-Partial Correlation	Total Model Adjusted R2
1	Problem-Based Learning Rating (PBL)	Interest in Future STEM Career	0.05	0.007 *	0.0001	0.45
2		Science Intrinsic Motivation	0.04	0.005 *	0.006	0.55
2		Science Ability Beliefs	0.03	0.006 *	0.005	0.64
3		Math Intrinsic Motivation	−0.03	0.06	0.0008	0.62
3		Math Ability Beliefs	0.02	0.15	0.00005	0.67
4	Science Ability Beliefs	Interest in Future STEM Career (IFSTEMC)	0.27	<0.001 **	0.028	0.45
4	Science Intrinsic Motivation		0.23	<0.001 **	0.09	0.45
5	Math Intrinsic Motivation		0.16	<0.001 **	0.04	0.45
5	Math Ability Beliefs		0.10	<0.001 **	0.002	0.45

Note: * significant at p < 0.01; ** significant at p < 0.001.

). The overall saturated model explained 45% of the variance in students’ IFSTEMC.

3.2. PBL and STEM Intrinsic Motivation and Ability Beliefs

Results support Hypothesis 2—higher PBL ratings significantly predicted students’ science intrinsic motivation (p = 0.005) and ability beliefs (p = 0.006). However, results did not support Hypothesis 3: a nonsignificant (but marginal) negative association (p = 0.06) occurred between PBL and math intrinsic motivation and a nonsignificant relationship occurred between PBL and math ability beliefs (p = 0.15).

Model results suggest that if PBL is mediated by science intrinsic motivation and ability beliefs, the mediation is partial. A significant direct of effect of PBL on interest in STEM future is retained even when including math and science intrinsic motivation and ability beliefs in the model (p = 0.007).

3.3. STEM Intrinsic Motivation, Ability Beliefs and Interest in a Future STEM Career

Results also demonstrated that science intrinsic motivation and ability beliefs (Hypothesis 4) and math intrinsic motivation and ability beliefs (Hypothesis 5) are significantly associated with IFSTEMC, even when controlling for ratings of PBL and other covariates. Table 4 lists these effects.

3.4. Mediation Model

To complete a robust analysis, we examined the mediating effect of PBL through science ability beliefs and intrinsic motivation on IFSTEMC using Hayes [106] MEDIATE macro. The effects were retained in the more rigorous bootstrapped MEDIATE analysis, indicating that science intrinsic motivation and ability beliefs partially mediate the relationship between student ratings of PBL and IFSTEMC. As recommended by Hayes, indirect effect estimates of confidence intervals that do not cross zero are considered significant.

Table 5. Indirect (Mediation) Bootstrap Test Effects.

Indirect Effect of PBL on IFSTEMC through:		Effect	Lower Bound (LLCI)	Upper Bound (ULCI)	Significance
	Science Intrinsic Motivation	0.024	0.0140	0.0363	Significant
	Science Ability Beliefs	0.029	0.0181	0.0415	Significant
	Math Intrinsic Motivation	−0.0048	−0.0128	0.0019	Non-Significant
	Math Ability Beliefs	0.0002	−0.0032	0.0040	Non-Significant

Note: Effects are considered significant if the lower and upper bounds of the confidence interval do not cross zero.

highlights these results. The omnibus test of the direct effect of PBL rating on interest in a future STEM career was significant, F = 5.99, p = 0.014. The omnibus direct effect of PBL on IFSTEMC was significant (F = 5.4983, p = 0.0191). In addition, the analysis showed support for Hypothesis 6: science intrinsic motivation and ability beliefs mediate the positive effect of PBL rating on IFSTEMC. Hypothesis 7, however, was not supported. Although multivariate regressions suggested a mediating relationship, the more robust MEDIATE analysis indicates that math intrinsic motivation and ability beliefs do not mediate the relationship between PBL rating and IFSTEMC. Table 5 lists the confidence intervals for all tested mediation variables.

3.5. Direct Effect of PBL Career Content on IFSTEMC

To further explore the direct effect of PBL on IFSTEMC, we conducted an exploratory item-wise multivariate regression of 12 PBL scale items on IFSTEMC (maintaining our prior covariates and controls). Results showed that one item in the PBL scale drove this direct effect: PBL projects give students a chance to think about future careers (p = 0.007). Using a Bonferroni correction, this estimate falls fails to exceed the adjusted p-value for significance (p = 0.004).

3.6. Race and Gender

We revisited multivariate regression analyses to understand potential race and gender interactions, examining the interaction between race and gender dummies with the effect of PBL on STEM attitudes and interest in a future. None of these interactions effects were significant, suggesting no direct relationship between (the interaction of) race or gender and PBL on STEM attitudes or IFSTEMC. Implications are discussed below.

4. Discussion

The findings from this study raise several important issues in understanding the development of positive student attitudes toward future STEM careers, and also highlight the importance of rigorous mediation analyses in examining such potential relationships. While multivariate regressions provide initial insight, the MEDIATE strategy provides a more precise, bootstrapped method of identifying mediation relationships, resulting in a clearer and more accurate picture of the relationships examined.

Prior research has clearly demonstrated the important role that intrinsic motivation and ability beliefs play in student success [12,72]. As a result, policy-makers and practitioners seek to incorporate instructional approaches that can enhance these particular student attitudes. From our results, it appears that PBL may be one such strategy for increasing students’ interest in STEM fields.

PBL showed a direct effect on interest in a future STEM career (IFSTEMC); when controlling for STEM attitudes, PBL had a reduced but still significant effect on IFSTEMC. Item-wise multivariate regression suggests that this remaining direct effect is primarily driven by career content in PBL projects—when controlling for STEM attitudes, “PBLs give students an opportunity to think about future careers” was the only item to significantly predict interest in a future STEM career. When corrected using a Bonferroni adjustment, this finding was no longer significant (p = 0.007 at a threshold of p = 0.004). However, many researchers argue that the Bonferroni correction is too conservative, particularly for exploratory hypotheses [107]. Thus, further study should more purposefully examine the relationship between career content in PBL experiences and student attitudes toward STEM careers.

Our analyses also indicated that higher overall student ratings of PBL predict science intrinsic motivation and ability beliefs, in turn predicting higher student interest in a future STEM career. Even if there is an assumption that students who attend inclusive STEM high schools are predisposed to higher intrinsic motivation and ability beliefs in science or other STEM subjects than students in non-STEM schools (an assumption that, it should be noted, is so far unconfirmed), we find that student perceptions of their PBL experiences do relate to science attitudes. Thus, there is sufficient variability within this sample to identify this significant relationship.

Interestingly, we did not find a similar relationship in the mediation analysis between ratings of PBL and math intrinsic motivation or ability beliefs. Researchers have long known that high school math courses have long-term impacts on students’ postsecondary outcomes [75] and that students’ self-beliefs about math predict high-school math course-taking [108]. Our findings are consistent in showing that math intrinsic motivation and ability beliefs do predict student interest in a future STEM career; however, neither in our mediation analysis are themselves predicted by PBL ratings or student demographics. Others have acknowledged that there are additional challenges (i.e., high-stakes testing, a packed curriculum) to implementing PBL in math classes [109] and that math teachers are often “at a loss” in how to instruct with PBL methods [80]. Unpublished qualitative data from our own work has indicated that high school math teachers in STEM schools implement PBL less often than science and humanities teachers. Thus, this relationship should be interpreted with caution. Similarly, the finding that neither math intrinsic motivation or ability beliefs mediate the relationship between PBL ratings and interest in a future STEM career suggests that perhaps students’ conceptions of future STEM careers do not include math (or, that they do include science but possibly not careers in all of the STEM disciplines). However, as these math attitudes are nonetheless clearly important for future outcomes, future research should focus on strategies to improve student self-perceptions of math.

Findings here demonstrated that race/gender did not moderate the relationships between PBL and STEM attitudes; that is, the effect of PBL on STEM attitudes manifests similarly across all students in this sample. However, forthcoming analyses that focus on a deeper exploration of race and gender effects across a variety of strategies (including and beyond PBL) indicate the potential for some race/ethnicity effects. Further research should carefully examine the interaction of race/ethnicity and gender effects with instructional and other STEM school strategies.

As is common in education research, demographic variables and fixed effects in our study exhibited the largest contributions to variance accounted for by models. While the unique effects (i.e., semi-partial correlations) of PBL on STEM attitudes are statistically significant, they are small relative to variables such as race, gender and school effects. In addition, because this research is based on student self-report data, we were unable to include a variety of other variables that likely contribute to the true implementation of PBL (such as teacher experience, student mobility and other administrative variables). Other demographic data was too sensitive to collect via student survey, such as ELL or special education status. Further research should utilize administrative data to more fully account for additional person and contextual variables.

Self-report survey data also requires us to consider PBL as reported by students. While this perspective is a valuable one, further research should incorporate external assessments of PBL, such as through observations by trained researchers.

As noted earlier, the constructs of intrinsic motivation and ability beliefs are also highly related to one another, since students tend to be more motivated when they feel that learning outcomes are within their control [58]. Certainly, we observed a degree of correlation between ability beliefs and intrinsic motivation items, though analyses supported each measure as a separate entity. Given the correlational nature of this study, we are unable to pinpoint whether ability beliefs predict intrinsic motivation or if these happen in tandem, in part as a function of PBL. Further research can disentangle this relationship via time-series methodology, examining how or if ability beliefs facilitates intrinsic motivation for STEM subjects or these outcomes occur in parallel (i.e., PBL facilitates both simultaneously).

Given the predominance of PBL in the study population (students in inclusive STEM high schools), we were unable to discern the effect of PBL’s presence in the school (i.e., as a dichotomous variable) compared to the students’ perceptions of PBL as measured by our scale. Our findings suggest that higher ratings do in fact predict higher IFSTEMC; thus, we would argue that true implementation of PBL (vs. merely including PBL in the curriculum) is likely a critical piece of this relationship. While many schools claim to incorporate PBL in their school models, our findings suggest that quality of PBL is essential. (Here, we use student ratings to estimate quality.) The amount of PBL instruction that student receive likely also plays a role here and is worthy of further investigation. However, full implications cannot be determined from this analysis. Further research should examine the implementation of PBL and compare effects between PBL and non-PBL classrooms (in the context of both STEM and non-STEM schools), as well as more closely examining implementation.

Despite predicted limitations of this study, findings clearly demonstrate a link between student perceptions of PBL and ultimate student interest in future STEM careers (IFSTEMC). While this is a marked goal for inclusive STEM high schools, these findings are relevant beyond this population. STEM-specific schools are but one channel through which we may increase and enhance our future STEM workforce; considering the huge number of students in comprehensive or otherwise non-STEM focused schools, it is important for the broader education community to also carefully consider strategies that may support and bolster students’ interest in STEM careers. The findings presented here on the qualities of PBL which support that interest may be applied in any classroom where such approaches are employed, regardless of whether it resides in a STEM-focused school.