Methodological Choices When Assessing Summer Bridge Programs in STEM Majors: A Scoping Review

Abstract

Summer Bridge Programs (SBPs) support students’ personal, organizational, content-related, and social challenges as they transition from high school to postsecondary education. However, the research is not conclusive on the overall effect of these programs on academic performance. Prior systematic reviews and meta-analyses have raised critical methodological concerns. Following the PRISMA framework, this scoping review examines the methodological aspects when assessing the academic effects of SBPs on incoming STEM (Science, Technology, Engineering, and Mathematics) students. The inclusion criteria were studies that quantitatively assessed the academic performance of SBPs offered during the summer prior to the start of classes for incoming first-year STEM students and that included a control group. We searched ERIC, Web of Science, SciELO, and ProQuest databases. This review included 37 studies evaluating 33 programs. The majority of academic evaluations relied on course performance, graduation or retention rates, and GPA (Grade Point Average) outcomes. Moreover, the evidence is mainly descriptive or observational, with no control variables. We found a lack of studies that used an experimental or observational approach while also controlling for participants’ background variables. Finally, the reported outcomes are mainly short-term, with only a small number of studies assessing longer-term effects of SBPs, such as graduation rates.

1. Introduction

The number of students in Higher Education Institutions (HEIs) has increased over the past decade and is expected to continue rising in the coming years (OECD, 2018). With the increase in access to tertiary education, students who were traditionally excluded from higher education are now enrolling in HEIs (Reed, 2016). For example, as significant progress has been made in closing historical gender gaps, women are now more likely to pursue post-secondary studies after graduating from secondary education (GEM Report UNESCO, 2024; OECD, 2018, 2021). This massification and democratization of higher education bring new challenges of equity, innovation, adaptation, and quality of education, among others (Hornsby & Osman, 2014).

In the transition to higher education, students face several challenges related to personal, organizational, content-related, and social aspects (Trautwein & Bosse, 2017). While addressing the traditional requirements of students, HEIs must also focus on the issues arising from increasing demographic diversity (Maringe & Sing, 2014). For example, a high proportion of students enter HEIs unprepared for the academic rigor of college or university courses (Bettinger & Long, 2005; Venezia & Jaeger, 2013). Students with marginalized identities (e.g., women, people who identify as sexual minorities and first-generation students) are more likely to experience chronic stressors during their transition to college (Kroshus et al., 2021). Additionally, first-generation students, unfamiliar with university culture and expectations, are more likely to drop out than their peers who are not first-generation students (Reed, 2016). Finally, women are underrepresented in engineering, manufacturing, and construction, as well as in information and communication technology studies (UNESCO, 2017), and they leave STEM disciplines at higher rates and are less likely to travel abroad for studies compared to men (GEM Report UNESCO, 2024).

In STEM disciplines, there are several challenges related to academics, performance and access. A recent meta-analysis found several interconnected barriers experienced by underrepresented groups in STEM education, such as socioeconomic factors, limited parental support, constrained school environments, and sociocultural norms (Liu & Kringos, 2025), which in turn impact students’ persistence in the field. Some factors identified are the lack of preparation or support students have when entering the tertiary education system (Burt et al., 2023; James & James, 2025; Masjutina et al., 2025) and lack of diversity (Blackburn, 2017; Botella et al., 2019; Scott & Martin, 2014).

Strategies to address these challenges include interventions that help students make sense of their academic experiences (Harackiewicz & Priniski, 2018). Many institutions that offer STEM programs have implemented Summer Bridge Programs (SBPs) to address these challenges in the context of higher education (Odeleye & Santiago, 2019). As Liu and Kringos (2025) state, targeted interventions specifically designed to support students from disadvantaged backgrounds can be aligned with specific barriers, such as the lack of preparation among students. SBPs typically occur during the summer before the first term starts, acting as a bridge from high school to college or university (U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse, 2016). However, the literature shows little consistency across definitions of SBPs (Sablan, 2014) because each program is tailored to an HEI’s specific goals and constraints, making each SBP unique. Sablan (2014) characterized SBPs by describing how they are similar and different across institutions and identified characteristics of program participation, administration, funding, and curriculum. Those characteristics can impact program evaluations (Baron, 2004; Calcagno & Long, 2009). For example, some SBPs aim to facilitate the transition to careers requiring a two-year degree (e.g., pharmacy students in Verdone et al., 2020), improve course placement in engineering (e.g., J. R. Reisel et al., 2012), or increase students’ self-efficacy (e.g., Hanlon & Schneider, 1999; Kilian, 2010). Other SBPs focus on a targeted subgroup, such as Latina women (e.g., Johnston et al., 2021), students from rural areas (e.g., Sheffield et al., 2018), migrant students (e.g., Gilbert, 1982), and African American students (e.g., J. M. Johnson, 2016).

Several variations in SBP implementation also exist within STEM fields. Some programs aim to increase the preparation of high school students by enhancing self-efficacy, motivation, and sense of belonging (e.g., Barth et al., 2021; Kassaee & Rowell, 2016), fostering interest in STEM careers and aspirations (e.g., Kitchen et al., 2018; Pong et al., 2011) or strengthening STEM identity (e.g., Knight et al., 2013).

1.1. Methodological Considerations When Evaluating Summer Bridge Programs

SBPs, like any other educational program, should be subject to evaluation. Evaluation aims to improve the quality of interventions (Owen, 2020). The evaluation process not only involves defining criteria and standards but also requires decisions about the type of evidence to collect and how to synthesize information. Behind any evaluation are different questions—epistemological, ontological, and normative—that guide methodological choices and method selection (Brandon & Singh, 2009; Chouinard & Cousins, 2009). Ultimately, the research design should align with specific questions being asked about a program (Cohen et al., 2008; Creswell & Creswell, 2023; Slavin, 2020). In this section, we outline some differences in the implementation of SBPs may pose challenges to evaluating them as a single, uniform intervention. Through examples of different STEM-specific SBP implementations, we show how different implementations translate into distinct methodological considerations when evaluating SBPs. We do not aim to create a comprehensive list of methodological considerations for each program, nor do we prioritize one method over others.

1.1.1. Differences in Implementation

Variances in implementation, such as format, social component, and duration, differ significantly across institutions depending on their goals. Some SBPs are entirely online (e.g., Tempelaar et al., 2011), others in-person or residential (e.g., Walter & Wade, 2024; Strayhorn, 2011), while other institutions have decided to implement a hybrid format with both online and in-person settings (e.g., Gibson et al., 2021). In a fully online or hybrid setting, students benefit from self-paced courses that provide greater autonomy and flexibility in their learning process compared to traditional face-to-face setting. However, online programs present unique challenges, such as differences in students’ time management, activity design, academic integrity, and assessment of online collaboration (Kearns, 2012). Thus, when evaluating these programs, some argue that research should avoid excessive focus on comparison studies between face-to-face and distance-delivered instruction, as the factors influencing these formats are not equivalent (Rovai, 2003).

In contrast, in-person (or residential) programs expose students to the culture of the university or college and foster relationships with peers and faculty members. These non-academic experiences, or “college knowledge” (Conley, 2008), can influence students’ social experiences prior to enrollment and enhance their sense of preparedness. For instance, if the social component includes female role models, SBPs may have a distinct impact on women (Herrmann et al., 2016; Rosenthal et al., 2013; Stout, 2013). Similarly, programs offering academic counseling or mentoring can help first-generation students set realistic expectations and strengthen their sense of belonging and self-efficacy (Hagler et al., 2024). Evaluating such programs requires incorporating diverse perspectives to analyze educational outcomes for varied learners. When examining the experiences of marginalized groups, broader structural factors should be considered to explain differences in academic outcomes (Jordan, 2010). Evaluations that account for cultural diversity are increasing globally, leading to the development of new methods, instruments, and statistical techniques (Boyce, 2017; Hood et al., 2015). For example, Nachbauer and Kyriakides (2020) found that decisions about equity indicators in educational outcomes involve choices such as whether to use longitudinal or cross-sectional measures, models of analysis, and class size considerations.

Finally, little attention has been given to program duration and the time allocated to specific activities (Sablan, 2014). Duration varies across SBPs, often depending on budgetary and logistical constraints. In some cases, SBPs serve as the starting point for extended interventions that include additional forms of support (Bradford et al., 2021), such as peer mentoring or extracurricular activities (e.g., Ikuma et al., 2019; M. Johnson et al., 2013; Lisberg & Woods, 2018).

1.1.2. Differences in Program Participation

Program participation and self-selection depend heavily on each institution’s decision regarding which students to target for each SBP. As institutions identify the characteristics of students who will receive support (Sablan, 2014), each SBP may target different subgroups. Some SBPs focus on all incoming students, regardless of their level of preparedness or academic skills (e.g., Ikuma et al., 2019). Others target students based on their high school performance (e.g., GPA or standardized entrance test such as the Scholastic Assessment Test, SAT in the US). Evidence shows that improving academic skills depends on the degree of support students need (Boatman & Long, 2018; Calcagno & Long, 2009).

Moreover, self-selection bias can occur if the program is offered to all incoming students, but registration is voluntary. In such cases, background characteristics may differ between students who register and those who do not (Baron, 2004), which could also impact participants’ future performance, regardless of their participation in the program. For example, a study conducted by Pain (2016) explored the relationship between the voluntary nature of remediation programs and students’ grades and completion rates, finding a decrease in math and English course grades after remediation became voluntary. Researchers should therefore consider selection bias when assessing the program effectiveness. This can be addressed by using an experimental approach or, if that is not feasible, employing carefully designed matched experiments to minimize selection bias (Gopalan et al., 2020; Slavin, 2002) or matching techniques such as Propensity Score Matching (PSM) (Adelson, 2013; Rosenbaum & Rubin, 1983). Consequently, decisions regarding how background variables are identified and measured, models used, stratification techniques, and effect estimation methods should be carefully planned (Fan & Nowell, 2011; Steiner et al., 2011). Other techniques to address selection bias include systematic sampling (Creswell & Creswell, 2023), statistical tests of group equivalence, or adding covariates in the analysis (Graham & Kurlaender, 2011).

1.1.3. Differences in Academic Content

Program academic content, completion criteria, and credits or placement in subsequent courses can also impact how students engage with the program and their post-program success. Most SBPs focus on mathematics and English because these subjects are foundational across most activities in HEIs (Henry et al., 2014; Peters, 2023). However, other programs focus on courses with low retention rates or high failure rates among first-year students (e.g., Harrington et al., 2016). Some programs include formative quizzes or tests to help students reflect on their learning (e.g., Gleghorn & Stwalley, 2011), which can also impact their performance at the end of the SBP, as students have the opportunity to address misconceptions identified during these evaluation (Morris et al., 2021).

Finally, students’ motivation to complete the program changes when completion is tied to a grade or placement in an advanced course (Chamberlin et al., 2023; Demszky et al., 2024; Rodríguez Rincón et al., 2024). Students who have an incentive to complete the program will differ in their engagement and effort from those who are intrinsically motivated to learn for skill improvement and college preparation.

1.2. Prior Research and Existing Knowledge

Despite the variation among different implementations of SBPs, research on specific programs shows that these programs can positively impact students’ experiences during their first year (Barth et al., 2021), persistence in higher education (Ikuma et al., 2019) and pass rates (Wathington et al., 2011). However, findings on the effectiveness of SBPs are mixed and often inconclusive (Calcagno & Long, 2009; Odeleye & Santiago, 2019), making it difficult to evaluate their overall effectiveness. A Randomized Control Trial (RCT) conducted by Barnett et al. (2012) in eight HEIs found that the programs did not affect the average number of credits attempted or earned. However, they did improve completion rates for first-year college-level courses in math and writing, but no evidence suggested an effect on persistence.

Nevertheless, there have been efforts to systematically collect evidence on SBPs. For example, Ashley et al. (2017) reported the goals of 30 STEM programs, categorized these goals into academic success, psychosocial, and department-level objectives, and evaluated the SBP’s success in meeting institutional goals. Finally, regarding the effect on academic outcomes in STEM majors, especially in GPA and retention rates, Bradford et al. (2021) synthesized 16 studies in the first meta-analysis of SBPs and reported that program participation has a moderate positive effect on first-year academic achievement. Similarly, the report by What Works Clearinghouse only found one study that met the group design standards (U.S. Department of Education, Institute of Education Sciences, What Works Clearinghouse, 2016).

Regarding methodological aspects, previous reviews and meta-analyses have identified several issues in evaluating the effectiveness of SBPs. For example, Odeleye and Santiago (2019) highlighted the need for research that focuses on longitudinal and quantitative assessments of SBPs and compares participants’ outcomes with those of a control group. Sablan (2014) also emphasized this issue and stressed the importance of considering differences and similarities among SBPs. Bradford et al. (2021) noted the challenge posed by the limited availability of detailed information about the methodologies employed.

Despite the limited number of studies addressing the effects of SBP on academic achievements, a significant gap persists in understanding the methodological approaches used to evaluate these programs. While earlier systematic reviews have identified common methodological limitations, no comprehensive review has systematically compared the diverse methodologies employed. This scoping review aims to provide an overview of how the literature assesses the evidence of the academic impact of SBPs offered to incoming students of STEM-related majors. Specifically, this review focuses on key elements such as outcomes of interest, study design (experimental or non-experimental), analytical approaches, confounding variables controlled in the analysis, sample size, and data collection period.

We focused solely on quantitatively measured academic outcomes for two main reasons. First, this provides a solid foundation for organizing existing research using comparable indicators relevant to SBP evaluation and HEI accountability (Mfengu & Raju, 2024; U.S. Department of Education, 2023). Second, there is significant diversity and little consensus on SBP implementation and its effects on students, particularly regarding non-academic outcomes (e.g., programs with role-modeling components influencing STEM identity). By emphasizing academic results, we also align with a consensus established by prior systematic reviews of SBP effectiveness (Bradford et al., 2021; Barnett et al., 2012).

Finally, the scope of this review is not to synthesize the overall effectiveness of SBPs but rather to identify studies that meet minimum methodological criteria for evaluating these programs. By building upon prior reviews and studies, this paper offers a more in-depth examination of the methods used and provides guidance for future researchers on how similar programs have assessed SBP effectiveness.

2. Materials and Methods

We selected the Systematic Scoping Review as the methodology for this review because we aimed to provide an overview of the evidence in the literature regarding the quantitative assessment of SBPs for STEM majors. To conduct this review, we followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guidelines to establish the search protocol and selection criteria for relevant studies (Tricco et al., 2018). Specifically, we organized the review into the standard stages of (i) search and identification of records, (ii) eligibility definition using the Population Intervention Comparison and Outcome (PICO) framework (National Institute for Health and Care Excellence [NICE], 2014) to define inclusion/exclusion criteria, (iii) screening for title and abstract and duplicate removal, (iv) data extraction and (v) data synthesis of the studies included in the review (Tricco et al., 2018).

2.1. Search

To capture literature from leading engineering and STEM education journals, we searched a broad range of databases, including ERIC, Web of Science (Core Collection), SciELO, and ProQuest. Each database was queried using combinations of keywords and phrases, including logical connectors (AND, OR) with refined search parameters or filters applied to the title and abstract fields (see Table 1).

Table 1. Databases and their corresponding search strategy.

Databases	Search String
ERIC, ProQuest, SciELO (Web of Science)	(bridg* OR summer OR prefreshm* OR freshm* OR remediation) AND (program* OR class OR course) AND (engineer* OR STEM OR scien* OR chem* OR tech* OR math* OR biolog* OR physic*) AND (college OR university OR undergraduate)
Web of Science	((bridg* OR summer OR prefreshm* OR freshm* OR remediation) NEAR/3 (program* OR class OR course)) AND (engineer* OR STEM OR scien* OR chem* OR tech* OR math* OR biolog* OR physic*) AND (college OR university OR undergraduate)

Because we wanted to map the full range of methodological approaches, we chose not to apply a time restriction to publication dates. The last search was conducted in January 2025. We also searched Google Scholar for additional relevant papers to extend the search coverage to gray literature. Finally, as a supplementary search, we manually reviewed programs included in the review by Ashley et al. (2017) and the meta-review conducted by Bradford et al. (2021) to ensure inclusion of as many programs as possible.

2.2. Inclusion/Exclusion Criteria

The PICO framework helped organize the relevant parameters of the review. We structured the inclusion/exclusion criteria by clearly defining the population of interest, the intervention or activity undertaken by the population, comparisons or alternatives to the intervention, and outcomes measured. Table 2 provides a detailed description of each aspect.

Table 2. PICO framework for the current review.

PICO Framework	Description
Population	All incoming first-year undergraduate students in any discipline within the field of Science, Technology, Engineering and Mathematics area.
Intervention	A Summer Bridge Program conducted during the summer prior to the first fall semester. The duration may vary but must occur exclusively during the summer. Program format (online or in-person), voluntariness, and participation criteria are not restricted.
Comparison	Other incoming first-year undergraduate STEM students who did not participate in any SBP activities.
Outcome	Any academic achievement measurement.

Generally speaking, all studies included in this review compare an experimental group with a control group and quantitatively evaluate outcomes related to academic performance. The SBP must occur exclusively during the summer prior to the fall semester and should not extend beyond that period. As part of the inclusion criteria, we limited the review to reports published in English. Table 3 outlines the reasons for exclusion.

Table 3. Exclusion criteria reasons and description.

Reason	Description
Language	The paper is not written in English.
Non-empirical study	The paper describes one or more programs without an appropriate study design to measure academic outcomes. It focuses on program design, implementation, or cost analysis rather than evaluation.
Not a primary study	The paper synthesizes existing primary studies, is a systematic or meta-review, or analyses multiple programs.
Not for STEM majors	The paper examines a program not intended for STEM-major students; it does not mention any targeted major or is an institution-wide program offered to all students.
Other intervention	The paper addresses a different type of intervention, such as a program that does not occur immediately after high school, a yearlong program, a research initiative, or a broader intervention that extends beyond the summer.
Other study design	The study design does not align with the aim of this review. For example, it uses a purely qualitative approach, measures non-academic outcomes (e.g., belonging, confidence), or lacks a control group.

2.3. Study Selection/Screening

The screening, the second phase, followed the inclusion criteria described earlier. Two reviewers independently screened each record from the search results to assess its eligibility for inclusion. When there was no consensus on whether to include the paper in the next phase, the team met and discussed until an agreement was reached.

2.4. Data Extraction

For each eligible study, we extracted information about the SBP and the methodology used to assess the relevant outcomes. SBP-related information included the institution offering the program, country, program duration, eligibility criteria or target students, selection criteria, participants’ major, subject taught, voluntariness, and program name. For the methodology, we recorded the verbatim research question or aim of the study, dependent variables, covariates or controlled variables, study design, analytical approach, description of treatment and control groups, number of participants in each group, number of cohorts involved, and the data collection timeframe. Finally, we extracted the publication type and the publisher’s name.

2.5. Data Synthesis

We aimed to describe the methodological aspects of the literature in three areas: (1) outcomes of interest, (2) study design and analytical approach, and (3) participants and cohorts involved. Some papers addressed several research questions with different data collection sources and methods that could complement the methods described in the eligibility criteria. Those complementary methods and analyses, such as pre-post comparisons of the SBPs, were not reported in the results.

For outcomes, only quantitative academic measures were extracted. When a paper reported multiple measures using different methods, only the information about the methods related to the outcome of interest were extracted.

For study design, we adopted a broader categorization of research methods based on biomedical traditions. Thus, each record was classified as descriptive or analytical, depending on whether it aimed to establish or quantify a relationship between variables (Noordzij et al., 2009; Ranganathan & Aggarwal, 2018, 2019). Descriptive designs summarized outcomes using traditional descriptive statistics such as mode, mean, or median (Loeb et al., 2017; Thompson, 2006). These studies primarily identified trends in experimental and control groups but did not attempt to establish statistical relationships. Analytical studies were further categorized as experimental or observational. In experimental studies, participants were randomly assigned to experimental or control groups, whereas in non-experimental studies, no random assignment occurred (Creswell & Creswell, 2023; Thompson, 2006). Observational studies involved no manipulation of variables or participation in the intervention (i.e., SBP). For analytical studies, we identified whether they controlled for confounding variables beyond the group variable. If no controls were applied, outcomes were examined solely in relation to group assignment (experimental vs. control). We also recorded data analysis techniques, including statistical tests, models, and descriptive statistics.

Finally, the total number of participants in each group was aggregated, including cases where papers reported multiple control groups or disaggregated participant numbers by cohort.

3. Results

3.1. Search Results

As shown in Figure 1, we initially retrieved 4041 papers across the four databases of interest. After removing duplicates, 3789 records were considered for screening. Next, we applied the screening criteria described in Table 2, which resulted in a subset of 226 articles for further review. Ultimately, 37 reports were extracted (33 from the database search and 4 from the supplementary search). Figure 1 also illustrates the frequency of exclusion reasons for the remaining 192 studies that were not included.

Figure 1. PRISMA flow diagram.

The 37 reports corresponded to a total of 33 unique programs included in this review: one report considered two different programs (Wheatland, 2002), and three programs were considered by multiple reports—four reports (Cançado et al., 2018; J. Reisel et al., 2011, 2012, 2014), two reports (Wischusen et al., 2010; Wischusen & Wischusen, 2007), and two reports (Chariker et al., 2013; Hieb et al., 2015).

3.2. Description of the Included SBP

Table 4 summarizes the SBPs included in the review and their distinctive characteristics. As expected, most of the SBPs considered in the review were offered by institutions in the US. Although we identified several programs across different regions in the screening phase, only four met the inclusion criteria: two in Germany (Eitemüller & Habig, 2020; Zorn et al., 2011), one in Australia (Schmid et al., 2012), and one in Italy (Bozzi et al., 2019). Program duration ranged from five days to ten weeks, with an equal number of SBPs (n = 13) lasting one or two weeks and five or six weeks. Most SBPs were in-person (n = 23); five included both online and in-person activities, and only three were entirely online.

Table 4. Description of included studies.

	Program	Institution (Country)	Duration [Weeks]	Format	Eligibility Criteria/Target Students 1	Selection to the Program Criteria	Participants’ Major—Subject Taught 2	Reference
1	WISE Summer Bridge Program	Arizona State University (USA)	1	In-person	Incoming female students	Not mentioned	Eng—Math, Phys, Chem, Computer-programming	Fletcher et al. (2001)
2	Early Bird Program	Berlin Institute of Technology (Germany)	9	In-person	Incoming students required to take Calculus I and Linear Algebra for Engineers in their first semesters	Not mentioned	Eng—Math	Zorn et al. (2011)
3	Summer Math and Language Arts Enrichment Program	California State University (USA)	5	In-person	Incoming students, URS, with a high school GPA of 3.0 or above; underrepresented and economically disadvantaged students	High school science and math preparation, the strength of the recommendation and interest in research careers	Basic biomed science—Math, English	Oppenheimer et al. (2020)
4	SMILE	Cañada College (USA)	2	In-person	Students who have expressed interest in pursuing engineering and STEM but performed low in the placement test	Not mentioned	Eng and other STEM—Math	Enriquez (2011)
5	Summer Diversity Program	Delaware State University (USA)	8	Online and in-person	Incoming students placed into Introduction to Algebra or College Algebra	Not mentioned	STEM—Math	Harrington et al. (2016)
6	Not mentioned	Drexel University College of Engineering (USA)	2	In-person	All incoming female and URS in engineering	All students who applied were accepted	Eng—Math, Chem, Phys and Computer Labs	Erickson-Ludwig and Clyne (2014)
7	Challenge Program	Florida International University (USA)	NA	In-person	Enrolled incoming students	Not mentioned	Eng—Eng design	Kornreich-Leshem et al. (2015)
8	Summer Math Jam	Georgia Tech (USA)	5	In-person	All first-time incoming students, but actively promotion of incoming minority students (based on ethnicity)	Not mentioned	Eng—Math, Chem, CS, English	Murphy et al. (2010)
9	SEEP	Jackson State University (USA)	10	In-person (residential)	First time incoming students with ACT scores from 17 to 25	Not mentioned	Eng—Math	Whalin and Pang (2014)
10	BIOS	Louisiana State University (USA)	5 days	In-person	All first-time incoming students and pre-enrolled in the Introductory Biology Science Majors I	FCFS until maximum of 60 students is reached	Bio—Bio	Wischusen et al. (2010); Wischusen and Wischusen (2007)
11	Online Bridge Course for Science/Engineering	Manhattan College (USA)	7	Online	Students in border case based on TRAM scores	Not mentioned	SEM—Math	Jura and Gerhardt (2022)
12	AMP Summer Math Bridge Program	Morgan State University (USA)	4	In-person (residential)	Admitted MSU students with math SAT score between 470 and 570	All applicants were accepted when pools were small. Otherwise, a 20% cap was applied	SEM—Math, English, Chem and Introduction to computers	Wheatland (2002)
13	NASA Morgan Engineering Enrichment Program	Morgan State University (USA)	5	In-person (residential)	Admitted MSU students with SAT score of 1000 or more and high school GPA of 3 or more	Not mentioned	Eng—Math, English, Chem, Phys, introduction to computers	Wheatland (2002)
14	Not mentioned	(USA)	6	In-person (residential)	Incoming engineering students	Not mentioned	Eng—Math	Doerr et al. (2014)
15	OSTEP	Ohio State University (USA)	6	In-person	First-generation students and URM	Not mentioned	STEM—NA	Tomasko et al. (2016)
16	Not mentioned	Politecnico di Milano (Italy)	2	Blended (online and in-person)	All incoming students	Not mentioned	Eng—Phys	Bozzi et al. (2019)
17	ABC	Purdue University (USA)	5	In-person	All incoming students	First come, first served	Eng—Chem, Math, MATLAB and English	Gleghorn and Stwalley (2011)
18	Jump Start program	St. Mary’s University (USA)	12 days	In-person (residential)	Incoming students placed into precalculus (based on readiness placement test or SAT/ACT score)	Not mentioned	STEM—Math	Lecocke et al. (2019)
19	E-MAP	University of Alabama (USA)	5	In-person	All incoming students, targeting 60% of incoming students (placement scores to pre-calculus course)	Not mentioned	Eng—Math	Gleason et al. (2010)
20	SP-Chem	University of California—UC Davis (USA)	NA	Online	All incoming students	Randomly	STEM—Chem	Dockter et al. (2017)
21	Summer Bridge Residential Program	University of Cincinnati (USA)	6	In-person (residential)	URS (race, ethnicity, and gender) and disadvantaged (socioeconomically and educationally) groups	Not mentioned	Nursing—Anatomy, Physiology, Chem, Math	Pritchard et al. (2016)
22	Summer Algebra Intervention course	University of Louisville (USA)	5	Onsite and online (self-guided)	Low Performing Students (below 80% based on readiness exam)	Not mentioned	Eng—Math	Chariker et al. (2013); Hieb et al. (2015)
23	Not mentioned	University of Maryland Baltimore County (USA)	6	Online	Incoming students with low placement scores	Randomly	STEM—Math	Chingos et al. (2017)
24	Exploring Engineering—E2	University of South Alabama (USA)	2	In-person	High-achieving incoming students (based on ACT)	Based on academic achievement and interest (demonstrated through an essay)	Eng—Interdisciplinary Eng topics	Steadman et al. (2014)
25	Not mentioned	University of Sydney (Australia)	7 days	Hybrid	All students enrolled in Fundamentals of Chemistry 1A who do not have university entry-level chemistry knowledge	Not mentioned	Any major that has the course—Chem	Schmid et al. (2012)
26	JSM	University of Texas (USA)	8	NA	Underrepresented groups	Not mentioned	Eng—Math	Weatherton et al. (2011)
27	Not mentioned	University of Wisconsin-Milwaukee (USA)	4	In-person (residential)	Low-performing students (low placement score)	Not mentioned	Eng—Math	Cançado et al. (2018); J. Reisel et al. (2011, 2012, 2014)
28	Not mentioned	University of Duisburg-Essen (Germany)	2	In-person	All students with courses with an affinity to chemistry (except for engineering sciences)	Not mentioned	Med, Bio, Chem, Phys, Water Science, Energy Science—Chem	Eitemüller and Habig (2020)
29	FREE	University of Alabama (USA)	2	In-person	High achieving students (ACT ≥ 28) and URS with high math scores or high school coursework in Calculus	Not mentioned	Eng—Electrical, computer engineering and mechanical and materials eng	Jefferson et al. (2013)
30	SoarCS	University of Massachusetts Lowell (USA)	1 or 4	In-person	All first-year undergraduate students with special encouragement of minority students	The program was non-selective	CS—CS	Dones et al. (2022)
31	RDI Summer Institute	North Carolina Central University (USA)	5	In-person (residential)	All pre-college students who have been accepted for admission with minimum high school GPA of 2.5	All who complete the application process are accepted	STEM—Eng design, entrepreneurial thinking and knowledge skills supporting engineering design	Jackson et al. (2024)
32	HSSM	Mills College (USA)	2 or 4	In-person (residential)	Based on their interest in STEM and math SAT score of 550-650	Not mentioned	STEM—Bio, Chem, Calculus	Walter and Wade (2024)
33	Not mentioned	Auburn University (USA)	4	NA	Traditionally URS in STEM, ACT score of 18-28	Based on an application, essay, transcripts, and two letters of recommendation	Science—Math, Chem	Evans (2014)

¹ URS: Underrepresented Students. ² Eng: Engineering; STEM: Science, Technology, Engineering, and Math; CS: Computer Science; SEM: Science, Engineering and Math; Med: Medicine; Bio: Biology; Chem: Chemistry; Phys: Physics.

The eligibility criteria for participation varied significantly across programs. Programs established criteria based on academic performance, students’ major, targeted subgroups, and/or specific curricular courses. Most programs (n = 18) set criteria related to academic readiness, such as high school grades (e.g., Jackson et al., 2024), standardized entrance tests (e.g., Lecocke et al., 2019), or placement tests (e.g., Chingos et al., 2017); four of these programs targeted high-performing students (Jefferson et al., 2013; Steadman et al., 2014; Wheatland, 2002). Seven programs were open to all incoming students within the targeted major, and eight programs were restricted to specific subgroups, such as female students (e.g., Fletcher et al., 2001), particular ethnic groups (e.g., Pritchard et al., 2016), socioeconomically disadvantaged students (e.g., Oppenheimer et al., 2020), or first-generation students (e.g., Tomasko et al., 2016). Finally, four programs based their eligibility on enrollment in a single course during the first or second semester, such as Calculus I or Linear Algebra (Zorn et al., 2011) or introductory Biology Science (Wischusen et al., 2010; Wischusen & Wischusen, 2007).

As Table 4 shows, most SBPs targeted Engineering majors (n = 17), with strong emphasis on math preparation. Nine SBPs included STEM majors, two were oriented to SEM (Science, Engineering and Math) (Jura & Gerhardt, 2022; Wheatland, 2002), two targeted health science majors: biomedical basic science (Oppenheimer et al., 2020) and nursing (Pritchard et al., 2016), one focused on biology students (Wischusen et al., 2010; Wischusen & Wischusen, 2007), two were open to any student requiring an introductory chemistry course (Schmid et al., 2012; Eitemüller & Habig, 2020), and one targeted Computer Science students (Dones et al., 2022). Consequently, most programs included math classes (n = 20). Interestingly, four SBPs for Engineering majors did not include traditional first-year engineering courses (i.e., math, chemistry, or physics); instead, they focused their programs on interdisciplinary engineering topics (Steadman et al., 2014), engineering design (Kornreich-Leshem et al., 2015), electrical and computer engineering alongside mechanical and materials engineering (Jefferson et al., 2013) and engineering design combined with entrepreneurial thinking (Jackson et al., 2024). Other courses taught in the SBPs included chemistry (n = 12), computer science or introductory computing/software (n = 7), English (n = 5), physics (n = 4) and biology (n = 2).

Regarding program voluntariness, we identified only one program that required participation as a condition for college admission (Cançado et al., 2018; J. Reisel et al., 2011, 2012, 2014).

3.3. Methodological Considerations When Assessing SBP

Although programs varied in implementation, we identified common methodological elements used to assess them. We organized these aspects into outcomes, study design, analytical approach, and participant/cohort characteristics.

3.3.1. Outcomes

The evidence reported primarily addresses short-term effects of SBPs, with the most common outcome being performance in subsequent courses related to summer content (n = 21), followed by retention or graduation rates (n = 20) and GPA (n = 12). Interestingly, Chingos et al. (2017) also evaluated placement into higher-level courses as an academic outcome.

Regarding the first outcome which focuses on subsequent course performance, the program’s evaluation was based on students’ performance in the corresponding course during the first term or year. That performance was measured using course passing rate (e.g., Oppenheimer et al., 2020), average score on the final exam (e.g., Dockter et al., 2017; Harrington et al., 2016), average course grade or the percentage of students achieving a specific grade threshold, such as C or better (e.g., J. Reisel et al., 2011, 2012, 2014) or DFWI rates, the percentage of students who receive a D or F or withdraw (e.g., Jura & Gerhardt, 2022).

The second outcome was the retention or graduation rates. These outcomes were calculated as the likelihood of graduation (e.g., Murphy et al., 2010), as the ratio of students who remained in the same major, or as the percentage of students on track to graduate within a timeframe (e.g., Wischusen et al., 2010). Of the 16 studies reporting retention or graduation rates, 15 focused on retention; two reported both (Cançado et al., 2018; Oppenheimer et al., 2020); and only one assessed graduation (Murphy et al., 2010). Accordingly, the studies assessing graduation rates collected data longitudinally, from 3 to 6 years after SBP participation; the rest of the studies collected data at the end of the fall semester (e.g., Steadman et al., 2014), at the end of the first year (e.g., Pritchard et al., 2016), or after four years (e.g., Tomasko et al., 2016).

Finally, academic performance was also measured using cumulative GPA (e.g., Wheatland, 2002), GPA in a particular semester(s) (e.g., Wischusen & Wischusen, 2007) or GPA for a set of courses aligned with the SBP content, such as math courses (e.g., Chingos et al., 2017) or gatekeeper STEM courses (e.g., Jackson et al., 2024). These studies usually collected data at the end of the first semester or year.

3.3.2. Study Design and Analytical Approach

Considering study design, the evidence was mainly analytical, i.e., observational or experimental. We classified 14 SBP studies as descriptive. There is almost no experimental study design in the analytical studies, as only two studies randomly assigned participants to the experimental or control group (Dockter et al., 2017; Chingos et al., 2017). All other analytical studies were classified as observational (or non-experimental).

Within the analytical papers, most studies assessed the outcome of interest while controlling for one or more variables. However, we identified 10 studies whose inferential analyses did not involve any controlling variables. Figure 2 summarizes the study design and analytical approach used in the included studies.

Figure 2. Summary of different study designs and analytical approaches of included studies.

We found 14 studies that followed a descriptive approach. Those studies analyzed the academic performance of the SBP participants by comparing the experimental and control groups using traditional descriptive methods. For categorical outcomes, such as course grade or retention rate, the comparisons were made by examining the percentage of students in each group (e.g., Gleghorn & Stwalley, 2011). For numerical variables, such as GPA or exam scores, the mean (and sometimes the standard deviation) and median statistics were used to describe and compare the two groups (e.g., Pritchard et al., 2016). Studies that followed a descriptive approach provided insights by capturing trends and patterns. However, these studies offered limited analytical depth, as they primarily rely on percentages, ratios, and means (and occasionally standard deviations), with no manipulation of variables, which restricted interpretation. As a result, these studies lacked the ability to establish causal relationships or explain the underlying factors behind observed differences.

There were 23 analytical studies, most of which used an observational design (n = 21). Of the 21 studies, three used a matching technique to obtain two balanced groups because they did not use randomization to allocate participants to the groups (Kornreich-Leshem et al., 2015; Wischusen & Wischusen, 2007; Jackson et al., 2024). Some studies (including the experimental ones) also considered confounding variables that could explain students’ academic performance besides participation in the SBP. Those considering confounding variables used previous academic performance as a covariate or variable to be controlled in their analysis. Academic performance was often measured using external instruments such as SAT scores (e.g., Doerr et al., 2014), American College Testing (ACT) scores (e.g., Cançado et al., 2018), or high school GPA (e.g., Chingos et al., 2017). In contrast, in the study by Schmid et al. (2012), the variable controlled was students’ self-reported experiences and prior knowledge about the subject to be taught in the SBP; similarly, Evans (2014) considered the pre-health concentration of subjects. Furthermore, the studies by Hieb et al. (2015) and Jura and Gerhardt (2022) used an internal validated instrument developed by department members to measure academic readiness. After academic performance, the other covariates used were gender (n = 5), Socio-Economic Status (SES) or income (n = 2), ethnicity (n = 2), and major or discipline (n = 2). Interestingly, Hieb et al. (2015) also considered metacognitive and motivational factors in their analysis. Finally, three studies addressed the limitation of the lack of randomization in the groups by controlling for confounding variables: in their multivariate matching method, Kornreich-Leshem et al. (2015) considered SAT Math score, high school GPA, gender, ethnicity, and major as the covariates to be matched between treatment and control groups; Wischusen and Wischusen (2007) and Jackson et al. (2024) also considered high school GPA, ACT or SAT scores, major, and gender.

For studies that did not control for confounding variables, the analytical approach primarily involved statistical tests comparing means or proportions between groups. ANOVA or T-test was used for numerical outcomes, such as GPA (e.g., Steadman et al., 2014), whereas the Chi-squared test was used for proportions, such as graduation rate (e.g., Tomasko et al., 2016). Although no explicit controlling variables were used, five studies considered different control groups to compare to the treatment group. Oppenheimer et al. (2020) considered two groups as controls to compare to the treatment; one was students who were accepted into the program but did not attend, and the second was the group of students who applied but were not selected. Wischusen et al. (2010) considered students who were in the subsequent course in the fall, as well as those who remained on the program’s waiting list. Jura and Gerhardt (2022) considered three groups: the first group included all students taking the subsequent courses in the fall and spring terms, the second consisted of students who were borderline cases for SBP based on the readiness exam, and the third included students who scored high in the readiness exam and were placed in a higher math course. Tomasko et al. (2016) compared the treatment of all university STEM-major students and also compared it to that of all underrepresented STEM-major students. Finally, Gleason et al. (2010) compared the treatment group with students with similar math placement scores.

Table 5 shows that experimental studies provided evidence of short-term outcomes but not retention or graduation rates. The observational studies with controlling variables provide evidence for all three outcomes equally, whereas there is a slight tendency towards retention or graduation rates in the observational studies that did not control for confounding variables.

Table 5. Study design, analytical approach disaggregated by outcomes.

Study Design	Controlling Variables?	Subsequent Course Evaluation	Retention and/or Graduation Rates	GPA	Total (Row)
Descriptive	No	9	7	3	19
Analytical
Observational	No	4	7	3	14
	Yes	6	6	5	17
Experimental	Yes	2	0	1	3
Total (column)		21	20	12	54

3.3.3. Study Participants and Cohorts of Students

The aggregated number of subjects included in the studies also varied considerably. The average size of the treatment group was 123, compared to 1204 for the control group. Of the studies considered, 13 had more than 100 participants in the treatment group. For those studies that reported the number of subjects in the control group, 20 studies had more than 100 participants. Seven of those had more than 1000 subjects in the aggregated control groups.

Regarding the cohorts, most studies reported on a single cohort (n = 10). Twelve studies had two or three cohorts. Notably, one study considered 6 cohorts of students (Evans, 2014) and three studies considered 10 or more cohorts (Murphy et al., 2010; Oppenheimer et al., 2020; Walter & Wade, 2024).

As expected, the studies that used matching techniques (Kornreich-Leshem et al., 2015; Jackson et al., 2024; Wischusen & Wischusen, 2007) included a rather equal proportion of participants and non-participants. The same is true for the experimental study conducted by Chingos et al. (2017).

4. Discussion

This systematic scoping review aimed to collect and describe the methodological aspects of quantitative studies assessing the academic outcomes of SBPs for STEM majors. This review examined 37 studies that assessed the academic outcomes of SBPs intended to support first-year incoming students of STEM majors. The programs were only for students right after their high school education and ended before the fall semester started. As part of the eligibility criteria, the studies had a control group for the treatment condition. Our findings indicate a lack of studies that use an experimental or non-experimental approach and control for participants’ background variables. Only two studies followed an experimental design. Of the remaining studies, only three considered matching techniques to balance disparities in background variables. We also found that the academic evaluation relies on graduation or retention rates, GPA and course performance, as previous research also pointed out. The course performance included course passing, course grade, final examination pass or grade of all the courses in the subsequent Fall semester or for a particular course that the SBPs provided bridging to, such as Calculus I or Algebra.

This review has some caveats. First, we focused only on the academic effects of the SBPs, which means this review considers only quantitative outcomes. Other outcomes, such as STEM identity, motivation, and self-efficacy are missing and could provide new insights into how SBPs impact other aspects of students’ trajectories. Second, this review does not assess whether the included interventions positively or negatively affect students’ academic outcomes, as we focused on describing the methods used to assess those programs. Third, although we attempted to look for gray literature, we mainly included studies published in journals, conferences, and book chapters and only one doctoral thesis. Nevertheless, there is a possibility of publication bias. Also, the programs included in this paper have relatively short durations and do not include programs where the SBPs are part of an extensive intervention. Finally, although the scope of this paper is not to provide a validated categorization of methodologies, we used common terminology of classification of different study designs in a broader perspective, following biomedical categories: descriptive or analytical (observational or experimental) (Ranganathan & Aggarwal, 2019). Other classifications could be used; for example, Hsu (2005) included in their review 30 different research methods/designs and 34 data analysis procedures.

Implications and Next Steps

Consistently with our review objective, we found that the current body of evidence is predominantly descriptive or observational, with limited control of variables and only two experimental designs. As research implications, these results provide limited evidence supporting a causal interpretation of SBPs’ impacts on academic performance (Loeb et al., 2017; Odeleye & Santiago, 2019; Baron, 2004; Thompson, 2006). This imbalance toward non-experimental designs aligns with previous attempts to systematize the body of evidence (Bradford et al., 2021). Although the aim of scoping reviews is to map the current body of evidence and does not evaluate the quality of the studies or effect of the interventions (such as the meta-review of Bradford et al., 2021), we note that the five studies with the highest methodological rigor, the two experimental designs and three observational designs with matching techniques to balance the participants’ background variables, are quite diverse in terms of implementation and effectiveness. For instance, two programs focused on Engineering design; one reported positive effects on retention and academic performance in courses, especially among women (Jackson et al., 2024), while the other found no impact on GPA (Kornreich-Leshem et al., 2015). Similarly, among two experimental studies on online SBPs, one reported that SBP did not improve math skills (Chingos et al., 2017), whereas the other found positive effects on chemistry performance and persistence (Dockter et al., 2017). The meta-analysis conducted by Bradford et al. (2021) found that program participation had a medium-sized effect on first-year overall grade point average and first-year university retention. This review complements Bradford et al. (2021)’s work by including more studies (37 vs. 16), synthesizing and categorizing approaches, allowing us to map which SBPs have been studied in which ways and how their varied implementations may influence outcomes. Finally, this lack of evidence could lead to erroneous conclusions due to inadequate comparisons (Slavin, 2020; Odeleye & Santiago, 2019; Baron, 2004).

More than twenty selected studies did not control for any variables in their analysis. This research gap highlights the need for more research to support evidence-based decision-making, providing valuable information for HEIs and policymakers. This kind of research should not only consider more rigorous methodology but also explore different outcomes other than graduation or retention rates, GPA and course performance, include more longitudinal measures, and evaluate how different aspects of the implementation impact on the students’ academic achievement, such as duration, online vs. in-person materials, presence of social activities, or voluntariness of the program.

Also, due to the study design, there is a clear imbalance in the subjects considered in the treatment and control groups. On average, there are 10 times as many subjects in the control group as in the treatment group. In this aspect, researchers are encouraged to find subjects to fairly compare the treatment when evaluating these programs. Some studies considered students who applied to the program but were not accepted, thereby reducing selection bias surrounding these programs. Other studies used different techniques to match a student who did not participate with a participant who is similar in terms of background variables.

Finally, the clear dominance of U.S studies reflects the distinctive role of Developmental Education (DE) in the American higher education system (Arendale, 2011; McGee et al., 2021) and the use of placement/selection practices that may not generalize to other educational systems (Conley, 2008; Tempelaar et al., 2011; Büchele et al., 2024). Future research should address how differences in educational institutions and culture affect the delivery of different SBPs and their effects on students’ academic outcomes, using more rigorous methods of analysis, considering local conditions, and changes in the social environment that could make prior evidence irrelevant (Berliner, 2002). Thus, different approaches to research design and analytical choices are needed to provide evidence.

For practice implications, this review revealed a wide variety of SBP implementations, as programs across institutions differ in duration, target population, ultimate goals, and program structure. Thus, HEIs supporting this type of intervention should be aware that the uniqueness of each program makes it impossible to conduct a one-size-fits-all program evaluation. Therefore, institutions that wish to evaluate their programs should consider how similar programs have been proven effective in achieving their goals and aspirations. This review provides an initial list of programs that have been academically evaluated, along with their corresponding methodological approaches. For this review, the only two studies that used an experimental design were fully online (Dockter et al., 2017; Chingos et al., 2017). Thus, this review highlights the limitations of the evidence to guide policy and practice. Second, HEIs that wish to evaluate their programs should be aware that, although the true experimental design is the “gold standard” for providing high-quality evidence, it is not always possible due to ethical considerations, especially when the program targets historically excluded groups. HEIs must consider that, to understand why SBPs impact students’ academic achievement, both causal and descriptive analyses are needed. Although experimental research usually has higher costs, these costs need to be funded by institutions (Gopalan et al., 2020). Finally, due to the inclusion criteria of this review, we did not consider interventions that took place beyond the summer preceding the start of class, or if they did, they did not isolate the effect of the SBP on students’ academic performance. To do so, more complex techniques are needed to attribute the real impact of the intervention to students’ academic performance and, therefore, evaluate if a program is effective in reaching its goals.

Finally, the evidence gathered in this review has limitations for guiding policy. Many SBPs rely on federal or governmental support. This contrasts with the goal of different initiatives that have supported and prioritized the development of evaluation and evidence-informed policy and practice, such as the Institute of Education Sciences (IES) in the USA, the Education Endowment Foundation (EEF) in England (Slavin, 2020) and the European Commission (EC) (Pellegrini & Vivanet, 2021), which can help HEI be held accountable to different actors (Macheridis & Paulsson, 2021). As Liu and Kringos (2025) state, creating clear and accessible pathways from secondary to higher education, such as many SBPs reported here, is an area where governments can make a significant impact.