Improving the Success of First Term General Chemistry Students at a Liberal Arts Institution

Abstract

General Chemistry is a high impact course at Benedictine University where a large enrollment of ~250 students each year, coupled with low pass rates of a particularly vulnerable student population from a retention point of view (i.e., first-year college students), make it a strategic course on which to focus innovative pedagogical development. Although our institution is not alone in the challenges that this particular course presents, we have prioritized implementing interventional strategies targeting academically underprepared students to increase their success by providing a preparatory course prior to this gateway course. Focusing on the persistence framework to guide curricular decisions, progress towards aligning our general chemistry curriculum to the academic profile of our students has afforded much higher pass rates than even two years ago.

1. Introduction

Benedictine University is a small liberal arts college in the western suburbs of Chicago, IL that has ~2000 undergraduate students across four colleges. The department of chemistry resides in the College of Science, and ~40% of undergraduates enrolled at the University declare majors housed in this College. At institutions nationwide, general chemistry is known to be a barrier to success among many first-year students [1,2]. This is also true at Benedictine, where the general chemistry sequence courses are high impact courses within the College of Science with ~250 students enrolled each year. One natural consequence of low success rates in a high impact gateway course like general chemistry is that it affects retention of students in science, technology, engineering, and math (STEM) disciplines during the vulnerable transition from freshman to sophomore academic years [3]. A correlation between failure in introductory courses and university attrition is not unusual for institutions [4]. In fact, according to a 2012 report by the President’s Council of Advisors on Science and Technology (PCAST), less than half of students intending to major in a STEM discipline graduate with their intended major [5]. In particular, first generation college students, underrepresented minority groups, and students with substantial family or work responsibilities and commitments outside of school are especially vulnerable to leaving STEM fields of study prior to graduation [3,6]. Benedictine University serves a diverse student population with 19% self-identifying as members of minorities underrepresented in the sciences, 23% as first-generation college students, and 36% of incoming freshmen in the sciences are Pell Grant eligible along with 81% of our students commuting to campus. Therefore, in an effort to address low retention rates in the sciences, first semester general chemistry is a natural target for interventional strategies and innovative pedagogical developments [6]. In this study, we report our efforts over the past several years to improve academic outcomes for students in the general chemistry sequence at Benedictine. These efforts involved determining a mechanism to identify STEM students who lack adequate preparation to pass general chemistry and developing a Preparatory General Chemistry course to help these students build the skills needed to succeed. In addition, the preparatory course was designed with a commitment to incorporate best practices in STEM education, such as active learning in the classroom. Although we did not change the content in our General Chemistry I and II courses, as our course aligns with the American Chemical Society standards for content, these more engaging active learning practices were integrated to improve our traditional general chemistry sequence [7,8,9]. The results of these efforts are reported here.

A single reliable method to identify students who will struggle in general chemistry remains elusive in large part due to the variety of factors that can influence student learning, many of which, as mentioned above, are nonacademic [10,11,12]. Many strategies have been employed to identify underprepared students in order to provide them adequate preparation prior to entering rigorous STEM major course work. Chemistry placement exams [13], Math course placement [14], ACT/SAT college entrance exam scores [13,14,15], high school grade point average (GPA) [13], and other types of identifiers [10] have been utilized to gain insight into academic preparation and make predictions about future success in general chemistry. The Toledo Placement exam is a nationally normed chemistry placement exam offered by the American Chemical Society Exams Institute that has been shown to be an effective mechanism for identifying academic preparation predictive of success in first semester general chemistry coursework [13]. Although this exam is effective at finding underprepared students, there are barriers to it being widely adopted as a placement exam. Most significantly, it is a pencil and paper exam that requires the use of adequate proctoring to have consistent results. Depending on the size of an institution, this can be a large administrative burden to put in place, and instating a required on-campus exam prior to matriculation could be seen as an undue barrier to sensitive incoming student populations, which could potentially affect enrollment.

Although it was determined that the Toledo Placement Exam would not meet our long-term need for a method to differentiate between incoming students who are prepared and underprepared for general chemistry, we still felt it would be a useful instrument to gather important insights about our current students. With this in mind, the Toledo Placement Exam was administered to General Chemistry I students during the first week of class during the spring 2015 and fall 2016 semesters, and their scores and final course grades were tracked. The results from the 228 participating students indicate that the student population and course expectations at our institution align with national norms where students placing in the 50th percentile or above have an 80% chance of success in their first term general chemistry course [13]. While this exam is an adequate predictive metric for success, the burdens stated above led us to use these results to identify a less cumbersome predictor of success. In this study, it will be shown that there exists a correlation between success in general chemistry at our institution and the Math ACT scores of our students.

Math ACT scores were then used as a tool to target academically underprepared students to provide them the support needed to facilitate success in general chemistry. The support system that was implemented at Benedictine University has two main objectives. The first was to design and offer a semester-long, on-site Preparatory General Chemistry course for underprepared students to take before enrolling in General Chemistry I. This course was piloted in the fall of 2016 (vide infra, pg. 8). The second involved using STEM education research to inform the redesign of the whole general chemistry curriculum at Benedictine, including the new Preparatory General Chemistry and existing General Chemistry I and II courses. In particular, these efforts were modeled on the persistence framework, which is an established guide to best practices to increase retention of undergraduate STEM majors [3]. The framework identifies three interventions that increase student learning and professional identification as scientists: early research experiences [16], active learning in the classroom [6,17,18], and enrollment in STEM learning communities [4,19,20]. At our university, we are in the beginning stages of putting the framework into action. Efforts to facilitate research in foundational chemistry labs will be discussed elsewhere, but establishing learning communities by strategically linking key courses and transitioning to learner-centered classrooms have been pivotal changes that have allowed our curricular redesigns to be successful. One example of faculty commitment to incorporate active learning in introductory science courses is the establishment of a learning assistant program. Our learning assistants engage with students inside and outside the classroom creating a structured peer-to-peer teaching and learning mentorship.

2. Materials and Methods

2.1. Data Extraction

All the historical data was extracted from a central university database where grade information is stored. To keep the data anonymous, the data was pulled from the database in aggregate with only student ID numbers included as possible identifiers. The information that was extracted from the database was student ID numbers, Math ACT scores, course grades, and term. For the graduation rate analysis, only fall semesters in 2010, 2011, and 2012 were analyzed using student ID’s, major, and graduation date. Numerical course grades were obtained from the instructors who taught the course with student ID’s as the only possible identifiers in order to maintain confidentiality of the information. No identifiers are used in this report and all results are reported in aggregate.

2.2. Statistical Analysis

Students who withdrew from or were repeating the 1st semester of general chemistry were removed from Math ACT and Preparatory General Chemistry analyses. Pass rate analysis included all students taking the course in the specified semester. Any student who completed the course was included in hypothesis testing. A 2-sample t test was utilized to test the null hypothesis that the preparatory chemistry group average was less than or equal to the traditional group. The variance between the groups was assumed to be unequal. A confidence interval of greater than 95% was required to reject the null hypothesis.

3. Results and Discussion

The chemistry department was motivated to examine both our evolving student academic profile and general chemistry curriculum in light of continued decreasing pass rates in first semester general chemistry (Figure 1). Students earn grades of A, B, C, D, or F that are based on a common grading scale in general chemistry while a W represents a student who withdrew from the course by the end of the 12th week of a 15-week semester. The pass rate is defined as the percentage of students earning an A, B, or C since a prerequisite for continuation in the course sequence is a “C” or better in the first semester course. In the span of only six years (fall 2010 to fall 2015), the pass rates decreased from 69.6% to 55.6% even though our curriculum was largely unchanged. This observation that, while the general chemistry curriculum remained the same from year to year, the pass rates of our freshmen were declining led us to reexamine how we can better serve a student body whose academic profile appears to have changed.

3.1. The First Semester General Chemistry Curriculum

General chemistry courses at Benedictine University are multi-section, multi-instructor courses with a common syllabus and common final exam designed to maintain consistency between sections. A statistical analysis showed no statistically significant difference between sections of general chemistry ensuring consistency of the analyzed data (data not shown).

The general chemistry curriculum at Benedictine University is a two-semester course sequence, consisting of General Chemistry I and II (CHEM 113 and 123, respectively), that is required for all students in the College of Science except for Mathematics and Computer Information Science majors (~95% of students) making it one of the most impactful courses offered at the university. General Chemistry I and II each have an associated one credit hour laboratory course that meets three hours a week; the lab courses are separate from the lecture courses, receive a separate grade, and are not a part of this study. Prior to our curricular revisions described in this report, the prerequisites to enroll in General Chemistry I required students to have taken high school chemistry or our introductory chemistry survey course and to have placed into trigonometry or higher level math course, thereby demonstrating competency in college algebra. First semester General Chemistry I is a three credit hour course, and the curriculum is based on a traditional sequence of content beginning with atomic structure, stoichiometry, and reactions of aqueous solutions, followed by thermodynamics, and concluding with electronic structure, periodic trends, and bonding models. This content corresponds to ten chapters from a common general chemistry textbook. Until recently, the mode of content delivery has primarily consisted of didactic and passive lecture-based classroom experiences. In the spring of 2015, the chemistry department began taking deliberate steps to include more active learning in the general chemistry classroom by, for example, incorporating think-pair-share opportunities and scaffolded problem sets in class [21,22]. Additionally, utilizing learning assistants in the classroom has enhanced these efforts, which has provided additional mentorship and resources to students [23]. The course content for General Chemistry I and II remained unchanged during this time, but we hoped that providing a more engaging classroom experience would improve student outcomes.

3.2. Higher Failure Rates can Be Correlated to Higher Attrition Rates

The freshman to sophomore year transition is the most vulnerable transition in higher education from a retention viewpoint [1,3]. The general chemistry course sequence is one of the mutual experiences of traditional freshmen in the College of Science at Benedictine University, and low pass rates can lead to attrition of at-risk students. The graduation rates of students enrolled in General Chemistry I over three years (fall semesters of 2010, 2011, and 2012) were analyzed by students’ earned grade in the course. Shown in Figure 2 are graduation rates as of spring 2017 (N = 781) for each possible grade earned for first semester general chemistry, which is mainly populated by first semester freshmen. Taken in aggregate, students who earn an A, B, or C (passing) are twice as likely to graduate from the university than those who earn a D, F, or W, which translates to failure of the course leading to repeating the course in order to complete the general chemistry sequence. The combined graduation rate for students who pass first semester general chemistry is 73% while the graduation rate is 43% for those that do not pass. An alternative path towards graduation could involve a student changing their field of study. After an initial failure in a gateway science course, a student may become successful in a major that is in more aligned with their talents, and that is considered a positive outcome. However, our institutional analysis indicates that this outcome is rare. Only 14% of students over the three-year time span who were unsuccessful in general chemistry changed their majors to areas outside of the sciences and eventually graduated from the university.

3.3. Identifying Mathematically Underprepared Students

There are many methods that have been used to try to identify academically underprepared students [10,13,15]. Because of the quantitative nature of general chemistry, it has been shown that math preparedness plays a large role in success in this course [14]. Strategies to determine which students may be mathematically underprepared for general chemistry have been based on high school GPA, standardized college admission tests such as ACT and SAT exams, and chemistry placement exams that are usually administered online [10,13,15,24]. In order to better understand our science student population’s readiness for general chemistry, the Toledo placement exam, which is an American Chemical Society Exams Institute exam for placement of students into general chemistry, was administered to our General Chemistry I students in the first week of class during spring 2015 and fall 2016 semesters. The exam is a 55 min test comprised of three 20-question sections that cover arithmetic and algebra, general chemical knowledge, and specific chemical knowledge. A score at or above the fiftieth percentile on this placement exam translates to an 80% success rate in first semester of general chemistry [13]. According to our results, our students’ exam scores and course grades aligned with these national norms providing evidence that our student population is performing similarly to other undergraduates on the national level.

Early on, the feasibility of offering a properly proctored pencil and paper exam was examined and found to be an undue administrative burden to academic advisors, chemistry faculty, and students. It was therefore imperative to identify academically underprepared students without a placement exam in order to address their deficits. Since the chemical education community has discussed the importance of math fluency for success in general chemistry, we decided to investigate if Math ACT scores are a viable measure of math preparedness and predictive of success in general chemistry. The ACT exam is a standardized test that measures college readiness. ACT scores, along with other metrics such as high school GPA, are utilized by college admission offices to make undergraduate admission decisions and award scholarships [25]. The ACT exam is accepted at all four-year colleges and universities in the United States, and all high school students in Illinois take the exam, so the overwhelming majority of Benedictine University students already report their ACT scores to the university. The Math ACT can also provide information of math proficiency to certain levels [25].

Since our students were historically performing similarly to national norms on the Toledo placement exam, it is reasonable to expect that if there is a similar correlation between Math ACT scores and pass rates of first semester general chemistry that a simple analysis of these scores would determine if an undergraduate student interested in a science major is ready for such a rigorous course. Historical Math ACT scores of four years of students (data from students enrolled in General Chemistry I between 2012 and 2015, N = 914) and final grades were analyzed, grouped by individual Math ACT scores, and pass rates were determined at each level. Shown in Figure 3 is the correlation between pass rates in first semester general chemistry and Math ACT scores (R² = 0.9538). The high correlation suggests that: (a) math preparedness is a significant determinant in success of a student in general chemistry and (b) utilizing this score alone, rather than a proctored chemistry placement exam, could serve as an entrance requirement to first semester general chemistry. While we are aware of other institutions that use Math ACT scores as a metric for placement in general chemistry, we were pleased to find such a strong correlation between Math ACT scores and historical course performance for our students. Therefore, if other institutions want to use Math ACT scores as a metric for placement at their own institution, it will be important for these institutions to first establish their own historical academic profile. Other colleges and universities might consider utilizing these scores since they are often already collected from students during the admissions process.

Based on our analysis of our students’ academic preparedness, we used a Math ACT score of 23 or above, indicating an 74% predicted success rate in General Chemistry I, as a prerequisite to enroll in General Chemistry I beginning in the fall 2017 semester. Once Math ACT scores are used to identify a subset of underprepared STEM students, there must be a mechanism in place by which these students can become ready to take General Chemistry I. Commercial online prep courses, online tutorials [24], participation in summer bridge programs [26], and preparatory chemistry courses [24,27,28,29] are some examples of what other institutions have put in place to increase success of their academically at-risk students. Students choose to matriculate at a small liberal arts institution like Benedictine University because of personalized interactions with faculty. Therefore, an on-site preparatory general chemistry course taught by a full-time faculty member or science learning specialist, rather than other methods of preparation, such as an online tutorial, was determined to best fit our institutional mission, size, and the academic profile of our students. In developing a preparatory course, we agreed that placing students into our existing introductory chemistry survey course, which is populated by nutrition and exercise science majors, would not serve the intended function since this course is intended to provide only broad chemistry content [30]. Some institutions experiencing similar pass rates as our institution have simply placed at-risk students into introductory chemistry [1], but the purpose of an introductory survey course does not align with the goal of a preparatory course which is to prepare students for the mathematical rigor of a general chemistry course.

3.4. Learner-Centered Pedagogies

In the spring of 2015, motivated by declining pass rates in General Chemistry I, the chemistry department at Benedictine University began to reexamine the general chemistry sequence. Guided by best practices described in the STEM education literature, the general chemistry curriculum delivery has transitioned from a passive lecture model into a more student-centered learning environment. The persistence framework outlines three established interventional strategies to improve retention among science scholars including active learning and engagement in STEM learning communities [3]. In light of this framework and the academic profile of our students, several curricular changes were implemented. A new Preparatory General Chemistry course was designed with the goal of preparing vulnerable students for success in general chemistry and mathematically rigorous disciplines. Additionally, the new preparatory course, as well as General Chemistry I and II, all began to incorporate more active learning into the classroom. The specific active learning strategies employed vary between courses and instructors depending on individual teaching styles, but, in general, more class time was dedicated to students working through problems and concepts individually and in groups. Some examples of activities that were incorporated into the classroom include think-pair-share questions and scaffolded problem sets. Having learning assistants assigned to these courses has been vital in facilitating more student engagement both in an out of the classroom.

Learning Assistant Program

The transformation of the general chemistry curriculum at Benedictine University has been enhanced by the participation of learning assistants in both Preparatory General Chemistry and General Chemistry I and II. Based on the model developed at the University of Colorado, learning assistants (LAs) are undergraduate students who work closely with a specific class or course to facilitate collaborative, small-group activities in the classroom and supplementary problem-solving recitations [23]. LAs enroll in a weekly pedagogy seminar and work alongside faculty members to develop active-learning assignments that help students engage and develop strategies to take responsibility for their own learning. The Learning Assistant Program was initiated within the College of Science at Benedictine University during the fall of 2014 with funding through a Robert Noyce Teacher Scholarship Program grant (DUE-1240091), and in the subsequent years, the program has grown exponentially to offer pedagogical assistance in courses across the STEM disciplines. Currently, the program utilizes 29 LAs, working with 19 faculty members, to augment the educational experiences of students across 13 courses. Learning assistants commit to 7–10 hr of work per week and receive a stipend for their efforts. Students generally find the LAs very approachable, and LAs often can provide valuable feedback about students’ experiences to faculty members.

In the fall of 2016, the Preparatory General Chemistry pilot class of 33 students was paired with a learning assistant. This LA was a high-performing student with a self-declared motivation to help students learn and an inherent rapport with students. The primary role of the LA in the Preparatory General Chemistry class was to facilitate group work and problem solving. Another benefit of incorporating a learning assistants into our curriculum overhaul is that they allow us to provide additional continuity for the at-risk Preparatory General Chemistry students. The LA for this course followed the students into first semester general chemistry and served as a learning assistant for that class in the spring of 2017.

3.5. Preparatory General Chemistry

Preparatory General Chemistry (CHEM 108) is designed as a one semester course to be taken by underprepared STEM students prior to taking General Chemistry I and II. It is a three credit hour course with no associated laboratory, and it also fulfills a physical science general education requirement. The purpose of the Preparatory General Chemistry course is to prepare students for success in general chemistry by developing academic study skills, mathematical reasoning, and problem-solving skills, as well as teaching selected chemical concepts and fostering on-campus connections, which can support and enrich their undergraduate careers [31]. The course was designed with the persistence framework in mind and utilizes active learning and learning communities to foster success in its target at-risk student population. For example, the class is taught as a mixture of lecture and active learning carried out with the help of LAs, and the course is paired with an Organismal Biology class as a learning community. Additionally, students who do not yet meet the math prerequisites for General Chemistry I can enroll in Preparatory General Chemistry, which allows these vulnerable students to take a chemistry class during their first semester. This course was first offered at Benedictine University during the fall of 2016 as a pilot.

3.5.1. Preparatory General Chemistry Course Design and Pilot

Preparatory General Chemistry is linked to a section of Organismal Biology as a formal learning community. At Benedictine University, a learning community consists of two classes, often from different disciplines, that have many of the same students in them, and the two courses have common assignments and activities. Because both Preparatory General Chemistry and Organismal Biology tend to enroll first-semester STEM students, many of the common assignments focused on helping students become aware of campus resources and fostering involvement with the campus community. For example, students in the learning community are assigned to go to eight on-campus activities, such as tutoring, study skills workshops, club meetings, speakers, music concerts, and sporting events during the semester. An additional benefit of the learning community is that it cultivates connections between students. Since students are in at least two classes with many of the same class mates, they begin to develop friendships and form study groups.

Upper level, high-performing students serve as LAs to the Preparatory General Chemistry class. The LAs attend classes and increase the level of student engagement in the classroom. LAs assist the instructor and help to answer student questions during the active learning activities. As experienced students, the LA’s serve as mentors, and they can advise students and provide examples of how to improve study skills and achieve academic goals. The LAs also lead help sessions held outside of class time, which are an additional resource that gives students the opportunity to ask questions about lecture materials and get help with homework questions. The fall 2016 pilot class of Preparatory General Chemistry had one LA who then followed the students to General Chemistry I in the spring of 2017.

During the fall 2016 semester, Preparatory General Chemistry was run as a pilot class at Benedictine University. The inaugural class was populated with 33 students who were advised to take the class because they met at least one of the following criteria: (1) they either had not taken high school chemistry or self-identified as being weak in chemistry; (2) they did not yet meet the General Chemistry I math prerequisite of College Algebra; or (3) they opted to enroll in the course after performing poorly on the Toledo Placement Exam that was administered during the first week of General Chemistry I. In addition to these academic weaknesses, many of the students also had other risk-factors that made them a vulnerable population for STEM retention. The students in the pilot cohort were twice as likely as other students in the College of Science to be first generation college students (50% vs. 22.5%) and 25% of them work more than 10 hr per week. The 33 students who took the class during the fall of 2016 earned the following grades: 22 students got an A or B, five students got a C, and six got a D, F, or W. This corresponds to a pass rate of 82% for the pilot of Preparatory General Chemistry, and all students who earned an A, B, C, or D completed the physical science general education requirement towards graduation even if they decide to change to a non-STEM major.

3.5.2. Preparatory General Chemistry Course Content

Despite the name, Preparatory General Chemistry’s primary emphasis is not to cover a significant amount of chemistry content or preview all of the topics that will be covered in General Chemistry I. The course focuses on two skills that have been determined to lead to success in course work: student academic skills and quantitative reasoning [32]. In fact, quantitative reasoning has been determined to be crucial to a twenty-first century education such that the Association of American Colleges and Universities has named Quantitative Literacy as one of its essential learning outcomes [33]. Academic skills are introduced through in-class presentations, activities, and homework assignments. Skills covered include understanding syllabi, time management, note taking, and textbook navigation. Quantitative reasoning skills are also a priority because students enrolled in Preparatory General Chemistry typically lack adequate preparation in numeracy and many students self-declared their struggles with math concepts. To address these deficits, the first few weeks of the course review the basic math skills needed in general chemistry including multiplication and division of fractions, graphing, conversion between fractions and decimal numbers, negative numbers in addition, subtraction, multiplication, and division, order of operations, mental math for numbers in scientific notation, and algebraic skills such as solving equations for a variable and solving equations. The chemistry content that is presented in Preparatory General Chemistry is included either because they are topics that a well-prepared student should be familiar with from high school chemistry (e.g., balancing chemical equations, writing electron configurations), or they are topics that frequently challenge General Chemistry students due to the level of problem solving or math involved (e.g., unit conversions, reaction stoichiometry).

3.5.3. Effectiveness of Preparatory General Chemistry in Improving Performance in 1st Semester General Chemistry

The primary goal of Preparatory General Chemistry is to prepare students who are identified as unlikely to succeed in general chemistry based on their mathematical abilities. Knowing the historical academic profile of our students was key to determining how effective the preparatory course was at this central task. In designing the course, we focused on developing students’ academic, mathematical reasoning, and analytical skills hypothesizing that these skills are critical in STEM courses. The important question to address is: will taking the additional Preparatory General Chemistry course help academically at-risk students succeed in their future science courses, in particular, their first semester general chemistry course, which was the target of this study. The Preparatory General Chemistry course’s effectiveness was assessed by analyzing the academic outcomes of the students who participated in the fall 2016 pilot of Preparatory General Chemistry. Of the 33 students who took Preparatory General Chemistry during the fall of 2016, 27 students passed, and 22 of those went on to enroll in General Chemistry I during the spring of 2017 (referred to as the pilot cohort). The effect of the preparatory course on the pilot cohort was determined by comparing their performance in General Chemistry I to that of students who had not taken the preparatory course. The students enrolled in first semester general chemistry without taking Preparatory General Chemistry are referred to as the traditional group.

Figure 4 shows the relationship between average numerical grade in General Chemistry I and Math ACT scores for both the Preparatory General Chemistry pilot cohort and the traditional group. The results for the traditional group, which are based on a subset of historic data from fall 2013–fall 2016, are not surprising given that pass rates and Math ACT scores show a similar near linear correlation (Figure 3), and the analysis here represents only a closer examination of whether Math ACT scores also align with a numerical final grade. More interestingly, it is important to note the relatively flat relationship between Math ACT scores and General Chemistry I grades for the pilot cohort, which indicates a reduced influence of their math skills prior to enrolling in undergraduate courses on their performance in first semester general chemistry. This provides evidence of the effectiveness of Preparatory General Chemistry to overcome previous deficits and that students in the pilot cohort retained and applied academic skills developed in this course.

Table 1. Overall comparison of the traditional group to the preparatory general chemistry with Math ACT scores < 26. The table highlights the difference in average scores, statistical significance of the difference and effect size.

Traditional Group (N) 1	Preparatory Chem Group (N) 1	Average Grade, Traditional	Average Grade, Preparatory Chem Group	Difference	p-Value of Difference	Effect Size	Class Std Dev
244	22	0.646	0.708	0.062	<0.001	0.47	0.13084

Note: ¹ Preparatory General Chemistry group limited to students with Math ACT < 26 therefore only these Math ACT scores were analyzed.

shows the average numerical final scores of the traditional (N = 244) and Preparatory General Chemistry pilot cohort (N = 22) and the difference in their average numerical final course scores. The statistically significant difference in the scores of the traditional and preparatory general chemistry groups represents a whole letter grade in improvement for students who had completed preparatory general chemistry prior to enrolling in first semester general chemistry, 0.062 (p < 0.001). Effect size was used as a measure of comparison between the two groups.

$$ES=\frac{Avgscoreofprepgenchemgroup-Avgscoreoftraditionalgroup}{standarddeviationofallfirstsemestergenchemenrollees}$$

(1)

An effect size = 0 determines no difference in the comparison groups, an effect size <0 indicates that a negative effect on the comparison groups, an effect size >0 indicates positive effect on the comparison groups. As reported in the literature, effect sizes with absolute values of 0.2 are small, 0.5 indicates a moderate effect, and 0.8 is a large effect. On average the preparatory general chemistry group shows a 0.47 effect size indicating a moderately positive effect on their prior preparation for general chemistry [24,34].

Table 2. Comparison of the traditional and preparatory general chemistry groups aligned with their Math ACT scores as analyzed by the difference in scores and effect size.

Math ACT	Traditional Group (N) 1	Preparatory Chem Group (N)1	Average Grade, Traditional	Average Grade, Preparatory Chem Group	Difference	p-Value of Difference 2	Effect Size	Class Std Dev
18	12	3	0.525	0.698	0.173	0.008	1.15	0.1506
19	14	4	0.634	0.735	0.101	0.016	0.90	0.1118
20	20	2	0.623	0.725	0.102	<0.001	0.82	0.1243
21	19	2	0.644	0.748	0.104	0.067 3	0.88 3	0.1184
22	22	2	0.642	0.707	0.065	0.015	0.52	0.1242
23	39	2	0.673	0.591	−0.082	0.953 3	−0.75 3	0.1098
24	55	3	0.688	0.718	0.030	0.252 3	0.28 3	0.1080
25	45	2	0.663	0.766	0.103	0.011	0.74	0.1386

Note: ¹ Preparatory Chem Group limited to students with Math ACT < 26 therefore only these Math ACT scores were analyzed; ² Null Hypothesis Average Grades Prep Chem Group minus Traditional group ≤ 0; ³ Indicates a value that is not statistically significant.

shows a closer examination of the two groups broken down by individual Math ACT scores as analyzed by the difference in their numerical average final course grades and the effect sizes. The points 21, 23, and 24 were deemed not statistically significant. There is a clear distinction between the preparatory general chemistry and the traditional groups for the lower Math ACT scores: 18, 19, and 20. These showed statistically significant differences (p = 0.008, 0.016, and <0.001, respectively) and large effect sizes (1.15, 0.90, and 0.82, respectively). This trend is clearly shown in Figure 4 showing that the weaker mathematically prepared students performed much above the students in the traditional group in the same Math ACT group. Importantly, these students were unlikely to succeed in their first semester general chemistry course according to our historical academic student profile—20%, 34%, and 45% chance of passing for 18, 19, and 20 Math ACT scores, respectively. Aligning with the results of our study, closing the achievement gap by incorporating problem solving skills and active learning into the curriculum has been shown to disproportionately benefit academically weaker students in introductory STEM courses [6].

4. Conclusions

General Chemistry sequence courses are high impact courses for our university as ~250 students enroll in these courses each year. Although it cannot be proven as a causal effect, performance in first semester general chemistry may be linked to attrition of STEM majors as shown by decreased graduation rates for students who are unsuccessful in this pivotal course. Between 2010 and 2015 we observed a significant decline in General Chemistry I pass rates. To curb this trend, the chemistry department at Benedictine University redesigned the general chemistry curriculum to improve the outcomes and learning experiences for all students with particular efforts focused on helping underprepared students. Key features of this curricular update included determining a metric to correctly identify at-risk students, understanding their academic weaknesses in order to provide a mechanism to prepare these students for rigorous STEM majors, and utilizing learner-centered pedagogies to engage students in these gateway chemistry courses. Specifically, Math ACT scores were found to be a satisfactory indicator of student preparedness for General Chemistry I. A Preparatory General Chemistry course that focuses on developing academic skills, quantitative reasoning, and on-campus connections was designed for underprepared students to complete before enrolling in General Chemistry I, and this course was piloted in the fall of 2016. Additionally, active learning and learning assistants have been incorporated into the new Preparatory General Chemistry and existing General Chemistry I and II courses to facilitate student engagement.

Based on preliminary evaluation of these programmatic updates, we have provided evidence of increased success of students who are academically underprepared for rigorous college science courses by focusing on their first semester general chemistry course, as it is universally known as a gateway course in science. Students with poor math preparedness who went through the Preparatory General Chemistry course as part of the pilot have been more successful in General Chemistry I than predicted by their Math ACT scores. On average, the pilot cohort scored a letter grade higher in General Chemistry I than students with comparable Math ACT score who did not take Preparatory General Chemistry I, which provides evidence for developed and retained academic skills. Based on these positive results from the pilot, we offered Preparatory General Chemistry during the fall 2017 semester to continue to provide a supportive framework for at-risk students, and we will track their General Chemistry I performance, retention, and graduation rates. A surprising finding from our data is that our pass rates in General Chemistry I are 20% higher than two years ago (compare fall 2015 pass rate of 55.6% to fall 2016 pass rate of 75.0%). The origin of the improvement is not known. Over the course of our revisions, the General Chemistry I content remained the same. However, starting in the spring of 2015, the course instructors began a concerted effort to increase student engagement in the classroom. It is possible that the incorporation of more active learning improved student outcomes, but other explanations are certainly possible. For example, in the fall of 2016, some students who met the prerequisites but were unlikely to be successful in General Chemistry I opted to take Preparatory General Chemistry instead. Regardless of the origin, pass rates up to 75% in General Chemistry I mean an additional ~30 students a year pass, and this trend may lead to improved retention for our university.