Getting Past the Gateway: An Exploratory Case on Using Utilitarian Scientific Literacy to Support First-Year Students At-Risk of Leaving STEM

Abstract

First-year students who enter college pursuing a STEM degree still face challenges persisting through the STEM pipeline (Chen, 2013; Leu, 2017). In this case study, researchers examine the impact of a utilitarian scientific literacy based academic intervention on retention of first-year students in STEM using a mixed methods approach. A sample (n = 116) of first-year students identified as at-risk of not persisting in STEM were enrolled in a for credit utilitarian scientific literacy course. Participants of the semester long course were then compared with a control group of first-year students identified as at-risk of persisting in STEM. A two-proportion z test was performed to assess the mean differences between students and participants of the course were given a survey to gauge student experiences. Quantitative results (φ 0.34, p < 0.05) indicate that the utilitarian scientific literacy course had a statistically significant impact on retention among first-year students at-risk of persisting in STEM. Moreover, qualitative data obtained from participant responses describe internal and external growth as positive outcomes associated with the intervention.

1. Introduction

Gateway courses serve as the initial barrier to persistence in the science, technology, engineering, and math (STEM) pipeline among college students [1]. These introductory courses are critical for STEM majors because they serve as the first encounter to coursework related to the major [2]. The students who succeed in gateway courses are likely to continue moving through the STEM pipeline, while students who do poorly become susceptible to changing majors or dropping out of higher education entirely [3]. Students who lack strong academic backgrounds and exhibit low self-efficacy in STEM coursework often leave the STEM pipeline before acquiring a degree or entering the workforce [4,5]. Astin and Astin [6] reported that well-prepared students are more likely than other students both to persist in their initial choice of a science major or career and to be enrolled into STEM majors and pursue a STEM career during and after college. Understanding challenges that students face in introductory courses is requisite to helping students persist in their major [7]. A student’s academic background gives details on the student’s academic preparedness. Students with stronger academic backgrounds are more likely to persist throughout the pipeline to STEM careers, yet most times these students are not African-American or Hispanic [8,9].

The leaky pipeline references student attrition in STEM. The leak in the pipeline to STEM majors and careers is most prominent among minority groups and most frequent at the postsecondary level [10,11,12]. Students either opt out of their majors in STEM or leave postsecondary institutions altogether. Because retention at postsecondary institutions may have many extraneous variables, the greatest concern for researchers are students who opt out of majors in STEM. Results from a 2017 study show that nearly a third (35%) of first-time associate and bachelor degree seeking students opt out of their majors in STEM within three years [13].

The following paragraphs will synthesize literature about persistence throughout the STEM pipeline, STEM-focused academic interventions, and the importance of utilitarian scientific literacy.

1.1. Persisting in STEM

In the United States, less than half of the majority of students who enter into STEM undergraduate curricula as freshmen graduate with a STEM degree, and only one-fourth of minority students who start as STEM majors graduate with a STEM degree [14]. Ethnicity/race, household income, low academic performance, and first-generation college students are not the only predicting characteristics of STEM students who leave STEM majors [3,8]. Students also leave STEM majors because of: (1) interest in another non-STEM field, (2) perceived difficulty of STEM classes, (3) poor learning environment (no team/community of learners), (4) low STEM self-efficacy (not having a scientist’s identity), (5) poor pedagogy (teaching about science and not how to do science), and (6) lack of support systems [15,16]. Development of STEM self-efficacy or scientist identity is a critical variable for the persistence in a STEM major [15,16]. Specifically, the more a student feels that being a scientist is an authentic and important part of who they are and that they have the ability to perform well in that discipline, the more likely they are to perform well, graduate, enroll in graduate school, and become economically productive in a STEM career.

The factors that typically influence performance and persistence in STEM may be difficult to measure among first year students because they are just beginning their academic career. A study conducted by Mitchell [17] revealed four major factors that were statistically significant contributors of underrepresented minority students’ success in STEM disciplines. They included college GPA, academically rigorous curriculum, percent of hours completed, and percent of hours passed; however, freshmen are just beginning to take courses. A major retention issue for first year STEM majors is that college level introductory science courses do not do an adequate job of increasing scientific literacy among STEM majors resulting in decreased interest in STEM due in part to boredom produced by courses focusing on learning terminology [18,19]. Many students become dissatisfied with STEM and the perceived difficulty of memorizing facts leading to a lack of interest in the subject matter [3]. Innovative course work for at-risk first-year STEM majors has been reported to be effective [20], but these general first-year scientific courses are not specifically designed for students to learn all of the concepts deemed necessary for scientific literacy. Science education, in this country, primarily consists of lecturing to students about science and then requiring them to recall facts [21,22]. This approach does not necessarily foster thinking scientifically or building a strong scientific literacy foundation. Scientific literacy happens when students think for themselves as a scientist would [21]. Teaching science in a way that shows the big picture might allow students to learn the concepts of scientific literacy which helps prevent dissatisfaction and fosters success and retention in STEM majors.

STEM students’ success in school may be influenced by the amount of information and the level of assistance they receive in choosing their STEM major. A scientific literacy curriculum may provide increased exposure to opportunities to learn concepts that foster understanding of STEM disciplines, majors, and careers.

1.2. Academic Interventions in STEM

Academic interventions are strategically constructed to educate student participants on skill building and promote skill fluency, or to motivate the application of existing skills in new circumstances and settings [23]. According to Harackiewicz and Priniski [24], the sphere of higher education has experienced an increase of academic interventions developed to improve academic outcomes in student performance. Researchers have proposed academic interventions and implemented programs for STEM students with hopes of finding multiple solutions to the low persistence rates within STEM majors and careers [25]. Current academic interventions in STEM have been developed to support populations at-risk of leaving the STEM pipeline. While some have not experienced success, others have experienced some form of improvement because of their intervention.

Researchers at Middle Tennessee State University describe the challenge of retention in STEM among these populations as a two-front battle. Not only do students need to be attracted to the field of STEM, but also need to be motivated to complete their major. These researchers implemented a motivationally informed intervention that targeted 36 incoming first-year students identified as at-risk of persisting in STEM [26]. The intervention FirstSTEP was designed to assist incoming first-year students on improvements to their math scores. To inform interventions for improving their college success, a student motivational assessment was used along with the Science Motivation Questionnaire. Scores gained in the FirstSTEP program were then compared to participants in a 2011 study by Glynn, Brickman, Armstrong, and Taasoobshirazi [27]. When compared to the 2011 study, scores for participants in FirstSTEP program were stronger in all levels of motivation albeit self-determination. The strength of the program was that students had high grade and career motivation; however, the researchers noted that there was not enough intrinsic motivation, self-efficacy, and self-determination to ensure their success as a STEM major.

In another study [14], an academic intervention that focused on the effects of mentoring students in STEM was assessed. Researchers at the University of Louisiana developed a program to address STEM pipeline leakage. This program was developed By incorporating coaching and vital scholastic mediations into an organized research program, the creative model has been created to guide STEM undergrad studies receiving the techniques that enable them to exceed expectations in their projects of study, as they figure out how to acknowledge and comprehend science more effectively. Researchers utilized a mentoring ladder to strategically support learning styles and learning strategies among students [14]. Majority of participants in this study were first-year and sophomore students. Mentoring was crucial to the program design because literature indicated that undergraduates with mentors often persist in STEM as well as gain higher GPAs when compared to peers without mentors [28]. Consistent with the literature, findings from this study support the critical role of mentoring in student success throughout the STEM pipeline.

1.3. Utilitarian Scientific Literacy

Scientific Literacy is a necessary skill in this age of digital literacy. Basic science processing skills include observing, classifying, measuring and using numbers, making inferences, predicting, communicating, and using the relations of space and time [29]. Educating students on scientific literacy fosters the growth of confidence and comfortability in understanding basic science processing skills.

Scientific literacy is considered a moldable term due to the various definitions for specific populations [30]. By definition, it refers to the knowledge and understanding of scientific concepts and processes required for personal decision-making, participation in civic and cultural affairs, and economic productivity [31]. Definitions that relate to the overall population can be found in functional and civic scientific literacy, which depict one’s comprehension of science vocabulary and fundamental scientific principles [32,33].

The form of scientific literacy that has been incorporated in this academic intervention is utilitarian scientific literacy, designed specifically for first-year STEM majors [34,35]. The concept of utilitarian scientific literacy is characterized by becoming literate as to what it takes to become a successful STEM major, as well as possessing the growth mindset identity of a successful student in STEM.

An extremely unique and original scientific literacy curriculum was developed for freshman STEM majors. The curriculum of the scientific literacy course was designed to address the factors necessary for the retention of freshman STEM majors by establishing student support systems, promoting interest in STEM, fostering a community of learners, improving STEM self-efficacy, and teaching students how to do science (rather than simply teaching students about science). The aim of this exploratory case study is to richly describe an institution-level academic intervention targeting first-year undergraduate STEM majors. This case study utilizes an exploratory mixed methods approach to examine the impact of a utilitarian scientific literacy based academic intervention on retention of first-year students in STEM. To achieve this, we analyze qualitative and quantitative data associated with a utilitarian scientific literacy academic intervention for first-year STEM majors [36].

2. Materials and Methods

2.1. The Methods

2.1.1. Development of the Intervention

In 2007, a team of faculty and staff at a small, minority-serving college in the southeast sought to address the rising STEM dropout rates amongst male STEM majors at the institution. Before leaving high school, black students lag behind their white and Asian counterparts when examining access to a full range of math and science courses that are needed for college success [37]. The utilitarian scientific literacy course was developed to address this problem at the minority-serving institution.

The initial goal of the utilitarian scientific literacy course was designed with the overarching purpose of increasing the number of students who receive baccalaureate degrees in STEM. To achieve this, the team of researchers, faculty, and staff focused on retaining STEM majors beyond the first year of college. The course curriculum and programmatic changes were established to provide these students with skills that build academic resilience.

2.1.2. Course Curriculum

The utilitarian scientific literacy course was designed with ten chapters that introduced students to topics around scientific literacy and STEM careers. The course had three focus areas: Quantitative literacy, personal future building, and academic skills. The course objectives include; developing quantitative literacy skills, understanding the research discovery process, improving note and test taking skills, and introducing students to a plethora of STEM careers and opportunities. The intended course outcomes for the students included being able to demonstrate a more complex understanding of contemporary science and acquire the necessary skills to perform science at an advanced level with an interest in pursuing a STEM career. A key component to the scientific literacy course was the upper level student leaders who were there to support and challenge the students because mentorship has been known to be an effective part of student success in STEM [37,38]. Overall, the course was designed to deliver new information to students that will build upon their scientific literacy and retention in STEM.

Each chapter within the course was specifically developed to provide students with the foundational skills that will build upon their scientific literacy. The ten chapters throughout the curriculum were centered around the three focus areas mentioned previously. The chapters were designed to be taught independently and cohesively as a complete unit. Some of the chapters include; Scientific Literacy and College Retention, How the Discovery Process Works, and Careers in STEM. Each chapter included a video that was designed to teach students the material, activities to support student development, and assessments to gauge student learning. The scientific literacy course was accepted at the institution as a 1 hour for credit course that meets two times a week throughout the semester. The structure of the course was designed to replicate a natural science lab setting so that students can work in teams with faculty to conduct research simulation case studies. Approximately 60% of the course deals with attitudinal and behavioral domains such as science identity, commitment, and confidence. The other 40% of the course involves developing scientific reasoning and critical thinking skills.

The objective of the course was to provide a solution to the STEM retention problem at the institution. During the first three years of the program there were several components that were not continued through the latter years. From 2009–2011 the course was comprised of additional components including; Guaranteed 4.0 and upper-level student mentorship. The Guaranteed 4.0 program was partnered with the course to help students learn different study skills that will support their growth and development as a student. The program included a workbook and a video to guide students through the program. Another component of the course were the upper-level STEM students who were given the title “student leaders.” These students were responsible for supporting students and modeling for them best practices when persisting as a STEM major.

Regression analysis was conducted to identify academically at-risk incoming freshmen science, technology, engineering, and mathematics majors who had a 75% probability of not passing the first introductory STEM courses. Students enrolled in the course represented the treatment group, students not enrolled represented the control group.

A sample of (n = 116) African American male first-year STEM students were enrolled in the utilitarian scientific literacy academic intervention. Students were sought from STEM disciplines at a minority-serving institution in the southeast. To identify participants, SAT scores of incoming students were compared with current students using a regression analysis. The success index equation and a logistic regression analysis was used to predict the probability of incoming first-year students being at-risk for failing/low-performances in gate keeper STEM courses. More specifically, the regression analysis compared the SAT scores of current freshmen who had not persisted in STEM to their following academic year with the SAT scores of the incoming freshmen. Incoming freshmen that shared similar scores with students who had not persisted in STEM were then prioritized and selected to be participants of this study. The study tracked students from the first semester of their first year to sophomore year through four year and six year graduation rates from 2009 to 2015 with there being no students tracked in 2014.

2.2. Data Collection

2.2.1. Quantitative Data

The number of participants in each control and treatment group was varied each year in response to the fluctuating number of enrolled students identified as at risk of not persisting in their STEM major. Students were tracked based on their classification and major. The study followed multiple cohorts of freshman students from the first to second year and through four and six year graduation rates. Data on participants was collected from 2009 to 2015 with there being no student data recorded in 2014.

Table 1. Instruction and cohort breakdown.

Delivery Type.	Fall Entry
Face-to-Face	2009
Face-to-Face	2010
Face-to-Face	2011
Flipped Classroom	2012
Flipped Classroom	2013
Online	2015
(with some face-to-face)

provides an illustration of how the course changed from 2009 to 2015.

2.2.2. Qualitative Data

Participants answered four open-ended questions around program evaluation. These responses were used as the qualitative data source for the study. The survey responses were open coded through inductive analysis to determine the impact of the scientific literacy course on student retention through the STEM pipeline [14,39]. Researchers then analyzed the data to record consistent themes that emerged for the student’s survey assessments. Two levels of codes appeared within the data around the four themes of the questions. Through a qualitative analysis of student responses two level codes emerged and researchers came to 100% agreement for all codes.

2.2.3. Research Subjectivity

When conducting research qualitative researchers must remain aware of personal biases which could sway their interpretation of the data [40]. To ensure interrater reliability three researchers open coded the data for consistent themes that emerged within the responses [41]. At the completion of the coding process, the researchers reconciled, and unanimously agreed on both levels of codes. This form of analysis was utilized as a means to conduct unbiased research.

3. Results

3.1. Quantitative

A two-proportion z test was performed to assess the mean differences between students enrolled in the utilitarian scientific literacy course and those who were not enrolled. Students exposed to the utilitarian scientific literacy course in a face-to-face setting had a statistically significant increase in retention, (φ 0.34, p < 0.05) when compared to students who were not enrolled in the course (

Table 2. First semester at-risk student retention.

Delivery Type	Entry Year	Treatment	Control	1st to 2nd Year Retention Rate:	1st to 2nd Year Retention Rate:
				Treatment (%)	Control
					(%)
Face-to-Face	2009	39	39	74.4% *	48.70%
Face-to-Face	2010	16	16	81.3% *	37.50%
Face-to-Face	2011	13	13	69.20%	38.50%
Flipped Classroom	2012	22	22	54.50%	36.40%
Flipped Classroom	2013	14	14	57.10%	35.70%
Online (w/some face-to-face)	2015	12	12	50.00%	25.00%

* p < 0.05.

). Retention at the institution of the control group increased by only 10% while the treatment group had a 27.4% increase. The treatment cohorts in the first three years (2009 to 2011) had very high retention rates compared to the treatment cohorts in the last three years (2012–2013, 2015).

3.2. Qualitative

At the conclusion of the semester, each male freshman student who participated in the course was given questions to respond to surrounding the scientific literacy course. Six first level codes were identified within the responses and then coded into second level codes. The first level codes included: Inner academic growth, external academic growth, course comments, academic preparation, careers in STEM, and engagement. Each of these codes were then placed within four themes (e.g., strengths and benefits, recommendation, modifications, and weaknesses and challenges) that were identified within the assessment questions.

3.2.1. Theme 1: Strengths and Benefits

Student responses revealed they experienced inner and external academic growth that contributed to their success within the course. One student commented “I am a better student and drive myself harder” while another student stated, “This class helped me understand science better, as a whole, and encouraged and drew me into wanting to do something”. An increase in confidence as a scientist was alluded to by the comment that “The class has helped me to think like a scientist”. Additionally, students made mention of outer academic growth stating that “The class gave a great overview of what scientists do. I now know how to study better” and “I feel that I have a better understanding of how real scientist’s problem solve. This allows me to have an upper hand on students who just jump into Biology. With practice, I’ll be able to follow along in research presentations and provide more scientific questions”. An overall analysis of the strengths and benefits showed a demonstrated change in students internal and external academic growth.

3.2.2. Theme 2: Recommendation

There was an overall majority consensus amongst the student participants that they would recommend the course to other students. Students expressed comments such as “Yes, because it builds a foundation for science” and “Yes, because they will have the opportunity to look at science in a professional manner. This class should also be open up to all majors so that all students will have the ability to look at whatever major they are in, in a scientific way”. In addition to those comments, some students referenced how the course offered insight into the realities of a scientist. “Yes, because it helps you to vision and see yourself being a scientist and shows you what science is really about” and “Yes, I think this is a good program that helps students, not only to understand the nature of science, but also helps them to think creatively as scientist”.

3.2.3. Theme 3: Modifications

In order to continue to increase the impact of the class students were asked to give suggestions on what changes they would like to see for the course. These comments ranged from logistics to lesson implementation including, “Like I said, the time of the 4pm time slot should be shifted from evening to morning” to “more group research and more case studies”. Students consistently mentioned comments around making the class more innovative and stimulating. “More interactive activities and presentations”, “Include more fun activities in the lecture. Allow the students to actually do research”, and “Include more fun activities in the lecture. Allow the students to actually do research”.

3.2.4. Theme 4: Weaknesses and Challenges

Within some of the response’s students expressed some issues and concerns they had with the course and the way some of the material was presented. Several students stated that they had trouble grasping the vocabulary that was being used in the lessons. Some of the comments were, “The presentations by students were hard to follow because most people didn’t understand the scientific vocabulary in which they referred” and “the lecture should be a little less complex, although I know the purpose of the class is to encourage us to think”. Students mentioned the case studies as being one of the most challenging aspects of the curriculum. One student remarked, “The case studies were fairly challenging” and another stated “I had a hard time understanding the case studies”. Despite the weaknesses and challenges for the course, the students felt that the course was beneficial to their growth as a scientist and it increased the likelihood of them persisting from year one to year two as a STEM major.

4. Discussion

4.1. Findings

The purpose of this study was to show the impacts of a utilitarian scientific literacy curriculum on freshman at risk of persisting in the STEM Pipeline. The findings from this case study support the hypothesis that utilitarian scientific literacy based academic interventions improve retention of first-year students in STEM. Data from this study suggest that incoming freshman college students who choose a STEM major have a relatively low level of scientific literacy. Evidence that a statistically significant effect of student persistence in STEM gateway courses occurred as a result of utilitarian scientific literacy-based academic intervention implementation. This evidence is defined as students successfully getting a C letter grade or higher in STEM gateway courses and enrolling in second-year STEM courses. Mean differences between the control group and participants provide further evidence of researchers [3], which claim that students who succeed in gateway courses are likely to continue moving through the STEM pipeline.

The scientific literacy course, with a focus on utilitarian scientific literacy, encompassed important components that are known to foster STEM student success and retention, especially amongst African American students. According to previous research, the attention on mathematics, mentoring and scientific literacy within the course, created a belief that students within the experimental group would experience an increase in retention from year one to year two [14,18,19,26]. The interdisciplinary approach used to construct the curriculum was intricate to the success of students enrolled in the course and for STEM retention at the institution. Utilitarian scientific literacy, although a new construct, only previously described by Yuenyong and Narjaikaew, has shown some promise that it can impact student retention [35].

Students provided a wide range of responses regarding the strengths and weaknesses of the course, as well as, recommendations and suggestions for improvements. Students submitted open-ended survey responses that listed academic preparedness as a strength/benefit of the utilitarian scientific literacy intervention along with internal and external academic growth. Overall, a majority of the students stated that they would recommend the course to other students, despite identified challenges. It is interesting to note that students experienced inner and external growth that may have contributed to their persistence within the pipeline. This finding has further strengthened our conviction in the role that utilitarian scientific literacy plays in a STEM student’s success and retention. The growth of students’ confidence as a scientist after the completion of the course was identified through their qualitative responses. This increase stresses the importance of scientific literacy and the impacts that a scientific literacy course can have on a student’s confidence as a scientist. Researchers [15,19] posit that students who have a perceived difficulty in STEM classes and low STEM self-efficacy leave STEM majors. Participants in this study have responses consistent with the literature as they note to have a better understanding of scientific material and an ability to think as a scientist.

4.2. Implications and Limitations

This study provides additional support for further evidence that mentorship may have an impact on higher student retention when persisting from year one to year two within the STEM pipeline. According to researchers, mentorship contributes to the motivation and persistence of African American students within STEM majors and careers [14,42,43]. Funding for the course was plentiful in the beginning but as the course continued some of those funds were depleted. Due to the lack of funds, there was no way to pay the mentors during the latter years of the course. Therefore, the responses of students differed as the course progressed without mentors’ present.

Students within this study consistently expressed the need for authentic research experiences, which have been appearing in more science laboratories and STEM curriculums around the country. Authentic research experiences can be defined as a process that involves critical thinking, hands-on participation, experimental design, and overall science procedures and communication [44]. The FirstSTEP program at Middle Tennessee State University and the program implemented at the University of Louisiana included some key interventions that have been deemed successful for STEM student persistence [14,26], however, neither of these interventions included authentic research experiences. The impact of authentic research experiences where students get to act and think like a scientist has proven to be successful for students, particularly for students at-risk of not persisting within STEM [45,46]. Including this type of intervention within STEM programs and curriculums should deliver positive results for students and the future of STEM careers.

As with the majority of studies, the design of this study is subject to some limitations. The course started with several different grants funding the project, yet once the grant was complete there were issues with paying the professor for the course which forced it to be put online for students to take. The change in the delivery of the course may have had an impact on lower student retention outcomes in the last three years when compared with the first three years. This decrease in funding also impacted the role of upper-level student mentors and the ability to compensate them for their time. At the conclusion of the study, the researchers realized that there needed to be more contextual data collected about the participants academic and environmental backgrounds prior to entering college. Knowing this contextual information could have allowed for more analysis and drawn conclusions when looking at the data and student responses. Including questions that will capture demographic information should be considered when moving forward with this work.

4.3. Conclusions

Over the past decade STEM production has been considered one of the most pressing issues within education and the challenges students face within the pipeline have not wavered. Higher education institutions must take the charge in changing the narrative for those students seeking STEM degrees and professions. The utilitarian scientific literacy was designed and delivered to students at-risk of persisting and it has shown some improvement in the retention of STEM students but there is still work to be done. Scientific literacy can serve as a gateway to increasing the retention of STEM students at minority serving institutions and provide supplemental education to those students who want to persist in STEM but do not possess the necessary foundational knowledge to be successful in the field.