A Statistical Analysis of GRE/GMAT Data for Admission to Master’s Degree Programs

Abstract

In this paper, we investigate the waiving of GRE/GMAT for admission to master’s degree programs in a state university in Florida, USA. Standardized tests, such as GRE/GMAT, were required for admission to the master’s degree programs in 2019/2020, waived in 2020/2021, and removed or modified in 2021/2022. We analyzed the application, enrollment, and performance data to assess the impact of these changes. The data show that the number of applicants and enrolled students exhibit an upward trend from 2019 to 2021. The undergraduate GPA of new applicants who did not submit the GRE in 2021 tends to be statistically significantly higher than for those who did submit the GRE in 2019 (p < 0.001). The new students’ first-semester graduate GPA in 2021 (no GRE requirement) tends also to be statistically significantly higher than the new students’ first-semester graduate GPA in 2019 (GRE requirement) (p < 0.01). The study employed random forest feature importance using the Gini index to analyze the predictive power of GRE and undergraduate GPA for forecasting first-semester graduate GPA. The results show that undergraduate GPA is a more significant factor than GRE. Overall, the study’s statistical evidence indicates that waiving GRE/GMAT requirements for master’s degree programs did not affect applicants’ performance, as measured by their undergraduate GPA, nor did it lead to a decline in student performance, as measured by first-semester graduate GPA.

1. Introduction

The Graduate Record Examination (GRE) or Graduate Management Admission Test (GMAT) is a standardized test many graduate programs use to assess a candidate’s preparedness for advanced study. It is important to note that while many universities require submitting GRE or GMAT scores for admission into their graduate programs, some do not. Research on graduate admission criteria, particularly in business and technical fields, has consistently highlighted the importance of standardized test scores, such as the GMAT and GRE, in predicting academic success [1,2,3]. However, using these scores has also been associated with potential fairness and diversity issues [4]. Research has shown that removing the GRE as a requirement for graduate admissions does not negatively impact student success [5]. From a psychometric perspective, it is essential to consider the bias, fairness, and validity of various admission measures, including the GRE [6]. Despite these concerns, the predictive value of these scores in graduate school performance remains a key consideration in the admissions process.

In the last decade, the number of universities eliminating the GRE/GMAT requirement for admission has steadily increased. Studies suggest no substantial correlation exists between general GRE/GMAT scores and the success of graduate students [7]. Kass 2019 [8] found no significant differences in outcomes between GMAT and non-GMAT students, particularly for those with high GPAs and work experience. Similarly, Kolluri et al. [9] have reported no significant difference in graduation GPA for students who waived the GMAT requirement. These findings challenge the validity of the GMAT as a selection tool. However, it is important to note that these studies focused on MBA programs, and further research is needed to determine the generalizability of these findings to other graduate programs. Additionally, Millar [10] highlights the need for a diverse public health workforce and the potential role of GRE waivers in achieving this goal.

The GRE/GMAT has been known to be at the center of conflicting results regarding its predictive power for graduate student success [11,12,13,14]. Therefore, universities are opting to drop the GRE/GMAT requirement and focusing on better ways to assess students’ potential for success in their programs [4,15]. Furthermore, Hooker et al. [16] have argued that removing the GRE and GMAT improved diversity in the graduate program. Further research is needed to explore these issues and to identify practical steps for improving admissions decisions.

Admissions requirements have become increasingly flexible due to the rise and availability of online graduate programs [17]. This made it crucial to rethink assessing students’ potential for success in their desired programs. Lightner et al. [18] have suggested that the skill sets necessary for success in online graduate programs may differ from those in traditional programs and that entrance requirements should be re-evaluated. Nye and Ryan [19] have proposed supplementing the GRE and GMAT scores with measures of conscientiousness and vocational interests to improve the predictive validity of admissions decisions and enhance diversity.

The university conducting this study provides 35 master’s degree programs, catering to approximately 4000 students in the fall semester of 2022. Out of these 35 programs, 24 are known as “programs of strategic emphasis” (PSE), according to the Florida Board of Governors (FL-BOG). Graduates of PSE programs tend to have better job prospects and are likely to secure higher-paying jobs [20]. As a result, most students graduating from these programs choose to enter the job market, with only a tiny percentage pursuing further studies in doctoral programs. All programs required applicants to submit GRE/GMAT scores for admission during the 2019/2020 academic year. However, the COVID-19 outbreak was declared a pandemic by the World Health Organization (WHO) in March 2020 as it spread to over 170 countries [21]. As a result, all institutions in the Florida State University System (FL-SUS), along with many other higher education institutions, were forced to halt face-to-face teaching and learning. This led to students having to complete all courses online during the latter part of the spring semester, and the campuses were subsequently closed after March 2020.

The graduate school at any university is committed to maintaining healthy enrollment and graduation rates while adequately supporting graduate students to achieve academic success. However, the COVID-19 outbreak presented significant challenges to the graduate school’s operations and the graduate programs. Due to the closure of most test centers, it became increasingly difficult for students to take the GRE/GMAT. Although online testing was an option, some students needed computer equipment, and others faced financial difficulties [22]. As a result, many students requested that the GRE/GMAT admission requirement be waived. Most program chairs and directors supported the students’ request. They recognized that the condition of GRE/GMAT could unfairly disadvantage economically disadvantaged students and those who lost jobs due to COVID-19. After careful evaluation and consideration, the graduate school, with final approval from the Office of the Provost, temporarily suspended the GRE/GMAT admission requirement for all 35 master’s degree programs for the 2020/2021 academic year. This decision would not have been possible without the impact of COVID-19. It is important to note that this change only applies to master’s degree programs, as doctoral and specialist programs will continue to require GRE scores for admission.

During the fall semester of 2020, 26 out of 35 programs submitted a request known as a “Curriculum Change Request” (CCR) to permanently remove the GRE admission requirement. Two programs submitted a proposal to waive the GRE/GMAT requirement for another year and planned to submit a CCR in 2021/2022. Seven programs proposed a CCR to waive the GRE requirement if the applicant’s undergraduate GPA meets a specific threshold for 2021/2022 and beyond. These requests were thoroughly debated, discussed, and voted on by the “College Council”, the “University Graduate Council”, and the “University Faculty Senate”. Finally, in May 2021, the Provost’s Office approved the CCRs.

The Scope of the Paper

This study aims to examine the effects of eliminating or modifying the GRE/GMAT admission prerequisite for all master’s degree programs offered by a regional comprehensive state university in Florida, USA. The objective is to determine how this change has impacted the programs. Prior to 2020, the university required all applicants to submit GRE/GMAT scores for admission. However, this requirement was waived in the academic years 2020/2021 and permanently removed from most programs in the academic year 2021/2022. There are a few exceptions where the requirement can be waived based on the applicant’s undergraduate GPA meeting a specific threshold. Our investigation will center on the following research questions, specifically looking at the disparities between 2019/2020 and 2021/2022:

• How did the GPAs of new applicants compare?
• Did the yield rate change?
• Has there been an increase in overall enrollment?
• Did newly enrolled students perform differently in their first semester?

In addition, we will use machine learning to model the relationship between the GRE requirement, first-semester graduate GPA, and undergraduate GPA. The findings from this research will inform deans and directors in making informed decisions on graduate admission.

This paper is structured as follows: Section 2 outlines the methods, while the analysis and results are presented in Section 3. Section 4 provides a discussion, and our conclusions are given in Section 5.

2. Methods

A quantitative approach was employed within the positivist paradigm. Descriptive statistics were reported for all outcomes (GPA, counts, and proportions of students). The nonparametric Wilcoxon rank sum test and the bootstrap $95\%$ confidence interval for medians were used to evaluate the differences hypothesis in the GPA scores. The Wilcoxon rank sum test determines if there is a significant difference between the values of two independent samples. This test is suitable for non-normally distributed data, where the samples are independent, and the variable is continuous (e.g., GPA). It can detect if one sample tends to have higher values than the other. The chi-square test was used to test the distribution of the number of students across years and semesters. We set statistical significance in advance at $0.05$.

We utilized random forests [23] to predict the first-semester graduate GPA. Random forest is a machine learning method that combines multiple decision trees to create a robust predictive model. Random forests aggregate the predictions of diverse randomized trees to improve accuracy and reduce overfitting. Each tree independently predicts the target variable during prediction, and the final prediction is obtained through majority voting (classification) or averaging (regression) across all trees. Random forests exhibit robustness, handle noisy data effectively, and provide feature importance analysis [24].

The random forest classifier is a helpful feature selection method that provides multivariate feature importance scores. This method has been successfully used to analyze high-dimensional data from various domains. The classifier performs implicit feature selection by using a small subset of variables for classification. One of the measures used is the Gini index derived from the training of the random forest classifier. The Gini importance is a general indicator of feature relevance. This metric measures the significance of a feature’s discriminative value for a particular classification problem in terms of how often it was selected for a split.

3. Analysis and Results

The changes associated with waiving or modifying the GRE/GMAT admission criteria on the number of student applicants and enrollments and GPA data were evaluated using the undergraduate GPAs of applicants and the new students’ first-semester graduate GPAs for the academic years of 2019/2020, 2020/2021, and 2021/2022. Our comparison of these years was based on each year’s unique circumstances. In 2019/2020, all programs required GRE/GMAT scores for admission. In 2020/2021, although the GRE/GMAT admission criteria were waived, some applicants had already submitted their scores. Therefore, the results were mixed. In 2021/2022, the GRE/GMAT admission criteria were either removed or waived if the applicant’s undergraduate GPA met a certain threshold. While applicants could submit their scores to enhance their applications, it was not mandatory.

The findings will undoubtedly give valuable information to graduate programs, informing them whether the decision to remove or waive the GRE/GMAT is suitable and pinpointing any areas needing modification or improvement. Since our university has yet to experience such a significant alteration in its graduate admission requirements, the study findings will also provide helpful insights to our top management.

3.1. The Number of Applications

In 2019/2020, all thirty-five master’s degree programs required GRE/GMAT for admission. In the 2020/2021 academic year, the GRE/GMAT was waived, but some applicants submitted GRE/MAT scores before the GRE/GMAT waiver was announced. The applicants could submit GRE/GMAT scores to enhance their applications in 2021/2022, but it was not required. Figure 1 shows the number of applications from the nine semesters of 2019/2020, 2020/2021, and 2021/2022. The number of applicants significantly changed across the semesters in 2020/2021, 2021/2022, and 2019/2020 (chi-square test $p<0.001$), with the highest being in the fall semester.

The application numbers for 2021/2022 compared to 2019/2020 increased by 64%, 61%, and 55% for the fall, spring, and summer semesters, respectively. However, the proportions of applicants in 2019 compared to 2021 stayed the same for each semester. The 95% multinomial confidence intervals for each semester are provided as follows: For the fall of 2019, the proportion is 43.2% (95% CI from 41.6 to 44.8), and for the fall of 2021, the proportion is 44.1% (95% CI from 42.8 to 45.3). For the spring of 2019, the proportion is 32.1% (95% CI from 30.6 to 33.7); for the spring of 2021, the proportion is 32.2% (95% CI from 31 to 33.5). For the summer of 2019, the proportion is 24.6% (95% CI from 23.1 to 26.2), and for the summer of 2021, the proportion is 23.7% (95% CI from 22.5 to 25).

Figure 1 shows an upward trend in the number of applicants from 2019/2020 to 2021/2022, corresponding to years of waiving or removing the GRE requirements. However, we should remember that the post-pandemic demand could also impact the number of applicants during those years [22]. Since many people lost their jobs during this challenging time in the USA, some chose to join graduate programs to enhance and learn new knowledge to be more marketable. One can also see from Figure 1 that the application numbers for the spring and summer semesters of 2021/2022 were about the same as those of 2020/2021. Here are some reasons. The first reason is that the job market was much better in the USA in 2021/2022 than in 2020/2021, so some students might have returned to work instead of attending school. The second reason is that those high-demand programs at this university reached their capacity in the fall semester, so they had to raise their admission standards for the spring and summer semesters. The acceptance rate has gone down significantly. That may not encourage people to apply for these programs. The third reason is that most of these 35 programs have more seats for new students in the fall semesters than in the spring and summer semesters. Some programs only accept students in fall semesters. That is why the fall application numbers were much more significant. We predict this will continue to be the case for a while.

3.2. The Applicants’ Undergraduate GPA

The applicants’ undergraduate GPAs were collected every semester. The averages and standard deviations of the undergraduate GPAs are presented in

Table 1. The applicants’ undergraduate GPA; average (SD).

	2019/2020	2020/2021	2021/2022
Fall	3.33 (0.42)	3.38 (0.41)	3.32 (0.44)
Spring	3.38 (0.40)	3.39 (0.40)	3.32 (0.44)
Summer	3.39 (0.41)	3.36 (0.42)	3.34 (0.41)

.

The distribution of the undergraduate GPA is presented in Figure 2 using violin plots. The violin plot is a hybrid of a box plot and kernel density plot, which shows peaks and how the data are distributed.

The number of applicants increased from 2019 to 2021, as expected from a post-pandemic demand increase; also, the average GPA was higher in 2021. Figure 2 indicates that the two distributions of undergraduate GPA are different. The undergraduate GPA of applicants with GRE/GMAT submissions (mean = 3.16; median = 3.1; median 95% CI from 3.02 to 3.15) tends to be smaller than GPA without GRE/GMAT submissions (mean = 3.34; median = 3.37; median 95% CI from 3.36 to 3.4). The Wilcoxon rank sum test p-value was <0.001, showing evidence of a statistically significant difference. The undergraduate GPA of applicants in 2021 (no GRE requirement) tends to be higher than the undergraduate GPA of applicants in 2019 (GRE requirement). This supports that the GRE requirement is not a reliable predictor of the quality—measured by their undergraduate GPA—of graduate student applicants. Graduate programs can remove GRE requirements and still receive outstanding graduate student applications.

3.3. The Yield Rate and Overall Enrollment

The yield rate is the number of newly enrolled students over the number of applicants, i.e., the rate of newly registered students.

Table 2. The yield rates.

	2019/2020	2020/2021	2021/2022
Fall	853/1991 = 42.8%	1044/2479 = 42.1%	1054/3259 = 32.3%
Spring	633/1481 = 42.7%	858/2280 = 37.6%	808/2382 = 33.9%
Summer	455/1135 = 40.1%	636/1740 = 36.6%	588/1756 = 33.5%

shows the yield rate over nine semesters in the three academic years. In Table 2, the numerator number is the new student enrollment, and the denominator is the number of applications received. For example, $853/1991$ means that 853 new students enrolled, and 1991 applications were received for the fall of 2019. The yield rate is $42.8\%$, representing a ratio of 853 over 1991.

Table 2 shows that the yield rates of the fall, spring, and summer semesters in 2021/2022 compared to the fall, spring, and summer semesters in 2019/2020 and 2020/2021, respectively, are significantly lower. The programs were highly competitive since the applicant pools were significantly large in 2021/2022. Many of the 35 programs reached their capacities, especially those high-demand programs, because each program could only take a certain number of students. Some high-demand programs must raise their admission standards to admit higher-quality students since the seats available are limited. Even though a student was admitted to a program, the student might need help finding a spot to register for the courses required or wanted because the course might be full. When the students cannot take the classes they want or need, they may not want to enroll at the university. This also indicates that removing the GRE/GMAT admission requirement played an important role. Figure 3 shows the enrollments in the three academic years. The enrollments of the fall, spring, and summer semesters in 2021/22 went up significantly compared with 2019/2020 and 2020/2021. The percentages of enrollment change in 2021/2022 over 2019/2020 for the fall, spring, and summer semesters were up by 34%, 38%, and 41%, respectively. Since the pools of applicants were much more significant than ever, some programs, especially those traditionally low-demand programs, could benefit from these increases to have more students in their programs.

3.4. New Students’ First-Semester Graduate GPA

New students’ first-semester GPAs were collected by the end of their first semester to evaluate their success in the 2019/2020 (GRE requirement) and 2021/2022 (no GRE requirement) academic years. Figure 3 shows that new student enrollments in the fall, spring, and summer semesters of 2021/2022 increased by 23.6%, 27.6%, and 29.2%, respectively, compared with those in 2019/2020. These numbers are a significant increase in enrollments, so we eagerly wanted to know how well the new students performed by the end of their first semester. This is a crucial question that most researchers would like to know the answer to. The average GPAs of new students by the end of their first semester were collected for the nine semesters in the three academic years. The means and the standard deviations were calculated for each semester and are presented in

Table 3. First-semester graduate GPA; average (SD).

	2019/2020	2020/2021	2021/2022
Fall	3.61 (0.68)	3.69 (0.72)	3.62 (0.78)
Spring	3.60 (0.75)	3.72 (0.58)	3.62 (0.72)
Summer	3.65 (0.61)	3.68 (0.66)	3.65 (0.6)

.

In Figure 4, the distributions of the new students’ first-semester graduate GPAs in the 2019/2020 to 2021/2022 academic years are presented. The graduate first-semester GPAs with GRE/GMAT requirements (mean = 3.595; SD = 0.62; median = 3.83; median 95% CI from 3.81 to 3.96) tend to be smaller than GPAs without GRE/GMAT requirements (mean = 3.63; SD = 0.70; median = 3.9; median 95% CI 3.87 to 3.95). The Wilcoxon rank sum test p-value was $0.002$, showing evidence of a statistically significant difference. The new students’ first-semester graduate GPA in 2021 (no GRE requirement) tends to be greater than that in in 2019 (GRE requirement). This indicates that removing the GRE/GMAT requirement did not affect the graduate student’s performance, measured by first-semester graduate GPA. Programs can select higher quality students from larger applicant pools, resulting in better performance and higher average GPA.

3.5. Predicting the First-Semester Graduate GPA

In this section, we shall propose a predictive statistical model to determine whether a student will obtain a GPA greater than $3.0$ in the first semester of the master’s degree program. We also aimed at extracting variable importance. Our data contain the following variables:

• GRE: Whether a student submitted GRE scores in their application to the graduate school
• Undergpa: Undergraduate GPA
• Year: Year to enter the master degree’s program: 2019, 2020, 2022
• Semester: Semester to enter the master degree’s program: Fall, Spring, Summer

Table 4. Data Description.

Variable	N = 5157
Year—n (%)
2019	1408 (27.3%)
2020	1722 (33.4%)
2021	2027 (39.3%)
GRE—n (%)
No	4459 (86%)
Yes	698 (14%)
First-Semester Graduate GPA—Mean (Range)	3.65 (0.00, 4.00)
Undergraduate GPA—Mean (Range)	3.36 (2.01, 4.00)
First-Semester Graduate GPA > 3—n (%)
No	567 (11%)
Yes	4590 (89%)
Undergraduate GPA > 3—n (%)
No	983 (19%)
Yes	4174 (81%)

presents descriptive statistics of the variables, including two categorical transformations of continuous variables (First-Semester Graduate GPA and Undergraduate GPA). Our binary target variable is “First-Semester Graduate GPA > 3”. We employed a random forests model with 500 trees to find variable importance using the Gini index. Larger values of the index are an essential predictor of whether a student will obtain a first-semester graduate GPA greater than 3.

The analyzed data have 5157 observations and three predictors (Semester, GRE, Undergraduate GPA as a categorical variable) split into training–testing 80–20%. The twinning technique [25] was employed to split the data into twins, statistically similar to the original dataset. Five-fold cross-validation was performed to tune the parameter of each model using a search grid of 20 values. The ROSE (Random Over-Sampling Examples) technique was utilized [26] to handle imbalanced data.

The testing performance of the predictive random forest model is shown in

Table 5. Random forest model for Graduate GPA > 3 (yes or no); higher values of the metric (close to 1) indicate better performance.

Metric	Value
Accuracy	0.745
Negative Predictive Value	0.140
Positive Predictive Value	0.920
Sensitivity	0.787
Specificity	0.337
F-score	0.848

. The computations were performed using Tidymodels [27] packages in R [28]. Overall, the accuracy is $74.5\%$ and an F-score of $84.8\%$ is obtained. These are excellent numbers given our data only have three predictors. The model does very well in predicting students with a first-semester graduate GPA greater than 3 (sensitivity of $78.7\%$). The positive predictive value (PPV) indicates the likelihood that a student who is predicted to obtain a first-semester graduate GPA greater than 3 will actually obtain it (PPV of $92\%$). However, the model does not do well to identify students with a first-semester graduate GPA less than or equal to 3 (NPV of $14\%$ and specificity of $33.7\%$).

Figure 5 highlights the feature importance of all variables. It was observed that undergraduate GPA is the most important predictor of “first-semester graduate GPA greater than 3”. Interestingly, the GRE (yes/no) variable was not ranked first, indicating that it has less predictive power for these data, providing more evidence that requiring the GRE for master’s admissions is not predictive of success in graduate programs.

4. Discussion

The distribution of GPA can be different because of the type of data. For example, applicants without the GRE seem to have a larger proportion of GPAs greater than 3 than applicants with the GRE. We can also observe that undergraduate GPA peaks at 3 (Figure 2), the required minimum GPA for admission in some programs. These characteristics make the two distributions have different shapes. The second reason is the sample selection; these data relate to a regional comprehensive university, and therefore, the shape of the distribution can be specific to the population of students applying to this university and its programs. The third reason is that the pre-post-COVID-19 return to graduate school could have impacted the shape of the distributions. Our statistical analysis has taken care of the non-normality of the distributions using nonparametric methods and providing confidence intervals for the medians. Note that the statistical significance is not the practical significance, as we can observe that the difference in medians is not practically significant (3.1 for the GRE group vs. 3.37 for the non-GRE group).

GRE/GMAT is a standardized test that evaluates general verbal, quantitative, and analytical writing skills. While these skills are essential, they may not accurately predict success in a particular area of graduate study. For instance, a high GRE score may not forecast how well an individual would perform in a research-intensive or applied program. As with any assessment, the GRE evaluates test-taking skills and the ability to adapt to the format rather than knowledge or aptitude in a specific discipline. This can disadvantage students who cannot afford expensive preparation courses or have limited time to prepare.

Based on this study, GRE scores have limited predictive power for first-year GPA in graduate school. This indicates that a high GRE score may not directly result in strong academic performance later on. Other factors, like research experience, may be more dependable indicators of success. These findings have been supported by other studies.

A major obstacle in the field of education analytics is the unavailability of high-quality data that are easily accessible to the public. In the future, it is important that more data are made available for conducting more studies on the value of GRE/GMAT scores, particularly given the current trend towards online education, the increase in the number of programs waiving GRE/GMAT requirements, and how that plays into the potential mathematics crisis that higher education may face in the coming years as a result of the knowledge gap caused by COVID-19.

5. Research Limitations

It is worth noting that the study only focused on master’s degree programs at a regional comprehensive university. As a result, the findings and conclusions of the study may not be relevant to programs where a significant number of students pursue doctoral studies, universities of varying sizes, or various student populations. Although this university currently requires GRE scores for its doctoral programs, removing this requirement for its master’s programs has been successful. It is essential to remember that what works well for one institution may not work for others, and this study did not examine how programs selected students after the GRE requirement was removed or waived.

6. Conclusions

The study found minimal statistical evidence supporting the importance of the GRE/GMAT for master’s admissions and its relationship to student success measured by first-semester graduate GPA. The undergraduate GPA of non-GRE new applicants in 2021 tends to be statistically significantly higher compared to those those who did submit the GRE in 2019. The new students’ first-semester graduate GPA in 2021 (no GRE requirement) also tends to be statistically significantly higher than the new students’ first-semester graduate GPA in 2019 (GRE requirement). This study demonstrated that waiving the GRE/GMAT requirements for master’s degree programs did not negatively impact application quality (measured by undergraduate GPA) or new student performance (measured by first-semester graduate GPA).

This study also found that the number of quality applicants to graduate programs and the yield rates can still increase even if admission barriers such as the GRE/GMAT are removed. With a larger pool of applicants, graduate programs can select higher-quality students, leading to success for both the students and the program.

In addition, random forest was utilized to quantify the importance of predicting student GPA at the first-semester graduate level. According to the findings, undergraduate GPA is a better predictor of success in graduate school than the GRE test.