Predicting First-Year College Student Retention: Validation of the College Persistence Questionnaire in a Spanish Sample

Abstract

This paper analyzes the factorial and predictive validity of a Spanish adaptation of the College Persistence Questionnaire (SCPQ) on the retention of first year students (FYS) at university. The participants were 490 FYS from two public universities in Southern Europe (Spain). Factorial analysis of the SCPQ (χ²/gl = 1.66, CFI = 0.92, NNFI = 0.91, RMSEA = 0.04) showed six reliable dimensions, similar to those found in the original study: Academic Integration, Social Integration, Supportive Services Satisfaction, Academic Conscientiousness, Degree Commitment and Institutional Commitment. Logistic regression showed that institutional commitment significantly predicted FYS’ permanence. The results support SCPQ’s validity as an adequate and useful tool for assessing FYS’ academic experiences involved in student retention.

1. Introduction

Higher Education institutions around the world are concerned about students’ permanence and academic success. In the North American context, only 40.3% of college students complete their degree, and 28–35% drop out of postsecondary education in their first year [1]. In Spain, the system is also inefficient, and more than 50% of dropouts occur in the first year of university [2,3,4]. Thus, a large number of studies have been designed to predict permanence between university freshman and sophomore year or student retention [5,6,7].

More traditional studies analyze the predictive capacity of students’ pre-university variables on academic success, concluding that High School GPA (HSGPA) results on standardized achievement tests (SAT-ACT) and socioeconomic status are significant predictors of first year students’ (FYS) achievement and permanence until graduation [8]. Another broad set of studies analyze the predictive capacity of students’ socio-personal characteristics (e.g., personality, learning styles and strategies), contextual influences (e.g., family support, financial difficulties), FYS experience variables (e.g., academic and social integration), and the effectiveness of actions taken by universities to foment students’ learning and involvement [9,10,11]. Recent meta-analytical studies highlight the incremental validity of these variables on precedents predicting both academic achievement and permanence [12,13,14].

From this latter perspective, Davidson, Beck and Milligan [15] developed and validated the College Persistence Questionnaire (CPQ), showing its psychometric adequacy and predictive capacity on student retention, as well as its incremental validity regarding HSGPA and SAT-ACT scores. The dimensions incorporated into the final predictive model for permanence were Institutional Commitment, Student’s High School Rank, Academic Conscientiousness and Academic Integration. Gore [16] and Lindheimer III [17] also showed the structural validity and predictive capacity of the CPQ on student retention.

The CPQ is currently a basic reference in the research on retention and persistence in university [15,18,19,20,21]. These studies focus both on determining its relationship with different psychoeducational constructs (e.g., psychological well-being [22], basic needs -competence, relatedness and autonomy- [23], academic and social overload [24], cultural intelligence [25], dropping out of post-graduate studies [5,26,27]), and on evaluating the effectiveness of different intervention programs at the university [28,29]. Moreover, its simplicity and ease of application have contributed to its use by more than 40 universities seeking to evaluate the adequacy and quality of the FYS university experiences stemming from the student’s interaction with the institution’s academic and social environment [10].

Thus, from the perspective of providing necessary conditions and services for first-year university students’ success, and in order to have tools with contrasted validity and efficacy that can evaluate FYS experience dimensions involved in retention and permanence in Spanish degrees and facilitate the development of corresponding intervention programs, we propose the Spanish adaptation and validation of the College Persistence Questionnaire (SCPQ) in the Spanish FYS university population. More specifically, we intend to analyze its factorial structure, psychometric characteristics, and capacity to predict student retention.

2. Materials and Methods

2.1. Participants

The participants were 490 FYS from two public Spanish universities during the 2012–2013 academic year. The distribution by sex was 276 females (52.3%) and 214 males (43.7%), with an average age of 18.47 years (SD = 1.77, range 18–30 years). Both the sex and age distributions of the sample reflect what is generally found in these particular degree programs. 85.5% re-enrolled in the 2013–2014 academic year as sophomores.

2.2. Measures

The CPQ is a self-reported instrument that integrates 34 items evaluating six dimensions: (a) academic integration (8 items; e.g., “How concerned about your intellectual growth are the faculty here?”, α = 0.81); (b) social integration (8 items; e.g., “How much do you think you have in common with other students here?”, α = 0.82); (c) supportive services satisfaction (6 items; e.g., “How fairly do you think students are treated here?”, α = 0.74); (d) institutional commitment (4 items; e.g., “How confident are you that this is the right university for you?”, α = 0.78); (e) degree commitment (5 items; e.g., “At this point in time, how certain are you that you will earn a degree?”, α = 0.78); and (f) academic conscientiousness (3 items; e.g., “I am not interested in academic work and do as little as possible”, α = 0.63). The items are answered on a 5-point Likert-type scale ranging from 1 = “very dissatisfied”, to 5 = “very satisfied”.

Students’ socio-demographic variables (gender and entrance age) were also considered in this study. The response variable considered was Permanence (PER), defined by whether or not students enrol in the second year of the degree program. Socio-demographic and permanence data were provided by the Information Systems Services of both universities.

2.3. Procedure

Before undertaking the research, the study was reviewed and approved by the Ethics Committee of the University of Valencia. The CPQ was translated into Spanish using back-translation design [30]. The definitive version of the SCPQ adaptation was applied collectively in the first three weeks of the second trimester of the school year in the students’ normal classrooms. Participation was voluntary.

According to the Declaration of Helsinki, the students provided their informed consent after the study objectives had been explained to them. The confidentiality of the information was emphasized, as well as its exclusive use for research purposes and for consideration in improving the services of FYS’ university programs.

2.4. Analysis

First, in order to analyze whether the original CPQ six-factor structure proposed by Davidson et al. [31] show a good fit for the Spanish data, a Confirmatory Factor Analysis was performed using EQS 6.1 program.

The original six-factor oblique model was tested since these dimensions are correlated in explaining FYS retention [31]. For that, the robust procedure was applied as estimation method, using the Satorra-Bentler Scaled Chi-square (SB-χ²) [32], which is specifically recommended for the analysis of multivariate non-normal data [33].

Furthermore, other robust indices were used: The comparative fit index (CFI) [34], the non-normed fit index (NNFI) [34] and the root mean square error of approximation (RMSEA) [35] with a confidence interval of 90%. For CFI and NNFI, values >0.95 indicate a good fit and values >0.90, an acceptable fit [35,36]. For RMSEA, values <0.05 indicate a good fit, values between 0.05 and 0.08 indicate an acceptable fit, and values >0.08, a poor fit [37].

After that, considering the CPQ scales and students’ socio-demographic variables, a logistic regression analysis was applied in order to analyze their predictive validity on student retention.

3. Results

3.1. Factor Structure

Preliminary item analysis showed some problems of skewness and kurtosis, indicating absence of multivariate normality [32]. Results from CFA showed that the oblique six-factor 34-item model proposed by Davidson et al. [31] was not supported by all the fit indices considered.

Although SB-χ² ratio values and RMSEA showed a plausible fit to the model, the CFI and NNFI values suggested that the model fit is inadequate: SB-χ²₅₀₇ = 1113.23, p < 0.001, CFI = 0.85, NNFI = 0.83, RMSEA = 0.05 [0.05–0.06]. Model trimming then consisted of removing the weakest path coefficients from the original model, which included four items (15, 20, 25 and 33).

Removal of these items produces a six-factor oblique model for the Spanish data (SCPQ) that was supported by all the fit indices: SB-χ²₃₈₆ = 642.08, p < 0.001, CFI = 0.92, NNFI = 0.91, RMSEA = 0.04 [0.03–0.04] (for clarity, errors are not shown and correlations among factors are shown in

Table 1. Correlations and internal consistency (Cronbach’ alpha in the diagonal). Means (M), SD (standard deviation).

Dimensions	M	SD	1	2	3	4	5	6
1. AI	3.60	0.58	0.77
2. SI	3.58	0.68	0.31 ***	0.83
3. SS	3.53	0.66	0.46 ***	0.36 ***	0.73
4. DC	4.61	0.59	0.33 ***	0.26 ***	0.25 ***	0.70
5. IC	4.09	0.87	0.32 ***	0.26 ***	0.23 ***	0.45 ***	0.70
6. AC	1.89	0.83	−0.38 ***	−0.20 ***	−0.18 ***	−0.35 ***	−0.32 ***	0.66

Note: AI = Academic Integration, SI = Social Integration, SS = Services Satisfaction, DG = Degree Commitment, IC = Institutional Commitment, AC = Academic Conscientiousness, *** p < 0.001.

; see Figure 1).

3.2. Psychometric Properties

Table 1 shows the descriptive statistics, levels of association and internal consistency (Cronbach’ alpha) for the scales of the SCPQ. Results indicated a good reliability of the SCPQ scales, showing no large variations with respect to the original structure in the Social Integration, Supportive Services Satisfaction, Institutional Commitment and Academic Conscientiousness scales. Cronbach alpha in the Degree Commitment scale increases from 0.65 in the original CPQ structure to 0.70 in the SCPQ.

3.3. SCPQ and Permanence

In order to determine the predictive validity of the SCPQ on permanence, a logistic regression analysis was carried out. The variables were introduced in two blocks. Block 1 considers the variables Gender and Entrance Age, and Block 2 also considers the SCPQ dimensions. No significant relationship was found between Permanence and Gender (χ²₁ = 3.11; p = 0.08, V_Cramer = 0.08), and Permanence and Entrance age (t₄₄₄ = −0.27, p < 0.97, d = 0.02). Regarding the SCPQ dimensions, the students who continued their studies show significantly higher scores on Academic Integration (t₄₄₈ = −3.42, p < 0.001, d = 0.56), Social Integration (t₄₄₈ = −3.39, p < 0.001, d = 0.58), Degree Commitment (t₄₄₈ = −4.54, p < 0.001, d = 0.65), Institutional Commitment (t₄₄₈ = −8.61, p < 0.001, d = 1.40), and lower scores in Academic Conscientiousness (t₄₄₈ = 3.23, p < 0.001, d = −0.51). No significant differences were obtained on Supportive Services Satisfaction (t₄₄₈ = −1.12, p < 0.26, d = 0.18).

The final model shows a significant global fit (χ²₁ = 56.49, p < 0.001), and the explanatory capacity of Block 2 was significantly superior to that of Block 1 (χ²₆ = 53.53, p < 0.001). The percentage of variance in Permanence explained by Block 1 is 0.04, while the additional consideration of the SCPQ dimensions allows it to reach a value of 0.28. The final model makes it possible to correctly classify 82.5% of all the cases in the response variable (χ²_{8 Hosmer-Lemeshow} = 5.61, p = 0.69), showing a sensitivity of 68.8% to detect subjects who do not persist in their studies and a specificity of 83.7% to identify subjects who do continue with them. The statistical modeling process led to a final model (see

Table 2. Logistic regression analysis of permanence.

	Exp (β)	IC 95% Exp (β)	χ2 Wald	gl	p
Intercept	0.16		0.95	1	0.30
Gender *
Male	0.77	[0.33–1.80]	0.35	1	0.55
Entrance Age	0.99	[0.90–1.10]	0.06	1	0.81
Academic Integration	1.04	[0.93–1.14]	0.34	1	0.56
Social Integration	1.04	[0.96–1.13]	1.12	1	0.29
Services Satisfaction	0.91	[0.79–1.05]	1.61	1	0.21
Degree Commitment	1.03	[0.86–1.24]	0.13	1	0.71
Institutional Commitment	1.48	[1.28–1.71]	27.97	1	0.001
Academic Conscientiousness	0.96	[0.83–1.12]	0.26	1	0.96

* Categories of reference: Gender→Female.

) that includes only the main effects of Institutional Commitment, revealing that permanence in is significantly more pronounced among students with higher levels in this dimension.

4. Discussion and Conclusions

The basic objective of this study was to adapt and validate the SCPQ through its application to a sample of FYS in two Spanish universities. The results showed the factorial validity and psychometric adequacy of the SCPQ, obtaining an underlying structure quite similar to the original proposed by Davidson et al. [31]. More specifically, after removing four items of the original scale, the AFC highlights that the six-factor oblique model corresponding to its original structure—Academic integration, Social Integration, Supportive Services Satisfaction, Degree Commitment, Institutional Commitment and Academic Conscientiousness—showed a satisfactory fit to the data. The results also show the psychometric adequacy of the SCPQ, given that the set of subscales presents satisfactory levels of internal consistency, with the exception of Academic Conscientiousness, which obtained a value of 0.66, although it is slightly higher than the one described in the original study. Moreover, the levels of association among the subscales are similar to those found in previous studies [16,17,31].

Furthermore, significant relationships were obtained in the expected direction between practically all of the dimensions of the SCPQ and Permanence. However, as in the original studies, the levels of association are reduced, with the dimension of Institutional Commitment reaching the highest value. Both results are congruent with the previous research [10,31].

The second objective of this study was to determine the predictive validity of the psychosocial dimensions evaluated through the SCPQ. The results show their predictive and incremental validity on the socio-demographic variables considered, together explaining 28% of the variance in the Permanence between the first and second years of university studies, and making it possible to correctly classify 82% of all the cases in the response variable. These results obtained are higher than those found by Davidson et al. [31] in their study with students in the US, where these variables explain 23% of the variance in permanence and make it possible to correctly classify 66% of the students. More specifically, the statistical modeling process highlight the relevance of Institutional Commitment, a unique dimension that is introduced in the final predictive model obtained.

Given that the SCPQ evaluates a wide range of experiential dimensions, and the factors that predict persistence in university studies can vary between institutions [8], this tool can be especially useful in the early identification of subjects at risk of dropping out or changing majors, in designing proposed actions for promoting permanence in them, and in evaluating the effectiveness of the measures taken to improve the psychosocial dimensions related to this phenomenon.

The limitations of the study are related to its cross-sectional nature, the relative small sample size, and the fact that these measures were administered at a specific time point in the academic year. Future studies should be carried out with broader and more representative samples, also considering more academic institutions and different university cohorts. On the other hand, future studies must consider other socio-demographic and pre-university variables—many of them available in the databases of the universities themselves (e.g., family socioeconomic and educational level, preference, dedication to study, minority groups, modality of university access, entrance grade), and different characteristics, cognitive skills and motivational dimensions of the students (e.g., self-regulated learning, academic stress and coping strategies, social support, academic goals, personality) in order to obtain a more comprehensive view of student retention, increase capacity to predict academic success at university, improve services provided to students, and adjust interventions to the specific needs of different groups of subjects [10]. Lastly, other SCPQ psychometric characteristics should be studied with a longitudinal (time-related) perspective, including the temporal stability of the scale and the stability of the factor structure along time, a stability that is needed in order to make meaningful comparisons among time points (e.g., when psycho-educational intervention effects are to be analyzed) [38]. Moreover, in spite of the importance of having tools with contrasted validity and reliability to evaluate dimensions related to persistence in university studies, it should be kept in mind that both the theoretical models of reference and the requirements of psycho-educational intervention emphasize the need to consider temporal relations and cause-effect between variables. Future studies will have to address these questions in our context, always from the perspective of responding to the growing demand to improve the efficacy of the services, programs and actions undertaken by universities to foment students’ academic success [39].

In any case, the present study provides evidence on the factorial validity, psychometric adequacy and predictive validity of the SCPQ on student retention, confirming its original structure [16,17,31]. On the other hand, according to previous research, the results highlighting that Institutional Commitment is an important predictor of student retention also show its significant relationship with the other dimensions assessed by the SCPQ. Thus, these results emphasize (a) that universities must foster students’ sense of belonging and identification with their degree and university (Institutional Commitment), and (b) there may be different student profiles that can explain the students’ lack of institutional commitment, making it possible to adjust interventions to the students’ specific needs [10,15]. For example, students’ lack of institutional commitment may be explained by their negative evaluation of the curriculum or teaching quality in their degree (Academic Integration), the low value they give to obtaining the degree (Degree Commitment), or any combination of the different SCPQ’ dimensions. This information can be especially useful for designing action proposals to meet different students’ specific needs, while providing academic and institutional decision-makers with valuable information about the strengths and weaknesses of services and programs directed toward fostering students’ academic involvement and success [5,24].