Positive Mental Health: Psychometric Evaluation of the PMHI-19 in a Sample of University Student-Athletes and Dancers

Abstract

Background: Student-athletes and competitive dancers experience significant physical, psychological, and emotional stress, often coupled with academic and social pressures. These stressors may lead to unhealthy coping mechanisms, negatively impacting mental health, quality of life, and athletic performance. While assessing mental illness is important, measuring positive mental health (PMH) can offer valuable insights into overall well-being and resilience. The positive mental health Instrument (PMHI) was developed to assess PMH, but its psychometric properties in student-athletes and competitive athletes have not been explored. Methods: Collegiate student-athletes and competitive dancers completed a survey, including the PMHI-19 and a demographic questionnaire. Confirmatory factor analysis (CFA) was conducted to assess the factor structure of the PMHI-19. An exploratory factor analysis (EFA) was then conducted to identify a more parsimonious structure. Results: The CFA of the PMHI-19 did not meet recommended model fit indices. EFA resulted in two different parsimonious models: a 4-factor, 11-item model (PMHI-11) and a 3-factor, 9-item model (PMHI-9), both meeting recommended fit indices. Conclusions: The condensed PMHI-11 and PMHI-9 models may be more suitable for use in collegiate athletic populations. Further research is needed to refine these instruments and explore their applicability across diverse athletic groups.

1. Introduction

University athletes (i.e., student-athletes, competitive dancers) are expected to consistently cope with the stress of physical training and psychological and emotional exhaustion, while meeting rigorous academic demands and maintaining social relationships. Often, the stress from these expectations can become overwhelming, leading student-athletes to develop unhealthy coping mechanisms and engage in risky behaviors (Knettel et al., 2023; Yusko et al., 2008). Risky behaviors could include excessive alcohol or drug use (Donohue et al., 2018; Knettel et al., 2023), adopting unhealthy body image perspectives (Vani et al., 2021) or developing eating disorders (Donohue et al., 2018; Vani et al., 2021). University athletes also experience heightened levels of stress, anxiety, and depression (Knettel et al., 2023; NCAA, 2023). Engaging in risky behaviors and experiencing negative psychological factors can reduce quality of life (QOL; Hohls et al., 2021), increase injury occurrence (Brewer & Redmond, 2017), increase burnout (L. Jones & Zenko, 2021), and decrease performance (L. Jones & Zenko, 2021; M. Jones et al., 2009). Furthermore, experiencing chronic stress can increase the development of more complex problems such as insomnia, weakened immune systems, and depression (Kahn & Kahn, 2017).

The ability to measure relevant psychological variables could aid in enhancing mental health for university athletes. Measuring both positive and negative psychological factors is important for obtaining a comprehensive mental health profile. While it was once thought that mental health was simply the lack of mental illness (Lluch-Canut et al., 2013), researchers have recently suggested that mental health includes the presence of well-being—having the resources to cope with daily stress, self-efficacy, and being able to work and contribute to one’s community (World Health Organization, 2022)—while mental illness and mental health are distinct constructs, but exist on the same continuum as suggested by the dual-continua model (Keyes, 2005). Well-being or positive mental health (PMH) can be explained by “…the ability to build and maintain relationships, have active coping and interpersonal skills, provide and receive emotional support, pursue personal growth and autonomy, and participate in religious and spiritual practices.” (Vaingankar et al., 2016). PMH has been found to be a protective factor against mental illness (Sequeira et al., 2020) and may have a more powerful influence on health and physiology than on mental illness (Iasiello et al., 2019; Sequeira et al., 2020). Currently, several questionnaires capture components of PMH and well-being, such as the Mental Health Continuum Short Form (MHC-SF; Lamers et al., 2011) and the WHO-Five Well-being Scale (World Health Organization, 1998). Three scales, however, specifically target PMH: the Positive Mental Health Questionnaire (PMHQ; Lluch-Canut et al., 2013), the Positive Mental Health (PMH) scale (Lukat et al., 2016), and the Positive Mental Health Instrument (PMHI; Vaingankar et al., 2011).

Vaingankar developed the PMHI to measure positive mental health through six identified factors: general coping, emotional support, spirituality, interpersonal skills, personal growth and autonomy, and global affect. The 47-item higher order model demonstrated acceptable fit indices (CFI = 0.96, TLI = 0.96, RMSEA = 0.05, α ≥ 0.90). Subsequent studies created shortened versions of the PMHI, the Positive Mental Health-19 (PMHI-19; Vaingankar et al., 2014) instrument and the Rapid PMHI (R-PMHI; Vaingankar et al., 2020). While the original PMHI demonstrated efficient psychometric evidence, the high internal consistency suggested item redundancy, and completing a 47-item instrument was time consuming and burdensome on respondents. The PMHI-19 was developed by conducting a series of CFAs and Rasch Analysis on the initial 47 items; the CFA resulted in removal of 27 items. The goodness of fit indices for the 19-item, 6-factor higher order model were acceptable (CFI = 0.984, TLI = 0.979, RMSEA = 0.058). Additionally, convergent and divergent validity were established (e.g., r = 0.59, p < 0.0001 with well-being and r = −0.3, p < 0.0001 with depression), as well as test–retest reliability (ICC = 0.93).

While all three instruments (i.e., PMHI, PMHI-19, R-PMHI) have analyses which indicate evidence of sound psychometric properties, instrument measurement concerns have also been noted. For example, high alpha levels (≥0.90) may indicate item redundancy, construct underrepresentation, reduced construct precision, or parallel items (Pesudovs et al., 2007). Specifically, the 47-item PMHI had a high internal consistency (i.e., greater than 0.90) across the subscales and was found to have floor and ceiling effects, while the 19-item PMHI exhibited redundancy in one subscale, and the 6-item PMHI exhibited ceiling effects. Additionally, when using a classical test theory (CTT) approach to determine the underlying latent factors, the factor structure should be established in new samples where the instrument will be used (Kline, 2016) and all PMHI instruments were only validated in a Singapore general population or in a population of individuals with mental illnesses (Vaingankar et al., 2011); thus, there is a need to establish the psychometric properties of the scales in a collegiate athlete population to support the measurement of PMH in this population. Due to the response burden and redundancy of the 47-item PMHI (Vaingankar et al., 2011), and the recent development of the R-PMHI, the current study used the PMHI-19. The purpose of the study was to assess the psychometric properties of the PMHI-19 using a United States population who participated in collegiate sports. The first objective was to conduct a confirmatory factor analysis (CFA) on the PMHI-19 in a university student-athlete sample. Because the model did not meet fit indices, the second objective was to reassess the factor structure using alternative model generation (i.e., exploratory factor analysis, covariance modeling).

2. Materials and Methods

The present study was approved by the University Institutional Review Board and informed consent was obtained from participants prior to data collection. Collegiate student-athletes and competitive dancers enrolled in dance institutions were recruited for the study. A convenience sample of dance and athletic trainer faculty recruited healthy and injured individuals from 12 universities in the United States; participants were recruited from different collegiate competition levels (e.g., NCAA Division I, NAIA).

A survey that included participant demographics (e.g., age, sex, division level, injury status, etc.) and the Positive Mental Health Short Form (PMHI-19) instrument was created in identical paper and electronic forms. The electronic survey was developed using Qualtrics online software (Qualtrics, LLC, Provo, UT, USA). All paper responses were inputted into Qualtrics for data analysis.

2.1. Instrumentation

2.1.1. Positive Mental Health Instrument Short Form

The PMHI-19 (Vaingankar et al., 2014) is a first-order, six-factor, 19-item instrument. Factors include: (1) general coping (GC; e.g., “I try to solve the problem one step at a time”); (2) emotional support (ES; e.g., “I have people in my life who give me support”); (3) spirituality (SP; e.g., “I set aside time for meditation or prayer”); interpersonal skills (IS; e.g., “I get along well with others”); personal growth and autonomy (PGA; e.g., “I have confidence in the decisions I make”); and global affect (GA; i.e., happy, relaxed, enthusiastic). A total positive mental health score is calculated by summing the 19 items and dividing it by 19.

2.1.2. Demographic Questionnaire

Respondents completed a short demographic questionnaire. Variables collected included sex, age, ethnicity, year in school, sport or activity of participation, health status, and division participation (e.g., NCAA division I).

2.2. Data Analysis

Data were exported from Qualtrics (Qualtrics, LLC, Provo, UT, USA, Version 2020, https://www.qualtrics.com) and analyses were conducted in Statistical Package for Social Sciences (IBM SPSS Statistics for Windows, Version 27.0. Armonk, NY, USA: IBM Corp) and Analysis of Moment Structure (IBM AMOS, SPSS, Inc., Version 27.0, Chicago, IL, USA). Missing responses for each survey item were calculated for each respondent and those missing responses to more than 10% of the items (i.e., two or more) were removed from the dataset. The remaining missing data were replaced with the rounded mean score (Kline, 2016; Tabachnick & Fidell, 2001) of the respective item for analysis purposes. Histograms, and skewness and kurtosis values were evaluated to check normality of data. Univariate outliers were assessed using z-scores with a cut-off value of |3.3| and multivariate outliers were assessed using Malahanobis distance. The cut-off value was identified using a chi-square table with degrees of freedom and a p-value of 0.01 (Kline, 2016).

A confirmatory factor analysis (CFA) was conducted using the full sample. Because model fit indices were not met, an exploratory factor analysis (EFA) was conducted until a parsimonious solution was identified. The EFA solution was then assessed using a covariance model approach. Criterion validity of the refined PMHI-19 was assessed by conducting correlation analysis between the original PMHI-19 and refined PMHI-19, with a high correlation indicating that criterion validity is established (Kline, 2016).

2.2.1. Confirmatory Factor Analysis

A CFA was conducted using the Analysis of Moment Structures (AMOS) software (IBM Corp., Chicago, IL, USA, Version 27.0) on the PMHI-19. Model fit indices computed were evaluated based on a priori values. The relative goodness-of-fit indices considered were the comparative fit index (CFI; ≥0.95), the Tucker–Lewis index (TLI; ≥0.95), Root Mean Square Error of Approximation (RMSEA ≤ 0.06), and Bollen’s incremental fit index (IFI; ≥0.95; Hu & Bentler, 1999; Kline, 2016). The likelihood ratio statistic (chi-square or CMIN) was also assessed, but was not used as the primary assessment of model fit (Brown, 2015; Kline, 2016). Because model fit criteria were not met, alternative model generation was conducted to identify a more psychometrically sound factor structure.

2.2.2. Exploratory Factor Analysis

An EFA using maximum likelihood extraction and direct oblimin rotation was conducted using SPSS. The following criteria were used to determine the appropriate number of factors to retain: (1) factors with eigenvalues >1.0; (2) examination of the scree plot inflection point; and (3) factors that accounted for ≥5% of the variance (Brown, 2015). Following extraction, items were assessed and removed one at a time using the following recommendations: loading <0.40; cross-loading ≥0.30; low internal consistency (α < 0.69 >0.89); high bivariate correlations with another item (r > 0.79); theoretical or conceptual misfit (Brown, 2015; Pesudovs et al., 2007).

2.2.3. Covariance Model

The identified EFA solution was assessed in a covariance model using the same overall model fit criteria utilized for the initial CFA (Brown, 2015; Kline, 2016). Localized areas of strain and modification indices were evaluated, and further refinement conducted if necessary.

3. Results

A total of 605 collegiate student-athletes and competitive dancers completed the survey; however, 7 were removed because they did not provide responses for more than 10% of the items on the PMHI-19. Skewness and kurtosis values were all in acceptable ranges to support data normality; however, a total of 42 cases were identified as univariate outliers and 53 were identified as multivariate outliers and were subsequently removed. Thus, responses from 503 participants were retained for analysis and criteria to support data normality were met. Respondents were aged 18–32 (mean age = 19.85 ± 1.58) with females accounting for 80.3% (n = 404) and males accounting for 19.7% (n = 99) of the sample. A further breakdown of demographics is reported in

Table 1. Demographic information.

Characteristics	N	%
Sex
Male	99	19.7
Female	404	80.3
Ethnicity
Caucasian	391	77.7
African	61	12.1
American	46	9.1
Hispanic	17	3.4
Asian	7	1.4
Athlete status
Student-athlete	219	43.5
Dancer	284	56.5
Injury status
Healthy	372	74.0
Acute injury	19	3.8
Sub-acute injury	24	4.8
Persistent injury	82	16.3

.

3.1. Confirmatory Factor Analysis

The CFA of the PMHI-19 did not meet recommended model fit indices (CFI = 0.941; TLI = 0.926; IFI = 0.941; RMSEA = 0.059; Figure 1). Item-factor loadings were statistically significant (p ≤ 0.001) and ranged from 0.48 to 0.98 with latent factor correlations ranging from 0.08 to 0.57 (Figure 1). Modification indices indicated cross-loadings for items 3, 7, and 13 on multiple dimensions.

3.2. Exploratory Factor Analysis

Initial EFA of the PMHI-19 using the full sample extracted five factors that accounted for 62.81% of the variance. A total of eight items were removed due to low loadings, high inter-item correlations, and inflated Cronbach’s alpha levels. The refined four-factor PMHI (PMHI-11) accounted for 74.09% of the variance with all factor loadings ≥0.440 (

Table 2. Exploratory factor analysis of the refined four-factor PMHI-11.

Item	1	2	3	4
PMHI_24R	0.989
PMHI_22R	0.517
PMHI_4		0.908
PMHI_16		0.894
PMHI_5		0.760
PMHI_10			0.993
PMHI_9			0.751
PMHI_7			0.453
PMHI_20				0.932
PMHI_21				0.711
PMHI_19				0.515
Eigenvalue	3.20	2.24	1.54	1.13
Cronbach’s alpha	0.665	0.890	0.766	0.744
Omega	-	0.894	0.821	0.771

). Factor 1 contained three items from the original emotional support factor (α = 0.77), factor 2 contained three items from the original spirituality factor (α = 0.89), factor 3 contained three items from the original personal growth factor (α = 0.78), and factor 4 contained two items from the global affect factor (α = 0.62). Because Cronbach’s alpha for factor 4 was not within the recommended range (i.e., <0.69) and only contained two items, a refined 9-item version was also identified, with removal of the two items of the fourth factor (

Table 3. Exploratory factor analysis of the three-factor PMHI-9.

Item	1	2	3
PMHI_4	0.909
PMHI_16	0.895
PMHI_15	0.760
PMHI_10		0.939
PMHI_9		0.804
PMHI_7		0.469
PMHI_20			0.933
PMHI_21			0.707
PMHI_19			0.522
Eigenvalue	2.90	2.21	1.50
Cronbach’s alpha	0.890	0.766	0.744
Omega	0.894	0.821	0.771

); however, both the PMHI-11 and PMHI-9 were tested in a covariance model.

3.3. Covariance Models

3.3.1. PMHI-11

The covariance model of the PMHI-11 met all recommended model fit indices (CFI = 0.968; TLI = 0.954; IFI = 0.969; RMSEA = 0.060; Figure 2). Item-factor loadings were statistically significant (p ≤ 0.001) and ranged from 0.40 to 0.96 with latent factor correlations ranging from 0.06 to 0.29 (Figure 2). Modification indices indicated a cross-loading on item 3 with factor “Personal Growth” with the MI > 10 (Byrne, 2016). Scores on the PMHI-11 had a strong correlation with scores on the PMHI-19 (r = 0.95).

3.3.2. PMHI-9

The covariance model of the PMHI-9 met all recommended model fit indices (CFI = 0.968; TLI = 0.952; IFI = 0.969; RMSEA = 0.073; Figure 2). Item-factor loadings were statistically significant (p ≤ 0.001) and ranged from 0.40 to 0.98, with latent factor correlations ranging from 0.06 to 0.28 (Figure 2). Modification indices indicated a cross-loading on item 3 with factor “Personal Growth” with the MI > 10 (Byrne, 2016). The PMHI-9 scores had a strong correlation with the PMHI-19 scores (r = 0.92).

4. Discussion

Collegiate student-athletes and competitive dancers encounter stressors that may lead to involvement in unhealthy and precarious behaviors (Knettel et al., 2023; Yusko et al., 2008). These resultant behaviors may contribute to the deterioration of their mental health, manifested through the development of mental disorders, (Donohue et al., 2018; Vani et al., 2021) and physical injuries (Brewer & Redmond, 2017). Therefore, it is critical to understand and assess mental health in this sub-population. Thus, our study objective was to assess the psychometric integrity of the PMHI-19 in collegiate student-athletes and competitive dancers in the United States so providers can accurately assess and treat student-athletes accordingly. The CFA of the PMHI-19 indicated the instrument was not psychometrically sound in our target population. EFAs and subsequent covariance model testing identified two viable models: a 4-factor, 11-item model (PMHI-11); and a 3-factor, 9-item model (i.e., PMHI-9).

Our EFA and covariance model analysis results preliminarily support the use of the PMHI-11 and PMHI-9 as more suitable options for a collegiate, athletic population. The PMHI-11 and the PMHI-9 also provide further benefit due to reduced length, thus lessening response burden and enhancing practicality for clinical use. The shortened scales save time for both patients and healthcare professionals, and maintain the integrity of patient interaction while efficiently identifying potential mental health issues in health examinations. The PMH-11 and PMH-9 accomplish this, as evidenced by the high correlation with the PMHI-19.

The PMHI-19 was originally designed for the adult population of Singaporean residents (Vaingankar et al., 2014). Establishing instrument reliability and validity is an ongoing and sample-dependent process (Byrne, 2016; Kline, 2016); our study emphasizes this dependence by producing a distinct structure for our sample of US student-athletes and competitive dancers. While the brevity of a short form (e.g., PMHI-11 and PMHI-9) brings advantages, it is crucial not to compromise the theoretical integrity of the instrument. The PMHI-19—operationalized to encompass general coping skills, emotional support, spirituality, interpersonal skills, personal growth and autonomy, and global affect—upholds the comprehensive nature of the original 47-item instrument (Vaingankar et al., 2020). The original 47-item instrument was shortened, but still included all six subfactors. However, the reduced PMHI-19 had poor internal consistency in at least one of the factors; poor internal consistency likely indicates item redundancy, construct underrepresentation, reduced construct precision, or parallel items (Pesudovs et al., 2007). Further, the even more condensed 6-item rapid form (Vaingankar et al., 2020) contained only a unidimensional structure. In contrast, the PMHI-11 preserves four essential subfactors—spirituality, personal growth and autonomy, emotional support, and global affect—while demonstrating good internal consistency for three factors. In contrast, the PMHI-9 preserves three essential subfactors—spirituality, personal growth and autonomy, and emotional support—with good internal consistency for all factors.

The PMHI-9 and PMHI-11 present the inherent advantage of being more concise than the PMHI-19, alleviating response fatigue among individuals. While the PMHI-11 global affect factor did not meet internal consistency recommendations; this may be in part because the items, though related (i.e., they are emotions), are measuring distinct concepts and are heterogenous enough for respondents to respond less consistently producing a low Cronbach’s alpha. Furthermore, we retained two items in the factor which only captures a fraction of the multifaceted experience of emotion. Emotions are a complex phenomenon, which are experienced subjectively and capable of influencing physiology and behavior (Bailen et al., 2019). While universal similarities exist in how emotions are felt, individual experiences vary in terms of intensity, pleasantness, and frequency (Bailen et al., 2019). So, although our results found that the PMHI-9 model met all recommended analysis indices compared to the PMHI-11, we opted to retain the fourth factor because we felt the PMHI-9 may lose some of its theoretical integrity by only having three of the six original factors. Therefore, we recognize that in certain situations, it may be advantageous to consider using the PMHI-11, which maintains the global affect subfactor.

Global affect assesses positive mood, encompassing aspects such as relaxation, happiness, and enthusiasm; all significant contributors to overall mental health (Vaingankar et al., 2014). However, retaining global affect within the PMHI-11 comes with a potential downside: Cronbach’s alpha falls below the recommended range, suggesting less than desirable internal consistency (Lance et al., 2006). Therefore, the frequency of two emotions experienced (i.e., happiness and excitement) will be captured, but may not provide a consistent, comprehensive understanding of the breadth and depth of human emotional experience. Moreover, this factor is evaluated using only two items, and at least three items are recommended to adequately represent a factor (Kline, 2016). While including global affect is valuable for preserving more elements of the original PMHI-19, the limited number of items assessing it should be considered when interpreting results. Future research may explore ways to refine the assessment of global affect within the PMHI-11 (e.g., develop new items to measure the construct) or weigh the advantages and limitations of its inclusion in this shorter instrument.

Future Research and Limitations

While the current study identified two acceptable factor structures derived from the PMHI-19 in a new sample (e.g., collegiate athletes), the sample largely consisted of female (80.4%) and White (77.9%) individuals. Future research needs to assess the PMHI-9 and PMHI-11 in a more diverse sample to support use of these scales in those populations. Further, responding to all 19 items could influence how the participants responded to the identified 9 and 11 items (Casanova et al., 2021). Thus, future research should have participants only respond to the 9- or 11-item versions of the scale. Additionally, future research could consider using the PMHI-9 and 11 for evaluating an alternate demographic, such as high school student-athletes or non-student-athletes. Ongoing research can also include analyses aimed at confirming the psychometric integrity, such as multi-group and longitudinal invariance, and establishing further measurement properties (e.g., reliability, minimal clinically important differences, etc.). Determining additional measurement properties of the proposed scales would confirm that the scales can be used to assess group differences or changes over time, such as examining between-group (e.g., sex, athletic participation status, etc.) differences, and how those differences should be interpreted, which is valuable in practice and research.

5. Conclusions

The PMHI-19 did not meet model fit indices in our sample of collegiate student-athletes. Two refined models, the PMHI-11 and PMHI-9, were identified and met model fit indices recommendations. The refined scales have increased practical advantages, such as reduced response burden. While the PMHI-11 retained the global affect subfactor, caution is warranted due to its lower internal consistency and limited item representation. Conversely, the PMHI-9 offers a more streamlined approach but risks losing theoretical integrity. Further research should assess the PMHI-9 and PMHI-11 in a more diverse sample, as well as conduct further measurement testing to guide use of the scales.