Development and Validation of a Novel Instrument to Measure Campus Climate in College Students

Abstract

There is increasing interest in perceived campus climate in the postsecondary education literature. However, conceptualizations and measurements of this construct have lacked reliability, statistical rigor, and validity. This study aimed to develop and validate a novel instrument to measure perceived campus climate. Across two studies with more than 1500 participants, we developed a 46-item novel instrument. This instrument has five factors: University Responsiveness (10 items), Community and Connection (10 items), Equity and Belonging (15 items), Campus Culture and Student Experience (8 items), and Student Satisfaction (3 items). We evaluated the psychometric properties of the novel instrument using exploratory factor analysis, confirmatory factor analysis, hierarchical regression, paired-samples t-tests, correlation, and ANOVA. Overall, the results suggest that the novel instrument is reliable, valid, and provides additional predictive power compared to previous iterations of campus climate instruments.

1. Development and Validation of an Instrument to Measure Campus Climate in College Students

As universities accept a wide variety of students onto their campuses, they must be aware of student needs for welcoming environments, which involves asking current students what it is like to be a student on their campus. Previous research has consistently found that perceived campus climate significantly predicts psychological outcomes in college students, such as psychological functioning, stigma, anxiety, depression, help-seeking behaviors, trauma, and discrimination (e.g., Fink, 2014; Gannon et al., 2025; Huang & Fan, 2024; Koo, 2021; Shalka & Leal, 2022). Climate is tied to belonging and engagement, two key predictors of college student persistence and learning (Bentrim & Henning, 2023). Ensuring some of these positive psychological outcomes can be associated with costs. For example, a study examining spending trends at more than 1000 institutions found that the average spending per full-time equivalent (FTE) student was $3334 for student services (e.g., mental health services, career services) and $4198 for academic support services (Studocu, 2025). These figures represent increases in spending of approximately 89 percent and 83 percent in these areas, respectively, since 2010. Universities may also want to understand the perceived campus climate at their university to consider areas of improvement for the student experience. Despite the importance and practicality of campus climate, instruments to assess it remain limited in psychometric rigor and theoretical coherence. Most existing measures are adapted from K–12 school climate frameworks, which do not capture the complexity of collegiate social and academic environments.

2. Existing Approaches and Criticisms

Campus climate measurement and conceptualization have been criticized in the literature specifically for their lack of rigor and validity (Hart & Fellabaum, 2008; Lee et al., 2024). Variation in definitions and conceptualizations of campus climate in research limits the academic community’s ability to build off this research to develop targeted interventions and make meaningful comparisons across college institutions (Hart & Fellabaum, 2008). In social science research, many variables of interest are abstract constructs that require reliable and valid instruments to measure them. Correspondingly, the quality of social science research is directly impacted by the quality of our instruments (Kimberlin & Winterstein, 2008). The term ‘campus climate’ has been used to refer to different forms of campus climate such as campus climate for mental health (e.g., Shalka & Leal, 2022), campus racial climate (e.g., Rowan-Kenyon et al., 2022), digital campus climate (e.g., Pryor & Nachman, 2022), attitudes toward rape and sexual assault (e.g., Oberweis et al., 2023), differences between types of college campuses (e.g., Bonnesen et al., 2025), and college diversity climate (e.g., Timmerman & Volpe, 2023).

Some researchers have used the concept of campus climate to describe students’ thoughts about their university within a specific context. For example, the 2019 American College Health Association (ACHA) National College Health Assessment II (NCHA-II) purported to measure campus climate but instead measured students’ perceptions of their university’s climate surrounding health and well-being, as well as their perception of the prioritization of students’ health (Soederberg-Miller et al., 2023). Sample items include “I feel that students’ health and well-being is a priority of my college/university” and “I feel that the climate of my college/university encourages free and open discussion about students’ health and well-being.” Given this context, it requires attention to detail and specific search parameters to conduct literature searches regarding perceived campus climate. Without specifying the type of campus climate, it would be accurate for a researcher to posit that a positive perceived campus climate is associated with less food insecurity, more feelings of belongingness and safety, fewer experiences with microaggressions, and less psychological distress (Samek et al., 2024; Shalka & Leal, 2022; Soederberg-Miller et al., 2023; Timmerman & Volpe, 2023). Certain institutions have employed quantitative approaches in an attempt to measure campus climate using institution-specific instruments (e.g., Baylor University, Northeastern Illinois University). However, these instruments are not all public or generalizable across academic institutions; therefore, their scope and usefulness to education researchers and policymakers are limited. Furthermore, no researchers are currently examining the associations between a perceived campus climate construct and student outcomes.

Validity issues with existing measures abound. A review of institution-specific instruments with publicly available items suggests their construction makes it nearly impossible to use them in quantitative analyses (e.g., lacking a clear factor structure, inconsistent response anchors). Others have developed instruments, including the Diverse Learning Environments Survey (DLE; Hurtado & Guillermo-Wann, 2013), which measures aspects of campus climate; however, it appears to do so in a limited manner, relying heavily on information provided by the institution rather than student self-reports. While gathering institution-level data is not inherently an issue, failing to obtain student data prevents researchers from truly examining the perceptions, perspectives, and behaviors of students. These previous attempts to measure campus climate have been narrowly focused on the context of one campus, leaving a gap in available assessment instruments. The National Center on Safe Supportive Learning Environments (NCSSLE) maintains a compilation of reliable and valid measures of school climate (NCSSLE, n.d.). However, this compilation of instruments only lists one measure for postsecondary education, an instrument used in the Beginning Postsecondary Students (BPS; Wine et al., 2011) Longitudinal Study. Unfortunately for researchers, this instrument is a structured interview that does not lend itself to large-scale studies across universities. In addition to the differing forms of campus climate, Hart and Fellabaum’s (2008) content analysis of 118 campus climate studies found that 53.4% focused solely on faculty. Faculty perspectives should be considered in assessing the perceived campus climate; however, as a university’s mission is to educate students, student perspectives should also be included. Given the need for reliable and valid instruments and the substantial differences among campus climate measures, the current use of these measures does not meet the level of scientific rigor required.

3. The Current Study

We developed and validated a quantitative instrument to measure perceived campus climate for use in postsecondary education research, using a mixed-methods approach across two studies. The first study used a qualitative approach to develop a preliminary set of items and Exploratory Factor Analysis (EFA) to generate a proposed factor model and reduce items. The second study used Confirmatory Factor Analysis (CFA) and various statistical methods to confirm the factor structure and establish reliability and validity. The instrument development and validation processes were guided by the theoretical framework developed by Peterson and Spencer (1990). The authors defined organizational (i.e., campus) climate as the typical patterns of key aspects of organizational life or members’ perceptions and attitudes toward those aspects. They identified three central features of campus climate, including (1) a focus on current patterns of beliefs and behaviors, (2) a primary emphasis on common views of organizational phenomena that allow for longitudinal comparisons across groups, and (3) a pervasive, broad array of organizational phenomena.

Objectives

This study contributes to the higher education and psychological literature by:

• Developing a theoretically sound and psychometrically robust instrument to assess perceived campus climate;
• Providing empirical evidence for the concurrent and incremental validity of the campus climate instrument; and
• Offering a tool for researchers and practitioners to evaluate campus interventions and programs, and monitor institutional change over time.

4. Study 1

The purpose of Study 1 was to determine the latent constructs of the item pool generated by the research team.

4.1. Materials and Methods

Participants

The sample for the first administration of the survey protocol included 1027 college students aged 18 to 71 (M_age = 30.84, SD_age = 10.15). Approximately 54.82% (n = 563) of participants identified as women, 38.07% (n = 391) as men, 5.30% (n = 55) as non-binary, and 1.8% (n = 19) declined to answer. Regarding relationship status, 41.97% (n = 431) of participants identified as single, and 57.55% (n = 591) were in a relationship. Approximately 39.82% of the participants (n = 409) reported an annual income of less than $30,000. Approximately 64.07% (n = 658) of participants reported attending a public four-year college, while 17.23% (n = 177) reported attending a private four-year college. One-third of the participants (n = 441) were from the southeastern United States, specifically Arkansas, Louisiana, Mississippi, Alabama, North Carolina, Tennessee, Kentucky, West Virginia, Delaware, Maryland, Virginia, the District of Columbia, Florida, Georgia, and South Carolina.

4.2. Analyses

Study 1 identified a theoretical framework to guide the instrument development process and developed an initial overly inclusive item pool that subject-matter experts reviewed (Peterson & Spencer, 1990). A total of 204 items were generated by the research team to represent the proposed factors and domains from the Peterson and Spencer work (1990). We used ChatGPT-4 omni to evaluate the grammar, syntax, and appropriateness of the proposed items. ChatGPT did not generate, revise, or propose potential items for the instrument. ChatGPT, along with Grammarly, evaluated the grammar of the items, including an estimated reading level for each item. Three independent experts in postsecondary education research were identified and recruited by the study team. The item pool and theoretical framework were sent to the experts and they were independently asked to identify items that did not reflect that framework, were inappropriately worded, were loaded, double-barreled, or reductive. These experts were given wide latitude to evaluate the items based on their experience, the literature, and the theoretical framework provided. An item was removed from the item pool if multiple experts recommended doing so. Any expert could recommend revising an item, and it would be considered for revision. The primary role of the subject matter experts was to provide an external review of items before any statistical analyses were conducted. This external review served as a reliability check of the items compared to the theory, ensuring they aligned with the operational definition of perceived campus climate and the theoretical framework. The first author oversaw the final decisions regarding the item pool. In total, the experts identified 140 items for removal from the item pool, leaving 64 for the EFA.

Analyses were conducted in Stata statistical packages 17 and 19, with guidance from Watkins (2021). We selected an exploratory factor analysis (EFA) to identify the latent constructs of the item pool and to address uncertainty about the structure of our campus climate scale. The method of analysis was iterated principal factor analysis and Promax rotation with Kaiser normalization. The authors used Promax rotation because the items were conceptually related and because it allows for correlated factors. The cutoff for retaining factors was based on a pattern coefficient of 0.4. The appropriateness of the data for EFA was verified through Bartlett’s Test of Sphericity and the Kaiser–Meyer–Olkin test. Parallel analysis and a visual scree test were used to determine the number of factors to retain. During the EFA process, item numbers shifted and changed as the instrument evolved. See the supplemental materials to view each iteration of the instrument labeled by study.

4.3. Results

The 64 campus climate items were hypothesized to reflect the theoretical understanding of academic climate outlined in Peterson and Spencer (1990). Before conducting the analysis, we analyzed the appropriateness of the data using Bartlett’s test of sphericity, which yielded a χ² of 48,250.96 (df = 2016, p < 0.001). Additionally, a KMO test of sampling adequacy of 0.98 demonstrates the appropriateness of the data for exploratory factor analysis. The number of factors for extraction is based on a Parallel analysis of eigenvalues ≥ 1 and a visual scree test demonstrating that five factors should be retained. After conducting an iterated principal factor (IPF) extraction and a Promax rotation with Kaiser normalization, eleven items (2, 5, 13, 31, 32, 33, 36, 38, 74, 76, 79) were dropped because their loadings were below the cutoff of 0.4. Sample removed items include “I feel race is discussed in a thoughtful way,” “religious diversity is respected on campus,” “the university’s goals align with student needs,” “social events on campus are welcoming to all students,” and “I feel proud to be a student at this institution.” The EFA was repeated with the removed items for a final five-factor model with 53 items: with Factor 1 consisting of 10 items and accounting for 34% of variance, Factor 2 consisting of 16 items and accounting for 33% of variance, Factor 3 consisting of 15 items and accounting for 31% of variance, Factor 4 consisting of 8 items and accounting for 28% of variance, and Factor 5 consisting of 4 items and accounting for 13% of variance. Communalities ranged from 0.29 to 0.81 (see

Table 1. Items, Factor Loadings, and Communalities.

	Factor Loading (Assigned Factor Bolded)					Communalities
	1	2	3	4	5
Factor 1
1. The institution is proactive in addressing student concerns	0.68	0.06	0.06	−0.01	0.03	0.56
3. I feel safe expressing my ideas on campus	0.66	0.08	−0.06	−0.03	−0.04	0.46
4. The university actively promotes student well-being	0.61	0.03	0.22	−0.03	−0.09	0.66
23. I trust the campus administration to act in students’ best interests	0.71	0.13	−0.03	0.05	0.02	0.65
24. Student voices are included in decision-making	0.73	0.10	0.14	−0.10	0.01	0.70
25. Leadership responds quickly to student concerns	0.75	0.03	−0.01	0.07	0.02	0.63
26. The university’s goals align with student needs	0.72	0.08	0.20	−0.09	0.05	0.72
27. Administrators engage regularly with students	0.58	0.00	0.04	0.18	−0.01	0.55
28. I feel informed about what is happening on campus	0.65	0.12	0.14	−0.05	0.02	0.62
29. Leadership shows concern for student well-being	0.70	0.18	0.02	−0.09	−0.02	0.66
Factor 2
2. My cultural background is respected on campus	0.15	0.61	0.08	−0.10	−0.10	0.59
9. The university actively promotes equity and inclusion	0.21	0.54	0.08	−0.08	−0.08	0.55
10. People from all backgrounds feel valued here	0.31	0.55	−0.08	−0.04	−0.06	0.56
11. The institution takes action against discrimination	−0.05	0.59	0.00	0.19	0.01	0.45
12. The university recruits and supports diverse faculty and staff	0.14	0.63	−0.07	0.12	0.03	0.55
13. I feel that institutional policies promote fairness	0.32	0.65	−0.14	0.01	0.03	0.65
14. There are spaces that support underrepresented groups	0.02	0.65	−0.10	0.21	0.06	0.50
15. The institution recognizes the needs of first-generation students	−0.03	0.62	−0.01	0.16	0.04	0.45
16. Multilingual support is available when needed	0.02	0.69	0.08	0.02	0.02	0.57
17. The university is responsive to social issues	0.03	0.70	0.03	0.13	0.05	0.63
18. Students are taught to value cultural differences	0.20	0.56	0.00	0.08	0.01	0.58
19. Administrators respond effectively to bias incidents	0.08	0.59	0.02	0.03	−0.12	0.54
20. I feel race is discussed in a thoughtful way	0.05	0.59	0.05	0.07	−0.02	0.50
21. Religious diversity is respected on campus	−0.20	0.76	0.16	−0.09	−0.02	0.49
22.My identities are affirmed on campus	−0.02	0.73	0.12	−0.15	−0.05	0.56
38. I am involved in activities that are meaningful to me	0.08	0.59	0.13	0.07	0.07	0.57
Factor 3
30. Student clubs and organizations are welcoming	0.19	0.05	0.45	0.20	−0.11	0.68
40. The university affirms intersectional identities	0.12	0.12	0.70	−0.01	0.03	0.70
41. I feel proud of the campus culture	0.07	0.04	0.67	0.15	−0.02	0.72
42. Students are treated as partners in campus life	0.14	0.07	0.75	−0.07	0.00	0.73
43. I have found one or more communities or groups where I feel I belong at this school	0.07	0.03	0.64	0.18	0.00	0.70
44. I am satisfied with my experience at this institution	0.07	0.01	0.82	0.04	0.02	0.79
45. I would recommend this university to others	−0.01	0.02	0.90	−0.03	0.00	0.79
46. My overall impression of the campus environment is positive	−0.02	0.04	0.77	0.09	−0.04	0.73
Factor 4
5. I feel comfortable interacting with people from different backgrounds on campus	−0.06	0.00	0.05	0.70	−0.03	0.49
6. Students celebrate one another’s accomplishments	0.23	0.00	−0.08	0.59	−0.10	0.53
7. Campus organizations are inclusive	0.32	0.03	−0.03	0.50	−0.04	0.57
8. Faculty members are open to student input	0.11	0.08	0.09	0.49	−0.07	0.49
31. I can express my identity freely on campus	0.23	0.10	0.07	0.47	0.05	0.53
32. I feel like a value member of the campus community	0.18	−0.02	0.04	0.65	0.01	0.61
33. I feel connected to campus traditions	0.09	−0.02	0.23	0.61	0.04	0.64
34. My identity is affirmed by faculty and staff	0.30	0.03	0.02	0.51	−0.01	0.58
35. Campus events help me meet new people	0.14	0.19	0.10	0.41	0.02	0.49
36. Students are encouraged to start new organizations	0.08	−0.08	0.19	0.64	0.00	0.60
37. Campus events reflect a wide variety of interests	0.16	0.19	0.10	0.44	0.03	0.55
39. The university supports students’ personal interests	−0.05	−0.01	0.22	0.70	0.01	0.65
51. I have withdrawn from activities on campus due to emotional exhaustion	−0.16	0.04	−0.06	0.68	−0.02	0.36
52. I have withdrawn from activities on campus due to feeling anxious	−0.08	0.04	−0.13	0.65	0.03	0.31
53. I feel comfortable disagreeing politically with my peers on campus	−0.21	0.09	−0.03	0.61	0.04	0.29
Factor 5
47. The university meets my expectations	0.02	−0.08	−0.05	0.10	0.59	0.37
48. The campus is a good fit for me	0.11	−0.04	−0.06	−0.04	0.89	0.81
49. I am glad I chose this university	−0.04	0.00	0.08	−0.08	0.83	0.70
50. I avoid certain areas of campus where I feel unsafe	−0.13	0.08	0.00	0.01	0.64	0.46

Note. Bold indicates which factor the item loaded onto.

).

Table 2. Correlation Matrix and Reliability Estimates of Rotated Common Factors.

	1	2	3	4	5
Factor 1	(0.94)
Factor 2	0.67	(0.95)
Factor 3	0.66	0.57	(0.93)
Factor 4	0.57	0.51	0.59	(0.95)
Factor 5	−0.41	−0.37	−0.42	−0.30	(0.84)

contains Cronbach’s alpha and factor correlations.

In summation, through factor analysis, the number of items was reduced, and a proposed factor structure emerged. The proposed factor structure appeared valid and was supported by multiple analyses. The next phase of the research was designed to test the factor structure, evaluate the reliability and validity of the instrument, and to evaluate if there were any differences in perceived campus climate based on student characteristics (e.g., racial identity, gender identity).

5. Study 2

Study 2 assessed the revised factor structure, test-retest reliability, construct validity, divergent validity, and incremental validity of the new campus climate instrument. Study 2 comprised two separate data collection points separated by at least two weeks.

5.1. Materials and Methods

Participants

We first administered the survey to 487 college students aged 18 to 67 (M_age = 29.92, SD_age = 9.74) via Prolific. Approximately 56% (n = 280) of the participants identified as women, 51% identified as White (n = 287), and 36% (n = 179) identified as men. Regarding relationship status, 48% (n = 247) of the participants identified as single, and 48% (n = 239) of the participants were in a relationship. Approximately 47.03% of the participants (n = 230) reported an annual income of less than $30,000. Approximately 74% (n = 369) of participants reported attending a public four-year college, while 16% (n = 80) reported attending a private four-year college.

The sample for the second administration of the survey protocol consisted of 247 college students aged 18 to 68 (M_age = 30.28, SD_age = 10.01), also recruited via Prolific. Approximately 54% (n = 133) of the participants identified as women, and 41% (n = 102) identified as men. Regarding relationship status, 52% (n = 129) of participants identified as single, and 47% (n = 117) were in a relationship. Approximately 40% of the participants (n = 99) reported an annual income of less than $30,000 and 58% (n = 144) identified as White. Approximately 77% (n = 189) of participants reported attending a public four-year college, while 15% (n = 34) reported attending a private four-year college. These two analytic samples were relatively representative of the national demographics for college students (Nam & Bryant, 2025). According to recent trends, slightly more than half of college students identify as women, and approximately 40% identify as non-Hispanic White. These proportions are not exactly reflected in the collected samples, but they are close enough as to not raise concerns about external validity (i.e., generalizability; e.g., Bornstein et al., 2013).

Prolific is an online research platform that matches researchers with eligible participants, serving as an alternative to the Amazon Mechanical Turk (MTurk) platform (Peer et al., 2017). Peer and colleagues found that while Prolific’s response rate was slightly lower than MTurk’s, it was faster than that of a university sample. Additionally, their research found that Prolific samples replicated existing results and delivered higher-quality data than either a university sample or an MTurk sample. Prolific offers multiple subject pool management features for researchers and participants, including minimum fixed payments for participants based on anticipated time requirements, the ability for researchers to reject submissions, the ability for participants to allow submissions to time out (i.e., not complete a survey/instrument), and the ability for participants to return a submission (i.e., withdraw consent and participation) at any time they choose (Palan & Schitter, 2018). Previous research found that data from Prolific-recruited participants were of higher quality than those from participants recruited through MTurk, Qualtrics, or Sona (Douglas et al., 2023). These participants were more likely to pass attention checks, follow instructions, have a unique IP address, provide meaningful answers, and remember previously presented information. Previous research has also found that Prolific is a psychometrically sound platform for conducting psychological and substance misuse research (Stanton et al., 2022). To meet the inclusion criteria, participants needed to be current undergraduate students, adults, located in the United States, and speak English fluently. There were no additional restrictions on age, gender identity, racial identity, major, or any other participant characteristic.

5.2. Measures

5.2.1. Campus Climate

Novel Campus Climate Instrument. The novel Perceived Campus Climate Scale, developed in Study 1, is a 46-item instrument designed to assess the various domains of perceived campus climate. The instrument has five subscales including University Responsiveness (10 items; e.g., “Leadership shows concern for student well-being”; α₁ = 0.934, α₂ = 0.945), Community and Connection (10 items; e.g., “I feel like a valued member of the campus community”; α₁ = 0.902, α₂ = 0.925), Equity and Belonging (15 items; e.g., “My identities are supported on campus”; α₁ = 0.945, α₂ = 0.950), Campus Culture and Student Experiences (8 items; e.g., “My overall impression of the campus environment is positive”; α₁ = 0.910, α₂ = 0.924), and Student Satisfaction (3 items; e.g., “I am glad I chose this university”; α₁ = 0.908, α₂ = 0.900). These items use a five-point Likert scale, with responses ranging from 1 (strongly disagree) to 5 (strongly agree). Responses are summed with higher scores indicating a more positive perceived campus climate. This instrument had total score internal reliability coefficients of 0.980 and 0.983 in the first and second administrations of the protocol, respectively. The instrument’s subscales’ reliability estimates were more than acceptable in both administrations. For a complete list of the items, please review the supplemental files.

American College Health Assessment Campus Climate Instrument. We also used the American College Health Association—National College Health Assessment (ACHA-NCHA; Lederer & Hoban, 2021) campus climate instrument, a four-item measure. These items use a six-point Likert scale, ranging from 1 (strongly disagree) to 6 (strongly agree). Responses are averaged, with higher scores indicative of a more positive perceived campus climate. Sample items include “We are a college/university where we look out for each other” and “I feel that I belong at my college/university.” This instrument had internal reliability coefficients of 0.906 and 0.919 in the first and second administrations of the protocol, respectively. Previous research with this instrument demonstrates acceptable internal reliability (e.g., Gilchrist, 2023).

Healthy Minds Study Campus Climate Instrument. The six-item Healthy Minds Survey campus climate instrument evaluates respondents’ perceptions of their campus climate (Healthy Minds Network, 2024). These items use a six-point Likert scale, ranging from 1 (strongly disagree) to 6 (strongly agree). Responses are averaged, with scores ranging from 1 to 6; higher scores indicate a more positive perceived campus climate. Sample items include “I see myself as a part of the campus community” and “My institution cares about my health and wellbeing.” This instrument had internal reliability coefficients of 0.837 and 0.825 in the first and second administrations of the protocol, respectively. Previous research with this instrument demonstrates acceptable internal reliability (e.g., Gannon et al., 2025; Peoples et al., 2024; Shalka & Leal, 2022).

5.2.2. Academic Locus of Control

We used the 21-item Academic Locus of Control Scale for College Students (ALC; Curtis & Trice, 2013) as a measure of divergent validity. This instrument has 21 true/false items with a minimum score of 0 and a maximum score of 21. Higher scores are indicative of an external locus of academic control (i.e., belief that outside forces control one’s academic outcomes), while lower scores are indicative of an internal locus of academic control (i.e., belief that one’s own actions and behaviors control academic outcomes). Sample items include “I sometimes feel that there is nothing I can do to improve my situation” and “I never feel really hopeless—there is always something I can do to improve my situation”. This instrument had internal reliability coefficients of 0.769 and 0.770 in the first and second administrations of the protocol, respectively. Previous research with this instrument demonstrates acceptable internal reliability (e.g., Castillo et al., 2025; Villa & Sebastian, 2021).

5.2.3. Depression, Anxiety, and Stress

We measured depression, anxiety, and stress using the 21-item short form of the Depression, Anxiety, and Stress Scale (DASS-21; Lovibond & Lovibond, 1995). Each of the three subscales, one for each construct, contains seven items rated on a four-point frequency Likert scale ranging from 0 (Did not apply to me at all—NEVER) to 3 (Applied to me very much, or most of the time—ALMOST ALWAYS). Sample items for the depression subscale include “I felt that life was meaningless” and “I felt I wasn’t worth much as a person.” Sample items for the anxiety subscale include “I felt scared without any good reason” and “I felt I was close to panic.” Sample items for the stress subscale include “I found it difficult to relax” and “I was intolerant of anything that kept me from getting on with what I was doing.” The scores for each subscale range from 0 to 42, with raw scores from 0 to 21 multiplied by two. Higher scores indicate more endorsements of depressive, anxious symptoms, or stress (Lovibond & Lovibond, 1995). Previous research has consistently supported the reliability and validity of the DASS-21 in college student samples (e.g., Becker et al., 2018; Corona et al., 2017; Rose et al., 2025). In this sample, the reliability coefficients for the depression (α₁ = 0.95; α₂ = 0.95), anxiety (α₁ = 0.86; α₂ = 0.89), and stress (α₁ = 0.90; α₂ = 0.90) subscales were acceptable.

6. Results

6.1. Demographic Differences

To evaluate potential differences in campus climate perceptions by participant demographic characteristics, we used analysis of variance (ANOVA). There were no statistically significant differences based on geographic region (F(5, 214) = 1.739, p = 0.127, η² = 0.039), racial identity (F(5, 233) = 1.673, p = 0.142, η² = 0.035), gender identity (F(4, 444) = 1.928, p = 0.105, η² = 0.017), income (F(6, 442) = 1.460, p = 0.190, η² = 0.019), sexual orientation (F(6, 443) = 0.725, p = 0.629, η² = 0.010), college type (F(2, 447) = 1.807, p = 0.165, η² = 0.008), or relationship status (F(5, 442) = 1.004, p = 0.412, η² = 0.011).

6.2. Reliability

To evaluate the temporal stability (i.e., reliability across time) of the Perceived Campus Climate Scale (PCCS), participants (n = 199) completed the measure twice, at least 14 days apart. Test–retest reliability was assessed using the Pearson correlation coefficient, a paired sample t-test, and the two-way random-effects intraclass correlation coefficient (ICC[2, 1]). The ICC evaluates absolute agreement between administrations. Results indicated strong test–retest reliability over the two-week span with a statistically significant strong positive correlation between scores at timepoint 1 and scores at timepoint 2 (r(199) = 0.857, p < 0.001; ICC(2,1) = 0.921), suggesting good reliability of the instrument (Koo & Li, 2016). Results of the paired samples t-test analysis indicated a statistically significant difference in mean scores between timepoint 1 and timepoint 2; however, the results were practically insignificant (t(199) = −2.424, MD = −2.618, SD = 16.471, SE_mean = 1.168, p = 0.026, d = −0.159).

6.3. Concurrent Validity

To examine the concurrent validity of the Novel Campus Climate Scale, correlations were calculated between the NCCS total score and two established measures of perceived campus climate: the HMS Climate Scale and the NCHA-ACHA Climate Scale. All measures were administered concurrently to assess whether the Novel Campus Climate Scale demonstrated theoretically consistent associations with established indicators of campus climate perceptions. Pearson correlations were calculated to evaluate the strength and direction of the relationships between the Novel Campus Climate Scale and the criterion measures. Higher scores on the Novel Campus Climate Measure were expected to be positively associated with higher scores on both established measures, reflecting similar underlying constructs. As predicted, the Novel Campus Climate Scale was strongly and positively correlated with the HMS at Timepoint 1 (r(449) = 0.561, p < 0.001) and Timepoint 2 (r(221) = 0.544, p < 0.001) as well as the ACHA at Timepoint 1 (r(450) = 0.884, p < 0.001) and Timepoint 2 (r(219) = 0.881, p < 0.001; see

Table 3. Correlations between Novel Campus Climate Instrument and Previously Validated Instruments.

	1	2	3	4	5	6	M	SD
1. NCCS1	-						177.13	30.5
2. HMSC1	0.561 ***	-					23.20	4.0
3. ACHAC1	0.884 ***	0.594 ***	-				18.54	4.0
4. NCCS2	0.857 ***	0.545 ***	0.792 ***	-			180.20	30.4
5. HMSC2	0.428 ***	0.460 ***	0.478 ***	0.545 ***	-		23.46	4.3
6. ACHAC2	0.768 ***	0.556 ***	0.807 ***	0.881 ***	0.592 ***	-	18.83	3.8

Note. NCCS = novel campus climate scale; HMSC = healthy minds study climate measure; ACHAC = American college health assessment climate measure; *** = p < 0.001.

). These results indicate substantial theoretical overlap between the NCCS and established campus climate measures. Consistent with our hypotheses, strong positive correlations between the NCCS and both comparison measures provide evidence of concurrent validity. The NCCS appears to assess perceptions of campus climate in a manner that aligns closely with established instruments, while potentially capturing unique dimensions relevant to contemporary campus experiences.

6.4. Incremental Validity

Hierarchical multiple regression analyses were conducted to examine whether the Novel Campus Climate Scale explains unique variance beyond existing campus climate instruments. In each regression model, covariates were entered in Step 1 as predictors of mental health (i.e., depression, anxiety, and stress) and academic locus of control, followed by established measures of campus climate in Step 2, and finally the Novel Campus Climate Scale total score in Step 3. Age, gender identity, institution type, and sexual orientation were the covariates (i.e., control variables) in all models.

Regarding the incremental validity of predicting depression, the results indicated that in Step 2 of the analysis, existing instruments accounted for a significant portion of the variance in depression (R² = 0.1778, F(6, 231) = 11.09, p < 0.001; see

Table 4. Hierarchical Regression Incremental Validity on Depression.

	B	SE	t	ES	R2	ΔR2
Step 1					0.10
Age	−0.232 ***	0.056	−4.15	−0.198
Gender	0.150	0.770	0.20	0.014
Orien	2.229 **	0.741	3.01	0.237
CollType	−1.133	1.440	−0.79	−0.046
Step 2					0.18	0.08 ***
Age	−0.160 **	0.060	−2.68	−0.137
Gender	0.247	0.775	0.32	0.023
Orien	2.280 **	0.717	3.18	0.243
CollType	−0.911	1.291	−0.71	−0.037
NCHA	−1.078 ***	0.219	−4.93	−0.353
HMS	0.497 *	0.200	2.49	0.179
Step 3					0.23	0.05 ***
Age	−0.132	0.070	−1.89	−0.106
Gender	0.558	0.834	0.67	0.051
Orien	2.636 ***	0.724	3.64	0.278
CollType	−1.313	1.425	−0.92	−0.049
NCHA	−0.342	0.374	−0.92	−0.108
HMS	0.556 **	0.205	2.72	0.191
NCCS	−0.131 **	0.049	−2.65	−0.331

Note. Orien = sexual orientation; CollType = university/college type; NCCS = novel campus climate scale; HMS = healthy minds study climate measure; NCHA = American college health assessment climate measure; *** = p < 0.001; ** = p < 0.01; * = p < 0.05.

). Adding the Novel Campus Climate Scale in Step 3 significantly improved model fit (ΔR² = 0.052, F(7, 206) = 12.15, p < 0.001). In the final model, the Novel Campus Climate Scale remained a statistically significant predictor of depression (B = −0.131, ES = −0.331, p = 0.009), suggesting that it contributes to unique variance beyond existing measures.

Regarding incremental validity of predicting anxiety, results indicated that in Step 2 of the analysis, existing instruments accounted for a significant portion of variance in anxiety (R² = 0.1760, F(6, 232) = 10.16, p < 0.001; see

Table 5. Hierarchical Regression incremental validity on Anxiety.

	B	SE	t	ES	R2	ΔR2
Step 1					0.07
Age	−0.196 ***	0.046	−4.28	−0.206
Gender	0.664	0.591	1.12	0.076
Orien	0.958	0.517	1.85	0.126
CollType	−0.785	1.266	−0.62	−0.039
Step 2					0.18	0.11 ***
Age	−0.117 *	0.048	−2.43	−0.125
Gender	0.632	0.577	1.09	0.074
Orien	0.976 *	0.470	2.08	0.130
CollType	−0.583	1.105	−0.53	−0.029
NCHA	−0.942 ***	0.191	−4.94	−0.386
HMS	0.806 ***	0.190	4.23	0.364
Step 3					0.21	0.04 *
Age	−0.110	0.056	−1.96	−0.115
Gender	0.733	0.608	1.21	0.086
Orien	1.254 **	0.481	2.61	0.171
CollType	−0.468	1.206	−0.39	−0.022
NCHA	−0.150	0.324	−0.46	−0.061
HMS	0.806 ***	0.188	4.29	0.358
NCCS	−0.118 **	0.041	−2.86	−0.385

Note. Orien = sexual orientation; CollType = university/college type; NCCS = novel campus climate scale; HMS = healthy minds study climate measure; NCHA = American college health assessment climate measure; *** = p < 0.001; ** = p < 0.01; * = p < 0.05.

). Adding the Novel Campus Climate Scale in Step 3 significantly improved model fit (ΔR² = 0.038, F(7, 206) = 10.84, p < 0.001). In the final model, the Novel Campus Climate Scale remained a statistically significant predictor of anxiety (B = −0.117, ES = −0.385, p = 0.005), suggesting that it contributes to unique variance beyond existing measures.

Regarding incremental validity of predicting stress, results indicated that in Step 2 of the analysis, existing instruments accounted for a significant portion of variance in stress (R² = 0.1979, F(6, 231) = 10.76, p < 0.001; see

Table 6. Hierarchical Regression incremental validity on Stress.

	B	SE	t	ES	R2	ΔR2
Step 1					0.09
Age	−0.226 ***	0.053	−4.23	−0.224
Gender	0.999	0.591	1.69	0.108
Orien	1.024	0.536	1.91	0.126
CollType	−1.471	1.366	−1.08	−0.070
Step 2					0.20	0.11 ***
Age	−0.144 **	0.056	−2.58	−0.144
Gender	1.023	0.575	1.78	0.110
Orien	1.093 *	0.484	2.26	0.135
CollType	−1.051	1.237	−0.85	−0.050
NCHA	−1.133 ***	0.197	−5.75	−0.430
HMS	0.690 ***	0.184	3.74	0.285
Step 3					0.23	0.04
Age	−0.141 *	0.062	−2.26	−0.136
Gender	1.122	0.606	1.85	0.122
Orien	1.394 **	0.497	2.80	0.175
CollType	−0.676	1.346	−0.50	−0.031
NCHA	−0.184	0.366	−0.50	−0.069
HMS	0.694 ***	0.181	3.84	0.280
NCCS	−0.134 **	0.045	−2.99	−0.404

Note. Orien = sexual orientation; CollType = university/college type; NCCS = novel campus climate scale; HMS = healthy minds study climate measure; NCHA = American college health assessment climate measure; *** = p < 0.001; ** = p < 0.01; * = p < 0.05.

). Adding the Novel Campus Climate Scale in Step 3 significantly improved model fit (ΔR² = 0.036, F(7, 205) = 10.49, p < 0.001). In the final model, the Novel Campus Climate Scale remained a statistically significant predictor of stress (B = −0.134, ES = −0.404, p = 0.003), suggesting that it contributes to unique variance beyond existing measures.

Regarding incremental validity of predicting academic locus of control, results indicated that in Step 2 of the analysis, existing instruments accounted for a significant portion of variance in academic locus of control (R² = 0.2260, F(6, 227) = 12.84, p < 0.001; see

Table 7. Hierarchical Regression incremental validity on Academic Locus of Control.

	B	SE	t	ES	R2	ΔR2
Step 1					0.08
Age	−0.096 ***	0.023	−4.20	−0.237
Gender	−0.232	0.257	−0.90	−0.063
Orien	0.448	0.260	1.72	0.140
CollType	0.181	0.554	0.33	0.022
Step 2					0.23	0.15 ***
Age	−0.072 **	0.022	−3.20	−0.178
Gender	−0.189	0.241	−0.79	−0.051
Orien	0.480 *	0.230	2.09	0.150
CollType	0.291	0.518	0.56	0.035
NCHA	−0.468 ***	0.086	−5.42	−0.445
HMS	0.107	0.078	1.38	0.113
Step 3					0.28	0.05 **
Age	−0.065 **	0.024	−2.69	−0.156
Gender	−0.193	0.251	−0.77	−0.052
Orien	0.580 *	0.235	2.47	0.181
CollType	0.146	0.558	0.26	0.016
NCHA	−0.068	0.127	−0.53	−0.063
HMS	0.085	0.077	1.10	0.086
NCCS	−0.057 ***	0.017	−3.29	−0.427

Note. Orien = sexual orientation; CollType = university/college type; NCCS = novel campus climate scale; HMSC = healthy minds study climate measure; ACHAC = American college health assessment climate measure; *** = p < 0.001; ** = p < 0.01; * = p < 0.05.

). Adding the Novel Campus Climate Scale in Step 3 significantly improved model fit (ΔR² = 0.050, F(7, 203) = 13.09, p < 0.001). In the final model, the Novel Campus Climate Scale remained a statistically significant predictor of academic locus of control (B = −0.057, ES = −0.427, p = 0.001), suggesting that it contributes to unique variance beyond existing measures.

These findings provide evidence of incremental validity for the Novel Campus Climate Scale. Even after accounting for variance explained by established campus climate measures and covariates, the Novel Campus Climate Scale accounted for an additional 5%, 4%, 4%, and 5% in the variance in depression, anxiety, stress, and academic locus of control, respectively. This suggests that the Novel Campus Climate Scale captures unique aspects of perceived campus climate that are not represented by previous instruments.

In summation, the novel perceived campus climate instrument is reliable and valid. The results of various analyses suggest that the instrument is stable across time, measures campus climate as purported, and explains additional variance in student mental health compared to older campus climate instruments. The ANOVA results suggest that there were no differences in campus climate perceptions based on student characteristics (e.g., gender identity, racial identity). These findings suggest that this novel instrument is psychometrically sound and useful for researchers, practitioners, policymakers, and stakeholders.

7. General Discussion

Campus climate is a statistically significant predictor of essential outcomes in college students such as psychological functioning, psychological distress, discrimination, engagement, and belonging (e.g., Bentrim & Henning, 2023; Fink, 2014; Gannon et al., 2025; Huang & Fan, 2024; Koo, 2021; Shalka & Leal, 2022). However, conceptualization and measurement of campus climate have been criticized for their lack of validity and rigor. In social science research, due to the complexity and abstract nature of constructs of interest, measurement requires valid and reliable instruments. Therefore, the quality of social science research is directly impacted by the quality of the instruments used. The present study aimed to develop and validate a novel instrument to measure college students’ perceptions of campus climate. The development and validation processes were guided by the theoretical framework developed by Peterson and Spencer (1990). The preliminary version of the questionnaire, after EFA, proposed five unnamed dimensions. After changes were made during the refinement process using another sample and CFA, the results confirmed the five-factor structure, which was found to be reliable and valid.

The instrument was internally consistent and consistent across time (i.e., test–retest reliability). There were strong, statistically significant, positive correlations between participants’ scores at Timepoint 1 and Timepoint 2. Regarding the concurrent validity of the new instrument, statistically significant positive correlations were observed between scores on the novel campus climate instrument and those on previously established instruments. Concerning incremental validity, the novel campus climate scale explained additional variance of depression, anxiety, stress, and academic locus of control. Even when accounting for prior campus climate instruments, the novel instrument adds to the construct’s predictive power. These findings suggest that the novel scale captures unique aspects of perceived campus climate that are not represented by previous instruments, without sacrificing predictive capacity. The findings in Study 2 mirror those of previous research, with a negative association between perceived campus climate and adverse psychological outcomes. However, a greater proportion of the variance in psychological distress is accounted for using the novel instrument.

As previously noted, instruments that purport to measure perceived campus climate have been criticized for their lack of validity and rigor (Hart & Fellabaum, 2008; Lee et al., 2024). Previously, campus climate has been used as an umbrella term describing campus mental health, campus racial climate, digital campus climate, attitudes toward sexual assault and rape, differences between types of college campuses, and the diversity climate of colleges (e.g., Shalka & Leal, 2022; Rowan-Kenyon et al., 2022; Pryor & Nachman, 2022; Oberweis et al., 2023; Bonnesen et al., 2025; Timmerman & Volpe, 2023). This variation in the conceptualization and operational definition of campus climate in tertiary education research limits researchers’ and practitioners’ ability to develop and evaluate targeted interventions and to make meaningful comparisons across college institutions (Hart & Fellabaum, 2008). Variation in definitions and conceptualizations of campus climate in research limits the academic community’s ability to build off this research to develop targeted interventions and make meaningful comparisons across college institutions (Hart & Fellabaum, 2008). This novel instrument addresses those concerns by conceptualizing campus climate using a well-researched and respected model (i.e., Peterson & Spencer, 1990) and using statistical methods to evaluate the reliability and validity of the instrument and its factor structure. To address the concern raised by Hart and Fellabaum (2008), we focused on students’ perceptions rather than those of faculty and staff. This instrument fills the void identified by Hart and Fellabaum (2008), who found that more than half of the instruments purporting to measure campus climate were focused on faculty perceptions.

8. Practical Implications

There are multiple opportunities for researchers and practitioners to utilize this instrument. For example, university stakeholders can evaluate their students’ perception of campus climate to inform policy decisions. For example, postsecondary education leaders could implement an action research approach to examine and improve campus climate, especially for marginalized students at their institution (Taylor et al., 2018). Action research approaches are an ideal approach to employ when institutions attempt to translate data into practice (Dowd et al., 2015). These approaches are collaborations between stakeholders, researchers, and policymakers to collect, analyze, and interpret data to create and implement solutions to internally identified issues (Dowd et al., 2015). This novel instrument is a valid and reliable method to collect quantitative data about students’ perceptions of their campus climate in multiple domains, which could inform interventions, improvements, and policies for postsecondary institutions. Postsecondary leaders and researchers could also use the transformational tapestry model to evaluate and improve their students’ perceptions of campus climate (Rankin & Reason, 2008). This model is a five-phase framework for campus climate assessment, planning, and intervention, designed to assist institutions and campus communities in conducting comprehensive assessments of their climate(s) and developing strategies to improve them (Rankin & Reason). The second phase of the transformational tapestry model requires the collection of institution-level campus climate data. This is a unique opportunity to use the novel campus climate instrument as the length of the measure is not overly cumbersome, the domains allow for exploration of multiple campus contexts, and the instrument’s psychometrics are strong regardless of student characteristics.

9. Limitations and Future Research Directions

The research team did not evaluate differences in campus climate based on participant major, college/university type, geographic region, or residential student status. To explore and assess the instrument’s generalizability and its results, future researchers should employ intentional recruitment strategies to examine differences in reliability and validity across groups. The data were collected over approximately 45 days; future researchers should examine the progression of perceived campus climate over a longer timeframe. These researchers could employ longitudinal approaches to examine campus climate associations and outcomes during students’ matriculation, compare year-over-year growth, cross-lagged associations between campus climate and student outcomes, and track the associations between campus-level campus climate (i.e., mean student perception) and university outcomes (e.g., graduation rate, student retention, alumni donation percentage). Because data were collected via Prolific, specific information about participants’ campuses (e.g., Minority Serving Institution status) was not collected. Future researchers should intentionally survey multiple campus types, including predominantly White institutions and minority-serving institutions, to determine if there are statistically and/or practically significant differences in students’ perceptions and experiences. Future researchers could also use stratified random sampling on specific campuses to ensure that the analytic sample is representative of the campus demographics.

10. Conclusions

The novel campus climate scale has proven to be a valid and reliable tool for measuring students’ perceptions of campus climate. The scale has undergone numerous analyses to confirm its reliability and validity, making it suitable for future studies of campus climate perceptions among college students. This instrument can help researchers deepen their understanding of how perceived campus climate affects college students’ social, behavioral, and academic outcomes.