Validation of the Term ‘Cultural Community’ in the Culturally Engaging Campus Environments Survey

Abstract

Addressing issues of inequity and exclusion on college campuses requires a comprehensive assessment strategy inclusive of diverse student populations’ cultures and communities, as it is integral to understanding students and their environment. Developing instruments that can accurately and robustly measure student culture is necessary for campus leaders to contextualize their data. The purpose of this paper is to validate and describe the use of the term cultural community in the Culturally Engaging Campus Environments survey. We used cognitive interviewing to illicit racially and ethnically diverse students’ interpretations of the term and its use in the survey. Findings suggest that students’ interpretations are mitigated by larger societal and institutional discourses, although race was a common mitigating factor.

1. Introduction

Assessment is important in advancing equity in higher education because institutional agents can gather evidence-based data to identify inequities for student success, inform stakeholders about where to make changes, and hold stakeholders accountable for enacting initiatives [1,2,3,4,5]. Creating change and fostering a community where students see themselves meaningfully reflected on campus requires comprehensive planning and assessment to be effective. Educators and administrators use survey data and other assessment strategies to gather a breadth of information on students and their experiences. Data are used to inform policy and practice across campus. This includes creating and sustaining environments meaningfully reflecting students’ cultural communities [6]. Assessing culture is multifaceted since it does not exist as a monolith across communities and is differentially valued based on the proximity to established institutional norms and practices [7,8,9].

Before a survey can be used in assessment, we must ensure the data have been properly evaluated. When scholars create new surveys or survey constructs, they embark on rigorous approaches to validate survey items. Without prior survey testing, results and analysis are de-contextualized, which can lead to false conclusions and improper policy implementation or practices [10,11]. Further, those who may be harmed in the process are the students most in need of support and inclusion [10,11]. Education researchers employ multiple psychometric techniques to determine survey quality and validity. Statistics, such as factor analysis and inter-item reliability, are used to confirm the validity and reliability of survey items and the validity of constructs [12,13,14,15]. In addition, researchers incorporate qualitative assessments such as enlisting subject matter experts to support content validity about survey instruments and cognitive interviewing [11,14,16]. These techniques are beneficial for developing and refining tools designed to measure complex phenomena and ideas.

Respondents can struggle to interpret items, deciding whether it applies to them and questioning the clarity of the item itself due to ambiguous terminology [17,18]. This issue may be particularly true when researchers use language that can be interpreted in multiple ways. For this reason, researchers often employ cognitive interviewing where respondents talk out loud about how they interpret or make sense of survey items. For instance, Park and colleagues used cognitive interviews with 31 racially and ethnically diverse undergraduate students to reveal nuances in how they understood items about cross-racial interactions in the College Senior Survey mitigated by both personal and institutional factors [11]. They identified five themes in students’ verbalizations, including hesitancy or uncertainty about connections to race, straightforward responses, influences from diverse contexts, reflections on question–respondent fit (i.e., not all items felt applicable to all students), and questions about the nature of interactions. Multiple college student surveys incorporate aspects of diversity, equity, or inclusion in questionnaire form. Surveys on the college student experience, such as the National Survey for Student Engagement (NSSE), the Student Experience in the Research University (SERU), the Cooperative Institutional Research Program (CIRP), and the Freshman Survey (TFS) ask demographic questions related to students’ identity in terms of race or ethnicity, gender, ability, or status, etc. (Center for Studies in Higher Education, 2018; Higher Education Research Institute, 2017a; National Survey of Student Engagement, 2019). National data sets provide researchers with a snapshot of perspectives and attitudes of a large number of college students at a given time [11]. Researchers can then disaggregate student responses based on demographic items to compare student experiences on items that explore cross-racial interactions, interracial friendships, diversity attitudes, and experiences with diversity engagement [11]. However, these surveys do not include items related to cultural community and, in particular, culturally engaging environments [6].

The purpose of this paper is to inform the validation of the Culturally Engaging Campus Environments (CECE; pronounced see-see) survey developed from the CECE model [6]. Prior testing of the CECE survey confirmed the content and construct validity of the CECE Scale [19]. After developing the initial scale, derived from the nine CECE indicators, subject matter experts were employed and edits to the constructs were made in order to ensure content validity. Later, exploratory factor analysis was used to group items within the CECE scale, followed by reliability analysis to further refine the scale items. Lastly, a confirmatory factor analysis “yielded an excellent model fit”, ensuring the scale is both robust and accurate [19]. In addition, research demonstrates that environments reflective of the CECE indicators are correlated with motivation, self-efficacy, sense of belonging, and success among racially, ethnically, and socioeconomically racially diverse college students [20,21]. We were interested in contextualizing the use of cultural communities in the survey, as the term is left undefined in the original model and is defined only by examples in the survey. In this paper, we present how cognitive interviewing illustrated the unique relationship between the ways students verbalized their understanding of cultural communities and the campus environments they sought out as familiar and validating. One research question guided this study is: how do students interpret the term cultural community in the CECE survey?

Shifting to a model that focuses on cultural community moves away from an inherent focus on the external identity labels. Instead, it highlights the “psychological need for cultural and ethnic identity” predicated on shared experiences such as language, religion, location, and beliefs, etc. [22]. There is an emphasis on this shared understanding that creates a community in which students feel a sense of belonging. Culture can be examined from an individual level, predominately focusing on cognitive and behavioral aspects of culture, or from the collective or community, which adds a layer of complexity to defining the boundaries between one culture and another [8,23]. Institutional culture, including normed attitudes and behaviors, can significantly impact student success [8,24,25]. Institutional culture incorporates “collective, mutually shaping patterns of institutional history, mission, physical settings, norms, traditions, values, practices, and beliefs, and assumptions that guide behavior of individuals and groups” [26]. Institutional cultures are often predicated on history in terms of who the institution was created for or who the institution is striving to serve, e.g., predominantly white institutions (PWIs), religiously affiliated institutions, minority-serving institutions (MSIs) [8,27]. Power dynamics and racism mitigate the symbolism embedded in an institution’s culture [25]. Conflicting interpretations between institutional cultures and individual and group cultures can impede transformation for equity efforts [1,8,27].

Incongruence between home and institutional cultures can cause isolation, invalidation, and hostility towards racially and ethnically diverse student populations [8]. Those who can find sources of support and community are more likely to succeed. When students’ home cultures are incongruent with that of the institution, they can be pushed out, choose to leave, assimilate or, if lucky, find enclaves of subcultures on campus to which they can find the support and validation they need to be successful [7,28]. Attention to the quality and quantity of cultural enclaves increases students’ success academically and socially [29]. In summation, students’ cultural communities are important for their success in college and should inform institutional research and assessment.

2. Culturally Engaging Campus Environments

The CECE model serves as a tool for understanding the role of cultural communities and cultural agents in fostering success for minoritized student populations [6]. To generate the CECE model, Dr. Museus used over 30 years of literature on college success to amplify racially diverse student voices and provide insight into factors that contribute to the likelihood that all students achieve successful outcomes [6]. Students perceive the campus environment in both positive and negative ways that call on institutional leaders to create optimal learning environments that go beyond identity markers such as race, ethnicity, sex, and gender [6,8]. The CECE model identifies two clusters of culturally engaging campus environments that are necessary to positively affect student outcomes in higher education. The first cluster of five indicators relates to the extent in which learning environments engage and reflect the cultural backgrounds, communities, and identities of diverse college students. These five indicators are: cultural familiarity, culturally relevant knowledge, cultural community service, cross-cultural engagement, and cultural validation. The second cluster of cultural responsiveness contains four indicators of how campus programs and support systems effectively and intentionally respond to culturally diverse students’ needs, including collectivist cultural orientations, humanized educational environments, proactive philosophies, and holistic support.

The model emphasizes that these indicators effectively lead to student engagement and success, although students can elaborate on how that happens. The model also focuses on how campus educators engage students from diverse communities in culturally relevant ways, while reflecting those communities in their demographic makeup [6]. The original purpose of CECE was to support racially and ethnically minoritized students, but it has since been taken up by scholars to support the needs of students from multiple cultural communities [30,31,32,33]. As CECE’s applicability expands, it becomes even more imperative for scholars and educators to understand how the term is taken up in various contexts.

3. Research Design

3.1. Methods

Cognitive interview, an often underutilized qualitative method in education research, is used to understand better the quality and scope of survey items [16,34]. Beatty and Willis define cognitive interviewing as the “administration of draft survey questions while collecting additional verbal information about the survey responses, which is used to evaluate the quality of the response or to help determine whether the question is generating the information that its author intends” [35]. Researchers use cognitive interviewing to collect information about survey items by asking participants to take the survey while eliciting verbal responses from them. We employed the “think aloud” technique, a cognitive exercise where participants are encouraged to speak out loud their thoughts while reading through survey items [35,36]. The advantages of this method include less interviewer training and a more open-ended response model [36]. By applying cognitive interviewing to a simple survey model, we wanted to ensure we could obtain “the likely variability in a process that is often assumed to be standardized, straightforward, and objective” [11]. When interpreting survey data, it is important to recognize how students think about and respond to questions. Tourangeau and colleagues devised a four-step theory, consisting of comprehension, retrieval of information, judgment and/or estimation, and reporting of an answer [18]. Our selection of cognitive interviewing as a method sought to make this process more explicit than an inner monologue, by providing students with opportunities to not only name what cultural communities matter to them but how they are reflected in their interpretations of items about the CECE indicators on the CECE survey.

3.2. Data Source

Interview questions came from the CECE survey derived from the CECE model [6]. The survey consists of 32 items related to the nine CECE indicators, sense of belonging, and engagement, followed by demographic questions. For this inquiry, participants only responded to questions related to the CECE indicators and demographic questions. There are nine scales on the survey that reflect the nine indicators in the model. The survey includes three items per indicator, except for cultural familiarity, which includes six survey items. The demographics section includes one open-ended question which asks, “Which cultural communities came to mind when you answered the questions above?” Survey respondents are also provided a definition of cultural communities which states that the term “can mean many things. It can refer to a national community, a racial or ethnic community (Asian American, Black, White, etc.), a religious community, an LGBTQIA + community, or even a community in the neighborhood where you grew up currently live”. The open-ended question is provided at the end of the survey after respondents have answered the CECE scale items.

3.3. Participants

Students from a large public, predominantly White, research university in the Midwest who completed the CECE survey in the spring of 2016 were eligible to participate. Of the 3071 survey respondents, we contacted approximately 1100 students via email. To provide a more comprehensive analysis of the survey, we looked to include a racially diverse group of undergraduate students [37]. In total, we emailed 142 Black students, 120 Latinx students, and 11 American Indian and Indigenous students, which constitute the total respondent count for each racial or ethnic group. Furthermore, we contacted a random sample of 315 Asian students and 515 White students to reach saturation. Our initial goal was to interview four students from each of the racial or ethnic identities referenced above for a total of 20 participants. Our final number of participants included four Asian students, four Black students, three Latinx students, and four White students for 15 interviews. Although we contacted American Indian and Indigenous students, we received no response. Two Latinx students were recruited via email, while the third was recruited via personal communication from a member of the institution’s Latinx culture center. Blair and Conrad found that, although increases in sample size provide more information and reveal more inconsistencies or issues with a survey, the increase in gains begins to lessen at 15 interviews [38]. Therefore, although our numbers varied from the initial goal, a final sample of 15 participants provided information-rich cognitive interviews (see

Table A1. Demographic Information.

Participant	Demographic Information	Salient Cultural Communities
Isaac	Black Man	Race (Black) Region (Southern Florida)
Tia	Black Woman	Race (Black) SES (low-income scholarship recipient)
Kayla	Black Woman	Race (Black) Sexual Orientation (LGBT community) Region (native to the town the institution resides) Major (Film)
Mel	Black Woman	Race (Black) Job (Resident Advisor)
Amy	Asian Woman	Race/Nationality (Korean American) Gender (Woman) Region (Midwest)
Caite	Asian Woman	Race/Nationality (Filipino American)
Paul	Asian Man	Race/Nationality (Filipino) Sexual Orientation (Gay) Age (40) Relationship status (Single)
Nate	Asian Man	Gender (Man) Race/Nationality (Chinese) International
Stephen	Latinx Man	Ethnicity (Hispanic) Region (Native of a nearby city) Student organization affiliation
Diana	Latinx Woman	Race and Gender (Latina)
Gabe	Latinx Man	Nationality (American and Mexican) Social Class (High Culture)
Helen	White Woman	Sexual orientation (LGBT community) Job (Resident Advisor) Religion (former Catholic)
Meg	White Woman	Race (White) Disability community Straight community
Jill	White Woman	Veteran community Traveler
Scott	White Man	Nationality (French Serbian) Sexual Orientation (LGBT community) Major/Interests (Music)

).

3.4. Data Collection

Data collection consisted of individual cognitive interviews lasting approximately 45 to 90 min. All interviews were recorded and transcribed verbatim. Interviews were organized by pairing participants and interviewers who belonged to the same racial and ethnic group when available. Each participant was provided a description of the study and given instructions on the process of the cognitive interview. In employing this approach, we found that participants often had questions at the comprehension step of Tourangeau and colleagues’ theory (2000) and devoted significant energy and discussion to clarifying definitions of various terms within the survey items. We also encouraged participants to make their mental processes as explicit as possible [35]. When participants appeared lost in thought or provided responses that required elaboration, the interviewers asked probing or follow-up questions.

3.5. Data Analysis

For data analysis, we independently conducted open coding of the transcripts line-by-line to draft an emerging set of codes [39]. We ensured that two researchers independently coded each transcript. Reminiscent of the constant comparative method, we connected ideas that emerged across the codes by discussing the relationships of the codes and comparing clusters of codes to each other and participants’ experiences, noting similarities and differences until themes that represented all participants’ verbalizations in the cognitive interviews emerged [39]. We sought to strengthen the trustworthiness of the findings through triangulation of multiple data sources, including transcripts, coding reports, and research memos, to verify emerging themes [40]. Member checks were conducted by emailing transcripts to participants, inquiring if their voices were accurately communicated. No participants questioned or suggested altering information in their transcripts.

3.6. Positionality

We found it important to address our various positionalities and to be aware of the influences they may have on our research process and the participants themselves [41]. Each team member created positionality statements and discussed their interpretations of the term cultural communities. We examined our salient social identities as one Black cisgender man, one Black cisgender woman, one multiracial Filipino/White cisgender man, and two White cisgender women. We questioned each other if we made interpretations about students’ experiences that seemed to deviate from what participants said and worked together to return to these data until we could fully substantiate our analyses.

3.7. Delimitations

We offer several delimitations for this study. The single-institution context is an important consideration for this study. Consistent with the intent of the CECE survey, institutional leaders should use these data to inform how to improve their campus environments to be more culturally responsive and engaging. Therefore, while the findings may be transferable to other settings, conducting cognitive interviews about student interpretations of items on the CECE survey should be conducted in different settings. We conducted cognitive interviews six months after students took the original survey. While there are benefits to providing students with opportunities to reflect on their responses directly after completing a survey, researchers may also heighten participants’ fatigue. Therefore, we opted to conduct cognitive interviews months after participants originally completed the survey. Third, students sometimes interpreted ‘people at my university,’ ‘sufficient,’ and other similarly structured questions to have different meanings. While cognitive interviewing is traditionally used to edit surveys for clarity and consistency, we also found that this method provides increased depth and breadth of item interpretation. Our probing questions allowed us to gain a better understanding of how students interpret different phrases. While these renderings are important, we focused on presenting data on how participants interpret the term cultural community. Finally, the fatigue associated with cognitive interviewing proved to be a delimitation. Participants were more vocal and answered questions in length at the beginning of the interview period and were less talkative toward the end in some cases.

4. Findings

Participants verbalized a broad understanding of characteristics that make up various cultural communities. They interpreted the term cultural communities differently and went through a similar process in naming their cultural communities. The participants were methodical in verbalizing the cultural community by initiating a common understanding based on campus discourses, establishing credibility, and providing examples of their interactions as members of their self-identified cultural communities.

Participants first considered cultural communities by acknowledging their institution’s response to societal discourses. Participants’ conceptions of the term cultural communities were often shaped by the identity politics in the broader campus environment. For example, college and university leadership create cultural centers and offer programming as a response to the needs of students from diverse backgrounds. Students referred to cultural spaces when verbalizing how they fit into particular cultural communities. From this perspective, race, ethnicity, and socioeconomic status were often the first point of departure for verbalizing conceptions of cultural community. All except two participants, Scott and Paul, verbalized race, ethnicity, or socioeconomic status first when defining cultural community.

Second, participants talked through options of what cultural communities could mean based on the prompt from the CECE survey. Participants communicated their minoritized communities as relevant to understanding the term cultural community. Participants also articulated the need to parse out each possible cultural community as an option to express what communities “I say I fit into,” and some verbalized that you have to recognize or know their cultural community(ities) to give it credibility. When participants considered the minoritized cultural communities with which they identified, they began interpreting the phrase as connected to their disability status, geographic location/nationality, discipline, and other characteristics that were salient to them.

Finally, after recognizing and establishing credibility with their cultural communities, the participants reinforced their position within the cultural communities identified by sharing examples of their interactions with peers. The participants either shared experiences where they engaged in-group members, or they discussed encounters with non-members that confirmed their cultural community affiliation.

5. Conceptions of Culture Informed by Campus Discourse

During the cognitive interview, participants started by presenting race, ethnicity, and socioeconomic statements when interpreting and making sense of cultural communities on their college campus. Participants talked about their cultural communities concerning the presence of particular groups on their campus. They used language that reflected the campus identity politics, where they understood the campus to prioritize or in some cases disregard, representational diversity. The participants identified with identity politics by illuminating discourses around the college campus demographics and what that meant for them. Participants would either identify with identity discourses related to their cultural community or distance themselves.

Participants interpreted cultural community based on the enrollment discourses and identity politics in their campus environment. Identity politics in this case referred to the college leaders prioritizing affluent white student enrollment as the racial majority on campus. Participants were aware of the demographic composition of campus as it pertained to race, ethnicity, and SES, and would verbalize cultural community by referencing their position within that makeup. Mel, a Black woman, interpreted her cultural community based on the discourse around enrollment numbers and race, noting, “When I think of similar backgrounds as me, I’m thinking like racial like our ethnicity or race as the same because you know there’s like 4.5% of us on campus.” With this interpretation, Mel highlighted the representational identity politics present on her college campus that left her in the minority. Conversely, participant Meg connected the campus environment to her interpretation of cultural community, highlighting how she was in the majority by stating:

“People from different cultural backgrounds, the term cultural is a (pause) there’s a wide gamut … When we’re talking about specific ethnicities, socioeconomic backgrounds (pause). There’s, I’m from [sic] County, as is most everyone that goes to this freaking school. So, you know. My [high]school was 90% white, 90% upper or upper middle class, 90% married families. Single-family homes and I feel like that’s exactly how this university is. It’s 90% white, socioeconomically rich people, and there’s either that or there are the international students who are wealthy enough to come here.”

Not wanting to solely identify with the campus majority, Meg chose to offset who she was and why that mattered. She went on to share the cultural community that was most salient for her by stating: “Specifically to me, being born with a physical disability, I would consider myself part of the disabled community. So, I would consider that a cultural background”.

Participants also spoke explicitly about the political nature of how the institution functioned related to cultural communities. When asked about opportunities to discuss important diversity-related issues for people with different cultural backgrounds, one participant shared how the institution markets itself as diverse when they feel the reality is very different. Nate explained:

“It’s hard because how I have to look at this in many aspects because in one way politically [sic] is framed as a diverse town many cultural diversity, but at the same time I don’t see how it’s really well integrated. It’s like it presented to you separately. There’s no integrations that I can feel visibly. I don’t have people to talk to me asking me about things happening in China or things they might interest in and have questions on. I think they’re always focusing on making U.S. experience. For example, if I want to integrate well in my friend group of who are American, I have to assimilate myself.”

In this quotation, Nate reveals more information about the political discourses that occur to frame the campus as diverse, but now there is no visible integration. Another participant discussed the politics associated with cultural celebrations, such as Cinco de Mayo. When asked if the institution values knowledge from the cultural community, Stephen stated:

“So, the best way I could describe this is if you think of Cinco de Mayo. Cinco de Mayo is the day that a little town in Mexico called Puebla defended against the French very little weapons. Against a massive army, and they won. Here it’s a drinking holiday … And just the point where you know I wouldn’t say they value what that day means to us, I guess. But that’s you know it’s more of a cultural issue it’s not something this institution can really, I guess, hammer home. So that’s, that’s why I would say that I somewhat disagree. You know, because in general, you know people, normal people don’t really care.”

Acknowledging the institution’s role is limited when addressing how Americans celebrate Cinco de Mayo, the participant shares how the holiday is considered a “drinking holiday” on his campus. This example highlights how the politics established by campus leadership that prioritize white students inform interpretations of cultural community that can disregard cultural histories. Even Stephen’s reference to other people at the institution as “normal” has a political connotation. Stephen discussed Cinco de Mayo in the context of an event respected by members of his cultural community but declared that normal people do not tend to care about the historical context associated with the celebration.

In addition to acknowledging the political discourses connected to the campus environment, the process of naming and interpreting cultural communities also meant the participants wanted to share the things that related personally to them. They articulated what they saw on campus, such as a cultural center, as a way to discuss how their cultural community was valued by the campus leadership. For example, Caite, a Filipino American, works in the Asian American culture center and described how the center allowed her to see people from her community regularly. Caite shared: “…so, I tend to lump different cultural backgrounds. More than anything based on race it seems to me that we often make stereotypes based on that and here at [campus] interestingly enough it seems to be that people from different races do keep to each other that’s what I’ve seen at the very least on the [Business School] side of things”.

Gabe acknowledged that he lumped cultural backgrounds together in his interpretation but eventually considered race and could see the segregation of race on campus. Other participants listed the cultural communities that are assumed. For example, Kayla defined cultural communities by saying: “I mostly think of race, but I guess also international students and people that are from different economic classes”.

6. Minoritization and Self-Identification

Once participants went beyond interpretations of cultural community informed by the campus environment, they began to discuss other aspects of their culture by establishing credibility through self-identification. They often named race, ethnicity, or socioeconomic status first as a mechanism to claim dominant or minoritized identities and then elaborate on the minoritized identities that are salient to them. For example, Scott asserted that he is White by societal standards, but when he dug deeper, he explained that his French community was more salient. As Scott put it when asked about finding people with a similar background: “I mean, yeah I’m white so it’s pretty easy… but then if the background is French or even more specifically French Serbian because my dad’s Serbian, my mom’s French”.

The cognitive interview process allowed the participants to fully verbalize and establish credibility with the cultural background with which they identified, which was true for Amy, who said: “I’m thinking about like similar backgrounds meaning like Korean Americans are like I don’t yeah I guess that’s the only similar background.” Being more thoughtful about the question led Amy to interpret her cultural community as being more than just Asian American, but Korean American specifically. Stephen also confirmed his understanding of cultural community to be more nuanced when he stated:

“If I really wanted to find someone with the same background as me, I don’t think I could. But that’s just because I have a really different background or unique background, I guess. And then that would really entail as to what similar means in this context. So yeah, I guess. But I think that’s OK. You know, it’s not easy to find. Now if in this case finding people of similar background means finding the Hispanic community it’s very easy. [Cultural Center], it’s very very well represented. That’s just not necessarily my interest.”

Moreover, Kayla shared a similar sentiment, sharing: “I am African American, and I am also a member of the LGBTQIA community.” She elaborated to add credibility by saying: “A lot of my friends are part of those communities or are multicultural in relation to race”.

Jill also expressed her own identities: “I grew up kind of poor, white, military”. It was important for her to establish credibility about the cultural communities for which she wanted recognition. Again, we hear a participant express her minoritized identities and then how they are recognized or given credibility. In her case, her friend or peer groups offer confirmation of the cultural communities that she belongs to and why. Caite shared the following when answering a question related to people at the institution.

“So, I’m thinking about Filipino Americans. For Filipinos I mean. People though. (Inaudible) people I’m thinking about like Asians. It’s not us. I think about Chinese, Japanese, Korean. So that invisibility is frustrating. And then, in general people of this institution. Yeah, I think the invisibility is just like too big for me to agree with this.” Again, this process is to distinguish what communities can exist and how her Filipino American cultural communities is where she falls into and why.

Participants’ interpretations of the term cultural community, as it related to their affiliation or belonging, included both aspects of culture and community in their definitions and meaning making. Cultural aspects included shared beliefs or experiences, values, histories, etc. Community included both a sense of place and people. In her interview, “Tia” defined cultural communities as “certain groups of people that have similar interests or concerns”. Tia incorporates two important key features in her definition by indicating that a cultural community is based on being part of a group and having shared beliefs.

7. Culture and Community through Interactions with Others

In addition to establishing credibility for their minoritized identities, participants offered additional examples of their interactions with others to confirm their cultural community membership further. Participants gave examples to elaborate on how they defined culture and community, thus cementing what cultural community meant to them. Students expressed how their membership in a minoritized cultural community was reaffirmed by the institution and people within the institution, either through positive or negative experiences.

Participants often were drawn to individuals with whom they can relate. For Tia, she expressed feeling most valued when interacting with other members of her cultural community: “[My friends] share the same interest as me so they’re interested in the same things, or they’re concerned about [the] same issues as me. But I don’t know about someone from like the science department or someone that isn’t part of the same cultural community as me if they have the same concerns and maybe they don’t know that these issues or like events are going on”.

Earlier in her interview, Tia also mentioned her positive interactions with educators interested in diversity issues and “concerned about my experience of being one of the very few Black students at a PWI.” Tia highlighted how individuals with common interests or who share a concern for her and identify as Black can be viewed as members of her cultural community. Since Tia was unsure about members of the science department or others having concerns for similar issues as her, she could not confirm them as members of her cultural community.

Although shared understandings related to cultural communities were often positive, for some participants, histories of oppression and systemic racism create a shared reality that can only be understood by members of certain cultural communities. “Isaac” told a horrific story of a racist incident he experienced on a bus coming home from campus and the difficulties of attempting to have productive conversations around race.

“From my stance, we have those feelings of frustration and confusion and anger sometimes because of the personal experiences that happened with us but if it’s never happened to you then you can’t really relate and therefore it just seems so simple.”

Although this participant identified with multiple cultural communities, being part of the African American community was particularly salient because of history and experience that folks outside of the culture cannot fully understand. Isaac’s statement illustrates how individuals with a shared history who can relate to his experiences make up his cultural community. Similarly, Diana named three intersectional cultural communities, being a Latina woman in STEM, as she noted the limited number of women and Latinx students. Paul offers another example of relatability, describing American students’ perspectives on international issues. When asked if people at the institution value knowledge from his culture, Paul stated:

“Somewhat disagree, that would be my answer. Just one case for before coming to American [sic] I presented this idea that Hong Kong and Taiwan are part of China and I’m not saying that it is absolutely correct right now, but everything I’m learning from me just like, oh, Hong Kong and Taiwan are not a part of China because of this, this, this, this this. There, it’s like more perspective for the Western. But they don’t really understand, they don’t honestly know why we think Taiwan and Hong [Kong] are a part of China. They just think, OK if not… you’re wrong. They don’t really understand it don’t try to engage in that. So, I think it’s not really from my cultural background in terms of I mean like, international issues. It’s from a background of, you know it’s in the U.S. lens to me.”

Paul highlighted how, to be a member of his cultural community, individuals should be willing to engage international perspectives, rather than be dismissive. This example shows how individuals must go beyond being familiar with the issues but should be willing to learn and relate to the circumstances that make up differing perspectives.

The participants viewed their cultural communities in very specific ways and made clear distinctions between similar cultures. Within predetermined affinity groups, smaller cultural communities often emerge that more closely represent the values and needs of individuals. Helen, a queer woman, revealed her frustration with trying to find others who shared her background.

“I think in terms of like queerness … I interacted with a lot of gay marriage gays, and I was just kind of like annoyed. Just because it’s kind of like white feminism right it’s like people who are not thinking beyond like what actually directly affects them… Yeah, but I was like these are not my people so my junior year I joined roller derby, which is outside of the university, and found a whole bunch of the kind of Stonewall queers who are like in charge.”

Helen found an LGBTQ community on campus but felt that these relationships were not fulfilling her interests. She used different language to categorize her community, citing “gay marriage gays” and “Stonewall queers”, who she explained were more radical and critical. Feeling like you belong to a cultural community incorporates more than a shared social identity but also a sense of shared values.

Amy also shared how she identified with a distinct cultural community that exists beyond the cultural community most familiar to others, being Korean American. She clarified that “not just Korean American part of like just Korean or American, I do like to understand a lot about the Korean culture, but I know I don’t understand everything.” Scott had a similar view with finding other people on campus with a similar background: “and those that do come to tend to be, like international students who just came from France. It’s not quite the same as like somebody who sort of grew up in both countries, in both cultures”. For both participants, affiliation with and belonging to a cultural community was tied to a sense of place which incorporates important aspects of culture. Amy and Scott named familial countries of origin as important parts of their culture, but that their cultural communities were, in fact, not quite Korean and not quite French, respectively. Both of their cultural communities were physically located in closer proximity and blended aspects from multiple cultures.

Participants’ sense of belonging to a cultural community was also predicated on a sense of shared understanding with other members of the community and the culture of the community itself. Kayla shared that her most salient cultural community was being a native of the town she grew up in. Her reason for naming her town included aspects of place, people, and culture.

“I have a lot of history with this town and just kind of trying to find myself… I see really huge but loving community of people that really care about the arts and care about [the town] and want to make it a better place.”

Kayla located herself within a cultural community based on several factors. Her sense of personal development and who she is as a person is tied back to the town in which she grew up. Furthermore, she articulated a shared sense of values which includes valuing the arts and community improvement. Both of these factors, which are inextricably tied as the shared values of the town (i.e., the culture of the community), informed the participants’ own development and understanding of the world. How participants understood themselves was often with their cultural communities.

8. Discussion and Conclusions

In this study, we sought to deepen our understanding of the term cultural community in the CECE survey. Our findings suggest that students often turn first to race when discussing culture but include a variety of identities and communities, including gender, nationality, socioeconomic status, and sexual orientation. Several external factors informed participants’ verbalization of their cultural affinity, including campus discourse and their interactions with others. A deeper understanding of students’ cultural communities is necessary when seeking assessment strategies to improve student success.

Race was a mitigating factor as students sought to verbalize the cultural communities with which they were most strongly affiliated. Like Park and colleagues (2017), institutional factors and their proximity to racial diversity mitigated participants’ interpretation of survey items. Students may perceive a campus as racially diverse but cultivating meaningful interactions across races looks different throughout the entire institution and varies by students’ racial identity and racial consciousness [11]. Like culture, race consciousness and race evasiveness influenced participants’ answers about race. White students were more likely to distance themselves from their racialized cultural community at this PWI. In contrast, Black and Asian participants were more likely to identify race as an important cultural community due to both the strength of their community and the lack of support available across campus. None of the participants were ignorant to the discourse on race and racism but there were hesitancies about how to engage.

As the literature suggests, multiple interpretations of survey items are relatively common. Respondents can struggle to interpret items, deciding whether it applies to them and questioning the clarity of the item itself due to ambiguous terminology [17,18]. The ways educators and researchers interpret results do not always align with students’ interpretations, particularly if the language used is new or novel to the field of higher education. Without prior survey testing, results and analysis are de-contextualized, which may lead to false conclusions, and improper policy implementation or practices [10,11]. Further, those who may be harmed in the process are often racially minoritized students most in need of support and inclusion [10,11]. This is not an excuse to remain ambiguous but a recognition that further item testing may be needed when survey results are to be meaningfully interpreted on individual campuses.

Survey data are not a one-size-fits-all model in institutional research and assessment. Our findings remind us that respondents are not objective data points. They are students who bring a wealth of experiences that they rely on when interpreting and responding to survey items. Students’ understanding of and experiences with the campus environment vary greatly based on many factors. University leaders and higher education scholars should view data analysis and results through a humanized lens. When assessing the campus environment, comparative analysis between students’ responses on cultural communities should be avoided. Similarly, how responses are grouped for analysis should be intentional and determined in relation to students’ needs and the institutional context. Institutional leaders cannot default to demographic indicators without considering cultural influences in the campus environment that influence students’ survey responses. If administering the CECE survey, the open-ended response could be used to more deeply understand students’ needs and identities. When communicating results with a broader audience, institutional leaders need to situate results through naming and explaining how hegemonic systems shape students’ responses. While two student racial groups may report similar responses to a particular CECE indicator, institutional leaders need to interpret responses in relation to the culture of the institution. Students’ sense of belonging needs to be contextualized.