Navigating the Disproportionate Impact of COVID-19 in Community Schools

Abstract

The COVID-19 pandemic created major disruptions in schooling and education. This exploratory study examines state-standardized test scores in traditional and community schools between 2019 and 2023, and qualitative data from a large urban school district located on the East Coast of the United States. The findings indicate declines in test scores for all students in both English Language Arts (ELA) and mathematics. The 11 schools implementing a community school strategy reported slightly smaller declines than traditional schools in both ELA and math. Qualitative analysis reveals that community schools were able to pivot quickly to support student and family needs during the pandemic, which may have helped mitigate some of the negative impact of the COVID-19 learning disruptions. This exploratory study points to the potential of the community school strategy in providing holistic supports to alleviate some of the challenges associated with historically underinvested communities, and their responsiveness in times of crisis.

1. Introduction

The COVID-19 pandemic significantly disrupted formal education throughout the United States and the world, from official school shutdowns to individual- and classroom-level quarantining periods. During this time, schools, students, families, and communities in the U.S. faced immense challenges to continue formal schooling, including digital divides and inequity in access to technology, disruptions in learning continuity and declines in the quality of instruction, and economic hardships of families (Parveen et al. 2022). Unsurprisingly, this major disruption and related challenges have impacted student learning. For the first time since the 1970s, reading and mathematics scores on the National Assessment of Educational Progress declined between the assessment years of 2012 and 2020 for students aged 13 in the U.S. (Irwin et al. 2022).

During the years leading up to the pandemic, many large, urban school districts started implementing the community school strategy to address issues of inequity, including New York City (Johnston et al. 2020), Oakland (Klevan et al. 2023), Baltimore City (Durham and Connolly 2016), and Philadelphia (Provinzano et al. 2018). While great variability exists in implementation, community schools aim to create physical and social spaces that foster partnerships between schools and the community to enhance the well-being of students, families, and the broader community (Oakes et al. 2017). Community schools, with an emphasis on providing resources to support the whole child, may have been particularly critical during the pandemic to support students and their families (Kimner 2020).

The COVID-19 pandemic disproportionately impacted historically minoritized communities—often the communities in which the community school strategy is being implemented. To examine the relationships between the COVID context and community schools, the current exploratory study descriptively examines test scores in community and traditional schools between 2019 and 2023. Building off of the descriptive findings, we use qualitative data to examine how community schools supported their students and families during the COVID-19 pandemic. We hypothesize that community schools, on average, experienced smaller declines in standardized test scores compared to traditional schools. Moreover, we hypothesize that community schools were able to address pandemic-related challenges quickly and effectively because of their preexisting focus on the whole child, developing partnerships, and creating collaborative structures.

2. Literature Review

2.1. COVID-19 Learning Loss

From official lockdowns to supply line disruptions, the COVID-19 pandemic significantly disrupted all aspects of society. Formal education was certainly not an exception. In March 2020, schools in the United States closed in-person instruction and remained closed for the rest of the 2020–21 school year (Jack and Oster 2023). At the peak of the COVID-19 crisis, UNESCO reported that nearly 1.6 billion learners in over 190 countries, or 94% of the world’s student population, were impacted by school closures (United Nations 2020). The closure of schools was relatively uniform around the world, but the reopening process varied significantly. Some countries fully or partially reopened schools within the month of March, while others remained closed for over 660 days between January 2020 to December 2021 (Munro et al. 2023). On average, schools in the United States remained closed or partially closed for approximately 667 days between January 2020 and December 2021, including typical school and summer breaks (Jack and Oster 2023).

Still, these numbers hide the great variability within each country. Specifically in the United States, the decision to reopen was determined by each of the 13,000 public school districts (Jack and Oster 2023). Education Week (2020) tracked the reopening plans of 900 school districts for the start of the 2020–2021 school year, which included the largest districts in the U.S. Only 24% of districts fully reopened for in-person instruction for all students; about 27% of districts opted for hybrid instruction; and 49% continued fully remote learning (Marshall and Bradley-Dorsey 2022).

Just as much variability existed in the approaches that were taken to mitigate the spread of COVID-19. The National Academies of Sciences, Engineering, and Medicine (2020) recommended a variety of mitigation strategies including wearing masks, physical distancing, creating cohorts of students, and screening of body temperatures and symptoms. Some schools reopened at the beginning of the year, but closed during the winter surge (Jack and Oster 2023). Naturally, these created further disruptions in students’ lives and their learning. However, by March 2021, only 8% of districts used fully remote learning only (Marshall and Bradley-Dorsey 2022).

During this time, students experienced changes in various academic outcomes. Early research attempted to predict the amount of expected learning loss from a significant disruption. Scholars debated the efficacy of predicting learning loss by examining different disruptions in learning including seasonal analysis (e.g., summer learning loss), similar learning disruptions due to unplanned events (e.g., Hurricane Katrina), loss of in-school time due to absenteeism, and international comparisons (Engzell et al. 2021; Harmey and Moss 2023; Kuhfeld et al. 2020b). These estimates varied, but consistently predicted a significant negative impact of COVID-19 on student learning.

As data from the 2020–2021 and subsequent school years became available, more concrete estimates of learning loss and its unequal impact on historically minoritized students became apparent. Donnelly and Patrinos (2022) conducted a systematic review of the literature documenting learning loss between March 2020 and March 2021. In the seven studies identified by the authors, the findings suggested learning loss across various subjects, grade levels, and geographic regions, though there was variability in these findings. For example, learning loss seemed to be less pronounced in older students and in reading compared to students in the primary grades and in math, respectively (Donnelly and Patrinos 2022; Kuhfeld et al. 2020a). In another systematic review, Hammerstein et al. (2021) examined 11 studies that reported the impact of school closures in response to the COVID-19 pandemic. The authors found comparable learning losses to summer learning loss, suggesting that virtual learning may be similar to students receiving no formal teaching in the summer months. Of concern, Hammerstein et al. found larger negative impacts on younger children and children from lower-SES families.

Schools in the United States serving historically minoritized students and families were especially impacted by the challenges at every stage of the COVID-19 pandemic. Many minoritized students and their families experienced houselessness, hostility from immigration enforcement, unemployment, and financial instability as a direct result of the pandemic (Grooms and Childs 2021). Remote learning created additional challenges including access to technology and insufficient support for special education students and teachers (Leech et al. 2022; Spencer et al. 2023). Parents identifying as racial and ethnic minorities reported feeling more concerned with certain aspects of returning to in-person learning, such as the safety of their students and students bringing home COVID-19, compared to white parents (Gilbert et al. 2020).

Other research predicts longer-term impacts of COVID-19-related school closures on students. In a meta-analysis including data from 39 studies across 19 countries, Di Pietro (2023) found evidence of learning loss and identified that students did not fully recover from the initial learning loss a year or more following the outbreak of COVID-19. Özdemir et al. (2022) projects that the impact of school closures will reverberate to future adult skills in working-age populations in the following years and exacerbate inequalities between Global North and Global South countries, negating gains that have been made in the past several decades. Unfortunately, the literature both within the U.S. and globally highlights how the historic inequities in education have been further compounded by the COVID-19 pandemic.

2.2. Community Schools

Community schools (also referred to as full-service community schools) attempt to support the child holistically by providing supports extending beyond the traditional responsibilities of schools. Community schools are typically organized around four pillars: integrated student supports (e.g., wraparound services such as healthcare and mental health resources); expanded learning time and opportunities (e.g., extended learning day or school calendar); family and community engagement; and collaborative leadership and practices (Maier et al. 2017; Oakes et al. 2017). While there may be variability in the implementation of each of these pillars from school to school, each of the elements is considered necessary for high-quality implementation of community schools (Maier et al. 2017).

Many community schools serve historically disinvested communities (Oakes et al. 2017). Students living in historically disinvested communities have a greater likelihood of facing housing instability and food insecurity, encountering community violence, and experiencing health problems (Sibley et al. 2019). To effectively serve these students, community schools provide resources for the whole child to holistically support development, including health and dental care, mental health resources, academic enrichment, and mentorship (e.g., Dryfoos 2002; Dryfoos and Maguire 2002; Maier et al. 2018; Oakes et al. 2017).

Recent estimates suggest about 8000 to 10,000 community schools exist across the United States (Quinn and Blank 2020), but that number is likely increasing with growing interest in the community school strategy (Camera 2021). Many states (e.g., Maryland State Department of Education 2021) as well as the U.S. Department of Education (U.S. Department of Education 2024) have announced plans to invest in the community school framework. In a School Pulse Panel report released in 2023 by the U.S. Department of Education, 60% of public schools utilized a community school approach or provided wraparound services, up from 45% in 2022 (National Center for Education Statistics 2023). Notably, many large, urban school districts have started to implement the community school strategy to create greater educational equity (e.g., Durham et al. 2019; Johnston et al. 2020; Provinzano et al. 2018).

The COVID-19 pandemic exacerbated inequities that many students and their families face. The pandemic also showed that schools and families in disinvested communities faced challenges with food, housing, and financial insecurity; experienced an increase in mental health crises; encountered virtual learning and technology access gaps; and exacerbated barriers to communicating and building relationships (Parveen et al. 2022). Community schools, on the other hand, operate with a vision of schools as the central hub of communities, offering families improved access to physical and mental health services, as well as increased opportunities for academic enrichment (Blank et al. 2003; Oakes et al. 2017). Theoretically, community schools would be optimally positioned to create a safety net for students and their families, especially during a time of hardship such as the COVID-19 pandemic. To date, however, this protective, preventative role of community schools during a time of crisis has not been systematically examined. Thus, exploring how community schools functioned during the COVID context can provide additional insight into the potential impact of the community school strategy.

3. Theoretical Frameworks

The community school strategy is supported by Epstein’s (1987) theory of overlapping spheres of influence and Maier et al.’s (2018) framework of community schools. Epstein’s (1987) theory of overlapping spheres of influence states that students exist at the center of the school, home, and community spheres. Relationships and connections across these environments create greater overlap between the contexts, resulting in more cohesive and consistent student experiences. When students receive coordinated support, they are better positioned to succeed in school behaviorally and academically (Sheldon 2003, 2007; Sheldon and Epstein 2002). Community schools attempt to bring the school, home, and community spheres closer together through family engagement and community partnerships that holistically support students. This approach to student development became particularly important and impactful during the COVID-19 pandemic when students, families, and communities experienced unprecedented challenges.

This study is also informed by Maier et al.’s (2017) four dimensions of high-quality community schools, mentioned earlier to help define community schools. Maier and colleagues identified these four dimensions through a review of 143 research studies. The four dimensions, or pillars, include: (a) integrated student supports; (b) expanded learning time and opportunities; (c) family and community engagement; and (d) collaborative leadership practices. All four pillars became particularly relevant during the COVID-19 pandemic, with community schools providing supports such as mental health services for students; additional resources such as food giveaways for families; partnering with community organizations for resources and services; and collaborative leadership structures to facilitate the work of community schools. These factors were considered in the collection and analysis of data.

The four pillars of high-quality community schools also support the overlapping of the spheres of influence. Integrated student support and expanded learning time and opportunities acknowledge that students exist beyond just the school sphere and can benefit from schools supporting the whole child. Family and community engagement builds and strengthens the relationships, structures, and processes that bring these spheres of influence closer together. Finally, collaborative leadership practices can ensure that family or community voices can be represented in school decisions.

The two theories also suggest that community schools may have been optimally primed and positioned to respond to address pandemic-related challenges. While the family and community spheres of influence were pulled away from the school, pre-pandemic relationships likely created opportunities to quickly reconnect. While students and families faced inequitable challenges, the integrated supports and community resources that are already part of the overall community school strategy may have helped bridge some gaps. Indeed, community schools and the communities they serve faced challenges during the pandemic. However, the theories suggest that community schools may have been able to respond relatively quickly and effectively compared to traditional schools that may not have had similar systems and processes in place to support their students.

4. The Current Study

The current exploratory study examined data from a larger evaluation of the implementation of a community school initiative in a large, urban, East Coast school district in the United States. This school district implemented the community school strategy as a turnaround school reform effort, specifically targeting the lowest-performing schools on state test scores in the district. The 11 schools targeted for the community school strategy in this district served historically minoritized student populations in historically underinvested communities.

We first descriptively examined students’ academic performance between the years of 2018–19 (baseline) and 2022–2023 (“post”-pandemic years) alongside qualitative data. While some schools were fully reopened in the 2021–2022 school year, the inconsistency of formal education with quarantining procedures made student outcome measures unreliable. Therefore, we chose the 2022–2023 school year to examine students’ academic performance. Building off of the descriptive statistics, we examined interview and focus group data to investigate how community schools responded to the pandemic.

This exploratory study asked the following research questions:

• To what extent did schools experience changes in state-standardized test scores between 2019 and 2023?
• To what extent did changes in state-standardized test scores differ descriptively between community schools and traditional schools?
• How did community schools experience and navigate the COVID-19 pandemic?

Although these are descriptive and exploratory questions, we hypothesized that all schools experienced a decline in state-standardized test scores between 2019 and 2023. However, we expected this decline to be smaller in schools using the community school strategy because of their attention to supporting whole child development and wellbeing.

We build on our descriptive analyses using qualitative data to explore how community schools responded to the challenges of the COVID-19 pandemic. We hypothesize that community schools were able to address challenges related to the COVID-19 pandemic relatively quickly and effectively with their preexisting emphasis on developing family and community partnerships; providing resources for whole-child development; and establishing other collaborative structures.

5. Methods

5.1. Data

The data used in this study were collected as part of a multi-year, mixed methods evaluation of a community school initiative in a large, urban, East Coast school district. The quantitative data included 116 schools, 11 of which were identified as community schools. The district implemented the community school strategy in these 11 schools as a turnaround school reform strategy in response to sustained low student performance on standardized tests and/or concerns about student attendance.

The descriptive analyses use district administrative data from the academic years of 2018–19 and 2022–2023, excluding non-traditional students (e.g., evening students). The district provided administrative data for students enrolled in the 2018–19 and 2022–2023 school years. Scale scores for both English Language Arts (ELA) and mathematics for a state-standardized assessment were examined.

The evaluation team also conducted interviews and focus groups in the summers of 2021, 2022, and 2023 in three community schools and with district leaders. The three community schools included one elementary school and two middle schools that were previously identified by the district as the focal schools for the evaluation. Semi-structured interviews were conducted with the Community School Managers (CSMs), school principals, and central office staff. The evaluation team also conducted focus groups with teachers, families, and family or community engagement school teams. Both interviews and focus groups lasted between 50 and 60 min. See

Table 1. Qualitative data sources.

Stakeholder Role	Data Collection Format	2020–21	2021–22	2022–23
Connected School Managers	Interview	3	3	3
Principals	Interview	3	3	2
Central District Staff	Interview	3	5	4
Community Partner	Interview	0	0	4
Teachers	Focus Group	3	3	0
Families	Focus Group	3	3	0
Family or Community Engagement School Teams	Focus Group	0	0	2
	Total	15	17	15

for the list of qualitative data sources.

5.2. Missing Data

The analyses use school-level means of student scores on a state assessment. Student-level missing data were ignored. School-level missing data occurred with school closures and new school openings between the 2018–19 and 2022–23 school years. The 21 schools that closed and two schools that opened were excluded from the analyses. One school served solely pre-kindergarten students, and another school served solely students classified for special education. These two schools had no testing data and were removed from the sample. From the original sample of 135 schools, we excluded 24 schools, creating the final sample consisting of 110 total schools (99 traditional and 11 community schools).

5.3. Measures

The quantitative data include demographic data and state assessment scores. The data included student-level scale scores on math and ELA state assessments. The student-level scale scores are aggregated as means at the school level. The average scale score for the traditional and community schools was calculated using the means across school-level scores.

5.4. Analytical Approach

This exploratory study started as a quantitative study but evolved into a concurrent mixed-method design (Teddlie and Tashakkori 2006). We first examined the quantitative test score data but quickly ran into challenges (discussed below) and modified our analytical strategy. We pivoted our approach to include both quantitative data to examine the potential differences in test scores and qualitative data to examine the conditions within community schools during COVID-19.

5.4.1. Descriptive Quantitative Analysis

The quantitative approach examines descriptive statistics of achievement data across community and traditional schools. Due to challenges tracking students over time, across schools, and across school types (i.e., switching between traditional and community schools), we decided to aggregate student-level data at the school-level. While this greatly limited our sample size to only 11 schools, we still hoped the descriptive data would show some important patterns in the data.

Again, we used school-level ELA and math scores on the state assessment. The mean of individual student-level data was used as an aggregate school-level measure of school test scores. In addition to the descriptive statistics, we used various data visualizations to examine the patterns of schools’ state test scores across 2018–19 and 2022–23. We quantified differences in state assessment scores by subtracting the 2022–23 school-level average state assessment score from the 2018–19 school-level average state assessment score.

5.4.2. Qualitative Analysis

We also analyzed qualitative data that were collected as part of the larger program evaluation examining the implementation of community schools in three focal schools (one elementary school and two middle schools). For the larger evaluation, the qualitative data analysis included coding interviews; examining memos and notes (Emerson et al. 2011); and triangulation (Patton 1999). We conducted two cycles of coding: first cycle coding focused on open and values coding; second cycle coding applied pattern or analytical coding (Merriam 2009). We compared and analyzed emerging themes with our ongoing discussions, the theoretical frameworks, and our notes. Lastly, we triangulated these codes using the focus group data, administrative data, and ongoing conversations with district staff.

For this exploratory study, we largely used deductive a priori codes to examine how the three schools responded to the COVID-19 pandemic. We identified emerging themes around the pandemic response in community schools as part of the larger evaluation. From this, we first coded the qualitative data with very general codes such as “pandemic” and “COVID-19”, alongside codes from prior analyses, such as “relationships” and “structures”. In the second cycle of coding, we developed three main codes of “COVID-19 challenges”, “community school managers and COVID-19”, and “COVID-19 response” to identify the more specific themes directly discussing COVID-19.

6. Results

6.1. Demographics

The demographics of community and traditional schools differed in this district. The 11 community schools served larger proportions of students identified as special education status, at-risk status, and economically disadvantaged, while traditional schools served a larger proportion of students with ELL status. Community schools enrolled larger proportions of Black students, while traditional schools enrolled larger proportions of white, Hispanic, and students of other non-white races. Finally, the majority of the 11 community schools were middle schools. See

Table 2. Demographics of schools and students.

	Total (N = 110)	Traditional School (n = 99)	Community School (n = 11)
School Level n (%)
Elementary	70 (63.6%)	68 (68.7%)	2 (18.2%)
Middle (grades 6–8)	13 (11.8%)	7 (7.1%)	6 (54.5%)
High (grades 9–12)	17 (15.5%)	15 (15.2%)	2 (18.2%)
K-8	8 (7.3%)	8 (8.1%)	0 (0.0%)
Middle-High	2 (1.8%)	1 (1.0%)	1 (9.1%)
Student Demographics
Gender 1
Female	48.6%	48.7%	47.7%
Male	51.3%	51.2%	52.3%
Race/Ethnicity
Black	66.8%	64.3%	89.5%
White	26.2%	28.3%	7.1%
Hispanic	18.5%	19.5%	9.5%
Other non-white race 2	7.0%	7.4%	3.4%
Special Education Status	17.3%	16.8%	21.7%
ELL Status	14.7%	15.6%	7.0%
At-Risk Status	57.3%	54.8%	80.4%
Economically Disadvantaged	8.4%	8.1%	10.4%

Note: The demographic categories provided in this table were provided by the district, unless otherwise noted. ¹ Some students identified as nonbinary or unknown gender. To protect the identities of these students, we do not report these numbers. ² This category aggregated students identifying as American Indian/Alaska Native, Asian, Multiracial, Native Hawaiian/Other Pacific Islander to protect the identities of these students.

for the demographics of all schools, traditional schools, and community schools.

6.2. State-Standardized Assessment Scores

Figure 1 presents the state assessment scale scores in 2019 and 2023 for all schools, traditional schools, and community schools. As shown in the figure, all schools—regardless of school type—reported a decline in state assessment scale scores between 2019 and 2023. The figure also shows that community schools reported lower state assessment scores compared to traditional schools in both 2019 and 2023. For instance, traditional schools reported an average of 733.68 on ELA in 2023 compared to 718.25 for community schools, or a difference of about 15 points. The 2023 math scores show a similar difference of about 17 points in the average test scores, with traditional schools reporting 717.59 and community schools reporting 710.54. This is expected, as the community school strategy was used as a model for turnaround school reform in this district, which targeted the schools with the lowest academic outcomes.

Figure 2 reports the difference in state assessment scores for ELA and math between 2019 and 2023. Schools, on average, reported a 4.95- and 8.16-point decline in their state assessment scores in ELA and math, respectively. Following national-level trends (Irwin et al. 2022), schools had larger declines in math assessments than in ELA assessments. The declines in state assessment scores were slightly smaller for community schools compared to traditional schools in ELA and math, though the differences are minimal. While traditional schools reported a 5.09- and 8.38-point decline in ELA and math scores respectively, community schools reported a decline of 3.64 points in ELA and 6.20 points in math.

While we recognize that these differences are very small, the community schools served communities that were disproportionately impacted by COVID-19 challenges. In other words, without the community school strategy, we speculate that the community schools would have seen even larger declines in test scores. The comparable declines of test scores between traditional and community schools may reflect the efforts of community schools to address COVID-related challenges.

Finally, we visualized the slopes for each school. Contrary to expectations, some state assessment scores trend upward between 2019 and 2023. Twenty-six traditional (26%) and 5 community (45%) schools reported an increase in ELA state assessment scores; 22 traditional (22%) and 3 community schools (27%) reported an increase in math state assessment scores. Although both types of schools, on average, reported declines in state test scores, a notable portion of both community and traditional schools reported some gains. These findings also show the large variability in the outcomes at the school-level and the need for more data, especially on community schools.

6.3. Responding to the COVID-19 Crisis

The descriptive data presented above suggest that community schools exhibited similar declines in their state assessments compared to traditional schools, despite serving communities that were disproportionately affected by the pandemic. From these descriptive findings, we were curious about what community schools experienced during the COVID-19 pandemic and how they responded to these challenges that helped mitigate some of the negative impacts of the pandemic. The three major themes that emerged from our analysis include COVID-related challenges, the importance of community school managers (CSMs), and COVID response.

6.3.1. COVID-19 Challenges

Much like all schools around the world, the three focal community schools faced many challenges from the COVID-19 pandemic. In addition to the disruption in formal learning through school closures and quarantining procedures, the school staff reported that families felt uncertainty about the physical health and safety of their students. Families expressed concern over their decisions about whether to send their children back to school or keep them at home when schools officially reopened.

The community schools in our study served historically minoritized and underinvested communities, many of which were unequally impacted by the pandemic. Many families sending their children to a community school faced food, housing, and transportation insecurity even before the COVID-19 pandemic occurred. Before the pandemic, the community schools provided additional resources for families to meet their basic needs. The pandemic only exacerbated these challenges. One of the principals described the role of her school within the context of the community:

We’ve always had to support parents. I always tell people … the “hill” like catches a cold, the “hood” gets pneumonia… If you got COVID issues there then…it’s going to be more concentrated here… So we’re already used to, like, supporting families with housing or food. You’re trying to find them a doctor or like, like, those are things that we’ve always had to do.

(Principal Michelle)

Principal Michelle communicates two key dynamics that characterize the community her school serves. First, the school supported families who needed access to basic resources fundamental to healthy family functioning and child development even before the pandemic. Second, Principal Michelle describes how the poor and historically minoritized communities her school serves suffered disproportionately during the pandemic.

The economic impact of the COVID-19 pandemic hit the families with children in the community schools especially hard. Often, the families in the three community schools worked low-wage jobs and/or held jobs that paid hourly wages. Principal Stacy described how the families in her community were especially vulnerable to the negative economic impact of COVID-19:

For families who work hourly wages who get COVID, who are out of work for 14 days; those 14 days are a detriment that they can never recover from. Right, because I’ve lost 14 days of pay when I’m already paycheck to paycheck.

(Principal Stacy)

Many families served by the community schools had no choice but to face the uncertainty of working through the pandemic; potentially expose themselves and their families to COVID-19; and support their children at home. These challenges undoubtedly affected family-school relationships and student learning in the community schools we studied.

With official school closures, food, clothing, and other resources provided by community schools became either unavailable or harder to access. The increased physical and emotional distance between the school and home environments hindered the relationships and trust that the community schools developed with their families prior to the pandemic. Before school closures, Laura stated that students and family members felt comfortable coming to the school to ask for support:

So it is like, “I’m so used to just being able to go to Miss [Laura]’s classroom and say I need deodorant,” or a parent coming to my school and saying “Hey, we don’t really have any groceries. I don’t get my food stamps until the 15th. Do you have any grocery gift cards?” They could just walk to the school.

(Laura, CSM)

These casual and informal interactions helped build relationships and trust between Laura and the families. Laura continued to describe how COVID-19 school closures affected the relationships of CSMs, teachers, and families:

But now, they can’t, you know, I can’t be in contact with them, I can’t touch them, I can’t hug them. So that has been a difficult part, just that connection piece… How do we navigate this relationship when this is not what we’re used to?

(Laura, CSM)

Not only did the community schools serve as a hub to distribute resources for families, but the schools also served as a hub to develop relationships with families and school staff. Especially early on in the pandemic, the lack of in-person contact became a large hurdle to connecting with and supporting families.

Communication between the school and home also became more difficult during the pandemic. The pandemic exposed and exacerbated challenges such as access to technology and housing instability, which consequently disrupted communications between schools and families. Again, Laura elaborated:

So COVID put our parents in a really bad situation where technology wasn’t always accessible, and that was the only way that we communicate with them. So if their technology wasn’t up to date, if their cell phone wasn’t turned on, then now it’s no way to get in contact with them. If they’re going from house to house and we don’t have an accurate address, it is hard for us to get paperwork to them.

(Laura, CSM)

The community schools experienced challenges with communications before COVID-19, such as families changing phones or moving houses throughout the year. However, the pandemic exacerbated these challenges to a degree that made it nearly impossible to contact some families without the physical connection to schools.

While COVID-19 impacted all schools globally, we saw that the schools in our study were unequally impacted by pandemic-related hardships and challenges. Universal problems such as relationship-building and communication were doubly challenging for the schools in our study, given employment pressures, barriers to access resources such as technology, and housing instability. These challenges related directly to student learning experiences during the pandemic: lack of access to technology would inhibit virtual learning; inconsistency in at-home learning environments could disrupt student learning experiences; and navigating virtual learning alone can create myriad barriers for students.

Still, the school personnel, families, and community partners we interviewed worked tirelessly to support their students. Moreover, because of the pre-existing mindset and structures of the community school strategy, these schools may have been optimally positioned to address these challenges and quickly provide support for their families.

6.3.2. Importance of CSMs

Despite the challenges, COVID-19 also illustrated the value of community school managers (CSMs) to schools, students, and families. The importance of the CSM position for the successful functioning of the community school program was repeatedly spoken about in the interviews. Within community schools, CSMs serve as the main coordinator of partnerships and resources within their school. For the community schools we studied, the presence of CSMs was critical throughout the pandemic in providing schools with a specific point person to organize the collection and distribution of resources and services.

During the pandemic, the district received an influx of donations and partnership offers to help families and communities in need. Having the CSM at the school engendered confidence from others in the school district that donated resources would be distributed effectively and efficiently. Cathy, a district staff member who worked directly with the CSMs explained:

Just being able to, like, meet basic needs, I think it’s a huge thing because most [traditional] schools don’t have someone who has the time to like really coordinate that kind of stuff. I think in COVID, in particular, people were coordinating deliveries to homes, so that was really huge.

(Cathy, District Staff)

The CSMs at the community schools were able to support students and families with accessing basic needs and—perhaps more importantly—coordinate the allocation of these resources.

Terry, the district’s coordinator of business partners, confirmed Cathy’s perspective and echoed her sentiments. Terry described how the district was overwhelmed with donations at the beginning of the pandemic and “it was hard to figure out, quickly, how to best allocate those donations”. For her, offering donations to a community school first made the most sense because the CSM was there to coordinate the distribution of those goods in the most effective way possible. Terry explained:

When I work with the [community schools], like, at least they have these people there whose job it is to, like, do the stuff I do, okay. So it’s easier to work with them, than with everyone else…They could do a good job at distributing [donations] to people who need it most.

(Terry, District Coordinator of Business Partners)

These comments suggest that the families at community schools were more likely to receive donations and the donations they need most because of the role CSMs play within their schools. Again, receiving these resources was particularly critical for the community schools in our study as they served students and families from historically underinvested communities—those overwhelmingly impacted by the pandemic. While the families served by the community schools still faced many challenges during this time, the community schools were able to remain constant in the support they provided them.

As a result, the families felt supported by the school and developed trusting relationships with the school and staff. Principal Smith at one of the middle schools shared:

A lot of our parents struggle, a lot of our parents have been impacted by the pandemic, so a lot of them, they call and have a need…There’s a community [who] knows [them] by name and able to call and get support, as opposed to trying to figure out who’s going to support this family right this moment.

(Principal Smith)

Before school closures, CSMs developed authentic relationships with families by providing the support they needed. As a result of having pre-existing relationships, CSMs were comfortable and prepared to support families during the COVID-19 school closures. Despite the compounding challenges that the pandemic created for the schools in this study, the community schools were well-positioned to help families and students navigate these obstacles.

6.3.3. COVID Response

With processes and structures in place to support resource identification and allocation, community schools were able to respond relatively quickly to the needs that arose during the COVID-19 pandemic.

CSMs described identifying areas of need and quickly addressing these challenges. The community schools provided technology and hotspots; found temporary housing for families; organized drive-throughs for families to pick up groceries and schoolwork; and developed partnerships to provide access to therapy. Moreover, CSMs responded to shorter-term, immediate needs when families asked for help. Jasmine, a CSM explained:

[I am] just able to meet needs in a more immediate way. We have a family who needs food, okay, I got a, I got a gift card, boom. We have family who needs a Lyft ride because they got to get their kids to children’s hospital all the way across town, and they’re deaf and hard of hearing and so taking a bus is really, really hard. Okay, let me get this partner who I know does Lyft gift cards.

(Jasmine, CSM)

Having the CSM not only helped coordinate longer-term resources and services, but also allowed for quick responses to immediate needs.

Moreover, teachers knew they could rely on the CSMs to address the needs they identified in their virtual classrooms. Laura explained:

All of our parents and our students are going through so much, and they [the teachers] leaned on me so much this year. Like, you know, “my parents needed internet”, “my parents need a laptop”, “my parents need a computer”, “my parents they just got evicted”, “I got parents that’s not eaten”… So we just, like, [Ms. Laura] “I need this I need this I need that”.

(Laura, CSM)

Teachers could directly address challenges that their students faced in the virtual classroom using the community school strategy. The entire school community relied on the resources provided by the community school strategy to directly and quickly address challenges that impacted students’ academic experiences and beyond.

Because the community schools in this district were also supported at the district level, the district-level leaders were able to identify common needs across schools and quickly identify partners. Amy, a district-level staff working with the community schools, said:

There were certain like trends of needs that were coming up across all of our schools and [we] needed someone to kind of zoom out and build relationships and a network of partners that can support on that across all of our schools.

(Amy, District Staff)

Amy went on to describe how addressing food security was a pressing issue during and following school closures. As the district-level staff, she was able to identify the types of food (e.g., prepared meals, shelf-stable food) that schools needed and develop broader, citywide relationships. Families and students served by the community schools benefited not only from the school-level resources but also from the district-level network of partnerships.

The community school strategy developed longer-term partnerships that provided ongoing support to schools as well as shorter-term resources that addressed immediate needs. These partnerships, resources, and services helped community schools support their students and families during the pandemic. Several resources, such as access to technology and food distribution, directly impacted the learning experiences of students. Without the support provided by community schools, student test scores may have been impacted more negatively due to the disproportionate challenges that students and families in these schools faced.

7. Discussion

In this exploratory study, we examined how community schools in a large urban district experienced and responded to the COVID-19 pandemic. We first descriptively examined school-level performance in standardized exams between 2019 and 2023 across community and traditional schools in a large urban school district. On average, school-level standardized test scores declined over the years of 2019 and 2023 in both ELA and mathematics, aligning with national trends (Irwin et al. 2022). Also following national-level trends, schools reported larger declines in the average scale scores in math (8.16-point decrease) than in ELA (4.95-point decrease).

Community schools reported similar decreases in state assessment scale scores between 2019 and 2023 compared to traditional schools. The small sample size for community schools greatly limits the interpretability and generalizability of these findings. Still, the community schools that serve historically disinvested communities exhibited similar declines in test scores compared to the traditional schools in this district. Given that historically disinvested communities were disproportionately impacted by the pandemic, we speculate that the community school strategy may have helped mitigate some of this impact. Future studies can examine trends in assessments across traditional and community schools with larger samples.

The qualitative data show how community schools navigated the pandemic and supported their students. Again, the three focal schools we studied served historically underinvested and minoritized communities within the district. The COVID-19 pandemic exacerbated challenges that the community schools were already trying to address before the pandemic. Prior research links these environmental challenges, such as food insecurity and housing instability, with lower academic, social, and developmental outcomes for students (e.g., Alaimo et al. 2001; Jyoti et al. 2005; Obradović et al. 2009; Rafferty et al. 2004; Rubin et al. 1996). These challenges only increased during the pandemic.

Other challenges, such as access to technology, became more pronounced during the pandemic and directly impacted student experiences with virtual learning. Lack of access to technology or unstable Wi-Fi would inhibit students from accessing their virtual learning. Given these compounded challenges, one may expect the students to similarly experience a compounded decline in test scores. Yet, the community schools, on average, reported similar—if not slightly smaller—declines in test scores compared to traditional schools.

We believe the infrastructure that community schools started to build before the pandemic may have helped mitigate some of the negative impacts on student academics. The community schools had already started to establish the four pillars of community schools (integrated student supports; expanded learning time and opportunities; family and community engagement; and collaborative leadership and practices) to address challenges in their communities prior to the pandemic. The pandemic increased the urgency, amount, and types of support that families needed, but establishing the four pillars in these schools may have helped prime them to address many of the COVID-related challenges. On the other hand, the traditional schools in this district may have faced these challenges for the first time. Rather than creating something new, community schools may have been able to increase the breadth and volume of the supports they were already providing their families and communities, while also addressing short-term, immediate needs.

For instance, having a CSM made the schools more likely to receive donations and could more effectively distribute the resources to families. Prior research has documented the importance of CSMs at schools especially for facilitating resource allocation to students and families (Hine et al. 2023; Sanders et al. 2019). On the other hand, traditional schools would have had to identify a new person to take on the responsibility of gathering donations and creating a new process for distributing resources to families.

Community schools also quickly pivoted the types of support they provided their students, such as access to technology. Teachers, principals, CSMs, and district-level staff in community schools identified needs or gaps, communicated these challenges, and found ways to support their students. Again, because many of the structures (e.g., CSMs, teams) were in place before the pandemic, the overall process of supporting students and families remained relatively consistent, even if the content or amount of support changed. These resources directly and indirectly supported student learning and development. We speculate that students may have reported greater declines in learning if these supports were not available.

Even though the pandemic pulled the school, home, and community spheres apart, preexisting relationships, trust, and the community schools’ focus on the four pillars helped pull these spheres back together relatively quickly. As a result, families could communicate their needs to schools, schools could reach out to community partners for support, and students could receive cohesive support across all spheres. These supports both directly and indirectly supported student learning experiences.

These exploratory findings are consistent with other recent research suggesting that schools with strong family engagement navigated the COVID-19 pandemic more successfully. A recent study by Learning Heroes and TNTP (2023) found that schools with strong family engagement experienced smaller drops in student attendance and lower increases in chronic absenteeism than those with weak family engagement. These findings, combined with those presented here, provide emergent evidence of the critical need for our education systems to adopt a whole-child approach using well-implemented and equitable programs of family and community engagement to support student learning and well-being, especially through local and global crises.

In alignment with the four pillars, the community schools in this study were developing processes, structures, and relationships to holistically support students and families—even before the pandemic exacerbated some needs. Moreover, the community schools had been working to bring the school, home, and community spheres together through relationship- and trust-building prior to the pandemic. Having the community school strategy in place allowed these schools to turn up the volume of this approach, rather than reactively create new processes and structures. Thus, students were holistically supported both by the four pillars of community schools and by the overlap in their school-home-community environments.

While the findings are promising, this study emphasizes the need for further research and a deeper exploration of the relationship between community school strategies and student learning experiences. The qualitative findings point to specific resources or services that were provided to improve student learning experiences. Future research can examine if these are the only ways that community schools impact student learning, or if and how other factors (e.g., sense of belonging or safety, a more collaborative staff environment, more staff, etc.) also enhance student learning.

Interestingly, just under a quarter of the schools in this district experienced an increase in their state assessment scores. This increase may be reflective of various factors including learning recovery efforts implemented by schools and the district; some students preferring the virtual learning environment; trends in student enrollment; and general fluctuations in standardized test scores across years. Future research should examine schools that report “learning gains” to better understand these changes and to find ways to support learning recovery in the coming years.

A major limitation of this study is the small number of community schools included in the sample. The quantitative data are purely descriptive and we do not claim causal relationships. While the qualitative data provide further insight to how the schools navigated the pandemic, the generalizability of these findings is greatly limited. Future studies should include a larger sample of community schools by examining a school district with more community schools or finding multiple, comparable districts with community school initiatives. Our study was limited in the analyses we could conduct because there were only 11 community schools in comparison to the nearly 100 traditional schools.

Moreover, while we had student-level data, we experienced challenges tracking students over time, across grades/schools, and between school types (i.e., switching between traditional and community schools). Future analyses can find more sophisticated ways of including these nuances in their analyses to examine student-level experiences. In addition, future studies should incorporate data from families to investigate their perspectives on navigating community schools and their experiences or outcomes. Understanding the specific mechanisms through which community schools influence both school- and individual-level outcomes will help guide more targeted and effective interventions.

Despite these limitations, this exploratory study provides insight into community schools during the COVID-19 pandemic and their students’ outcomes. Undoubtedly, access to services and resources provided by the community school strategy helped address some needs during a challenging time. While community schools still faced many challenges during the pandemic, these schools were better positioned to accept and distribute donations, support families, and develop partnerships with community organizations. Ultimately, the community school’s whole-child approach that existed prior to the pandemic likely contributed to their speedy and holistic response to COVID-19-related challenges.