Do Community Schools Work for High-Needs Students? Evaluating Integrated Student Support Services and Outcomes for Equity

Abstract

This study examines whether and how community schools’ integrated student support services (academic, socioemotional, health, and family support) contributed to improving whole-child/youth development and reducing systemic inequalities of students’ learning/wellness outcomes across New York State under the Every Student Succeeds Act (ESSA). Applying a quasi-experimental method with propensity score matching to the state’s 2018–2023 school survey and report card databases, it provides new evidence on the efficacy of community school programs on average and by subgroups (race/ethnicity, poverty, disability, English language learner, and housing status). The results of matched comparisons between community schools and non-community schools are mixed, after considering their differences in terms of student demographics and baseline conditions. Overall, community schools showed policy implementation fidelity with more state funding, policy-aligned practices, and school-based health centers/clinics. However, community schools had no discernable impacts on academic achievement and chronic absenteeism overall, except that the operation of school-based health centers was associated with a reduction in absenteeism. In contrast, community schools had more positive impacts on high school graduation rates, particularly among disadvantaged minority students; the impacts are attributable to policy-aligned practices, set-aside funding, and school-based health center dental programs. Educational policy and research implications are discussed.

1. Research Objectives and Significance

The primary objective of this research project, HELP (Health-Education-Life Protection), is to investigate and improve community school programs for high-needs students. These programs involve integrated student support services and enriched whole-child/youth development and learning opportunities. In light of concerns about the post-pandemic crises of students’ learning/wellness losses and inequalities across the nation (Lee et al., 2024), it is crucial to inform and improve policy on full-service community schools as an evidence-based intervention strategy for promoting educational equity under the Every Student Succeeds Act (ESSA). According to the NY Commissioner of Education, one silver lining during the pandemic was breaking the silos across state government agencies (i.e., education, health/mental health, social work, labor) for the sake of whole-child support services (Rosa, 2022). Despite the promise of such a holistic cross-sector intervention approach, community schools must cope with the challenges of program/service incoherence, competition, and fragmentation (Adelman & Taylor, 2022).

In New York State, there are currently 829 community schools (about 17% of all public and charter schools in the state) according to the New York State Education Department (NYSED) 2022–2023 Basic Education Data System (BEDS) survey; the number is on the rise with increased state funding and policy support. However, it is not yet known how effectively they work and what needs to be improved. Prior research evidence on the efficacy of community schools exhibits limited generalizability, selection biases, and a lack of information on program/service variability and on the potential heterogeneity of treatment effects (e.g., subgroup differences by students’ race/ethnicity, poverty, disability, English language learners, and housing status). This calls for large-scale, state-representative, and equity-oriented data analyses of community school impacts that explain what types of community school programs and services work (or do not) and for whom and how.

This study is designed to explore the key mechanisms and effects through which the community school system enhances whole-child/youth development and contributes to the systemic reduction of racial and socioeconomic inequalities in students’ learning and wellbeing across the state. This study will help fill a gap in our knowledge base towards an evidence-based policy, particularly with respect to whether and how the provision of integrated student support services in community schools is empirically linked to the systemic improvement in whole-child/youth development outcomes among student subgroups.

2. Theoretical and Analytical Framework

The study is premised upon a theory of change that stresses the power of partnership for integrated student support services (academic, socioemotional, health, family/community support) as a mechanism to reduce risks (e.g., disengagement, misbehavior, course failure) and increase assets (e.g., self-efficacy, resilience, access to health care, and participation in after-school learning) simultaneously, thereby promoting desired whole-child/youth outcomes. It builds on the Positive Youth Development (PYD) framework, which encompasses characteristics across developmental contexts and strengths (Benson et al., 2011; Syvertsen et al., 2021). Based on the ‘hierarchy of needs’ model (Maslow, 1943), we posit that students’ needs for health, wellness, and a sense of belonging must be met before they can be both physically and mentally ready for academic learning and achievement. Further, we utilize the ideas of ‘shared accountability’ (Lee et al., 2019) and ‘collective impact’ (Karnia & Kramer, 2011) to help break the silos of the education and health care systems and promote school–family–community partnerships towards whole-child/youth development support.

Numerous studies suggest positive benefits of community schools in terms of improving chronic absenteeism, disciplinary referrals, reading/math achievement, and high school graduation rates (Anderson et al., 2019; Adams, 2010; Caldas et al., 2019; Biag & Castrechini, 2016; Heers et al., 2016; Houser, 2016; Johnston et al., 2020). Existing cost-benefit research also suggests an excellent return on investment of up to $15 in social value and economic benefits for every dollar spent on school-based wraparound services (Maier et al., 2017) or approximately $7 in net benefits for each $1 invested in a community school coordinator (Bloodworth & Horner, 2019). Notable examples include the Harlem Children’s Zone and Promise Neighborhood programs, which have shown promising results through comprehensive social services delivered in conjunction with schools (Dobbie & Fryer, 2011).

However, there is a dearth of studies examining community school partnerships for children’s health care (Johnson-Shelton et al., 2015), and only a few studies have examined the impact of school-based health centers (SBHC) on students’ health care access and academic outcomes (Lim et al., 2023). A meta-analysis of the relationship between SBHC services and student outcomes showed mixed results; it found either a weak or no relationship for academic outcomes such as better attendance, improved test scores, and higher grade point average but a strong relationship for health-related outcomes such as the use of alcohol, tobacco, and drugs, mental health problems, and high-risk sexual behavior (Geierstanger et al., 2004). Further, previous studies did not fully examine community school impact mechanisms and equity implications for disadvantaged and marginalized groups. Therefore, there is a pressing need for more comprehensive and systematic evaluation research that assesses the impacts of community school programs with integrated student support services on diverse student subgroups.

Specifically, this study addresses the following research questions: (1) What are the key characteristics of New York State community schools in terms of student demographics and needs? How do community schools differ from non-community schools in their pre-pandemic baseline status of student engagement and achievement? (2) What integrated student support services do community schools provide to students during and after the pandemic? How well do community schools align their practices with state policy, receive state set-aside funding, and implement student health support programs (school-based health centers, dental programs, mental health clinics)? and (3) How well did community schools work to improve student outcomes (academic achievement, graduation, absenteeism) over the 2018–2023 school years? Did community school programs work better for high-need students and thus help narrow the engagement/achievement gaps among diverse subgroups of students (by race/ethnicity, poverty, disability, English language learners, and housing status)?

3. Data and Methods

This study employs quasi-experimental and survey research methods to address the research questions outlined above. First, we utilize pre-existing statewide school survey and report card datasets. We begin with a descriptive analysis of school-level data (N = 4769 schools, including 4420 public schools and 349 charter schools), collected from the NYSED BEDS school survey (see questions below). Here, question number 2 refers to the following practices: (1) partnerships between schools and community resources; (2) integrated student support services (e.g., academic support, health and social services, family and community engagement); and (3) expectations of producing consistently better learning opportunities and results (e.g., the creation of safe and supportive learning environments and improvements in school attendance, academic achievement, socioemotional, mental and physical health).

• Is this school a community school?
• Is this school actively and intentionally working toward meeting practices articulated in the Community Schools description provided in the instructions?
• Does this School receive funding from the Community Schools Foundation Aid Set-Aside? (Note: Charter school BEDS forms do not collect this information as it is not applicable).
• Is there a New York State Department of Health-approved School-Based Health Center operating at this school’s location?
• Is there a New York State Department of Health-approved School-Based Health Center Dental Program operating at this school’s location?
• Is there a New York State Office of Mental Health-approved School-Based Mental Health Clinic or satellite provider operating at this school’s location?

Next, we link the BEDS survey data with New York State school report card data (2018–2023) to examine the association between community school status and student outcomes. These outcomes include (1) Grades 3–8 English Language Arts (ELA) and Math Proficiency on New York State Assessments; (2) High School ELA and Math (Algebra I, Algebra II, Geometry) Proficiency on New York State Regents Exams (Passing Required for a Regents Diploma); (3) High School Graduation Rates; and (4) Chronic Absenteeism Rates for both Elementary/Middle and High Schools. As New York State school report card data are not fully available for the pandemic period (2020 and 2021 years), we only use data from 2018, 2019, 2022, and 2023 in this study. Since the 2016–17 school year, New York State has initiated the Community Schools Set-Aside funding with $100 million, which gradually increased to $250 million in 2019–2020. The state also established three regional Community Schools Technical Assistance Centers to assist the implementation/development of community schools across 240 districts throughout the state.

We conducted a series of t-tests to compare student outcome gains from 2018 to 2023 between community schools (CSs) and non-community schools (non-CSs). We also examined and compared the features of community schools vs. non-community schools in terms of student/family background characteristics, which may affect the types of services provided and the outcomes achieved. Both statistical significance (p-values) and practical significance (effect sizes, including Cohen’s d and odds ratios) are reported to assess differences between CSs and non-CSs.

Considering CS vs. non-CS demographic differences and other pre-treatment differences, we used propensity score matching to explore the impact of community schools on integrated student services and outcomes. This involved an equity-oriented subgroup analysis to examine the heterogeneity of community school impacts by students’ race/ethnicity, poverty, English language learner (ELLs), disability (students with disabilities, SWD), and housing status. Appendix A provides detailed information on all the variables and analytic methods used.

To address potential selection bias for CS vs. non-CS comparisons, we used propensity score matching with inverse probability of treatment weighting (IPTW). IPTW adjusts for selection bias by assigning differential weights to units based on the inverse probability of receiving treatment at a given time, conditional on prior outcome history and other covariates (Hirano & Imbens, 2002; Rosenbaum & Rubin, 1984). Applying IPTW, we conducted multivariate, multilevel regression analyses using (1) Stata program version 18 (teffects ipw) and (2) Hierarchical Linear Models (HLM growth curve model) to examine the effects of community school programs on student outcomes. We also conducted mediation analyses of program effects via IPTW regression, exploring the effects of mediators (CS practices, funding, health, mental health and dental health support) on student outcomes. We conducted sensitivity analysis to address potential unobserved confounders by computing Rosenbaum bounds (Rosenbaum, 2002).

4. Findings

4.1. Descriptive Analysis of CS vs. Non-CS Differences

Table 1. School demographics, student support programs and services, and student outcomes by community school status.

	Community Schools (CS)				Non-Community Schools (Non-CS)				CS vs. Non-CS Gaps
Variable	N	M (SD)	Min	Max	N	M (SD)	Min	Max
Demographics (2022/23)									Cohen’s d
% Black	663	26.15 (24.95)	0	90	4054	16.31 (21.77)	0	90	0.44 ***
% Hispanic	663	35.70 (28.67)	0	100	4054	26.81 (24.46)	0	100	0.35 ***
% Asian	663	4.25 (7.84)	0	68	4054	8.16 (13.35)	0	93	−0.31 ***
% White	663	30.06 (37.09)	0	100	4054	44.48 (35.20)	0	100	−0.41 ***
% ELLs	663	13.38 (16.17)	0	98	4054	9.25 (12.62)	0	96	0.31 ***
% Eco. Dis.	663	77.02 (19.19)	0	99	4054	54.52 (27.37)	0	100	0.85 ***
% SWD	663	22.11 (10.38)	0	100	4054	19.71 (13.20)	0	100	0.19 ***
% Homeless	663	8.04 (8.17)	0	49	4054	3.77 (5.70)	0	51	0.70 ***
Student support programs/services (2022/23)									Odds Ratio
CS Practice	663	0.98 (0.15)	0	1	4072	0.03 (0.18)	0	1	1309.96 ***
CS Funding	624	0.73 (0.44)	0	1	3781	0.03 (0.16)	0	1	99.51 ***
Health center	663	0.30 (0.46)	0	1	4072	0.08 (0.27)	0	1	4.89 ***
Dental clinic	663	0.42 (0.49)	0	1	4072	0.01 (0.12)	0	1	49.66 ***
Mental clinic	663	0.39 (0.49)	0	1	4072	0.11 (0.31)	0	1	5.40 ***
Student outcomes (2018/19–2022/23 [gain])									Cohen’s d
% ELA proficient	601	3.02 (10.34)	−39.0	44.0	2800	1.12 (10.17)	−38.0	47.0	0.18 ***
% Math proficient	544	4.19 (12.59)	−52.0	49.0	2505	3.82 (11.22)	−47.0	67.0	−0.03
% HS Graduation	219	7.83 (9.62)	−12.9	39.8	931	2.88 (5.81)	−16.9	37.4	0.74 ***
% HS absenteeism	230	−11.15 (15.24)	−59.0	28.6	989	−11.71 (13.72)	−67.4	42.5	0.04
% EM absenteeism	618	−13.19 (10.40)	−55.7	19.7	2881	−11.37 (7.79)	−89.4	19.4	−0.22 ***
% Regents ELA passing	233	−4.96 (10.59)	−41.0	26.0	1021	−4.55 (8.79)	−38.5	35.5	0.05
% Regents Algebra I passing	359	−9.98 (16.72)	−74.0	52.5	1685	−7.29 (12.18)	−92.0	50.5	−0.21 ***
% Regents Algebra II passing	185	−17.42 (18.78)	−77.0	43.5	873	−15.28 (16.83)	−90.5	50.5	−0.12
% Regents Geometry passing	203	−14.71 (15.30)	−73.0	53.5	945	−14.72 (15.30)	−73.0	53.5	0.00

Notes. Eco. Dis. (=Economically disadvantaged students), ELLs (=English language learners), EM (=Elementary/middle school), HS (=High school), SWD (=Students with disabilities). *** p < 0.001.

shows the descriptive statistics of school demographics (2022/23), student support programs and services (2022/23), and student outcomes (2018/19–2022/23 gain) by community school status in 2022/23. In terms of demographic factors, community schools (CS) had a significantly higher percentage of Black students (26.2% vs. 16.3%, d = 0.44, p < 0.001), Hispanic students (35.7% vs. 26.8%, d = 0.35, p < 0.001), English language learners (ELLs) (13.4% vs. 9.3%, d = 0.31, p < 0.001), economically disadvantaged students (77.0% vs. 54.5%, d = 0.85, p < 0.001), students with disabilities (22.1% vs. 19.7, d = 0.19, p < 0.001), and students experiencing homelessness (8.0% vs. 3.8%, d = 0.70, p < 0.001) than non-community schools, whereas non-community schools had higher percentages of Asian (4.3% vs. 8.2%, d = −0.31, p < 0.001) and White students (30.1% vs. 44.5%, d = −0.41, p < 0.001).

In terms of student support programs and services, community schools had a significantly higher percentage of active and intentional efforts to follow the CS practice guidelines (98% vs. 3%, odds ratio = 1309.96, p < 0.001), set-aside funding (73% vs. 3%, odds ratio = 99.51, p < 0.001), school-based health centers (30% vs. 8%, odds ratio = 4.89, p < 0.001), dental programs (42% vs. 1%, odds ratio = 49.66, p < 0.001), and mental health clinics (39% vs. 11%, odds ratio = 5.40, p < 0.001) than non-CSs (see Figure 1).

As for the student outcomes, community schools had mixed results. On the one hand, CSs reported a significantly higher ELA proficiency gain (3.02 vs. 1.12, d = 0.18, p < 0.001), a higher graduation rate gain (7.83 vs. 2.88, d = 0.74, p < 0.001), and a higher elementary/middle school chronic absenteeism gain (−13.19 vs. −11.37, d = −0.22, p < 0.001), as well as a significantly higher loss in Regents Algebra I passing rate (−9.98 vs. −7.29, d = −0.21, p < 0.001) from 2018/19 to 2022/23 than non-CSs. On the other hand, there were no significant differences in overall ELA proficiency rates, overall math proficiency rates, and high school chronic absenteeism gains from 2018 to 2023.

4.2. Matching and Balance Check Analysis of Covariates

The above comparisons of student outcome gains are made before matching that takes into account the CS vs. non-CS differences of student demographics and baseline outcomes. In this section, we address this issue of selection bias based on observed confounders (covariates). To assess the group mean differences of all covariates between community schools and non-community schools before and after matching, we conducted independent samples t-tests and computed effect sizes (standardized group mean differences). In Appendix B, we report the results of balance checks between the two groups: community schools and non-community schools. Additionally, to compare differences across the three groups—non-community schools, one-year community schools (1-year CSs), and two-year community schools (2-year CSs)—we also performed two separate independent samples t-tests with effect sizes.

Before matching, community schools had significantly higher percentages of Black students, Hispanic students, English Language Learners (ELLs), economically disadvantaged students, students with disabilities, and students eligible for free or reduced-price lunch. Additionally, community schools had higher rates of teaching out of certification and higher chronic absenteeism at both the elementary/middle school and high school levels. In contrast, community schools had significantly lower percentages of White and Asian students, lower baseline (2018/19 school year) ELA and mathematics proficiency levels, and lower high school graduation rates. Community schools also had smaller school enrollments compared to non-community schools.

After matching, however, community schools no longer exhibited different characteristics in any of the above variables either statistically or practically, compared to non-community schools. Therefore, the results of the balance check analysis demonstrate that propensity score matching effectively eliminated observable selection biases.

4.3. Trend Analysis of CS vs. Non-CS Student Outcomes

We compared the trends in student outcomes among CS and non-CS samples over the 2018–2023 period, both before and after matching. Here, we highlight only a few selected line graphs for selected variables due to space limitations, but all the graphs are available upon request.

First, the trends of average Grade 3–8 ELA proficiency gaps in New York State remained relatively stable over time, with 14–15% (percentage point) higher performance for non-community schools than community schools. The average math proficiency was also consistently higher for non-community schools than for community schools across the years (see Figure 2). Proficiency rates dropped for both groups in 2022, perhaps due to the impact of the pandemic, but rebounded in 2023, returning to pre-pandemic levels. The matched trends eliminate CS vs. non-CS differences in their baseline status of proficiency but did not show differences in their growth rates over the study period.

Second, we compared the trends in high school Regents exams proficiency (passing) rates for ELA, Algebra I, Algebra II, and Geometry. High school students attending community schools consistently had lower passing rates than those in non-community schools—approximately 18% lower in Geometry, 16% in Algebra II, 14% in Algebra I, and 12% in ELA. While all four subjects saw notable declines in passing rates following the onset of the COVID-19 pandemic, the performance gap between community and non-community schools remained relatively stable from 2018 through 2023. The matched sample trends also show similar patterns.

Third, we compared the trends in the New York State chronic absenteeism for high schools and elementary/middle schools. High schools had higher levels of chronic absenteeism than elementary/middle schools regardless of community school status. For unmatched trends, high schools identified as community schools had about 10% point higher chronic absenteeism than non-community high schools (see Figure 3). Elementary/middle community schools also had similarly higher chronic absenteeism than non-community counterparts. Despite some recent recovery, the pandemic-peaked chronic absenteeism rate did not yet revert to its pre-pandemic level among both CS and non-CS groups. The matched trends also did not show any noticeable signs of narrowing the disparities between the two groups; the gaps in absenteeism rates between CSs and non-CSs persisted.

Lastly, we compared the trends in the New York State high school graduation rates across the years (see Figure 4). The graduation rates were higher for non-community schools than for community schools across the years. The rates were on the rise for both groups, but more so for the community schools, especially in 2022, indicating a reduction in the graduation rate disparity over time. A similar growth pattern was observed for the matched trends in high school graduation rates despite their relatively smaller gaps.

4.4. IPTW Regression Analysis of CS Treatment Effects

Building on visual inspections of the outcome trends above, we conducted IPTW regression analyses to estimate the average treatment effects of CS programs and services.

Table 2. IPTW estimates of community school (CS) average treatment effects on student outcome gains during 2018–2023.

	ELA Proficiency
	Grade 3	Grade 4	Grade 5	Grade 6	Grade 7	Grade 8	Grades 3–8
CS vs. Non-CS	−1.22 (0.70)	−0.12 (0.65)	−0.59 (0.63)	−0.54 (0.86)	−0.73 (0.93)	−0.29 (1.24)	−0.29 (0.47)
CS Duration: (1 vs. 0 Year) (2 vs. 0 Year)	−1.66 (0.98)	−1.39 (0.87)	−1.21 (0.82)	−0.27 (0.98)	−0.31 (1.11)	−1.54 (1.13)	−0.65 (0.60)
	−1.10 (0.79)	0.63 (0.82)	−0.52 (0.77)	−0.28 (0.85)	−1.06 (0.93)	−1.40 (0.99)	−0.44 (0.48)
	Math Proficiency
	Grade 3	Grade 4	Grade 5	Grade 6	Grade 7	Grade 8	Grades 3–8
CS vs. Non-CS	−1.56 * (0.77)	−1.31 (0.73)	−0.96 (0.68)	−1.50 (0.79)	−0.72 (0.91)	−1.52 (0.98)	−0.82 (0.47)
CS Duration: (1 vs. 0 Year) (2 vs. 0 Year)	−1.76 (1.01)	−2.41 * (1.00)	−1.66 (0.97)	−1.48 (1.16)	−0.78 (1.25)	−0.71 (1.61)	−0.99 (0.64)
	−1.30 (0.92)	−0.87 (0.98)	−0.28 (0.91)	−0.25 (0.94)	−0.54 (1.02)	−1.47 (1.15)	−0.93 (0.53)
	HS Regents Proficiency
	ELA		Algebra I	Algebra II		Geometry
CS vs. Non-CS	1.70 (1.28)		0.49 (1.07)	0.57 (1.71)		1.74 (1.68)
CS Duration: (1 vs. 0 Year) (2 vs. 0 Year)	−1.94 (1.25)		−0.12 (1.75)	−2.34 (2.07)		−1.46 (2.05)
	−0.34 (0.93)		0.52 (1.28)	0.50 (2.09)		−2.43 (2.06)
	Chronic Absenteeism and Graduation
	E/M Absenteeism		HS Absenteeism			HS Graduation
CS vs. Non-CS	0.03 (0.49)		−0.83 (1.03)			3.78 ** (1.16)
CS Duration: (1 vs. 0 Year) (2 vs. 0 Year)	0.05 (0.68)		0.53 (1.47)			−0.51 (0.71)
	−0.07 (0.61)		−1.87 (1.32)			1.49 * (0.65)

Note: * p < 0.05, ** p < 0.01.

below shows the results of CS effects for all outcomes, including the estimates of overall CS effects, as well as the estimates of one-year and two-year CS effects differentiated by program implementation duration.

First, Table 2 presents the IPTW-estimated effects of community schools (CSs) on grade 3–8 ELA and mathematics proficiency rates, both separately and combined. The results show that grade 3 students in CSs had significantly lower math proficiency than those in non-CSs (B = −1.56, SD = 0.77, p < 0.05), as did grade 4 students in one-year CSs compared to non-CSs. However, no significant differences were found in ELA or combined proficiency rates across grades 3–8, nor in math proficiency for grades 5–8. Second, Table 2 displays the IPTW-estimated effects of CSs on high school Regents exam passing rates (ELA, Algebra I, Algebra II, and Geometry), using both binary (CSs vs. non-CSs) and three-group comparisons (1-year CSs vs. non-CSs; 2-year CSs vs. non-CSs). No significant differences were observed in passing rates across any of the four subjects under either comparison. Third, Table 2 shows the IPTW-estimated effects on chronic absenteeism in elementary/middle schools and high school graduation rates across all groups. Students attending CSs had significantly higher high school graduation rates than their non-CS peers (B = 3.78, SE = 1.16, p < 0.01). Among multiple treatment comparisons, students in two-year CSs also had higher graduation rates than those in non-CSs (B = 1.49, SE = 0.65, p < 0.05).

For student subgroup results, we only report high school graduation outcomes here to show evidence on the heterogeneity of treatment effects. We did not find any major subgroup differences for other outcome variables (all other outcome subgroup reports are available upon request). Figure 5 presents subgroup analyses of the IPTW-estimated effects on high school graduation rates. In the binary comparison, Black, Hispanic, and economically disadvantaged students in CSs had significantly higher graduation rates than their counterparts in non-CSs (B = 6.44, SE = 2.53, p < 0.05 for Black; B = 7.29, SE = 1.91, p < 0.001 for Hispanic; B = 3.43, SE = 1.13, p < 0.01 for economically disadvantaged). In the multiple treatment comparison, Hispanic and economically disadvantaged students in two-year CSs also had higher graduation rates than those in non-CSs (B = 3.89, SE = 1.19, p < 0.01; B = 1.71, SE = 0.77, p < 0.05, respectively).

We also conducted mediation effect IPTW analyses by categorizing community schools (CSs) based on the types of programs and services they offered, including the presence of CS practices, dedicated CS funding, school-based health centers, dental programs, and mental health clinics. The results (not fully shown here but available upon request) indicated that the presence of CS practices, CS funding, and dental programs was positively associated with high school graduation rates (B = 3.19, SE = 0.82, p < 0.001 for CS practices; B = 3.18, SE = 1.06, p = 0.003 for CS funding; B = 2.77, SE = 1.02, p = 0.007 for CS dental programs). Additionally, the offering of school-based health centers was negatively associated with high school chronic absenteeism rates (B = −3.13, SE = 1.37, p = 0.02). The effects of mental health clinics were all statistically insignificant.

Additionally, we conducted Rosenbaum’s bounds sensitivity analyses by using the rbounds command in Stata for robustness check on our estimate of community school impacts, particularly high school graduation rates. The results indicated statistical significance up to gamma (i.e., hidden bias parameter) values of about approximately 1.2–1.3 for all students and for economically disadvantaged students, suggesting moderate robustness to hidden bias. This means that an unobserved confounder would need to increase the odds of treatment assignment by about 20–30% to nullify the observed effect at the alpha = 0.05 significance level. For Hispanic students, the results remained statistically significant up to a gamma of 1.7, indicating strong robustness to hidden bias. In contrast, for Black students, the results were only statistically significant up to a gamma of 1.0, suggesting that the findings are not robust to hidden bias.

4.5. Discussion of Findings and Policy Implications

There is no doubt that there was a sharp decline in academic performance and a sharp increase in chronic absenteeism across the state of New York during the pandemic. Although there were signs of recent recovery, measures of these student outcomes are still worse than pre-pandemic levels. These problems are particularly pronounced among high-need students, including disadvantaged minoritized students. Have community schools been better able to cope with post-pandemic crises of student learning and wellbeing declines and disparities? In light of these concerns, this study examined the efficacy of the community school approach as an evidence-based school improvement strategy under the ESSA that involves integrated student support services empowered by dedicated government funding and school–family–community partnerships to promote whole-child development.

We fully acknowledge that community schools were initially different from non-community schools in terms of school type/location, demographics, and baseline measures of student outcomes. Ignoring these pre-existing differences may cause potential biases in evaluating the impact of community school programs and services. Therefore, our study incorporated a wide range of covariates for matching and ensured balance checks to support fair program evaluation. This study relied on both survey and school report card data to compare community schools with non-community schools in terms of academic achievement, absenteeism, and graduation outcomes over the period of 2018–2023 years (excluding the 2020–2021 pandemic years due to data limitations).

The results have been mixed so far in our study. On the one hand, community schools have neither improved overall academic achievement nor reduced chronic absenteeism better than non-community school counterparts. An exception was that the operation of school-based health centers was associated with a reduction in chronic absenteeism at the high school level; this finding is consistent with some prior research (Cura, 2010; Lim et al., 2023). On the other hand, there were more positive impacts on high school graduation rates, particularly among disadvantaged minority student groups. The effects range from 3- to 7-percentage-point gains (average effect size d = 0.65) and seem to have contributed to raising CS high school graduation rates from the ‘70s to the ‘80s since the pandemic, nearly closing the gap relative to their non-CS counterparts.

Mediation effect analysis of the community school mechanism suggests that those positive impacts on high school graduation rates may be attributable to state policy-aligned best practices, set-aside funding, and school-based dental programs. On the latter finding, prior research found an association between children’s oral health and school attendance (Guarnizo-Herreño et al., 2019), and thus school-based dental care services may help to mitigate the oral discomforts that interfere with school engagement. However, no significant benefits were found from the provision of mental health support services. The lack of significant findings regarding mental health support deserves deeper exploration, as it may reflect data constraints, weak implementation, or limited student engagement.

These findings support community schools as evidence-based intervention strategies to facilitate the post-pandemic recovery of students’ learning and wellbeing. A large majority of community schools reported receiving set-aside funding support and actively aligning their practices with the state’s policy guidelines of best practices. This demonstrates the need to sustain or expand funding support for school–family–community partnerships and integrated student support services. At the same time, however, the study results raise some questions about why the current policy had limited impacts and how to improve student outcomes further. This study did not find any evidence of impacts of community school programs on academic achievement in either ELA or math across grade levels. It is possible that it takes longer to improve academic achievement due to the pandemic-induced learning interruption and delay. Further studies are needed to follow up on the longer-term academic trends. However, the finding is largely consistent with prior research that showed no effect or only a small effect on student achievement test scores in elementary and high schools (Johnston et al., 2020; Somers & Haider, 2017). This might be attributable to the fact that most of the community school programs do not focus on academic improvement such as improving state test scores or the quality of instruction provided during regular school hours (Somers & Haider, 2017). Prior research has found stronger effects on student engagement (e.g., absenteeism) and attainment (e.g., graduation rates) than on academic achievement as measured by standardized test scores. This discrepancy in outcomes warrants further investigation.

Communications with selected school district administrators and staff in the New York State Department of Education have revealed the current policy limitations and challenges of community school initiatives. First, it turned out that the amount of funding per district was minimal and would likely not have a great impact on sustainable programming in schools. Among community schools, fewer than half operated school-based health centers, dental clinics, and mental health clinics. Second, there is a concern about “fidelity of implementation” due to the lack of common, operationally defined parameters of community school practices. While there are guiding principles, there are no official binding requirements for schools to operate as community schools. Although this flexibility provides schools with autonomy, it may undermine program coherence and accountability. Third, there is a concern about “communication” between the key stakeholders of community schools. For instance, some community school coordinators did not even know that their schools received set-aside funding (part of the community school grant program started in 2016–2017 and set aside the amount of $250 million since 2019) from the state. Further, our review of all community school websites across the state reveals that they do not always provide clear and well-organized information on student and family support services. Fourth, there might be a shortage of “support infrastructure and personnel” to ensure program quality. While the state has provided funding for three regional Community Schools Technical Assistance Centers that are supposed to support schools’ professional development and capacity-building, these are not sufficient to serve more than 800 community schools across the state. Lastly, our finding of limited academic impacts raises a concern about the instructional efficacy of community schools, although the longer-term impacts across the state remain to be seen. We recommend better integration of community-based learning opportunities (e.g., hands-on learning activities about community environmental issues) as a way to improve students’ academic engagement and knowledge transfer. Active partnerships with community-based organizations (CBO) and local colleges/universities can enhance a wider range of learning opportunities for students.

5. Research Limitations and Recommendations

Our study has several limitations and caveats that warrant considerations for subsequent research. First, one challenge in evaluating implementation lies in the autonomy that community schools have to adapt programming based on local needs. This flexibility is important but can lead to inconsistencies across sites, which may complicate evaluation efforts. It also raises questions about the role of external support. Future research needs to address several questions: What types of support (e.g., technical assistance, training, partnerships) are available to schools, and how do these affect implementation? How might outcomes differ between schools that receive structured support and those that do not?

Second, the mixed findings overall and the concern about fidelity of implementation raise the issue of take up. Are the students who actually take up and use community school services experiencing better outcomes and is this effect masked by the students who do and do not take up the services being lumped together? Future research needs to address potential issues with students’ service access and use. How do student outcomes vary by the extent to which they take up those services and the extent to which the services actually meet their needs? This issue calls for subsequent research that takes into account not only students’ access to services but also their actual use of services in community schools. Our current estimates of program effectiveness are based on the analysis of “intent to treat” (ITT) effects as opposed to “treatment on the treated’ (TOT) effects—effects among students who fully received treatment as intended. Thus, this analytical approach for estimating the TOT effects needs to measure and differentiate the level of students’ service use (i.e., treatment dosage receipt) and their related characteristics. This will help remove potential selection bias, as students more actively engaged in community school programs/services may differ systematically in ways that also affect their outcomes.

Third, our study has data limitations regarding the measurement of student outcomes. We could not directly examine student health-related outcome measures due to the lack of available data. The impact of community school services may interact with children’s initial mental or physical health status—questions that need investigation. We recommend that school report cards collect and report data on students’ mental and physical health outcomes to better understand these relationships (Lee, 2020). Further, there were discrepancies in the list of schools that self-reported the operation of school-based health centers/clinics and the state-approved list of such; some schools may have mistakenly identified their regular nurse’s office as a school-based health center. This data accuracy issue requires further investigation. Additionally, the chronic absenteeism data have limitations as a measure of students’ academic engagement. Future studies need to examine data on students’ truancy (skipping or being late for classes), discipline, and homework problems. Lastly, future research evaluating the impact of community schools could assess family/parent-level variables (e.g., parenting, resilience, PTA, and school engagement) and explore how they might influence/moderate school effects.

6. Conclusions

Based on the results of our study, we find that New York’s community-school strategy produces some promising results. It aligns with other large-scale initiatives but differs in emphasis and implementation. For example, California’s recent expansion of community schools has been driven by competitive planning and implementation grants with required needs assessments, family engagement plans, and formalized partnerships through MOUs—a structure that foregrounds fidelity standards and technical assistance (Maier & Niebuhr, 2021). In contrast, New York has emphasized broader access to integrated supports within district schools, with flexible guidelines and relatively modest per-school funding, yielding stronger improvements in graduation rates than in standardized academic achievement. To further improve community schools in New York, we make some actionable recommendations below.

To accelerate post-pandemic recovery while closing subgroup gaps, the state should expand and target the community-school model as a community-based, equity-oriented strategy. First, it should concentrate new set-aside funds in high-need schools to scale up school-based health centers and dental services, which show the clearest links with reduced absenteeism and higher graduation rates; core services should be kept universal, but allocations should be weighted by poverty and other demonstrated needs. Second, implementation fidelity and accountability should be improved by issuing a set of minimum implementation standards and developing public dashboards to report service uptake and outcomes. Third, capacity and communication challenges should be addressed by requiring annual training for all coordinators, funding at least three additional Technical Assistance hubs (or regional coaching support) to reach all schools, and mandating a publicly accessible services webpage at every CS. Fourth, instructional integration should be strengthened by funding community-based, credit-bearing projects (e.g., those addressing local environmental or public-health problems) during the school day, co-designed with CBO and nearby colleges. Finally, mental-health support services should be relaunched via comprehensive strategies involving needs assessment, family engagement, youth co-design, and rapid-cycle testing.