Reliability and Construct Validity of a Self-Report Measure of SEL Capacities Among K-12 Educational Leaders

Abstract

Social and emotional learning (SEL) practices in schools endeavor to support wellbeing and emotional intelligence in young people; they work best when implemented well. Educational leaders in K-12 settings need to have capacities to provide SEL implementation support. Surveying SEL implementation capacity can identify specific strengths and areas for improvement and monitor progress. The current study assesses the validity and reliability of a 15-item self-report scale of capacities to support SEL implementation. A sample of 507 county-, district-, and school-level K-12 educational SEL leaders completed the scale in Fall 2023. Confirmatory factor analysis was used. The SEL capacities scale contains four unique dimensions with high internal reliability: mindsets (5 items), knowledge (3 items), skills (6 items), and efficacy (1 item). The SEL capacities scale also showed consistency (e.g., factor structure invariance) across school seasons, different educational settings, roles in the education system, years of experience among leaders, and recent levels of SEL supports received. Data generated by the SEL capacities scale can be used to inform practice decisions, make comparisons across people and over time, and unearth specific mechanisms of change related to developing adult SEL capacities to provide SEL implementation support.

1. Introduction

Social and emotional learning (SEL) has emerged as important for the holistic development of young people. When effectively implemented, SEL practices not only enhance emotional intelligence but also contribute to positive health and educational outcomes. Students exposed to well-implemented SEL programs exhibit improved academic performance, better emotional regulation, and reduced behavioral challenges (Domitrovich et al., 2017; Schonert-Reichl, 2019). Successful implementation of SEL programming depends on implementation leadership, with documented impacts of leadership on both implementation fidelity and student outcomes (Li et al., 2023; Shapiro et al., 2020). For the purposes of this paper, educational leaders are defined broadly as individuals, ranging from teachers to administrators, who explicitly provide support for SEL implementation, including those working in County Offices of Education (COEs), school districts, and at individual school sites.

Domitrovich et al. (2008) conceptualize multiple contextual and individual factors, including knowledge, training, and self-efficacy, as influencing SEL implementation quality. Models for SEL implementation identify leader “capacities” as a key mechanism of implementation leadership. These capacities—defined here as the mindsets, knowledge, skills, and sense of efficacy needed to take action—directly influence implementation success in SEL initiatives (Meyers et al., 2019). The assessment of SEL capacities among educational leaders is crucial for the continuous improvement of SEL initiatives through targeted professional development and support systems. Moreover, measuring these capacities enables the monitoring of SEL implementation conditions in diverse contexts, identification of specific strengths and areas for improvement in SEL implementation, monitoring of change in SEL benchmarks over time as efforts are made to implement and improve leaders’ ability to promote SEL for students (Todd et al., 2022), and pinpointing specific mechanisms of change within SEL capacity-building initiatives.

1.1. Measuring SEL Capacities Among Educational Leaders: Knowns and Gaps

High-quality SEL implementation is widely believed to be optimally achieved through “systemic SEL” (Mahoney et al., 2021), an approach to SEL that involves collaboration among multiple levels of the education system in carrying out the work. Systemic SEL involves enhancing adult SEL capacities, as well as establishing supports and plans, scaffolding student SEL competencies, and integrating data-based continuous improvement practices. A complementary approach to systemic SEL is transformative SEL (tSEL), which takes an equity lens toward the promotion of SEL competencies by considering the root causes of inequity (e.g., poverty, systemic racism) and providing culturally and linguistically responsive educational practices (Jagers et al., 2019). There have been growing calls for integrating systemic and transformative SEL approaches into practice, with many educational leaders interested in efforts to support SEL initiatives through a shared process of improvement. However, SEL initiatives often face implementation challenges, and little guidance is available as to how educators, scholars, and policymakers can work together to achieve implementation goals. One way these challenges can be addressed is through the standardization of processes within a multisystemic SEL support system, which can enable data-driven insights that inform the development of high-quality SEL implementation (Domitrovich et al., 2008). Standardized measurement is one key component of systemic SEL, providing valuable information about shared needs, goals, and innovations. It is a key process in the translation of SEL research knowledge to practice through intermediary supports and structures, such as school-based, multisystemic prevention systems (Metzger et al., 2024).

The SHIFT framework is one model used to guide systemic and transformative SEL implementation. It illustrates an approach to understanding SEL leader capacities as a mechanism of system change, which implicates the assessment of capacities within diverse educational settings (Shapiro et al., in press). The SHIFT model was generated through a systematic search and scoping review of the SEL implementation literature, conducted in collaboration with practice partners in K-12 public education. The framework is intended to provide guidance for regional implementation of SEL, with the goal of shifting SEL toward a more Systemic and Humanizing Implementation Focused on Transformation (SHIFT). The SHIFT model aims to facilitate individual thriving (e.g., engagement, performance, wellbeing) through enhancing favorable systemic conditions (e.g., supports, capacities, and structures and routines that promote learning, teaching, and leading). According to the model, SEL leader capacities—including mindsets, knowledge, skills, and efficacy—are improved when educational systems are able to establish a variety of partnerships across systems and access supports (e.g., tools, training, coaching, data-based feedback loops). These capacities, in turn, may support educational leaders in building the structures and routines of SEL implementation that enable system improvement, as well as promote student thriving. Highlighting a systems approach, the SHIFT model views leaders across multiple educational settings—including COEs, districts, and schools—as key agents in the provision of SEL. A conceptual figure of the SHIFT framework is illustrated in Figure 1. Detailed information about the development of the SHIFT model can be found in Shapiro et al. (2024, in press).

Input from educational leaders across various educational settings, about their own capacities, is seen as highly valuable as part of an aligned and comprehensive feedback system for guiding ongoing efforts to improve SEL implementation. A standardized scale for SEL capacities that can be used with people in different positions at different times is needed. Ideally, a multidimensional approach—capturing mindsets, knowledge, skills, and self-efficacy—would offer actionable data that enables tailored and effective supports addressing specific areas of need in leader provision of SEL implementation support. Existing measures of SEL capacity have primarily focused on classroom teachers, with few tools designed for broad use with educational leaders, including leaders across organizational levels, such as principals, district administrators, or county-level leaders. Most studies reviewed use adapted versions of the Teacher SEL Beliefs Scale developed by Brackett et al. (2012), which includes the following self-reported constructs: comfort teaching SEL, commitment to SEL, and perceptions of school SEL culture. Although this useful tool has been widely applied (e.g., Cooper et al., 2023; Collie et al., 2015; Domitrovich et al., 2019), it has only been validated with teachers, and it does not assess self-perceptions of SEL knowledge or skills, which are central to implementing and sustaining SEL infrastructure. Furthermore, many existing measures are program-specific—for example, focusing on perceptions of or preparation for implementing specific curricula such as PATHS (Domitrovich et al., 2019) or Harmony (Morrison et al., 2019)—rather than capturing general SEL capacity. Other tools (e.g., Ford et al., 2024; Huck et al., 2023; Thierry et al., 2022) capture a limited subset of relevant constructs, such as perceived support or openness to SEL, often in the context of specific initiatives or school settings. Only a few studies (e.g., LeVesseur, 2015; Domitrovich et al., 2019) have attempted to integrate organizational-level factors or to create multi-dimensional capacity assessments, but even these remain focused on school- and classroom-level implementation, rather than on multi-level leaders providing SEL implementation support.

1.2. A Scale for Measuring SEL Capacities Among Educational Leaders

In response to the need for a comprehensive assessment tool, a scale for measuring educational leaders’ self-perceived capacities for SEL implementation support was developed as part of the Berkeley Assessment of Social and Emotional Learning—Leader Voice (BASEL-LV; Shapiro et al., 2022). This survey instrument was used for the ongoing evaluation of educational leaders’ wellbeing and their conditions for thriving within CalHOPE Student Support—a statewide effort launched in 2021 to support SEL implementation in PK-12 schools across California (for more information: Eldeeb et al., 2025; Metzger et al., 2025a). BASEL-LV is a measurement tool for education sector leaders, which is part of a suite of tools that includes versions intended for students (BASEL—Youth Voice), staff (BASEL—Staff Voice), and teachers (BASEL—Teacher Voice). The BASEL-LV is intended for anyone in education settings who has a role in providing implementation support to others, which can include administrators as well as teachers who serve on leadership teams. Together, these measures aim to capture multiple perspectives on SEL implementation, and they have the potential to support informed decision making at multiple levels within the education system. An individual working in education can take multiple assessments if applicable (e.g., classroom teachers can complete the BASEL-TV to respond to questions about their classroom role, and also the BASEL-LV to respond to questions about their school-wide leadership). In California, the BASEL-LV has been administered periodically (including in Fall 2023 and Spring 2024) to educational leaders operating within three distinct, sometimes overlapping educational settings: County Offices of Education (COEs), district offices, and school sites. COE leaders typically engage in high-level agenda setting for SEL practices across regions, while district and school leaders focus more on local priorities and supporting the delivery of SEL interventions to students.

1.3. Theoretical Underpinnings of an SEL Capacities Scale

Within the context of program and policy evaluations, utilizing a multidimensional scale to measure educational leaders’ perceptions of SEL capacity would enable the pinpointing of specific strengths and areas for improvement in SEL implementation support. Furthermore, such a scale may help researchers determine whether particular types of SEL capacities among leaders, such as pro-SEL mindsets or specific SEL skills, serve as key mechanisms or moderators of more distal outcomes (e.g., high-quality SEL delivery; student emotional wellbeing, engagement, and academic performance). In developing an SEL capacities scale, scale developers drew on the research literature and collaborative work with practice partners in K-12 education to derive four key domains of SEL capacity (Shapiro et al., 2024). Items on the SEL capacities scale for leaders were developed to align with these four domains of capacities for providing support for SEL implementation, which are theorized to be unique dimensions of the scale:

• Favorable SEL Mindsets: Adopting a mindset that recognizes the importance of SEL for engagement, performance, and wellbeing among both students and adults.
• SEL Knowledge: Sufficient understanding to explain what SEL is and how it is implemented, including how to implement SEL equitably.
• SEL Skills: Possessing the skills necessary to practice SEL effectively, overcome challenges in SEL implementation, and to seek the promotion of equitable outcomes among young people.
• SEL Efficacy: Having confidence in one’s capacity to support SEL implementation.

Assessing whether the SEL capacities scale has sound psychometric properties is requisite of its utility for guiding decisions around SEL implementation leadership. It should be established that educational leaders’ ratings on the scale are reliable—in particular, that they can be interpreted in the same way across items that are aggregated into the full scale or into the same subscales (i.e., internal consistency), and across repeated administrations of the measure (i.e., test–retest reliability; Fowler, 2009). Having a reliable measure of SEL capacities can enable standardization in how SEL capacities are measured, compared, and monitored across a variety of settings and over time.

It is also critical to provide evidence of construct validity for the SEL capacities scale. Construct validity is the extent to which a measurement tool accurately reflects the theoretical construct it is intended to assess, and it is central for guiding meaningful interpretations of scores representing unobserved constructs (Fowler, 2009). For the SEL capacities scale, it is intended that four distinct unobserved constructs are reflected in ratings. However, it is possible that a different pattern of dimensionality could emerge in educational leaders’ ratings, necessitating a reappraisal of what the existing scale intends to measure or the refinement of the scale to reflect the intended dimensions more accurately. Further, it may be the case that all of the theorized constructs within the SEL capacities scale have strong conceptual overlap and that the scale is more appropriately measured as a unidimensional construct. In assessing construct validity, determining dimensionality of the scale is useful for guiding summary score recommendations—for instance, whether specific subscales have distinct implications for practice and should be aggregated separately, or whether the scale is unidimensional and item scores can be aggregated parsimoniously into a single summary metric.

Related to construct validity, establishing invariance in factor structure is critical for determining whether the SEL capacity scale’s measurements are generalizable across multiple populations of educational leaders (Putnick & Bornstein, 2016). It is important that ratings on such a scale can be interpreted in the same way across diverse populations with different work-related characteristics before interpreting or comparing scores. Within SEL initiatives, identifying specific populations of educational leaders who can benefit the most from the provision of SEL implementation support will aid in facilitating equitable outcomes in the building of implementation capacity (Jagers et al., 2019, 2021). To this end, use of a standardized, invariant SEL implementation scale would enable cross-group comparisons which could help direct resources to where they are most needed or likely to be beneficial.

One important characteristic that can impact scale properties is the point in time during which the measure is administered. Establishing temporal invariance of the SEL capacities scale would ensure that ratings can be interpreted in the same way over time, which is especially critical for evaluating the success of implementation support strategies (e.g., training, coaching), monitoring progress within ongoing capacity-building initiatives, and detecting any deterioration when attempting to sustain SEL implementation over time. Moreover, because job responsibilities and other realities can vary throughout the school year, it is useful to determine that ratings are consistent across school seasons—for example, between the beginning (i.e., fall term) and end (i.e., spring term) of a typical school year.

Establishing invariance across different subgroups of leaders is also important. These subgroups include educational leaders across diverse primary educational settings (e.g., COEs, districts, schools), who hold different roles in K-12 education (e.g., being an instructor, administrator, or holding other roles), and who vary in their experience level. By nature of these differences, leaders are often exposed to different degrees of SEL support and themselves support SEL practice in different ways. For example, county-level leaders and those who are predominantly administrators may have a more prominent role in high-level SEL agenda setting, while school-level leaders and those whose primary role is instruction are more likely to spend time engaging in on-the-ground daily practice of SEL with teachers and students. In addition, self-perceptions of SEL implementation quality have been shown to differ between more and less experienced educators (Domitrovich et al., 2019). Further, differences in capacities may not only occur naturally, but may be induced by adult SEL efforts, such as the degree of recent support received for SEL implementation (e.g., tools, quality assurance) from SEL experts. The receipt of SEL implementation support can lead to differences in SEL capacity to provide implementation support. Thus, invariance across different degrees of support received should be established in order to make comparisons across diverse contexts or explore the effectiveness of SEL implementation support strategies. Altogether, if SEL implementation capacities can be measured in the same way across these settings, roles, and experience levels, with evidence of validity, it can facilitate coherent progress monitoring within systemic SEL efforts that transcend naturally occurring differences in broad scale initiatives. Parenthetically, three items in the SEL capacities scale proposed in this paper have question stems that differ based on primary educational setting of the respondent (e.g., COE leaders refer to their own regions, district leaders to their own districts, and school leaders to their own schools). This renders the exploration of invariance by educational setting even more important. Finally, establishing the scale’s invariance can also help evaluators to conclude that any mean-level differences by group—such as might be expected between COE leaders and school-level leaders—are attributable to true population differences and not to differences in the ways in which leaders respond to the specific items.

1.4. The Current Study

The objective of the current study was to determine the reliability and construct validity of a self-report measure of SEL capacities measured using the BASEL-LV. This objective was addressed through three primary research questions (RQs).

1.4.1. RQ#1. Determine the Factor Structure and Dimensionality of the SEL Capacities Scale

To assess construct validity, we examined the factor structure and dimensionality of the scale using a combination of descriptive analysis (e.g., inspection of item means) and confirmatory factor analysis (CFA). Generally, we were interested in determining whether there was a factor structure that adequately fit the data and showed superior model fit. Our hypothesis, based on theory, was that four unique dimensions would be represented in the data: mindsets, knowledge, skills, and efficacy. In service of this hypothesis, we addressed three questions around the dimensionality of the scale. First, we examined mean structures to determine if the hypothesized mindsets dimension would emerge as a unique dimension by virtue of having higher means on its items compared with items intended to represent the other dimensions. Given the current study’s use of a convenience sample of educational leaders who were assuming roles to support SEL implementation, it was anticipated that the sample may have pre-existing mindsets and beliefs that are highly favorable to SEL, though not necessarily the knowledge, skills, and efficacy needed to effectively lead SEL implementation. Second, through a series of CFAs, we assessed whether efficacy may be considered a distinct sub-construct or whether it is better conceptualized as belonging to a latent factor for knowledge or for skills. Third, we aimed to identify whether SEL capacity is conceptualized more appropriately as a multidimensional or a unidimensional construct.

1.4.2. RQ#2. Determine Internal Reliability and Consistency of the Best-Fitting Model for the SEL Capacities Scale

We determined the reliability of the best-fitting model established in RQ#1, including internal consistency of specific factors and reliability over repeated measurement in Fall 2023 and Spring 2024.

1.4.3. RQ#3. Assess the Invariance of the Best-Fitting Model for the SEL Capacities Scale Across Time and Across Meaningful Participant Characteristics

Of interest was whether the factor structure in the best-fitting model determined in RQ#1 was consistent across Fall 2023 and Spring 2024 samples (temporal invariance), and across different populations of educational leaders in different support contexts. Specifically, we tested for invariance across different primary educational settings, primary roles in K-12 education, years of experience in one’s current role, and degree of recent support received for SEL implementation. We had no a priori hypotheses of non-invariance in factor structure across time or across different levels of these demographic characteristics.

2. Materials and Methods

2.1. Recruitment and Data Collection Procedures

Data for this study were collected primarily for the purpose of providing technical assistance to a statewide SEL effort. Using the data for research purposes was approved by the University of California, Berkeley Institutional Review Board. The collection of data used in this study commenced in the Fall of 2023, 40 months after a CalHOPE kick-off event, using the BASEL-LV with a convenience sample of education sector leaders participating in CalHOPE. The COE-level sample was initially obtained through a contact directory of designated educational leaders at the county level who are responsible for CalHOPE Statewide communications (see Metzger et al., 2025a). Any COE SEL leader known to have participated in CalHOPE at any time—as determined by the availability of contact information from practice rosters, attendance records, or prior survey invitations or responses—was emailed a personalized Qualtrics invitation to participate. This included COE SEL leaders (e.g., division heads, county office superintendents) who were not involved in the daily work of CalHOPE, but who worked in roles overseeing employees providing SEL implementation support. Contacted COE leaders received weekly reminders to respond to their invitation through Community of Practice meetings, emails, and other sources.

Within the Qualtrics invitation was a referral form, inviting COE-level contacts to provide contact information for leaders working on SEL implementation at the district and school levels within their counties. Contacted leaders at the COE, district, and school levels were invited to complete the BASEL-LV survey, with a field period of approximately one month. Contacted leaders received weekly email reminders to respond to their invitation. For all respondents, we confirmed that contact information provided by COE leaders on the Qualtrics form matched self-reported contact information provided within the BASEL-LV survey, ensuring efficiency in re-contacting prospective respondents in future waves of survey administration. A total of 1311 personalized survey invitations were distributed in the Fall 2023 field period.

2.2. Sample

The current study focuses on the sample of educational leaders generated during the Fall 2023 field period, the first time point of the CalHOPE system for which BASEL-LV data were collected at all three educational settings (COE, district, school). Leaders were screened out of the survey if they responded “no” to both of the following screening questions: “Are you in a role at your […region/district/school…] where you support social and emotional learning?” and “Are you on an SEL Leadership Team?” A total of 676 leaders responded to the personalized survey invitations (51.6%), and of these, 657 (97.2%) respondents met the screening criteria and were administered the full survey. We believe many non-responses were due to a self-screening process, whereas the invitation recipient did not actually identify as providing implementation support for SEL. The analysis sample comprises the subsample of respondents (henceforth, participants) who affirmatively met screening criteria and provided informed consent to have their data used for research purposes. Consent was indeterminate for 81 respondents due to non-completion of consent-for-research items on the primarily-for-practice-purposes survey; these cases were excluded from analyses. The research-consenting sample included 507 participants (88.0% consent rate; 38.7% of all contacted leaders), together representing 53 counties (out of 58 counties in California). Leaders were asked to describe their primary work by indicating whether their job is best described as situated within the (a) county office (COE), or otherwise serving students across a whole county, (b) district office, or otherwise serving students across a whole district, or (c) school site, or otherwise serving students within a single school. The distribution of participants’ self-reported primary work settings was 23.7% COE, 16.4% district, and 60.0% school. Although the current study treats COE and district/school work settings as discrete levels with independent samples, in practice some educational leaders across the state serve in dual roles (e.g., a rural county office administrator may also be a school principal; an urban district may be the only district in a county, and therefore administrators provide both district and county leadership). In the sample, 99.6% were in a role that supports SEL implementation, 83.2% reported having an SEL Leadership Team on site, and 92.7% of those who reported having an SEL Leadership Team on site served on their SEL Leadership Team. Many participants identified their educational role as primarily administrative/operations support (37.5%) or instruction/learning support (34.1%). Many participants (61.5%) interacted with students “most” or “almost all of the time” during their work week, and a plurality (44.2%) of participants had 6 or more years of experience in their current role. The majority of participants were non-Hispanic White/European American (71.0%) and female (79.6%). See

Table 1. Sample Description with Demographic Categories Disaggregated (Fall 2023).

		By Educational Setting
	Full Sample	COE	District	School
N	507	120	83	304
Role supports SEL implementation	95.9%	100.0%	100.0%	99.3%
Setting has an SEL Leadership Team on site	81.3%	85.8%	77.1%	83.9%
Respondent is on an SEL Leadership Team	92.3%	87.4%	89.1%	95.7%
Primary role in K-12 education
Administration	38.7%	50.0%	51.8%	28.6%
Instruction	30.4%	28.3%	10.8%	42.8%
Student Wellbeing	25.6%	19.2%	34.9%	25.7%
None of these	5.2%	2.5%	2.4%	3.0%
Frequency of student interaction during work week
Almost all of the time	41.5%	4.2%	16.9%	72.7%
Most of the time	13.3%	5.0%	18.1%	16.8%
Some of the time	11.9%	7.5%	19.3%	9.2%
Not a lot of the time	33.3%	83.3%	45.8%	1.3%
Years worked in current role
0–1 year	15.4%	17.5%	16.9%	11.8%
2–5 years	44.4%	45.0%	51.8%	37.8%
6–10 years	20.1%	23.3%	19.3%	20.4%
11–15 years	7.1%	10.0%	2.4%	9.9%
16–20 years	4.2%	3.3%	4.8%	6.6%
More than 20 years	8.7%	0.8%	4.8%	13.5%
Race and ethnicity
American Indian/Native Alaskan	3.8%	2.5%	3.7%	3.3%
Asian/Asian American	4.5%	5.9%	6.1%	4.6%
Black/African American	4.5%	3.4%	4.9%	2.6%
Hispanic/Latinx	19.4%	14.4%	28.0%	18.2%
Middle Eastern/North African/Arab American	1.4%	0.0%	3.7%	0.3%
Native Hawaiian/Pacific Islander	1.4%	1.7%	1.2%	1.0%
Non-Hispanic White/European American	69.1%	76.3%	61.0%	71.6%
Other	5.9%	2.5%	6.1%	5.6%
Gender
Woman	80.7%	79.8%	77.1%	80.3%
Man	17.5%	19.3%	20.5%	19.1%
Transgender	2.0%	0.8%	2.4%	0.0%
Non-binary	0.0%	0.0%	0.0%	0.3%
Other	1.6%	0.0%	0.0%	0.7%
Prefer not to answer	0.0%	0.0%	1.2%	0.0%

Note. Due to missing data, sample sizes within each educational setting differ among demographic variables. COE = County Office of Education.

for full details about participant characteristics for the Fall 2023 sample.

A sample from Spring 2024 comprised 386 educational leaders who were screened in and consented (88.7% consent rate; 31.1% of 1240 contacted leaders), including 257 participants who had also provided data for the Fall 2023 survey. The Spring 2024 sample did not differ significantly from the Fall 2023 sample in their distribution across primary work roles, years of experience in current role, race and ethnicity, or gender. There was a significantly higher proportion of participants whose primary work setting was at the COE level in Spring 2024 (33.2%) versus Fall 2023 (23.7%, χ² = 9.96, p = 0.002).

2.3. Measures

2.3.1. SEL Capacities

Scale Development. Item selection was guided by insights gained from the scoping review—conducted alongside practice partners—that informed development of the SHIFT model (Shapiro et al., in press). Specifically, the theorizing of dimensions was based on work by Domitrovich et al. (2008) that conceptualizes multiple contextual and individual factors—including knowledge, skills, and self-efficacy—as influencing SEL implementation quality. Relatedly, thematic analysis of data collected from California COE representatives within the CalHOPE Support System evaluation revealed both struggles and promises in the areas of building adult awareness, knowledge, and skills for SEL implementation (Shapiro et al., 2024). Further, systemic SEL and tSEL frameworks (Jagers et al., 2019; Mahoney et al., 2021) were considered in the selection of scale items, notably the inclusion of items that reflect a systems approach and which center key dimensions of equity. Face validity and content validity of items were determined qualitatively within a research-practice partnership (i.e., “CalHOPE Research Committee”), who worked together to conduct research, but also co-facilitate Communities of Practice meetings, and had deep understanding and shared language around the SEL capacities used in these settings.

Self-perceived capacities for implementing SEL are assessed along a 4-point Likert scale (4: “YES!” means definitely true for you, 3: “yes” means mostly true for you; 2: “no” means mostly not true for you; 1: “NO!” means definitely not true for you). A higher score on any given item indicates a stronger self-perceived presence of that capacity. This scaling technique has been validated and used in multiple studies over the past 40 years (Social Development Research Group, 2005).

The scale for SEL capacities includes 15 items, with five items intended to measure mindsets and beliefs about SEL (e.g., “I believe SEL is very important for student engagement”), three items for SEL knowledge (e.g., “I am able to answer questions from people in my […region/district/school…] about the practice of SEL”), six items for SEL skills (e.g., “I have the skills to remove obstacles to SEL implementation”), and one item for efficacy (“As a leader, I have confidence that I can support SEL implementation”). A list of all items can be found in

Table 2. Descriptive Statistics and Correlation Table for Capacities Scale Items (Fall 2023 Sample).

		M	SD	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
Mindsets
	Considering all other competing school priorities, I believe SEL is very important for … Student engagement.	3.83	0.39	1.00	0.79 **	0.72 **	0.73 **	0.67 **	0.27 **	0.30 **	0.26 **	0.23 **	0.21 **	0.16 **	0.21 **	0.17 **	0.12 **	0.29 **
	Considering all other competing school priorities, I believe SEL is very important for … Student performance.	3.80	0.42		1.00	0.64 **	0.70 **	0.62 **	0.24 **	0.28 **	0.25 **	0.18 **	0.17 **	0.17 **	0.21 **	0.18 **	0.12 **	0.23 **
	Considering all other competing school priorities, I believe SEL is very important for … Student wellbeing.	3.88	0.32			1.00	0.72 **	0.63 **	0.27 **	0.30 **	0.23 **	0.22 **	0.15 **	0.13 **	0.15 **	0.12 **	0.11 **	0.26 **
	Considering all other competing school priorities, I believe SEL is very important for … Achieving equity in student engagement, performance, and wellbeing.	3.79	0.43				1.00	0.74 **	0.26 **	0.28 **	0.29 **	0.14 **	0.14 **	0.17 **	0.20 **	0.19 **	0.14 **	0.27 **
	Considering all other competing school priorities, I believe SEL is very important for…Adult engagement, performance, and wellbeing.	3.73	0.50					1.00	0.30 **	0.35 **	0.36 **	0.23 **	0.26 **	0.17 **	0.18 **	0.21 **	0.13 **	0.29 **
Knowledge
	I am able to answer questions from people at my region/district/school about … The practice of SEL.	3.48	0.59						1.00	0.82 **	0.60 **	0.36 **	0.31 **	0.35 **	0.36 **	0.35 **	0.32 **	0.42 **
	I am able to answer questions from people at my region/district/school about … SEL implementation.	3.40	0.62							1.00	0.68 **	0.38 **	0.30 **	0.39 **	0.41 **	0.35 **	0.33 **	0.45 **
	I am able to answer questions from people at my region/district/school about … Equity in SEL implementation.	3.13	0.75								1.00	0.34 **	0.36 **	0.37 **	0.41 **	0.55 **	0.47 **	0.40 **
Skills
	I model adult SEL skills every day.	3.38	0.57									1.00	0.52 **	0.37 **	0.40 **	0.38 **	0.34 **	0.39 **
	I create opportunities for adults to practice SEL skills.	3.20	0.65										1.00	0.42 **	0.42 **	0.43 **	0.42 **	0.36 **
	I have the skills to remove obstacles to SEL implementation.	3.05	0.64											1.00	0.73 **	0.56 **	0.49 **	0.49 **
	I have the skills to openly and effectively address problems implementing SEL.	3.08	0.62												1.00	0.63 **	0.53 **	0.49 **
	I have the skills to facilitate challenging conversations about equity in SEL implementation.	3.05	0.68													1.00	0.66 **	0.48 **
	I have the skills to recognize and address root causes of educational disparities.	3.02	0.65														1.00	0.40 **
Efficacy
	As a leader, I have confidence that I can support SEL implementation.	3.48	0.56															1.00

** Correlation is significant at the 0.01 level.

. Three items intended to measure SEL knowledge have question stems that vary based on the respondent’s primary educational setting: COE participants are asked about their region, district participants about their district, and school participants about their school. Within the CalHOPE evaluation, the 15-item scale was administered to a sample of 117 COE-level leaders in a prior wave (Summer 2023). The scale had high internal reliability in this sample, and thus, these items were retained in future administrations of the BASEL-LV. A sixteenth item intended to measure SEL capacities was added into the BASEL-LV survey in Fall 2023, theorized to map onto the mindsets and beliefs factor: “I believe certain uses of SEL can perpetuate disadvantages among students.” Some respondents noted confusion with the wording and interpretation of this item, and exploratory work revealed that internal reliability coefficients (both in the mindsets subscale and in the full scale) were improved meaningfully with the omission of this item from the scale. Accordingly, this item was dropped from psychometric analyses in the current study and dropped from the BASEL-LV in future waves of data collection.

2.3.2. Recent SEL Supports Received

The BASEL-LV includes a 25-item scale to measure educational leaders’ self-report of supports received in the last 6 months. This support was provided to build capacity among leaders to support SEL implementation. Examples included funding, resources, training, tools, coaching, and feedback for continuous improvement received from statewide or regional SEL experts. Although the types of supports measured through these 25 items are identical for all leaders, the source of support differs by level. COE participants report on supports received from the CalHOPE statewide team, while district participants report on supports received from their COE, and school participants report on supports received from their district or COE. Items assessing supports use the same response options as items assessing capacities. Scores on all items were averaged to yield a single “supports received” score per respondent.

2.4. Analysis Plan

2.4.1. RQ#1. Determine the Factor Structure and Dimensionality of the SEL Capacities Scale

Our approach to confirmatory factor analyses (CFAs) was as follows. Prior to running CFAs, we examined the mean and variance structure of the scale’s items to determine potential groupings of items with similar means or variances (see Table 2 for item means, standard deviations, and correlations). Five items intended to measure mindsets had very low variability due to ceiling effects. Estimation problems in the CFAs were anticipated given the low variability of these items (e.g., some items with zero responses of NO!), and thus they were omitted from all factor analyses. For all remaining items, to circumvent estimation problems due to the low frequency of NO! responses, responses of NO! and no were collapsed into a single category, yielding three-level ordered categorical variables (NO!/no = 0, yes = 1, YES! = 2) to be estimated in CFAs. Although CFAs did not treat indicators on their original scale, the modified approach did preserve homogeneity in the item ranges and in the underlying meaning of item values across all indicators included in the models, and thus, the internal factor structure of the instrument was likely preserved. For all non-CFAss, items retained their original, intended four-point scales. CFAs were modeled using Mplus Version 8.4 (Muthén & Muthén, 1998). Missing data were rare (range 0.0–1.2%) across all analysis variables. One case with missingness on all variables was excluded from analyses, while maximum likelihood estimation was used to account for the remaining missing data. Based on the magnitude of factor loadings (RQ#1), number and high reliability of indicators (RQ#2), and the amount of missing data, sample sizes were determined to yield sufficient statistical power (Wolf et al., 2013). All CFAss specified a weighted least squares mean and variance adjusted (WLSMV) estimation method with a probit link to account for the ordered categorical nature of the scale items. Within the CFA models, because no single item was expected to load more heavily than other items, all indicators were specified to be freely estimated. Latent factor variances were fixed to 1 to ensure model identification and to standardize the factor loadings for interpretability (Muthén & Muthén, 1998; Bollen, 1989). Standard model fit indices for structural equation modeling, with established thresholds for acceptable fit, were used to determine goodness of fit: CFI/TLI (>0.90; Bentler, 1990; Tucker & Lewis, 1973), SRMR (<0.08; Hu & Bentler, 1999), and RMSEA (<0.08; Steiger, 1990).

A series of CFAs was estimated in order to identify the best-fitting model. Since it was pre-determined that five items intended to measure mindsets would be omitted from CFAs, model building was focused on assessing the dimensionality of the 10 remaining items. Altogether, four models were fitted:

Model 1 (M1). Theory-driven, two-factor solution for knowledge and skills, with efficacy conceptualized as a distinct sub-construct and therefore not specified in the model due to having only a single indicator.

Model 2 (M2). Two-factor solution for knowledge and skills, with efficacy conceptualized as a unit of knowledge whose observed indicator maps onto the latent factor for knowledge.

Model 3 (M3). Two-factor solution for knowledge and skills, with efficacy conceptualized as a skill whose observed indicator maps onto the latent factor for skills.

Model 4 (M4). Single-factor solution that conceptualizes SEL capacities as a unidimensional construct inclusive of knowledge and skills.

All models were compared to the theory-driven M1, which served as a de facto baseline model. Differences in fit indices were used for model comparison, with the following thresholds used as guidelines for determining meaningful differences in fit: a statistically significant difference in Chi-square values (p < 0.05), or CFI, TLI, and SRMR values with absolute differences of 0.01 or greater (Cheung & Rensvold, 2002; Hu & Bentler, 1999). Since M2 and M3 included 10 indicators and M1 included only 9, significance testing for Chi-square differences could not be completed for M2 or M3.

2.4.2. RQ#2. Determine Internal Reliability and Consistency of the Best-Fitting Model for the SEL Capacities Scale

Cronbach’s alpha estimates were used to determine internal reliability for each dimension that contains multiple items (Cronbach, 1951). Pearson’s R estimates were used to determine the strength of correlations among average scores for different dimensions of the scale. Pearson’s R estimates were also used to determine test–retest reliability for each dimension, using a subsample of participants who completed both the Fall 2023 sample and the later Spring 2024 sample. In both sets of analyses, high reliability was determined to be an alpha of above 0.80, which is a standard threshold for Cronbach’s reliability coefficient (Nunnally & Bernstein, 1994).

2.4.3. RQ#3. Assess the Invariance of the Best-Fitting Model for the SEL Capacities Scale Across Time and Across Meaningful Participant Characteristics

To test measurement invariance of the SEL capacities scale, we used a structural equation modeling approach called multigroup CFA, which is a widely used approach in educational research. Invariance testing models focused on identifying meaningful differences in factor structure and means between different groups within the overall sample of educational leaders. Altogether, group-based comparisons across five key characteristics were examined: time point (e.g., temporal invariance; Fall 2023 vs. Spring 2024), primary educational setting (COE vs. district/school), primary role in the education system (instruction vs. all other roles), years of experience working in education (0–5 years vs. 6+ years), and recent levels of support received for implementing SEL (fewer vs. greater, as determined by a median split on the supports received scale). A standard, model-building approach was used for invariance testing, which involved assessing three levels of invariance for the theorized two-factor solution (M1): configural invariance, weak factorial invariance (i.e., metric invariance), and strong factorial invariance (i.e., scalar invariance). Steps for invariance testing were as follows:

(1)

Configural modeling was used to determine if the scale has the same factor structure between groups (i.e., configural invariance). The configural invariance model specifies the identical structure of the factor-indicator relationships across subgroups while placing minimal constraints for model identification. Evidence of configural non-invariance suggests that different subgroups have different factor-indicator relationships and that the scale’s measurements should not be compared across subgroups. Configural invariance is a prerequisite for testing for metric and scalar invariance. Here, configural models specified the same latent factors as the baseline model M1, with the addition of the GROUPING option in Mplus to estimate factor loadings separately in each group (e.g., COE leaders as one group, district/school leaders as another group). Standard model fit indices (CFI/TLI, SRMR, RMSEA) indicated overall fit of the two-group model.

(2)

Metric modeling was used to determine weak factorial invariance: whether the scale has equal factor loadings between groups. In metric models, configural models were modified so that the magnitude of the factor-indicator relationship (e.g., the factor loadings) was constrained to equality between groups. Metric invariance was determined by comparing model fit indices between the configural model (where factor loadings are free to vary between groups) and the metric model (where factor loadings are constrained to equality). Substantial degradation in model fit for the metric model would indicate metric non-invariance. Evidence of metric non-invariance suggests that the educational leader ratings of specific items better represent the theorized dimensions of SEL capacities for one group than another and that subgroup comparisons on the scale would not be based on measurement of the same underlying construct, and thus, that valid cross-group comparisons cannot be made. Configural and metric invariance is a prerequisite for scalar invariance testing.

(3)

Scalar modeling was used to determine strong factorial invariance: whether the scale has equal intercepts (e.g., means) across groups. In scalar models, configural models were modified to constrain factor intercepts to equality between groups. Scalar invariance was determined by comparing model fit indices between the configural model (where factor intercepts are free to vary between groups) and the scalar model (where factor intercepts are constrained to equality). Metric and scalar models were also compared. As with the metric model, substantial degradation in model fit for the scalar model would indicate scalar non-invariance. Evidence of scalar non-invariance would suggest that educational leader ratings of SEL capacities are systematically lower or higher for one group than another. This type of invariance is important to establish even when group differences at the mean level are expected, as they were in the current study for groups of leaders with stronger engagement in SEL planning (e.g., COE vs. district/school primary settings) and a greater degree of recent SEL support. Scalar invariance establishes whether the scaling of responses is measured in the same way across groups, ensuring that a difference in means between groups is not due to differences in how the items are interpreted across groups.

Criteria suggested by Chen (2007) were used for statistical testing of model comparisons in tests of invariance. Chi-square difference tests were used to determine statistical significance of fit differences, with p-values at or above 0.05 suggesting invariance (e.g., for metric invariance, insubstantial degradation of model fit by constraining loadings to be equal across groups). Additional thresholds for determining invariance were CFI/TLI value differences of <0.01, and SRMR value differences of <0.02 (configural vs. metric/scalar) or <0.01 (metric vs. scalar). Weak invariance was identified for a key characteristic if metric but not scalar invariance was established. Strong invariance was determined if all model comparisons (configural vs. metric/scalar, metric vs. scalar) met invariance criteria. Because the Fall 2023 and Spring 2024 samples contained a combination of repeating and unique participants, it was important to account for possible effects of dependencies in the data among survey repeaters. In testing for temporal invariance, sensitivity analyses were performed in one subsample that included only the survey repeaters and in another subsample that included only unique participants across the two survey time points.

3. Results

3.1. RQ#1. Determine the Factor Structure and Dimensionality of the SEL Capacities Scale

Disaggregation of the SEL capacities scale into four subscales (mindsets, knowledge, skills, efficacy) is consistent with the data from the current sample. Model fit indices and comparisons of model fit for the four tested factorial solutions (three two-factor solutions, plus one single-factor solution) are detailed in

Table 3. Model fit indices for SEL capacities scale (Fall 2023 sample).

										Model Comparison (vs. M1)
Model #	Model	# Items	χ2	df	CFI	TLI	RMSEA [90% CI]		SRMR	Δχ2	ΔCFI	ΔTLI	ΔRMSEA	ΔSRMR	χ2 DIFFTEST
M1	2 Factor: Knowledge + Skills (Efficacy Omitted)	9	251.58	26	0.980	0.973	0.131	[0.116, 0.146]	0.058
M2	2 Factor: Knowledge (+ Efficacy), Skills	10	420.21	34	0.967	0.957	0.150	[0.137, 0.163]	0.077	168.63	−0.013	−0.016	0.019	0.019	N/A *
M3	2 Factor: Knowledge, Skills (+ Efficacy)	10	286.93	34	0.979	0.972	0.121	[0.109, 0.134]	0.056	35.35	−0.001	−0.001	−0.010	−0.002	N/A *
M4	1 Factor: Unidimensional Construct	9	604.51	27	0.949	0.932	0.206	[0.192, 0.220]	0.124	352.93	−0.031	−0.041	0.075	0.066	χ2 = 119.35, p < 0.001

* Chi-square difference testing parameter estimates were not possible for models M2 and M3 (vs. M1) due to having different numbers of indicators compared with M1.

.

The theory-driven solution (M1), treating efficacy as a separate sub-construct from knowledge or skills (and therefore not including the efficacy item in the CFA), had acceptable fit with the data on CFI/TLI (>0.95) and SRMR (<0.08) indices, but fit was not in the acceptable range for RMSEA (>0.08). This pattern of model fit held for the two-factor solutions treating efficacy as part of the knowledge sub-construct (M2) or as part of the skills sub-construct (M3). The two-factor solutions for M2 and M3 failed to improve model fit indices substantively relative to the theorized M1, with only small differences in model fit indices (<0.02 in absolute change) between M1 and M2/M3. The unidimensional solution (M4) that loaded knowledge and skills indicators onto a single factor showed adequate model fit on CFI/TLI (>0.90) indices, but poor fit on SRMR (>0.08) and RMSEA (>0.08) indices. Differences in model fit indices between M1 and M4 were meaningfully large (>0.03), with a Chi-square difference test yielding significantly poorer model fit for M4 (χ² = 119.35, p < 0.001). Overall, the theorized M1 is the best-fitting solution, and further psychometric testing focuses on this model only.

Table 4. Factor loadings for two-factor solution of SEL capacities scale (Fall 2023 sample).

	Factor Loading (SE)
Item	Knowledge	Skills
I am able to answer questions from people at my region/district/school about … The practice of SEL.	0.924 (0.012)
I am able to answer questions from people at my region/district/school about … SEL implementation.	0.968 (0.011)
I am able to answer questions from people at my region/district/school about … Equity in SEL implementation.	0.881 (0.016)
I model adult SEL skills every day.		0.683 (0.032)
I create opportunities for adults to practice SEL skills.		0.689 (0.028)
I have the skills to remove obstacles to SEL implementation.		0.869 (0.015)
I have the skills to openly and effectively address problems implementing SEL.		0.901 (0.014)
I have the skills to facilitate challenging conversations about equity in SEL implementation.		0.868 (0.017)
I have the skills to recognize and address root causes of educational disparities.		0.806 (0.020)

Note: Five items measuring mindsets are omitted from the CFA due to a lack of variability. Efficacy is omitted from the CFA due to having only a single indicator in the scale and being conceptualized as a district sub-construct.

details the factor loadings for M1, with three and six indicators loading onto knowledge and skills, respectively. Factor loadings were high (>0.60) for all indicators.

3.2. RQ#2. Determine Internal Reliability and Consistency of the Best-Fitting Model for the SEL Capacities Scale

Since the best-fitting model by theory and data, M1, specifies four subscales (mindsets, knowledge, skills, efficacy), correlations among these specific subscales were examined. Here, there were strong, positive correlations (Pearson’s R > 0.25, p < 0.001) among mean scores for all four subscales. All multi-item subscales showed high internal reliability: mindsets (Cronbach’s alpha = 0.91), knowledge (Cronbach’s alpha = 0.86), and skills (Cronbach’s alpha = 0.85). Test–retest reliability estimates within the subsample of survey repeaters were reasonably high, given an active capacity-building initiative interceded: mindsets (Pearson’s R = 0.37, p < 0.001), knowledge (Pearson’s R = 0.49, p < 0.001), skills (Pearson’s R = 0.61, p < 0.001), and efficacy (Pearson’s R = 0.36, p < 0.001).

3.3. RQ#3. Assess the Invariance of the Best-Fitting Model for the SEL Capacities Scale Across Time and Across Meaningful Participant Characteristics

See

Table 5. Model fit indices for invariance testing of SEL capacities scale.

Characteristic: Time Point (Fall 2023 vs. Spring 2024)
Model	Invariance Type	χ2	df	CFI	TLI	RMSEA [90% CI]		SRMR	Model Comparison	Δχ2 [p-Value]		ΔCFI	ΔTLI	ΔRMSEA	ΔSRMR	Meets Invariance Criteria
A	Configural	516.41	52	0.982	0.975	0.142	[0.131, 0.153]	0.063	B vs. A	−17.25	0.889	0.001	0.004	−0.013	0.000	Yes
B	Metric	499.16	59	0.983	0.979	0.129	[0.119, 0.140]	0.063	C vs. B	15.74	0.853	−0.001	0.002	−0.005	0.000	Yes
C	Scalar	514.90	66	0.982	0.981	0.124	[0.114, 0.134]	0.063	C vs. A	−1.51	0.947	0.000	0.006	−0.018	0.000	Yes
Characteristic: Primary Educational Setting (COE vs. District/School) *
Model	Invariance Type	χ2	df	CFI	TLI	RMSEA [90% CI]		SRMR	Model Comparison	Δχ2 [p-Value]		ΔCFI	ΔTLI	ΔRMSEA	ΔSRMR	Meets Invariance Criteria
A	Configural	284.77	52	0.979	0.972	0.133	[0.118, 0.148]	0.064	B vs. A	−15.39	0.885	0.002	0.005	−0.014	0.000	Yes
B	Metric	269.38	59	0.981	0.977	0.119	[0.105, 0.133]	0.064	C vs. B	36.59	0.000	−0.002	0.000	0.001	0.003	No
C	Scalar	305.97	66	0.979	0.977	0.120	[0.106, 0.134]	0.067	C vs. A	21.20	0.002	0.000	0.005	−0.013	0.003	No
Characteristic: Primary Role in K-12 Education (Instruction vs. Non-Instruction) *
Model	Invariance Type	χ2	df	CFI	TLI	RMSEA [90% CI]		SRMR	Model Comparison	Δχ2 [p-Value]		ΔCFI	ΔTLI	ΔRMSEA	ΔSRMR	Meets Invariance Criteria
A	Configural	259.60	52	0.982	0.975	0.126	[0.111, 0.141]	0.060	B vs. A	−2.44	0.542	0.001	0.004	−0.011	0.001	Yes
B	Metric	257.16	59	0.983	0.979	0.115	[0.101, 0.130]	0.061	C vs. B	8.09	0.764	0.000	0.002	−0.006	0.000	Yes
C	Scalar	265.25	66	0.983	0.981	0.109	[0.096, 0.123]	0.061	C vs. A	5.65	0.709	0.001	0.006	−0.017	0.001	Yes
Characteristic: Years Experience in Current Role (0–5 Years vs. 6+ Years) *
Model	Invariance Type	χ2	df	CFI	TLI	RMSEA [90% CI]		SRMR	Model Comparison	Δχ2 [p-Value]		ΔCFI	ΔTLI	ΔRMSEA	ΔSRMR	Meets Invariance Criteria
A	Configural	259.69	52	0.981	0.974	0.126	[0.111, 0.141]	0.061	B vs. A	−1.06	0.447	0.001	0.004	−0.010	0.000	Yes
B	Metric	258.63	59	0.982	0.978	0.116	[0.101, 0.130]	0.061	C vs. B	11.57	0.353	0.000	0.002	−0.005	0.001	Yes
C	Scalar	270.20	66	0.982	0.980	0.111	[0.097, 0.124]	0.062	C vs. A	10.51	0.426	0.001	0.006	−0.015	0.001	Yes
Characteristic: SEL Supports Received (Fewer vs. Greater) *
Model	Invariance Type	χ2	df	CFI	TLI	RMSEA [90% CI]		SRMR	Model Comparison	Δχ2 [p-Value]		ΔCFI	ΔTLI	ΔRMSEA	ΔSRMR	Meets Invariance Criteria
A	Configural	298.48	52	0.973	0.963	0.137	[0.122, 0.152]	0.070	B vs. A	1.25	0.071	0.001	0.005	−0.010	0.001	Yes
B	Metric	299.73	59	0.974	0.968	0.127	[0.113, 0.141]	0.071	C vs. B	21.65	0.002	−0.002	0.002	−0.003	0.002	No
C	Scalar	321.38	66	0.972	0.970	0.124	[0.110, 0.137]	0.073	C vs. A	22.90	0.003	−0.001	0.007	−0.013	0.003	No

Note: Models were compared based on Chen’s (2007) criteria: Δχ² p-value > 0.05, ΔCFI < 0.01, Δ TLI < 0.01, ΔSRMR < 0.02 (configural vs. metric/scalar) or < 0.01 (metric vs. scalar). * Fall 2023 sample only.

for full details of model fit indices for invariance testing of M1. Evidence of strong invariance for model M1 was found for time point (Fall 2023 vs. Spring 2024) and for two participant characteristics: primary role (instruction vs. all other roles) and years of experience in current role (0–5 years vs. 6+ years). For these characteristics, comparisons across configural, metric, and scalar models yielded non-significant estimates on Chi-square difference tests (p > 0.05), with CFI, TLI, and SRMR difference scores all < 0.01. For time point, strong invariance was also observed in sensitivity analyses that included only survey repeaters, as well as only unique participants across the two survey time points.

Weak invariance was confirmed for two characteristics: primary educational setting (COE vs. district/school) and degree of SEL supports received (fewer vs. greater). Here, Chi-square difference tests indicated statistically significant (p < 0.05) differences in model fit between scalar models and both the configural and metric models, suggesting scalar non-invariance. No significant differences were found when comparing configural and metric models, indicating metric invariance for the primary educational setting and SEL supports received. As detailed in

Table 6. Means and standard deviations of SEL capacities scale by study characteristic.

Participant Characteristic		SEL Capacities—Subscales
		Mindsets		Knowledge		Skills		Efficacy
	N	M	SD	M	SD	M	SD	M	SD
Time Point
Fall 2023	507	3.81	0.36	3.34	0.58	3.13	0.48	3.48	0.56
Spring 2024	386	3.86	0.29	3.47	0.55	3.23	0.47	3.57	0.53
Primary Educational Setting *
COE	120	3.86	0.32	3.50	0.58	3.33	0.44	3.62	0.52
District/School	387	3.79	0.37	3.29	0.57	3.07	0.48	3.44	0.56
Primary Role in K-12 Education *
Non-Instruction	334	3.81	0.35	3.35	0.58	3.15	0.47	3.52	0.55
Instruction	173	3.79	0.38	3.31	0.58	3.08	0.50	3.41	0.56
Years Experience in Current Role *
≤5 Years	283	3.80	0.38	3.32	0.57	3.14	0.48	3.49	0.56
6+ Years	224	3.81	0.33	3.36	0.60	3.11	0.48	3.47	0.55
SEL Supports Received *
Fewer	246	3.72	0.42	3.11	0.57	2.95	0.43	3.31	0.58
Greater	261	3.89	0.26	3.55	0.51	3.30	0.47	3.64	0.49

* Fall 2023 sample only.

, mean values for knowledge and skills showed meaningfully large differences as a function of educational setting and SEL supports received, while mean-level differences were minimal by time point, primary role, and years of experience. Pertinent to scalar non-invariance, these differences in mean values reflect ceiling effects that were observed for leaders in COE settings and with high degrees of SEL supports received. Scalar non-invariance for the first two study characteristics is likely attributable to these large mean-level differences.

4. Discussion

The current study assessed the validity and reliability of a 15-item self-report scale of capacities to implement SEL that is part of the BASEL-LV and is intended for educational leaders in a variety of work settings. Based on theory and on psychometric measurement within the current study, SEL capacities can be measured reliably and with an acceptable degree of construct validity by using the proposed SEL capacities scale, which measures educational leaders’ general self-perceived mindsets, knowledge, skills, and efficacy for providing SEL implementation support. As currently measured, the scale is intended to have broad applicability to multiple educational leader roles in multiple settings, and it is not limited to a specific SEL curriculum.

A specific aim of this study was to determine the factor structure and dimensionality of the SEL capacities scale. Initial insight into the scale’s dimensionality was derived from inspecting item means. For items intended to measure mindsets, consistently higher means across all of these items—relative to the mean structure of the theorized dimensions for knowledge, skills, and efficacy—as well as high internal reliability, suggest that items measuring mindsets map onto a distinct psychological construct. Within this population of educational leaders who were highly engaged in SEL leadership, educator mindsets about SEL—such as whether they believe SEL practices are important for promoting student wellbeing and achievement—could be meaningfully separated from the skills, knowledge, and efficacy needed to provide SEL implementation support. Next, using CFAs, it was determined that knowledge and skills were modeled appropriately as unique dimensions. The theorized two-factor solution for knowledge and skills (omitting the single item for efficacy) had high factor loadings, high internal reliability within each of the two dimensions, and acceptable fit on CFI/TLI and SRMR indices. Although RMSEA values for the theorized two-factor solution (and follow-up invariance models) were outside the acceptable range, this is likely owing to the simplicity of the models and to the low degrees of freedom, rather than serving as an indicator of poor compatibility between relationships within the model and relationships within the data. Moreover, the theorized two-factor solution fit the data meaningfully better than a one-factor solution that conceptualizes SEL capacities as inclusive of knowledge and skills in a single dimension. A remaining question addressed through CFAs was whether efficacy was most appropriately modeled as a unique dimension. Here, it was determined that model fit indices did not differ meaningfully among the three models: the theorized two-factor model omitting efficacy, one loading it onto a latent factor for knowledge, and one loading it onto a latent factor for skills. This suggests that perceived efficacy for implementing SEL could be conceptualized reasonably well as belonging to the knowledge dimension, to the skills dimension, or as its own unique construct. This pattern suggests a degree of conceptual overlap, with educational leaders who are strong in one of these areas also likely to be strong in the others.

There appears to be alignment between the data here (showing multidimensionality in the scale), and theoretical and empirical research on SEL capacities. Domitrovich et al. (2008) conceptualize knowledge, skills, and self-efficacy as different factors influencing SEL implementation quality. Within intervention science more broadly, there are evidence bases demonstrating the distinct importance of pro-intervention mindsets (Elias et al., 2003; Pankratz et al., 2002; Ringwalt et al., 2003), gained knowledge (Duane et al., 2025; Dusenbury et al., 2003; Pankratz et al., 2002; Rogers, 2003), specialized skills (Dusenbury et al., 2003), and self-efficacy in program delivery (Datnow & Castellano, 2000; Kealey et al., 2000; Ringwalt et al., 2003) in facilitating effective program implementation. Furthering this evidence base within an SEL programming context, the current study validates a multidimensional scale that can be used to address future questions about the additive or interactive nature of specific SEL capacities in promoting SEL implementation. Within the context of the SHIFT framework, it is conceivable that all of these dimensions need to be present in order to improve the structures and routines of SEL implementation (the next lever of system transformation in SHIFT, following capacities). The capacities may be mutually reinforcing; when leaders perceive themselves to be effective in program practice (self-efficacy) and attribute student improvements to the intervention (mindsets), they may be more motivated to continue developing their knowledge and skills in this area, as was observed in past research on SEL implementation quality (Han & Weiss, 2005). Alternatively, one or more of these dimensions—or their interaction—may be more important drivers of implementation success in different contexts. For example, many educators have pre-existing pro-SEL mindsets, and believe they could implement SEL effectively if trained, but lack the supports to build their knowledge and skills. In this situation, programs that target knowledge and skills may be more impactful than those that focus on shifting mindsets.

Another specific aim was to assess the invariance of the SEL capacities scale across time and across meaningful participant characteristics. First, invariance in both factor structure and mean structure was confirmed for the time point variable (between Fall 2023 and Spring 2024). Test–retest reliability from Fall 2023 and Spring 2024 was also determined to be high for educational leaders who provided data for both time points. Establishing temporal invariance of the scale has useful implications for progress monitoring in program evaluations (Lee et al., 2023b). With the current scale, evaluators can make meaningful inferences about changes in leader SEL capacities over time, which likely reflect real changes in SEL capacity, rather than changes in the meaning of capacities or in the way they are measured across time. It is notable that temporal invariance was validated by comparing responses across fall and spring seasons of the school year, given the dynamic nature of educational leader work responsibilities between the beginning and end of a typical school year. Invariance in factor structure was also confirmed for primary educational settings (COE vs. district/school), primary role in K-12 education (instructional role vs. non-instructional role, such as administration), years of experience in one’s current role (0–5 years vs. 6 or more years), and level of recent SEL supports received (fewer vs. greater). When group-based invariance is established, subgroup differences can be interpreted as reflecting differences in the construct measured rather than leaders’ differential rating behaviors based on group characteristics (Lee et al., 2024). The scale assessed in the current study enables valid cross-group comparisons of SEL capacities, both concurrently and over time. For progress monitoring purposes, users of the SEL capacities scale will have more trustworthy information to guide score interpretation and data use, in efforts to remediate any subgroup differences in SEL capacities. For example, COE leaders (who come from an implementation support system and focus on providing regional support)—and district or school leaders (who come from an SEL delivery system and tend to focus on local support)—may differ in their baseline levels of specific capacities, their change over time, or in the importance of specific capacities in driving implementation outcomes. These important differences can be assessed systematically with the proposed scale.

Among participant characteristics, invariance in mean structure was established for primary role and years of experience. There was non-invariance in mean structure across levels of primary educational setting and SEL supports received, which was consistent with data showing notably higher mean levels of SEL capacities among this sample of COE leaders and among leaders with more SEL support (Table 6). For these two characteristics, scalar non-invariance was not surprising, given that the mean differences between groups were driven by ceiling effects in the higher-scoring subgroups that restricted the scoring range. Educational leaders who are direct recipients of capacity-building efforts (e.g., COE leaders in the current sample) may be more likely than those who were indirect recipients of capacity-building efforts (e.g., district and school site leaders in current sample) to have greater capacity approximately three years into CalHOPE. Furthermore, statewide support for SEL implementation began with building capacity for COE leaders, and later shifted to building district- and school-level capacity (Metzger et al., 2025b). Relatedly, the instrument used to measure receipt of SEL implementation support represents specific supports—such as tools, training, and coaching—that were theorized within the CalHOPE logic model to aid in building capacity for implementation of SEL (Shapiro et al., 2024). Thus, it is logically consistent that fewer and greater levels of receipt of SEL supports are mapped onto lower and higher levels of SEL capacity, respectively. One could consider this further evidence of construct validity—in that the measure behaved in accordance with theory. Nevertheless, ceiling effects for the COE and high-support subgroups suggest limited room for positive change among the subgroups already at a high level, and uncertainty about the interpretability of comparisons, suggesting the need for further research in different samples or at different time periods.

4.1. Implications for Educational Policy and Practice

The study results have numerous implications for building systems of SEL implementation support to improve student emotional intelligence and academic outcomes. First, surveying SEL leader implementation capacity with the current scale has the potential to inform continuous improvement in practice through identification of specific strengths and areas for improvement, and to enable the testing of interventions aimed at shifting leader capacities to promote emotional intelligence in young people. Users of the scale may derive uniquely informative insights by summarizing data about mindsets, knowledge, skills, and efficacy separately. Second, SEL leader capacities are a purported mechanism of change facilitating student academic success and wellness (Domitrovich et al., 2017), but multidimensional constructs to measure these capacities—including implementation capacity—have been lacking. The current scale can be used to unearth specific mechanisms of change (e.g., leader SEL knowledge, skill) when evaluating a variety of SEL programs. For example, SEL program evaluations may reveal that adoption of favorable SEL mindsets is associated with change among educational leaders with less exposure to SEL, but among leaders who already have favorable SEL mindsets, it may be the knowledge and skills for implementation support, gained through deep professional learning, that drive positive change in student outcomes. Third, given the scale’s invariance over time, and the fact that the scale’s items are not specific to any given SEL program or initiative, the current measure represents an opportunity to standardize the measurement of SEL capacities for the sake of comparison, replication, and generalization. Standardization of processes within a multisystemic SEL support system enables research and is an important factor in high-quality implementation of SEL intervention (Domitrovich et al., 2008). The SEL capacities scale has the potential to be used as part of a battery of standardized measures for coordinated decision-making among diverse educational partners.

Fourth, data generated by the SEL capacities scale can inform practice decisions (e.g., for selecting professional development strategies, offering just-in-time support, etc.). The finding of factor structure invariance across a range of meaningful leader characteristics suggests potentially wide applicability of the scale to diverse populations of educational leaders across a variety of work settings. Educational researchers have consistently highlighted the promise of coordinated, multisystemic approaches for promoting emotional intelligence and academic achievement in young people (Domitrovich et al., 2008; Oberle et al., 2016; Shapiro et al., 2024). In particular, SEL initiatives need to be implemented well across educational settings to promote student success (Li et al., 2023; Shapiro et al., 2020). In response, CASEL has been facilitating collaborative initiatives to support districts and states that are interested in advancing SEL. There remains a need for standardized measurement approaches to aid progress monitoring in these multisystemic programs and efforts, and the current study fills an important gap by validating a measurement approach for SEL capacities that can be interpreted in the same way across educational settings and across roles in the education system. This study is one of the first to measure SEL capacities among educational leaders at different educational settings, particularly beyond school sites and including district and regional levels. Leaders at these levels play important roles in SEL practice planning and agenda setting, providing implementation support to those who deliver SEL. Future work focused on assessing capacity to provide SEL implementation support at these levels will enhance collective knowledge about how administrative leadership can support SEL practice improvements across multiple schools and regions (Kendziora & Osher, 2016; Mania-Singer, 2017; Oberle et al., 2016). For example, district and regional leaders may develop visions for SEL practice, advocate for or create policies that support SEL implementation, and provide SEL resources and professional development opportunities within districts and schools (Oberle et al., 2016).

A couple of relevant examples have emerged from within the CalHOPE Support System, demonstrating the promise of measuring SEL capacities systematically in service of improving SEL implementation. First, COE leaders who attended monthly statewide Communities of Practice meetings had opportunities to learn with and from each other on topics related to SEL implementation. Through these meetings, COE leaders were qualitatively assessed to have shifted their conceptions of SEL over six months from being primarily skills-based to having systemic and equity-centered components (Eldeeb et al., 2025). In future work, these qualitative insights on gained knowledge about systemic SEL can be triangulated with quantitative insights about this domain of knowledge as measured with the capacities scale, lending to an evidence base that informs future strategies for helping leaders to gain knowledge about systemic and transformative SEL in learning communities. Second, nearly 400 teachers who took a credit-bearing university extension course in SEL completed a version of the SEL capacities scale (for classroom teachers) at the beginning and end of the course. Results showed significant improvement in all four dimensions of the scale (Duane et al., 2025), providing evidence that capacities grew among educators who took the course. This course was designed to translate knowledge into practice; available evidence suggests it was successful in doing so.

Although users are encouraged to use the subscales to gather informative data about four general dimensions of SEL capacity, users may also benefit from taking an item-based approach to analyzing data from the instrument. Developers of the scale intentionally included individual items related to the promotion of equity, to be responsive to recent calls for tSEL practices that center equity and social justice (Jagers et al., 2019, 2021; Lee et al., 2023a). Further, as evidenced in Table 2, some mean scores within a given sub-construct are meaningfully lower than others—in particular, items within the knowledge and skills sub-constructs that reflect the concepts of equity in education. An item-level analytic approach may supplement the scale-based approach by helping users to identify more specific domains of SEL capacity that represent benchmarks of success or areas for improvement.

4.2. Limitations

Several limitations are noted. (1) The sample was limited to educational leaders, many of whom are leaders in SEL practice and were motivated to participate in CalHOPE. The overwhelming majority of participants reported concurrently serving in roles that support SEL implementation, having an SEL Leadership Team on site, and serving on that SEL Leadership Team (Table 1). This may have contributed to the lack of variability in the data, including ceiling effects for items related to SEL mindsets, and low frequencies of negative (NO! or no) responses for all items. As a result, CFA models were run without the mindsets items included, and by specifying all other items on a reduced three-point scale that combined NO! and no responses into a composite value instead of on the original, intended four-point scale. The scale is intended to gauge SEL capacities among more general populations of educational leaders, not just among those who are already providing SEL implementation support. More work is needed to establish the scale’s generalizability. In particular, ceiling effects for items related to SEL mindsets might not be anticipated in more general populations of educational leaders, which, if true, would open the possibility of identifying a different pattern of dimensionality in factor analyses of the data.

(2) The sample was predominantly white and female, which although common identities among educators, may not generalize to the larger population of educational leaders working in schools, districts, and in other educational settings. It is notable that on several other key characteristics where conceptions of and mean levels of SEL capacities could differ—primary educational setting, primary role in K-12 education, years of experience—the sample was more diverse and may be more representative of educational leadership statewide. The current study did not have/use meaningful data on different cultural contexts, such as different sources of community cultural capital, different workplace compositions by race and ethnicity, different cultural perceptions of education, or different culturally and linguistically responsive experiences in educational settings. These cultural variables may intersect with SEL mindsets, skills, knowledge, and efficacy in unique ways, and more validation work is needed to assess the cross-cultural invariance of the scale both within US states and internationally.

(3) The study design did not enable rigorous tests of the scale’s predictive validity in terms of theorized outputs and outcomes, notably SEL implementation and subsequent student outcomes. Predictive validity is especially important to establish for measures that serve as mechanisms of change for a certain outcome (Fowler, 2009), and that work will be central in establishing the extent to which practice experts can draw meaningful insights about the usefulness of the capacities scale for measuring leader capacity to support SEL implementation in service of promoting student thriving. Although the current study’s results suggest a four-dimension scale best represents the structure of the data, more work is needed to determine whether there is meaningful distinction among these constructs in determining differences in leadership practices, SEL implementation, or in later student outcomes such as wellbeing, engagement, and academic performance. More specifically, additional work will be informative for testing causal paths in the SHIFT model, such as whether the provision of SEL implementation supports enhances specific dimensions of SEL capacities, and whether enhanced capacities lead to an improvement in the structures and routines of SEL implementation.

(4) Reliance on self-report data may have contributed to bias in the conclusions that can be drawn about school, district, and county levels of SEL leadership capacity. Although mindsets and perceived self-efficacy may be limited to self-assessment owing to their subjective nature, knowledge and skills are two constructs for which more objective measures may be sought in future work. For example, SEL practice courses can assess knowledge through quizzes, and skills can be assessed through observational methods. Establishing the degree to which self-reported knowledge and skills align with these complementary assessments will be useful in pursuing further validation of the scale.

5. Conclusions

As systemic SEL initiatives take root in diverse educational settings, there is a clear need for standardized approaches to monitoring progress in the building of capacity for educational leaders who support SEL implementation. The current study is the first to introduce a brief, pragmatic scale for measuring SEL implementation capacities that can be used by educational leaders in multiple roles, working across multiple systems. The scale’s intended dimensions—mindsets, knowledge, skills, and efficacy—were determined to have internal consistency, test–retest reliability, construct validity, and factor structure invariance, but not scalar invariance, across meaningful educational leader characteristics. Data generated by the proposed multidimensional scale can inform practice decisions in multiple educational settings, aid in progress monitoring through identification of specific strengths and areas for improvement, and unearth specific mechanisms of change related to SEL capacities when evaluating SEL programs.