Is Fidelity of Implementation of an Anti-Bullying Policy Related to Student Bullying and Teacher Protection of Students?

Abstract

Bullying is a significant school problem. Policies have been developed to reduce bullying, yet little is known about their implementation, which must occur for these policies to have an effect. This study examines associations between the overall implementation of a state anti-bullying policy and implementation of specific components outlined in the policy with two outcomes: bullying among students and teacher protection of students. Data were collected from 588 educators in K-12 schools across North Carolina a year following the enactment of an anti-bullying law in the state. Results show that overall policy implementation fidelity is inversely related to student bullying and positively related to teacher protection. In addition, the implementation of certain policy components (i.e., educator and student knowledge of bullying reporting procedures, training of educators about protected classes from bullying, student knowledge of protected classes, and educators reporting and remediating bullying based on protected classes) is significantly related to the outcomes. Thus, the implementation of certain anti-bullying policy components may be more potent in addressing bullying. Future research should identify constellations of policy strategies that need to be activated in schools to eliminate bullying.

1. Introduction

Bullying is a prevalent threat to the well-being and school success of U.S. students, especially students from vulnerable groups. To combat bullying, a number of intervention and prevention strategies have been developed and implemented in recent decades, including policies. In the absence of federal legislation, all 50 states have enacted anti-bullying laws [1]. For an education policy to have an effect, it must be well-crafted and implemented as intended; however, research on the content and implementation of anti-bullying policies in schools remains vastly understudied, leaving policymakers and educators with little evidence to guide policy formation and implementation. The purpose of this study is to examine how the extent of implementation of various anti-bullying policy components is related to two bullying outcomes: bullying among students and teacher protection of students from bullying.

1.1. Definition and Prevalence of Bullying

Bullying has been defined as unwanted aggressive behaviors enacted intentionally over time by a student or group of students using some form of power to cause physical and/or psychological harm to another student in a school setting [2,3]. Findings from a nationally representative survey of U.S. youth ages 12 to 18 found that 21% of students were bullied at school [4]. In addition, the Youth Risk Behavior Survey has shown a constant prevalence rate of 19–20% for bullying victimization since tracking began in 2009 [5,6,7,8,9]. Thus, bullying is a prevalent and persistent school problem.

1.2. Negative Outcomes Associated with Bullying

Bullying is associated with a host of negative outcomes for students. Being a victim of bullying has been linked with a number of negative school and mental health outcomes, including absenteeism, low academic performance, low self-esteem, depression, suicidal thinking and behavior, anxiety, and psychosomatic problems (e.g., sleep difficulties, bed wetting, headaches, stomach aches, and neck or back pain) [10,11,12,13,14,15,16,17,18,19,20]. Students involved as perpetrators in the bullying dynamic also suffer. Bullying perpetration is associated with truancy, low academic performance, dropout, depressive symptoms, suicidal ideation and behavior, and violent and criminal behavior (e.g., assault, robbery, vandalism, weapon-carrying, and rape) [10,13,15,16,19,21,22,23,24]. Thus, bullying threatens the school success and well-being of students involved in different aspects of the bullying dynamic.

1.3. Student Groups Targeted for Bullying and Protected Classes

Bullying is often motivated by prejudice toward disadvantaged or stigmatized groups who are disproportionately affected by bullying [25]. Research demonstrates that the following population groups experience high rates of bullying victimization: students who are lesbian, gay, bisexual, transgender, or queer (LGBTQ) [26,27]; students with disabilities or health conditions [28,29,30]; students who are overweight or obese [31,32,33]; students from lower socioeconomic backgrounds [34,35]; students who are immigrants or the children of immigrants [36,37,38]; and students of color (i.e., American Indian, Asian, Multiracial, and Pacific Islander) [5,6,7]. Thus, bullying may even further disadvantage students who are already vulnerable to health and school problems. Given this, 21 states enumerate protected classes in their anti-bullying laws [1]. A protected class is a group of people with a common characteristic who are legally protected from bullying on the basis of that characteristic (e.g., race, gender, and national origin) [39]. Laws that enumerate protected classes or statuses make it explicitly clear that forms of bias-based bullying are prohibited, which is intended to protect vulnerable groups. For example, evidence shows high rates of bullying toward LGBTQ students [26,27], 20 states have anti-bullying laws that enumerate sexual orientation and gender identity as protected classes [1], and evidence shows that LGBTQ students in schools with enumerated policies reported less harassment and more frequent and effective intervention by school personnel [40,41].

1.4. Policy Interventions for Bullying

Currently, the United States does not have a federal law against school bullying. However, between 1999 and 2015, all 50 states passed anti-bullying laws [1]. The passage of these laws was spurred by the shootings at Columbine High School in 1999 and the increasing awareness and concern about bullying and school safety since that time [42,43]. These laws apply to approximately 98,000 PreK-12 public schools, with the goal of protecting over 50 million students from involvement in bullying [1,44].

1.5. Effectiveness of Policy Interventions for Bullying

A recent systematic review on the effectiveness of policy interventions for bullying found mixed results [40]. Some studies showed that having an anti-bullying policy was associated with lower levels bullying perpetration and victimization [45,46]. Conversely, another set of studies found no associations between anti-bullying policy presence and reductions in bullying [47,48]. Research is also mixed regarding the relation between having an anti-bullying policy and educators intervening in bullying [40]. One study found that educators in schools with an anti-bullying policy were more likely to respond to a bullying incident, compared to educators in schools without such a policy [49]. Conversely, two studies showed that having an anti-bullying policy was not connected with educator intervention in bullying [48,50].

A simple assessment of whether a school does or does not have an anti-bullying policy may not be sufficient in examining the effectiveness of a policy. What likely matters more are the quality of the strategies outlined in the policy and the extent to which the policy is implemented [40]. Emerging evidence does suggest that policy content is related to bullying outcomes among students. Hatzenbuehler and colleagues found that the number of components specified in state anti-bullying policies was not associated bullying victimization; however, three policy components were consistently found to be associated with a decreased odds of being bullied: (1) outlining the scope of school jurisdiction in regulating bullying; (2) describing bullying behaviors that were prohibited, and (3) requiring districts to develop and implement their own local anti-bullying policies [51]. In another study, Sabia and Bass identified two policy components associated with a decreased likelihood of being bullied: (1) procedures for investigating bullying incidents and (2) consequences for bullying perpetrators [52].

In addition to examining policy components individually, other researchers have created overall policy quality scores based on various content criteria and then used these scores to examine relations between policy quality and bullying outcomes. Sabia and Bass used policy content scores based on 16 criteria identified by the U.S. Department of Education: purpose of the policy, applicability or scope of the policy, prohibition of bullying behaviors, enumeration of protected classes, requirement for districts to implement policies, review of district policies by the state, definition of bullying behaviors prohibited, procedures for reporting bullying, procedures for investigating bullying, procedures for maintaining records of bullying, consequences or sanctions for bullying, mental health services for bullying victims and/or perpetrators, communication about the policy to members of the school community, training school personnel or bullying intervention and prevention, data collection and monitoring of bullying, and assurance of right to pursue legal remedies for victims [52,53]. Sabia and Bass found that higher policy content scores were associated with reductions in the probability of being bullied [52]. In another study, Ordonez evaluated the quality of anti-bullying policy strategies based on the following criteria: a definition of bullying; procedures and consequences for bullies; plans for disseminating the policy to students, school personnel, and parents; programs or practices that encourage acceptance of diversity, empathy for others, respect toward others, peer integration, and responsible use of power; supervision of students in school areas prone to bullying (e.g., playground, cafeteria, and hallways); and socio-emotional skills training for victims and bullies [54]. Ordonez found that students in schools with higher policy quality scores reported significantly lower rates of verbal and physical bullying victimization than students in schools with lower quality policies [54]. In a third study, Woods and Wolke analyzed the content of anti-bullying policies based on the following criteria: a definition of bullying; recognition of negative outcomes linked with bullying; notes of locations where bullying can occur; evaluation of the prevalence of bullying; involvement of stakeholders in policy development; supervision of students in school areas; formulation of a school task group to coordinate anti-bullying efforts; classroom rules about bullying; classroom sessions about bullying; discussion of bullying at parent-teacher association meetings; involvement of parents in bullying prevention efforts; and follow-up with victims and bullies after incidents [55]. Woods and Wolke found that students in schools with higher policy quality scores reported significantly lower rates of verbal and physical bullying victimization on the playground than students in schools with lower quality policies [55]. Findings from these studies suggest that merely having an anti-bullying policy is not sufficient to effectively reduce bullying; the quality of the content of the policy matters. In addition to content, the extent to which a policy is implemented may also shape its effectiveness.

1.6. Fidelity of Implementation of Anti-Bullying Policies

The passage of a policy by officials does not mean that a policy will be immediately and efficiently put into operation precisely as intended. Implementation is a complex, dynamic, and ongoing process involving a vast assortment of people, resources, organizational structures, and actions [56]. In order for a policy or program to accomplish its intended effects, it must be implemented with a high degree of fidelity [57,58,59]. For policy interventions, fidelity refers to the extent to which a policy is implemented as intended based on the directives expressed in the policy document. Directives outlined in state anti-bullying laws vary somewhat but often require school districts to formulate local bullying policies, train school personnel on the policy and bullying intervention, notify students and parents about the policy, establish procedures for reporting and investigating bullying incidents, establish appropriate consequences for bullies, and provide mental and behavioral health services for victims and bullies [53].

Researchers have found considerable variability in the fidelity of implementation of policy interventions for bullying. For example, 51% to 98% of educators reported that their school systems had adopted a local anti-bullying policy in compliance with their state’s policy [60,61,62,63,64,65,66]. In terms of training and notification regarding bullying policies, 39% to 94% of educators reported receiving training on the policy [60,61,64,65,66,67,68,69], and 56% to 84% of educators reported that students were notified about the policy [62,64,65,69,70]. Regarding school procedures, 60% to 94% of educators indicated that their school maintained procedures for reporting bullying [64,67,69,70], 78% to 92% of educators indicated that their school had procedures for investigating reports or complaints about bullying [65,67,69,70], and 52% to 80% of educators indicated that their school provided mental health assistance to students involved in bullying [61,65,69]. These findings show that implementation fidelity varies across locations and policy components.

Only one study was found in the literature that examined the extent to which an anti-bullying policy was implemented and bullying outcomes [67]. This study found that higher levels of implementation of strategies associated with a state anti-bullying law predicted lower levels of bullying severity, lower levels of harassment based on protected classes (e.g., race, sex, national origin, religion, disability, weight, sexual orientation, and gender identity), and more positive perceptions of school climate [67]. Given that this was the only study found in the literature to examine fidelity of policy implementation with bullying outcomes, more research is needed in this area.

1.7. Purpose of the Current Study

This study intended to fill gaps in the literature by examining the relationships between fidelity of anti-bullying policy implementation and two behavioral outcomes: bullying among students and teacher protection of students. To advance a more specific and granular understanding in this area, in addition to examining the relations between overall policy implementation fidelity and the outcomes, we also examined associations between the implementation of individual policy components and the outcomes. Based on the extant literature, it was hypothesized that there would be (1) a significant and inverse association between implementation fidelity and student bullying and (2) a significant and positive association between implementation fidelity and teacher protection of students. The current study involved data collected from educators about the School Violence Prevention Act of 2009, which is the anti-bullying law in North Carolina (NC) that applies to elementary and secondary public schools [71].

2. Methods

2.1. Policy Content of the School Violence Prevention Act (SVPA)

The SVPA was signed into law on 23 June 2009. In the law, bullying was defined as verbal, written, electronic, or physical actions that induced fear of harm or created a hostile environment for a student [71]. Such behaviors were prohibited as well as bullying behavior based on actual or perceived race, color, ancestry, national origin, religion, gender, socioeconomic status, academic status, sexual orientation, gender identity, physical appearance, and disability. The law applied to behavior on school property, at school-sponsored functions, and on school buses. According to the law, school personnel who witnessed or possessed information about bullying were required to report incidents to the appropriate school officials. On the other hand, students and school volunteers were encouraged but not required to report bullying incidents.

The law also required that school districts adopt their own local anti-bullying policies by 31 December 2009 and train all school employees by 1 March 2010. Each local policy had to include the provisions described above in terms of the definition of bullying, prohibition of bullying behaviors, enumerated protected classes, scope of the policy, and bullying reporting requirements, as well as other components:

• Behavioral expectations for students and school personnel.
• Procedures for reporting bullying incidents.
• Identification of a school employee designated to investigate reports of bullying.
• Procedures for investigating reports of bullying incidents.
• Prohibition of reprisal or retaliation against individuals who reported bullying incidents.
• Consequences and appropriate remedial actions for students who committed acts of bullying.
• Plans to publicize and disseminate the local policy.
• Inclusion of the local policy in student and employee handbooks.
• Inclusion of the local policy in employee training.

2.2. Study Design and Procedures

Because school-level educators are the primary implementers of education policy, we surveyed members of a statewide professional association of educators and school employees in NC. The cross-sectional survey was announced in an email message sent through the association’s membership listserv. The email invitation contained a brief description of the survey, stating that it was focused on bullying, was optional and anonymous, and could be completed within 15 min. The email also contained a link to the welcome and informed consent page of the online survey. An online survey format was selected because of several advantages: participants can respond to a Web survey at times and places convenient for them, participants can often complete Web surveys quickly, and participants may be less affected by social desirability bias in their responses because they are not directly disclosing the information to another person. In the survey, participants initially completed three demographic questions and were asked to identify the school and district in which they worked. Identifying their school allowed for the merging of the survey data with school-level data from the 2009–2010 school year (e.g., school size and student to teacher ratio). The remainder of the survey items assessed student bullying, teacher protection of students, and the implementation of the SVPA. No incentives were used to solicit participation. The survey was available from mid November 2010 to early January 2011. Research ethics were followed in the conduct of this study and it was approved by the authors’ institutional review board.

2.3. Sample

Of the approximately 5000 educators who were invited to participate, 664 (13.3%) responded to the survey to some extent. However, 76 respondents were excluded because they did not complete any of the dependent variable items, worked in private or charter schools, or worked in special education or vocational schools. Thus, a total of 588 participants were included for data analysis, which was 11.8% of the educators invited to participate. Bivariate analyses comparing the 588 included and the 76 excluded respondents showed no significant differences in terms of the proportions of White, non-White, male, and female respondents.

The sample of 588 educators included 79% teachers, 10% education support professionals, 4% school administrators, 3% school counselors, 3% school social workers, and 1% school nurses. The racial/ethnic breakdown of the sample was 80% White/Caucasian, 15% Black/African American, 1% Hispanic/Latino/Latina, 1% American Indian or Alaska Native, and 3% multiracial/multiethnic. The sample included 84% females and 16% males. These sample demographics are closely aligned with statewide representative demographic data of NC K-12 public school teachers, which shows that 81% of teachers were White, 16% were Black, 1% were Hispanic, 1% were American Indian or Alaska Native, 1% were Asian, 80% were female, and 20% were male [72].

Respondents were employed in 324 schools in 85 (74%) of the 115 school districts in NC. All of the participants worked in regular education K-12 public schools. The grade levels at the schools where participants worked varied with 40% in elementary schools, 2% in elementary-middle schools, 25% in middle schools, 5% in middle-high schools, and 28% in high schools. In terms of the geographic area of participants’ schools, 62% were in small town or rural areas, 22% were in urban areas, and 16% were in suburban areas. These figures are similar to those among all NC K-12 public schools: 51% were elementary schools, 18% were middle schools, 19–21% were high schools, 5% were elementary-middle schools, 3% were middle-high schools, and 2% were K-12 schools [73]. In terms of geographic area, 62% were in small town or rural areas, 22% were in urban areas, and 16% were in suburban areas.

2.4. Dependent Variables

Student Bullying and Teacher Protection of Students

The dependent variables of ‘student bullying’ and ‘teacher protection of students’ were assessed using the Bully Index [74], which is an educator-report measure with two subscales. The student bullying subscale consisted of six items (e.g., “Students in this school threaten others with physical harm”), and the teacher protection subscale consisted of four items (e.g., “Teachers in my school reach out to help students who are harassed by other students”). Participants were asked to think about the school in which they worked and rate their agreement with 10 statements using a five-point Likert-type scale ranging from 1 (disagree) to 5 (agree). A score for each subscale was calculated by averaging responses from the items for each subscale. Higher scores indicate higher levels of bullying among students at the school and teacher engagement in protection of students from bullying. In other studies, this scale has demonstrated acceptable to good internal consistency reliability (α = .73 to .96), as well as evidence of convergent and divergent validity [74,75,76,77]. In the present study, the internal consistency reliabilities of the student bullying and teacher protection subscales were α = .92 and α = .77, respectively.

2.5. Independent Variables

This study included two sets of independent variables: the main explanatory variables of interest related to fidelity of anti-bullying policy implementation, as well as covariates related to the school context (e.g., school size and student to teacher ratio). School contextual variables were included to control for their effects. Research has shown that contextual variables account for significant amounts of variance in bullying-related outcomes [68]. Indeed, certain school contexts may facilitate or inhibit aggressive behavior among students (e.g., low levels of adult monitoring of school areas prone to bullying versus strong socio-emotional bonds between students) [78,79,80].

2.5.1. Fidelity of Bullying Policy Implementation

Fidelity of implementation of the SVPA was measured using nine items designed by a group of four individuals (i.e., one educator, one parent, and two researchers) who were involved in advocating for the passage of the SVPA. The items were constructed based on the content of the SVPA and assessed the implementation fidelity of nine policy components. Fidelity of implementation related to protected social classes (i.e., race, national origin, gender, socioeconomic status, sexual orientation, gender identity, physical appearance, and disability status) was emphasized for two reasons. First, this aspect of the law was highly controversial during the formulation of the policy [81], and thus, might not be executed as intended. Second, youth who are vulnerable or members of minority groups are often targeted for bullying and report high rates of victimization [25,82].

To assess implementation fidelity, participants were asked the following:

• Whether or not they had received training on the SVPA, with response options of ‘yes’ and ‘no’;
• How often they knew whom to report incidents of bullying to at their school, with response options of ‘never’, ‘rarely’, ‘sometimes’, ‘most times’, and ‘always’;
• How often students at their school knew whom to report incidents of bullying to, with response options of ‘never’, ‘rarely’, ‘sometimes’, ‘most times’, and ‘always’;
• Which social classes were protected from bullying in their school’s local policy, and participants could select ‘I don’t know’ for the item or select ‘yes’ or ‘no’ beside each of the eight social classes;
• If they had received training about bullying based on the eight social classes, and participants could select ‘I don’t know’ for the item or select yes or no beside each of the eight social classes;
• If students in their schools had been informed that bullying was prohibited based on the eight social classes, and participants could select ‘I don’t know’ for the item or select ‘yes’ or ‘no’ beside each of the eight social classes;
• How often employees at their school reported witnessed bullying incidents based on the eight social classes to the designated school official, with response options of ‘never’, ‘rarely’, ‘sometimes’, ‘most times’, and ‘always’;
• How often school officials investigated reports of bullying based on the eight social classes, with response options of ‘never’, ‘rarely’, ‘sometimes’, ‘most times’, and ‘always’; and
• How often appropriate remedial action was given to students who perpetrated bullying based on the eight social classes, with response options of ‘never’, ‘rarely’, ‘sometimes’, ‘most times’, and ‘always’.

Fidelity scores are often expressed as percentages where 0% would indicate that an intervention was not at all implemented as intended and 100% would indicate that an intervention was completely implemented as intended [83]. Using this logic, participants’ responses were coded as percentages for the nine implementation variables. For item 1, responses of ‘yes’ and ‘no’ or ‘I don’t know’ were coded as 100% and 0%, respectively. For items 2–3 and 7–9, responses of ‘never’, ‘rarely’, ‘sometimes’, ‘most times’, and ‘always’ were coded as 0%, 25%, 50%, 75%, and 100%, respectively. For items 4–6, a percentage was calculated based on the number of ‘yes’ responses selected out of eight. Thus, no ‘yes’ responses received a 0% score, one ‘yes’ response received a 12.5% score, two ‘yes’ responses received a 25% score, and so on. Finally, an overall implementation fidelity score was calculated by averaging the nine percentages. The nine-item implementation fidelity measure demonstrated very good internal consistency reliability (α = .82) and has face validity because item content was derived directly from statements in the SVPA by a team of experts who were very familiar with the law.

2.5.2. School-Level Covariates

School-level data collected through the NC Department of Public Instruction for the 2009–2010 school year included the following variables:

• Size of the student body in terms of average daily membership;
• Student to teacher ratio, which was attained by dividing the average daily membership by the total number of classroom teachers in the school;
• Percent of economically disadvantaged students (i.e., those eligible for free or reduced-price lunch);
• Number of short-term suspensions per 100 students;
• Percent of students scoring below grade level on the end of grade math test;
• Percent of students below grade level on the end of grade reading test;
• Percent of teachers with advanced degrees (i.e., master’s, educational specialist, or doctoral degrees);
• Percent of teachers with less than 4 years of teaching experience;
• Percent of teachers with 4 to 10 years of teaching experience;
• Percent of teachers with more than 10 years of teaching experience;
• Teacher turnover rate (i.e., the percent of teachers in the school who left their positions in the past year); and
• Total per pupil expenditure in dollars (i.e., the sum of local, state, and federal expenditures per student).

2.6. Data Analysis

Prior to analysis, a number of diagnostics were performed using Stata (version 15) to examine the linearity between the independent and dependent variables, the distributions of the residuals, the distributions of the independent and dependent variables, influential outliers, heteroscedasticity, multicollinearity, and missing values. Plots showed no clear departures from linearity, and the residuals were approximately normally distributed for both dependent variables. All of the variables were approximately normally distributed except for the number of suspensions per 100 students. This variable had a positively skewed distribution, and thus, a natural log transformation was used to achieve a normal distribution. No cases of influential outliers were found, and no significant heteroscedasticity problems were found. However, two multicollinearity problems (variance inflation factor scores > 10) were found for the variables representing the proportion of students below grade level in math and reading. These two variables were highly correlated (r = .91, p < .001), and thus, were averaged together to create one new variable—the percent of students below grade level. In terms of missing data, 6.8% of values were missing; thus, full information maximum likelihood (FIML) was used to handle missing data. This procedure allows for all cases to be included in analyses, even if they are missing values on some variables. Analyses were run using Stata (version 15) and included descriptive statistics, bivariate correlations, and a series of regression models using maximum likelihood estimation.

3. Results

The means and standard deviations shown in

Table 1. Means, Standard Deviations, and Intercorrelations for Study Variables.

Variable	Mean	SD	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22
1. Training school personnel on the law	37.59	48.48	--
2. Educator knowledge of bullying reporting procedures	90.78	18.72	.20 *	--
3. Student knowledge of bullying reporting procedures	78.14	21.37	.21 *	.52 *	--
4. Inclusion of protected social statuses in the local bullying policy	72.65	41.26	.21 *	.26 *	.29 *	--
5. Training school personnel about protected social statuses	34.66	44.20	.49 *	.21 *	.27 *	.37 *	--
6. Student knowledge of social statuses protected from bullying	62.21	44.35	.26 *	.32 *	.39 *	.55 *	.45 *	--
7. School personnel reporting bullying incidents based on social statuses	78.22	22.75	.21 *	.48 *	.53 *	.32 *	.30 *	.41 *	--
8. Investigating reports of bullying based on social statuses	81.01	24.79	.26 *	.51 *	.53 *	.36 *	.30 *	.39 *	.74 *	--
9. Taking appropriate remedial action with bullying perpetrators based on social statuses	72.13	26.97	.30 *	.43 *	.50 *	.29 *	.36 *	.43 *	.71 *	.76 *	--
10. School size or number of students	719.16	441.46	−.05	−.13 *	−.19 *	−.05	−.09	−.09 *	−.15 *	−.12 *	−.18 *	--
11. Student to teacher ratio	14.52	2.61	−.01	−.03	.03	−.01	.01	−.01	−.03	.01	.00	.55 *	--
12. Percent of economically disadvantaged students	59.13	22.04	.03	.03	.06	−.02	−.01	.00	.09	.04	.08	−.46 *	−.47 *	--
13. Number of suspensions per 100 students	1.08	0.56	−.00	−.03	−.17 *	−.01	−.04	.01	−.07	−.12 *	−.15 *	.17 *	−.27 *	.17 *	--
14. Percent of students below grade level	72.50	13.77	.00	.00	.12 *	.02	.03	−.03	−.01	.06	.06	.31 *	.60 *	−.73 *	−.59 *	--
15. Percent of teachers with advanced degrees	27.15	9.56	−.02	−.01	.01	−.01	−.02	−.07	−.01	−.03	−.02	.12 *	.08	0.18	−.24 *	.21 *	--
16. Percent of teachers with 0 to 3 years of experience	20.20	9.55	.02	−.04	−.04	.01	−.00	.04	.04	−.03	.02	−.13 *	−.25*	.24 *	.27 *	−.42 *	−.37 *	--
17. Percent of teachers with 4 to 10 years of experience	29.41	9.35	.05	−.02	.01	−.09	−.11	−.06	.01	.02	−.02	.14 *	.22 *	.03	−.26 *	.12 *	.11 *	−.23*	--
18. Percent of teachers with 11 or more years of experience	50.53	11.80	−.05	.05	.03	.06	.09	.01	−.04	.00	.00	−.00	.03	−.23 *	−.02	.26 *	.22 *	−.63 *	−.61 *	--
19. Teacher turnover rate	11.76	6.47	−.02	−.03	−.05	.07	.02	.07	.03	−.03	.00	−.09	−.22 *	.26 *	.41 *	−.47 *	−.30 *	.47 *	−.17 *	−0.26 *	--
20. Per pupil expenditure in dollars	8800.00	1006.34	−.00	.07	.06	.05	.05	.07	.05	.03	.05	−.31 *	−.34	.22 *	−.02	−.05	.01	.04	−.23 *	.15 *	.09 *	--
21. Overall policy implementation fidelity score	65.06	25.52	.59 *	.56 *	.62 *	.64 *	.69 *	.73 *	.69 *	.72 *	.72 *	−.13 *	.02	.03	−.09	.04	−.06	.00	−.07	.05	.03	.07	--
22. Student bullying score	3.05	1.13	−.15 *	−.26 *	−.39 *	−.12 *	−.17 *	−.17 *	−.37 *	−.40 *	−.44 *	.17 *	−.10 *	.06	.39 *	−.21 *	−.08	.10 *	.03	−.11 *	.15 *	.01	−.32 *	--
23. Teacher protection of students score	4.23	.76	.16 *	.41 *	.47 *	.32 *	.32 *	.43 *	.53 *	.52 *	.54 *	−.16 *	.03	.01	−.23 *	.08	.00	.02	.05	−.05	−.02	−.00	.48 *	−.46 *

* p < .05.

indicate that the extent to which the SVPA was implemented varied across schools and policy components. Table 1 also shows the intercorrelations for the study variables. All nine of the policy implementation fidelity scores were significantly correlated with both of the outcomes, with inverse correlations for student bullying and positive correlations for teacher protection of students. Large correlation coefficients were found for certain policy components, namely student knowledge of bullying reporting procedures, school personnel reporting bullying incidents based on social statuses, investigating reports of bullying based on social statuses, and taking appropriate remedial action with bullying perpetrators based on social statuses (Table 1).

Multivariate linear regression modeling was used to examine the relationships between fidelity of implementation of the SVPA and the outcome variables. Two regression models were run predicting student bullying and teacher protection using the overall policy implementation fidelity composite scores and school-level covariates as the independent variables.

Table 2. Regression Analyses Predicting Student Bullying and Teacher Protection of Students (N = 588).

Independent Variable	Student Bullying Model			Teacher Protection Model
	B	SE B	95% CI	B	SE B	95% CI
Fidelity of Implementation of the Policy
Training school personnel on the law	−0.001	0.001	[−0.003, 0.001]	−0.001	0.001	[−0.002, 0.000]
Educator knowledge of bullying reporting procedures	−0.000	0.003	[−0.006, 0.005]	0.004 *	0.002	[0.001, 0.008]
Student knowledge of bullying reporting procedures	−0.008 *	0.003	[−0.013, −0.003]	0.004 *	0.002	[0.001, 0.007]
Inclusion of protected social statuses in the local bullying policy	0.001	0.001	[−0.001, 0.004]	0.000	0.001	[−0.001, 0.002]
Training school personnel about protected social statuses	0.000	0.001	[−0.002, 0.003]	0.001 †	0.001	[−0.000, 0.003]
Student knowledge of social statuses protected from bullying	0.001	0.001	[−0.002, 0.003]	0.003 *	0.001	[0.001, 0.004]
School personnel reporting bullying incidents based on social statuses	−0.005	0.003	[−0.010, 0.001]	0.005 *	0.002	[0.002, 0.009]
Investigating reports of bullying based on social statuses	−0.005	0.003	[−0.010, 0.001]	0.002	0.002	[−0.001, 0.006]
Taking appropriate remedial action with bullying perpetrators based on social statuses	−0.008 *	0.003	[−0.013, −0.003]	0.005 *	0.002	[0.001, 0.008]
Overall policy implementation fidelity composite score a	−0.012 *	0.002	[−0.015, −0.009]	0.014 *	0.001	[0.012, 0.016]
School-Level Covariates
School size or number of students	0.000	0.000	[−0.000, 0.001]	−0.000	0.000	[−0.000, 0.000]
Student to teacher ratio	−0.034	0.026	[−0.085, 0.018]	−0.005	0.017	[−0.038, 0.028]
Percent of economically disadvantaged students	0.004	0.004	[−0.003, 0.011]	−0.002	0.002	[−0.006, 0.003]
Number of suspensions per 100 students	0.654 *	0.108	[0.443, 0.864]	−0.240 *	0.067	[−0.372, −0.108]
Percent of students below grade level	0.007	0.008	[−0.009, 0.023]	−0.001	0.005	[−0.011, 0.009]
Percent of teachers with advanced degrees	−0.003	0.005	[−0.013, 0.007]	0.002	0.003	[−0.004, 0.008]
Percent of teachers 0 to 3 years of experience	0.088	0.084	[−0.076, 0.252]	0.034	0.053	[−0.069, 0.137]
Percent of teachers 4 to 10 years of experience	0.099	0.083	[−0.065, 0.262]	0.031	0.053	[−0.072, 0.134]
Percent of teachers with 11 or more years of experience	0.083	0.083	[−0.080, 0.247]	0.029	0.053	[−0.074, 0.132]
Teacher turnover rate	0.001	0.009	[−0.016, 0.019]	0.002	0.006	[−0.009, 0.013]
Per pupil expenditure in dollars	0.000	0.000	[−0.000, 0.000]	−0.000	0.000	[−0.000, 0.000]

* p < .05; ^† p < .08; ^a Separate models were run for this explanatory variable because it is a composite of the above nine individual fidelity scores.

shows the results of the regression analyses, including unstandardized regression coefficients, standard errors, and 95% confidence intervals. The independent variables accounted for moderate amounts of the variance in student bullying (R² = .258) and teacher protection (R² = .284). Overall policy implementation fidelity scores were significantly inversely related to student bullying (B = −0.012) and significantly positively related to teacher protection (B = 0.014).

Next, another pair of regression models were run predicting student bullying and teacher protection using the implementation fidelity scores of the individual policy components and school-level covariates as independent variables. The results are shown in Table 2. The independent variables accounted for significant amounts of the variance in student bullying (R² = .362) and teacher protection (R² = .446). In both of these models, student knowledge of bullying reporting procedures (B = −0.008 and B = 0.004, respectively) and taking appropriate remedial action with bullying perpetrators based on social statuses (B = −0.008 and B = 0.005, respectively) were significantly related to both of the outcome variables. However, for the teacher protection model, three additional implementation fidelity variables were significantly associated with teacher protection: educator knowledge of bullying reporting procedures (B = 0.004), school personnel reporting bullying incidents based on social statuses (B = 0.005), and student knowledge of social statuses protected from bullying (B = 0.003). Additionally, in this model, training school personnel about protected social statuses was marginally related to teacher protection (B = 0.001). Among the school-level covariates, the prevalence of suspensions among students was the only school-level covariate significantly associated with the outcomes (B = 0.654 and B = −0.240, respectively).

4. Discussion

The findings show significant associations between overall anti-bullying policy implementation fidelity regarding the SVPA and bullying-related outcomes. Specifically, higher levels of anti-bullying policy implementation fidelity predicted lower levels of student bullying and higher levels of teacher protection of students, with the predictors accounting for close to 30% of the variance in the outcomes. These findings align with those from Cosgrove and Nickerson, who found that higher levels of implementation of strategies associated with New York’s anti-bullying law were associated with lower levels of bullying, lower levels of biased-based harassment, and more positive perceptions of school climate [66]. Thus, findings from this New York study and the present North Carolina study indicate that higher levels of implementation of a collection of anti-bullying policy strategies is related to lower levels bullying among students and higher levels of school protective factors (i.e., teacher protection of students and a positive school climate). Across the New York and North Carolina laws, the collection of anti-bullying policy strategies assessed included receiving training on the policy, understanding social classes protected from bullying, understanding reporting and investigating procedures for bullying incidents, and having remedial plans or interventions to address bullying and harassment. Implementing collections of anti-bullying strategies have the potential to influence multiple bullying-related outcomes and improve the school environment. Indeed, other studies have found that interventions can promote a positive school climate (i.e., positive interactions between individuals in the school community, feelings of physical and emotional safety, mutual respect and support, and valuing learning and engagement in education), which is incompatible with bullying behavior [84].

The findings also underscore the importance of implementation in the education policy process. Although the policy process has been conceptualized in different ways by scholars, there is agreement that the process involves several key phases with constituent activities for each phase [85]. According to Jann and Wegrich, the policy process includes the following phases: (1) Agenda-setting: an educational problem is identified that needs to be addressed by officials; (2) Formulation: the problem is investigated and potential solutions are developed; (3) Decision-making: policymakers discuss and debate possible strategies and then select a policy solution; (4) Implementation: the new policy is put into action; and (5) Evaluation: the policy is evaluated to ascertain if it was properly implemented and if the targeted outcomes were achieved [85]. The current study focuses on the latter two phases of implementation and evaluation; however, this is not an endpoint. The policy process is iterative, and there is an ongoing need for improvement in education. Evaluation findings regarding policy implementation and the targeted outcomes can be used by policymakers, educators, and school officials to modify the policy, address implementation strategies, and provide additional necessary resources so that the goals of the policy can be fully attained. Given our results showing varying levels of implementation of policy components (i.e., 35–91%; Table 1) yet the importance of implementation to the bullying outcomes, policymakers, educators, and school officials should provide additional resources and guidance to address implementation fidelity issues to improve the attainment of the outcomes.

The findings also showed significant correlations between implementation of each of the nine policy components and the outcomes. Nonetheless, the regression results showed that only two of the policy components (i.e., student knowledge of bullying reporting procedures and educators taking remedial action with bullying) were significantly associated with the student bullying outcome, and six of the nine components were significantly or marginally associated with the teacher protection outcome: (1) educator knowledge of bullying reporting procedures; (2) student knowledge of bullying reporting procedures; (3) training school personnel about protected social statuses; (4) student knowledge of social statuses protected from bullying; (5) school personnel reporting bullying incidents based on social statuses, and (6) taking appropriate remedial action with bullying perpetrators based on social statuses. These findings suggest that the activation of certain policy components are more likely to have an effect on bullying-related outcomes, compared to other components. Furthermore, the results suggest that certain components may have a greater influence on the behavior of certain members of the school community (i.e., students compared with educators) in terms of bullying-related outcomes. For example, the results suggest that students understanding the bullying reporting procedures in their school and educators taking corrective action when bullying incidents are reported are likely to curb student bullying behaviors. The findings also indicate that educators who understand bullying reporting procedures in their school, educators being trained about protected classes from bullying, educators reporting bias-based bullying to the designated school official, and officials taking appropriate action to address the bullying incidents reported may result in the protection of students by educators. Considering these findings in the current context of public education where educators are often asked to do more without suitable resources, the implementation of certain policy components may need to be prioritized over others. The implementation of all of the individual policy components may have the potential to influence bullying-related outcomes; however, the regression results found that the implementation of certain components was found to be significantly related to the outcomes.

The only school-level factor significantly related to the outcomes was student suspensions. Thus, as the number of suspensions per 100 students increased, student bullying scores increased and teacher protection scores decreased. One explanation for this finding rests on the assumption that suspensions are a consequence of a range of serious behavior problems among students. Historically, suspensions usually resulted from physical fighting, insubordination, class disruption, skipping class, drug use and selling, vandalism, and weapon possession [86]. Thus, behavior problems resulting in student suspension also likely co-occur with bullying behaviors. Comprehensive interventions may be needed to address this range of problematic student behaviors.

4.1. Limitations

This study has several limitations. This study did not use probability sampling, and thus, the sample may not be representative of educators across NC. Another possible limitation was the somewhat low response rate; however, a low response rate does not necessarily lead to nonresponse error, and there are examples of surveys with lower response rates that were as accurate as or more accurate than those with higher response rates [87,88,89]. The representativeness of a sample is more important than the response rate [90], and in this study, the sample demographics parallel those that are representative of teachers in the state. In addition, there may have been selection bias because educators who took the survey may have been more interested in bullying, and therefore, may have been more alert to student bullying and overly critical in their assessment of the implementation of the SVPA and teacher efforts to protect students. On the other hand, participants’ responses may have been influenced by social desirability response bias because educators were asked to report on legally mandated actions related to the SVPA, negative behaviors among students, and actions by fellow educators to protect students from bullying. Another limitation is that the outcome of student bullying was based on educator perceptions. Although teachers are considered key informants who know what goes on in schools, bullying among students may have been more accurately measured using student-report data. A final limitation relates to the cross-sectional nature of the survey; implementation is an ongoing process, yet our assessment occurred at only one time point.

4.2. Future Research

To advance research in this area, several recommendations for future studies are outlined. First, researchers should comprehensively measure the implementation of individual policy components and examine their associations with bullying outcomes. Such research would reveal the combinations of implementation actions needed to effectively reduce bullying. Comprehensive and precise empirical examinations of the role of policy implementation can reveal key mechanisms and processes that occur between the adoption of an anti-bullying policy and the outcome of reduced bullying behavior. Furthermore, assessments of implementation should include multiple respondent groups, such as teachers, students, and administrators, which would allow for a multi-perspective assessment of implementation and the triangulation of results. A second recommendation is that a range of bullying-related outcomes should be assessed because the activation of different policy components may affect different groups within the school community; outcomes may include bullying perpetration by students, bullying victimization among students, school climate perceived by teachers, and disciplinary issues related to bullying perceived by school administrators. In addition, bullying is a multidimensional behavioral phenomenon [91]; thus, different types of bullying behaviors should be measured because the implementation of certain policy components may affect one form of bullying but not another (e.g., physical bullying versus cyber-bullying). It is also recommended that researchers include school-level and district-level contextual variables in their analyses when possible. Educators’ capacity to implement anti-bullying policies depends on barriers, resources, and environmental conditions present in schools, which may constrain or enable implementation [92]. In addition, certain implementation actions may work better in certain schools than others (e.g., elementary schools versus high schools). Finally, longitudinal studies are needed because policy implementation is an ongoing process and may be different immediately after a policy is enacted, 1 year after adoption, or after a policy has been in place for several years. Research on education policy implementation is a challenging endeavor but is extremely important and needed at this time to elucidate complex implementation processes that can lead to reduction of a pervasive threat to student success and well-being–bullying.