Effectiveness of Digital Health Tools to Prevent Bullying among Middle School Students

Abstract

Bullying is a widespread public health problem with significant behavioral and mental health consequences. The current study tested the effectiveness of combining interactive digital material for students, educators, and parents with class sessions to prevent bullying among middle school students. Fourteen middle schools were randomly assigned to intervention and comparison conditions. Both conditions received a classroom-based drug and violence prevention program that taught social skills, self-management skills, and social resistance skills. The intervention condition included class material on bullying and an educational video game for students that reinforced the classroom program; it also included digital material on bullying for parents and school staff. All students completed online pre- and post-test surveys to assess bullying-related behavior, knowledge, and life skills. Results indicated that students in the intervention schools reported significantly less bullying and cyberbullying perpetration and increased life skills knowledge relative to comparison schools. This study provides evidence that a school-based drug abuse and violence prevention program, when enhanced with a set of digital tools for students, parents, and school staff, holds considerable potential for addressing bullying among middle school adolescents.

1. Introduction

Aggression and violence during adolescence have been extensively examined in recent decades. A consistent finding from this body of research indicates that serious forms of aggression plateau from childhood through adulthood, but that less severe, more common forms of aggression may not begin until early or late adolescence [1]. Bullying is a pervasive subtype of adolescent aggression that has captured the attention of researchers, educators, mental health professionals, and policy makers [2]. Bullying involves persistent and intentional threatening and aggressive or verbally abusive behavior directed toward others at relative disadvantage, such as those who are younger, smaller, or weaker [3]. Targets who are perceived as “different” by their peers, including youth with obesity, chronic health problems, identified developmental disabilities, as well as lesbian, gay, bisexual and transgender youth are also at higher risk of bullying victimization [4,5,6]. Victims of bullying often feel unable to defend themselves because of a physical or social power imbalance [7]. Bullying can occur in almost any context although school-related incidents are most common for adolescents [8].

Bullying as a form of social aggression involves multiple roles including bullies, victims, or bully–victims (those who both perpetrate and are victimized), as well as bystanders, defenders, and reinforcers [9,10]. In traditional bullying, either physical or social in nature, offenders and victims are typically face to face or in close proximity. Cyberbullying involves harassing or threatening communication that is sent electronically from offender to victim via a variety of technologies including social media, text messages, mobile phone apps, internet forums, or online gaming. Cyberbullying is related to traditional school bullying, but there are important differences [11,12]. For example, in contrast to face-to-face bullying, where targets can leave a situation and escape to safety, those who are cyberbullied can feel continually victimized and unable to escape due in part to the ubiquitous, persistent, and permanent nature of online communications [13,14].

Bullying in all its forms is highly prevalent. Current estimates indicate that 30% of 6th, 24% of 7th, and 25% of 8th grade students report being bullied in the past 12 months and nearly 50% of these bullying experiences occur in school settings [15]. From 2007 to 2019, the percentage of individuals who have experienced cyberbullying at some point in their lifetimes has more than doubled, from 18% to 37% [16]. These prevalence estimates may not capture the true impact of bullying, as it is estimated that nearly two-thirds of bullying incidents, regardless of form, go unreported [17]. Furthermore, it is expected that cyberbullying among teens will continue to increase as new cohorts of youth obtain access to smartphones and the internet at younger ages [18,19].

1.1. Negative Behavioral and Mental Health Consequences of Bullying and Cyberbullying

Bullying and cyberbullying have significant short- and long-term behavioral and mental health consequences for victims and perpetrators that are pronounced, endure over time, and linked to a variety of problems. Victims are more likely to socially withdraw [20], perform poorly in school and have higher school dropout rates [21,22]. Victims are also more likely to report anxiety, depressive symptoms and suicidal ideation or attempts [23,24]. In addition to psychological problems, victims are also more likely to report physical symptoms including head and stomach aches, and sleeping problems [25,26]. Students who report frequently bullying others (perpetrators) are at increased risk of substance misuse, violence later in adolescence and adulthood, and suicide-related behavior [22]. For youth who both bully others and are bullied themselves, mental health problems are pervasive [27,28,29]. The negative effects of bullying are often persistent and can last into adulthood, contributing to a variety of problems including alcohol and drug dependence [30], violence, and offending later in life [31].

Although bullying and cyberbullying have a number of similar predictors and negative consequences [32] and can occur together [33], research shows that cyberbullies often exhibit risk profiles and characteristics that differ from those involved in traditional bullying [29,34]. For example, cyberbullies may engage in bullying behaviors online that, because of social norms and strictures, they would not necessarily exhibit in face-to-face encounters, reflective of online disinhibition—a lack of restraint one feels when communicating online in comparison to communicating in person. [35]. Cyberbullies also take advantage of the unique characteristics of digital communication technologies (e.g., anonymity and a potentially infinite audience) that are not applicable in face-to-face bullying contexts. Given the similarities and differences between in-person and cyberbullying along with the short- and long-term consequences of this aggressive behavior, it is imperative that effective preventive interventions are evaluated for their impact on this important public health concern.

1.2. Primary Prevention of Bullying and Cyberbullying

In the United States (US), preventing bullying has been recognized as an urgent priority for governmental action. Over the last decade, all 50 states and the District of Columbia have adopted or revised laws to address bullying. A landmark 2016 report by the National Academies of Science (NAS) [2] indicated that bullying is a public health crisis and recommended that rather than “adopting a different program to combat each new problem that emerges,” schools should integrate prevention efforts by using programs and services that address multiple skills to enhance resilience, promote positive behaviors, and prevent multiple risk behaviors with shared determinants. The NAS report [2] also indicated bullying prevention programs that promote punitive, zero-tolerance school policies (e.g., suspension or expulsion) are least effective, while those that promote social and emotional skill building produce more substantial reductions in bullying. Primary prevention research suggests that because bullying shares a common set of etiologic determinants with violence, substance use, and other externalizing behaviors, comprehensive school-based preventive interventions that address multiple problem behaviors within a social and emotional learning or positive youth development framework may be most efficient [36,37].

Several preventive interventions have been identified as promising or effective in reducing bullying behavior. A comprehensive meta-analysis examining 103 independent effect sizes of school-based anti-bullying programs reported reductions by approximately 20% for bullying perpetration and 16% for victimization [8], but that effect sizes varied widely across studies. Programs with the most substantial reductions were multicomponent interventions that targeted in-person bullying and cyberbullying, addressed various bullying roles, included active bystander intervention training, and provided relevant information and resources to parents [38]. Other promising and widely disseminated school-based bullying prevention programs include Positive Action [39], Steps to Respect [40], the Hazelden Cyberbullying program [41], and the Olweus Bullying Prevention Program [42]. Although these programs show effectiveness in preventing bullying among teens, they were not designed to comprehensively address multiple related risk behaviors—substance use, violence, and bullying—via a positive youth development framework that promotes resilience and addresses the shared risk and protective factors that are common across these outcomes.

1.3. The Promise of Educational Video Games

Bullying frequently occurs using online communications. Thus, using digital health tools to deliver bullying prevention content represents an alternative and innovative model for positive technology use [43]. In particular, interactive educational video games can be used to model developmentally appropriate online and in-person communication and engagement. Educational video gaming has been shown to be an effective means by which children and adolescents can learn to handle real-life difficulties and situations [43,44,45]. Video games have been used with success in health education and in preventing youth drug abuse, drunk driving, high-risk sexual behavior, and violence [46,47,48]. Approximately, ninety percent of teenagers in the US play video games [49], which presents an important opportunity for delivering pro-health behavioral interventions.

A growing body of research examining the effectiveness of educational video games indicates that multisensory, media-rich learning experiences make educational content more engaging and thereby increase student comprehension and retention, enhance intrinsic motivation, and facilitate behavior change [50,51]. The ability to commit to a goal and overcome challenges along with the capacity to be playful and have fun are common game elements that also motivate behavior change and promote a sense of wellbeing [52,53]. Motivational game features including immediate and continuous progress feedback (e.g., points scored, accumulation of badges, and level advancement), customizable avatars, and narratives with emotional and value-based rationales have been identified among the most effective video game interventions [54]. Games designed to be interactive and immersive can be suitable for communicating pro-health messages and promoting health behavior change because they not only entertain, but also increase intrinsic motivation to engage with and repeat health and behavioral lessons that might otherwise be less attractive if presented in more traditional, pedantic ways [55].

1.4. Life Skills Training

Several meta-analysis and systematic reviews suggest that cognitive-behavioral interventions that focus on changing internal thinking and affective processes to influence actual behavior are effective in improving adolescent problem behaviors [56,57], but are implemented infrequently [58,59]. Interventions based on the tenets of cognitive-behavioral therapy (CBT) hold significant promise for reducing aggression and bullying among adolescents [60]. A leading example of a drug and violence school-based prevention program based on cognitive-behavioral principles is Life Skills Training (LST). The LST model teaches youth personal self-management skills, social skills, drug refusal skills, resilience, and other life skills needed for healthy psychosocial development and successful navigation of key developmental tasks. LST targets both interpersonal and intrapersonal (such as knowledge, attitudes, and skills) processes through skills training methods designed to enhance social resistance skills, and a broad set of general skills that reduce problem behaviors [61]. Program content of the LST program is taught using interactive teaching methods including small group discussion with breakout sessions and skills practice through structured behavioral rehearsals.

LST has been tested in a series of randomized controlled trials reported in over 35 peer-reviewed publications [61]. These studies showed reductions of 50% or more relative to controls in cigarette smoking, alcohol use, marijuana use and use of illicit drugs among students receiving the LST program, as well as improvements in risk and protective factors associated with adolescent substance abuse. LST also has been shown to reduce violence, aggression, and delinquency among middle school youth [62]. Long-term follow-up data collected from students who participated in the LST program in middle school have shown that program effects lasted well into young adulthood [63]. Follow-up studies have also shown effects on behaviors not directly addressed by LST including risking driving [64], HIV risk behavior among young adults [65], and methamphetamine use [66]. Taken together, these findings indicate that LST can produce both immediate and long-term prevention effects that last into young adulthood, and that these effects can generalize to other risk behaviors not specially addressed in the program. The LST program combined with supplemental bullying prevention content offers considerable potential as a multimedia prevention tool for implementation in school-based settings.

1.5. Goal of Present Study

The present study was designed to test the effectiveness of the LST drug abuse and violence prevention program for middle school youth with and without a set of digital tools to provide students, parents, and school personnel with bullying-specific content and an educational video game for students that focused on the application of life skills to bullying and other high-risk situations. The bullying content added to the LST program was designed to help students identify and recognize bullying and its types (including physical, social, verbal, and cyberbullying); teach bystander intervention skills (e.g., distracting the bully); change normative beliefs and expectancies that support bullying; teach self-efficacy, self-control and coping skills as they relate to bullying; and build social, self-regulation, and healthy relationship skills. The main hypothesis examined whether LST plus the video game, supplemental bullying classroom materials, and anti-bullying informational e-learning modules for parents and school personnel would produce significant reductions in bullying relative to LST alone. It was also hypothesized that students who received the LST intervention with bullying content and the interactive video game would report improvements in health knowledge concerning the adverse effects of bullying, skills knowledge, and life skills, relative to students in the comparison group who received LST alone.

2. Materials and Methods

Using national lists of middle school principals, teachers, and district-level administrators, schools were randomly selected from different geographic areas across the U.S. and emailed recruitment fact sheets with a description of this study. The final roster of schools enrolled in this study were middle schools including students (ages 11–14) in grades 6–8 (42.9%), along with schools serving students from grades 7–8 (14.2%), grades 7–12th grade (7.1%), kindergarten to grade 8 (21.4%) or kindergarten to grade 12 (14.2%).

2.1. Sample

A total of 699 students from 14 middle schools voluntarily participated in the current study and completed the pre-test and post-test surveys. This sample was 44% male, 52.7% female, 1.1% other, and 2.2% preferred not to answer. The racial makeup of the sample was White (79.1%), Black (11.1%), American Indian or Alaskan Native (3.5%), Asian (4.3%), Native Hawaiian and Other Pacific Islander (1.3%), and 14.0% mixed or other races. Approximately 20.0% of participants reported that they were Latino/Hispanic. The mean age of the participants was 11.64 (SD = 0.63). Most participants were 11 (40.6%), 12 (42.3%), or 13 (11.7%) years old and in the 6th (60.9%) or 7th grade (32.6%).

2.2. Research Design

The effectiveness of the intervention was assessed through a cluster-randomized comparison group design in which schools were matched by geographical region and enrollment size prior to randomization. Schools were then randomly assigned to either receive the intervention (8 schools) or serve in the comparison group (6 schools). Students (n = 472) attending intervention schools received the standard LST prevention program with added bullying prevention content, an interactive video game, and e-learning modules containing anti-bullying content for parents/caregivers and school personnel. Students (n = 227) randomized to the comparison condition received the standard LST prevention program without bullying content or exposure to the video game.

2.3. Procedure

Study participants completed an online pre-test survey prior to the intervention and a post-test survey approximately 4 weeks after completion of the intervention. Data collection for students in the comparison group occurred at approximately the same time. The surveys assessed self-reported demographic information, bullying behavior, knowledge of adverse health and social effects of bullying, skills knowledge, and life skills. Unique identification codes were used to link pre-test and post-test surveys. To preserve confidentiality, student names were not included on any data collection materials. Students completed pre- and post-test surveys using computers, laptops, or other devices during a regular classroom period or computer lab session. All study procedures were similar to those used in previous prevention studies [67,68] and were approved by an Institutional Review Board. A referral protocol that provided a free and confidential 24 h hotline was available for any students who might have experienced emotional distress. No participating students reported distress and the protocol was not activated.

2.4. Intervention

Classroom Sessions. During class sessions (see

Table 1. Program Sequence for the LST Classroom Program with Bullying Content.

Lesson	Goals and Objectives	Key Skills	Sessions
Self-Image and Self-Improvement	Teach what self-image and self-improvement are and how they impact one’s behavior.	Self-analysis, self-improvement, goal-setting, reframing thoughts.	1
Making Decisions	Teach to use effective decision-making skills and solve problems independently.	3 Cs of effective decision-making (clarify, consider, choose); resisting group pressure.	2
Smoking: Myths and Realities	Debunk common myths and misconceptions about cigarettes and other forms of tobacco use.	Analyzing data; checking assumptions; considering pros/ cons.	1
Smoking and Biofeedback	Teach immediate physiological effects of smoking.	Measuring heart rate; understanding scientific method.	1
Alcohol: Myths and Realities	Debunk common myths and misconceptions about alcohol use.	Analyzing data; checking assumptions; considering pros/ cons; separating fact and fiction.	1
Marijuana: Myths and Realities	Debunk common myths and misconceptions about marijuana use.	Analyzing data; checking assumptions; considering pros/ cons; separating fact from fiction.	1
Advertising	Increase awareness of and teach resistance to techniques used by advertisers to manipulate consumer behavior.	Analyzing ads; recognizing persuasive techniques; separating fact from fiction and want from needs.	1
Violence and the Media	Increase awareness of media influences on violence and how to check media presentations against reality.	Analyzing perceptions about violence; comparing images in media and reality; resistance to media distortions.	1
Coping with Anxiety	Define anxiety and common situations which cause it.	Recognizing anxiety and its physical effects; healthy techniques to manage anxiety; progressive relaxation skills, mental rehearsal/ visualization, and deep breathing.	2
Coping with Anger	Teach common situations that trigger anger and how to cope with them.	Recognizing anger, its physical effects, and how to control anger.	1
Communication Skills	Teach how to communicate effectively.	Using verbal and non-verbal communication; techniques for avoiding misunderstandings; clarifying; asking questions; being specific; and paraphrasing.	1
Social Skills	Teach basic social skills to develop successful interpersonal relationships.	Making social contacts; giving and receiving compliments; effective listening; being persistent; having self-awareness; considerate of others	2
Assertiveness	Teach assertiveness and resist peer resistance skills for drug use.	Reflecting on types of responses, consequences; awareness of persuasive tactics; repertoire of refusal responses; verbal and non-verbal assertiveness.	2
Resolving Conflicts	Teach life skills for resolving conflicts.	Analyzing conflict resolution choices; controlling anger; building consensus; problem solving; negotiation and compromise.	1
Bullying and Cyberbullying	Identify types of bullying, roles in bullying situations, bystander intervention skills, and techniques for coping.	3 Rs of Bullying (Recognize, Respond, Report), 3 Ds Technique for Active Bystander Intervention (Direct, Distract, Delegate), RISE-UP Against Bullying, techniques for reporting bullying.	2
Total Class Periods			20

), which were approximately 45 min in length, students participated in didactic lectures, small-group discussion with breakout sessions, and behavioral rehearsals (e.g., role playing). The LST program is conceptually organized into four major components. The first component, personal competence, includes class sessions that focus on self-image and self-improvement, assertiveness, coping with anxiety, and coping with anger. The second component, social competence, includes classroom sessions on social skills, assertiveness, communication skills, and resolving personal conflicts. The third component, drug resistance, includes smoking myths and realities, smoking and biofeedback, marijuana myths and realities, alcohol myths and realities, advertising, and violence and the media.

Bullying resistance skills comprise the fourth component, which includes newly developed bullying prevention classroom sessions that taught anti-bullying information, attitudes and norms, challenges to expectancies for the benefits of bullying, and skills designed to deter bullying and cyberbullying. During several class activities and skills training, students analyzed real-world scenarios and practiced bystander intervention skills in behavioral rehearsals that involved responding to bullying and other high-risk situations. In the first of two class sessions, students participated in interactive didactic activities to discuss different types of bullying (physical, social, verbal, and cyberbullying) and their physical, emotional, and social effects as well the four roles common in bullying incidents (i.e., bully, target, passive bystander, and active bystander). Students also discussed and dispelled common bullying myths to reinforce what bullying is and is not. In a second class session, students practiced ways to respond and report bullying to school personnel and other trusted adults. Students were also taught to respond to a bullying incident using various active bystander techniques (e.g., direct response, distract the bully, and delegate someone to seek help from an adult). In the same sessions, students were taught coping mechanisms for victims or bystanders.

Teacher’s Manual and Student Guide. Classroom materials included a comprehensive, step-by-step teacher’s manual with session goals, objectives, content, interactive activities, and instructions for implementing the program. The teacher manual was supplemented with a set of presentation slides containing information and skills to guide students during class sessions and the video game modules. Each student received a student guide that was structured to mirror the format and layout of the LST middle school program materials in order to standardize the intervention, enhance fidelity, and facilitate integration with other intervention materials.

Interactive Educational Video Game. Galaxia is an interactive educational video game designed to prevent bullying and cyberbullying among middle school youth. The game included a variety of animated characters and engaging scenarios that addressed individual and group processes related to in-person and online bullying victimization, perpetuation, and witnessing. The learner observed various scenarios unfold, watched characters interact with one another, and interacted with the game by selecting responses that the character could say in a particular exchange. Based on the educational content, one statement was considered the correct answer. In addition to skills training, Galaxia focused on the antecedents, responses, and consequences of bullying and cyberbullying.

Content for Galaxia was adapted from the LST program in a careful iterative process to ensure integrity to the model. Galaxia was named so to reflect outer space, where the game scenarios took place, and to appeal to a wide adolescent audience with characters having nonhuman physical characteristics and extraordinary facial features (see Figure 1). Galaxia was designed to be graphically appealing, with character animation to immerse students in real-life situations that commonly occur in middle school settings. Media-rich features including the capacity for immediate feedback, transmission of multiple cues (sound effects paired with correct responses, color variety to focus on perpetrators and victims, etc.), language variety (humorous and nonsensical words) and individualization (user selects from a library of images to build an avatar) were key elements to enhance player engagement. At the start of each game module, players were given a brief story setup about a situation involving their own avatar and other Galaxia characters and teachers. Storylines involved everyday challenges faced by young adolescent students (e.g., academic and extracurricular performance, peer acceptance, and bullying). Players navigated through each branched scenario until they reached a decision point. They were then presented with three possible dialogue options. One dialogue option correctly utilized a skill covered in the LST program, while the other two options were either wrong or not the most effective response. Only when players selected the correct life skill were they awarded points. For reinforcement, the correct skill name was displayed on the screen (e.g., “Be Direct as an Active Bystander” or “Tell a Trusted Adult”). Players earned a cumulative score, which was the sum of all points earned from each scenario. Students were granted access to each successive module based on mastery of the previous module. The game included a dashboard that informed students of their progress for each individual module and for modules combined. Both teachers and the research staff were able to view student exposure and mastery of game scenarios and this information provided critical dosage and fidelity information.

Galaxia included four modules (each approximately 30 min of play time) that reinforced cognitive-behavioral skills taught during classroom sessions. In Module 1, the player started the game to find student characters stressed about a number of situations including which classes to register for, not disappointing parents, and figuring out who damaged school property with graffiti. Mastery of self-improvement, decision-making, and resolving conflict skills was required to advance to the next module. The player began Module 2 to find characters focused on final exams and a school-wide talent show. In the scenario, some characters took their end-of-term group projects responsibilities seriously, while others procrastinated, creating a potentially anxiety-provoking situation. Effective use of communication skills, assertiveness, coping with anxiety, and social skills were required to advance to the next module. As outlined in

Table 2. Interactive Video Game Branched Scenarios for Module 3: Bullying and Cyberbullying.

Scenario	Objective	Trigger	Correct Avatar Response	Points Earned
Gymnasium
A character is verbally aggressive toward other characters and player (through avatar) after receiving failing grade	Managing Anger	Teacher did not allow students to use most modern calculator	Reframe Situation to Stay Calm and Control Anger	75
	Stop Bullying	Character is verbally aggressive	Be Direct as an Active Bystander	150
	Managing Anger	Character is verbally aggressive	Remind Yourself to Stay Calm	225
	Stop Bullying	A character insults everyone because they are leaving school for Winter break, but she has nowhere to go.	Express Yourself Using ‘I’ Statements	300
Dorms
A character has received a number of negative text messages	Stop Bullying	Anonymous person threatens to send to the entire school unflattering pictures of a main character	Consider Consequences before Acting	375
Accusations and gossip about who is sending threatening text messages	Stop Bullying	One character is verbally abusive	Be Direct as an Active Bystander	450
	Stop Bullying	Without adult supervision, characters plan to investigate who’s sending anonymous texts	Tell a Trusted Adult or Ally	525
Counselor’s Office
Role playing with counselor on how to handle bullying situations	Stop Cyberbullying	Teacher offers to Help	Make a Specific Request	600
	Stop Cyberbullying	Bully ridicules other for asking for help from an adult	Tell a Trusted Adult or Ally	675
	Stop Bullying	Threat of physical aggression	Leave the Situation	750
Dorms
One character breaks controller on a game console	Managing Anger	Character broke game controller and demands that the owner hands over spare controller	Take a Deep Breath and Silently Count to 10	825
Character receives another bad grade	Self-Image and Self-Improvement	Character realizes she neglected to study which caused her poor academic performance	Don’t Focus on Negative Experiences	900
Character considers not going home for Winter break to avoid criticism from parents	Make a Decision	Character faces the prospect of not going home for Winter break	Consider Alternatives and Consequences	975
Character thinks she knows the identity of the cyberbully	Managing Anger	Characters considering retaliating against person believed to be sending anonymous text messages	Stop and Calmly Count to 10	1050

, in Module 3, the player navigated several situations involving different forms of bullying. To accomplish the objectives of this module, the player needed to demonstrate mastery of anger management skills, skills for stopping bullying, as well as self-improvement and decision-making skills in the context of bullying incidents. In Module 4, the player faced several scenarios including being pressured by peers to share prescription medication to gain advantage on a test, carry out a school prank, sneak into an adult-rated movie, and try alcohol and drugs at a party. To receive the maximum number of points, the player was required to demonstrate mastery of resistance skills to avoid peer pressure to engage in prohibited behaviors and use prescription and illicit drugs.

E-Learning Modules for Parents/Caregivers and School Personnel. To ensure that students were exposed to consistent anti-bullying messages at school and at home, e-learning modules containing information about bullying were developed and given to parents and school staff. Both parent and educator modules provided content on the frequency and pervasiveness of bullying, types of bullying and cyberbullying, bully roles, consequences of bullying as well as healthy strategies for coping with bullying. Modules emphasized the importance of expressing compassion toward victims, tips for communicating with teens, strategies to handle bullying, and information on how to identify signs of a bully and a target. Modules geared toward professionals (teachers, support staff, bus drivers, coaches, tutors) provided content on how to detect bullying-related problems and how to intervene in bullying situations and emphasizing reparative disciplinary practices. Each module included knowledge checks and took approximately 30 min to complete.

2.5. Program Sequence

The curriculum intervention materials were implemented by regular school teachers. The provider (e.g., teachers and counselors) to student ratio was approximately 1:25. Class sessions took place in a defined sequential order starting with the Self-Image and Self-Improvement lesson and ending with two 45-min sessions on bullying. As shown in Table 2, there were 15 Life Skills Training topics implemented over 20 class sessions: (1) Self-Image and Self-Improvement; (2) Making Decisions; (3) Smoking: Myths and Realities; (4) Smoking and Biofeedback; (5) Alcohol: Myths and Realities; (6) Marijuana: Myths and Realities; (7) Advertising; (8) Violence and the Media; (9) Coping with Anxiety; (10) Coping with Anger; (11) Communication Skills; (12) Social Skills; (13) Assertiveness; (14) Resolving Conflicts; and, (15) Bullying and Cyberbullying. After the classroom sessions were completed, providers assigned students to play the video game. Providers were given the flexibility to assign the game modules in or out of class.

2.6. Online Provider Training Workshop

Prior to intervention implementation, program providers completed an asynchronous six-hour, self-paced online training workshop which offered instruction about principles and practices of the LST program, and strategies for facilitating discussions, conducting skills practice, and teaching bullying sessions. As shown in

Table 3. Online Program Provider Training Workshop.

Module	Training Activities for Providers
Module 1: Introductions and Expectations	● Get acquainted through trainer-moderated discussion board to promote a cohesive and supportive learning community. ● Share workshop expectations and identify potential barriers to implementation for immediate feedback from the lead trainer.
Module 2: Orientation	● View archived presentation that describes study’s purpose. ● Ask lead trainer questions and concerns about provider roles and responsibilities.
Module 3: Principles and Practices of the LifeSkills Training Program	● View narrated video presentation about the theoretical framework and research supporting the LST program. ● Demonstrate understanding of how research principles and practices are incorporated into the LST instructional materials and are reflected in key teaching strategies.
Module 4: Teaching Skills for Facilitating Discussion	● Practice teaching skills, including facilitation, coaching, and behavioral rehearsal, to promote social emotional, cognitive behavioral learning, and advance the acquisition of pro-health ideas, skills, and attitudes. ● View video presentations of the teaching skills demonstrated by a master trainer.
Module 5: Becoming Familiar with LST Instructional Material and Resources	● Complete a virtual walk-through of a sample lesson noting key design elements to support fidelity delivery of the LST lessons and facilitate preparation to implement the program. ● Take a virtual tour of the teaching resources on the LST website.
Module 6: Exploring the LST Middle School Program	● Providers view video presentations of each unit in the LST Middle School curriculum. ● Providers analyze two units of their choice and identify key objectives, skill sets, points to make and summaries by completing and uploading a unit review worksheet for review and feedback by lead trainer. ● Discuss with other providers knowledge gained from each unit. ● Address specific challenges they foresee in delivering the lessons.
Module 7: Teaching Bullying and Cyberbullying Sessions	● Providers review key learning objectives and activities in bullying and cyberbullying units by completing check points and engaging in peer-to-peer discussion posts. ● Reviewed video game dashboard to monitor student progress.
Module 8: Prevention Education Resources	● Providers reviewed a virtual resource center with links to research articles and websites to enhance educator understanding of evidence-based practices that address risk and protective factors impacting adolescent health and wellness. ● Reviewed resources and how they could be used to support high-quality program implementation.
Module 9: Bullying and Cyberbullying Prevention Feedback Forms	● Providers reviewed tools and resources essential for delivering live sessions and documenting their activities in the research study. ● Examine fidelity, community events reports, and attendance forms to be submitted over the course of the program implementation.
Module 10: Course Feedback	● Providers gave feedback on the quality of the teacher training using an online survey instrument. ● Provider feedback used for continuous improvement of the online training.

, providers learned to effectively implement the intervention, including how to structure classroom sessions and the supplemental video game. Providers reviewed the scope and sequence of the program including the goals, objectives, and learning activities of all intervention materials. Providers explored the conceptual model of LST and its underlying theory, methods, and effectiveness and practiced CBT skills training methods needed to successfully implement the classroom sessions of the program (e.g., facilitating classroom discussions, demonstrating new skills, coaching students through small-group behavioral rehearsal or skills practice, and providing positive feedback and reinforcement). Next, providers were given instruction on practical implementation issues including classroom management strategies, how to respond to disclosure of sensitive information by students, and how to establish ground rules for the classroom sessions (e.g., all students should be given the opportunity to participate and everyone’s contributions must be respected without criticism). Finally, providers were given instruction on how to integrate components of the in-class intervention with the video game modules. The workshop also clarified roles and responsibilities for participating in the randomized trial.

2.7. Measures

Online self-report surveys were administered to collect data prior to the intervention and approximately 4 weeks after completion of the intervention. Data collection for students in the comparison condition occurred on a similar schedule to the intervention condition to ensure that the interval between the pre-test and post-test was similar for both conditions. The surveys assessed several study variables including demographics; behavior and knowledge regarding physical, social, verbal, and cyberbullying victimization and perpetration; hypothesized risk and protective factors; and, the skills, knowledge, attitudes targeted by the intervention.

2.7.1. Demographic Information

Data concerning the demographic characteristics of the participants were collected using standard survey items assessing gender, age, and race/ethnicity.

2.7.2. Bullying

The student survey included 22 items selected from the Bully Survey [69], an established age-appropriate tool which assesses experiences with bullying victimization and perpetration. All bullying items had a 9-point frequency response scale anchored by 1 (never) and 9 (more than once a day). The McDonald’s ω coefficient for the 11 bullying victimization items was 0.92 and for the 11 bullying perpetration items was 0.92.

Bullying Perpetration was assessed in four domains: Physical, Verbal, Social, and Cyberbullying. Physical Bullying Perpetration was measured by three items: “About how often (if ever)”: (1) Have you pushed or shoved another student to make them feel bad?”; (2) “Have you beat up another student to make them feel bad?”; and (3) “Have you broken another student’s belongings on purpose to hurt them?” Verbal Bullying Perpetration was measured by two items: “About how often (if ever)”: (1) “Have you made fun of other students?”; and (2) “Have you said mean things behind their back?” Social Bullying Perpetration was measured by two items: “About how often (if ever)”: (1) “Have you excluded or ignored another student on purpose?”; and (2) “Have you spread rumors about another student to try to hurt their reputation?” Cyberbullying Perpetration was measured by four items: “About how often (if ever)”: (1) “Have you written or commented mean things about another student online?”; (2) “Have you sent unwanted messages to another student in order to hurt them?”; (3) “Have you threatened to post someone’s personal information, photos, or videos online in order to hurt them?”; and (4) “Have you posted someone’s personal information, photos, or videos online in order to hurt them?”

Bullying Victimization was assessed in four domains: Physical, Verbal, Social, and Cyberbullying. Physical Bullying Victimization was measured by three items: “About how often (if ever)”: (1) “Have you been pushed or shoved by another student on purpose?”; (2) “Have you been beat up by another student?”; and (3) “Have you had your belongings broken by another student on purpose?” Verbal Bullying Victimization was measured by two items: “About how often (if ever)”: (1) “Have other students made fun of you?”; and (2) “Have other students said mean things behind your back?” Social Bullying Victimization was measured by two items: “About how often (if ever)”: (1) “Have other students excluded you on purpose?”; and (2) “Have students spread rumors about you to try to ruin your reputation?” Cyberbullying Victimization was measured by four items: “About how often (if ever)”: (1) “Have other students written or commented mean things about you online?”; (2) “Have other students sent you unwanted messages online?; (3) “Have other students threatened to post your personal information, photos, or videos online in order to hurt you?”; and (4) “Have other students posted your personal information, photos, or videos online in order to hurt you?”

2.7.3. Skills Knowledge

Thirty-six “True/False” items used in previous prevention research assessed Skills Knowledge [70]. Knowledge items measured key intervention skills including Bullying Prevention, Self-Image, Decision Making, Advertising/Media, Violence and the Media, Coping with Anxiety, Coping with Anger, Communication Skills, Social Skills, Assertiveness, and Conflict Resolution. There were six items that measured Bullying Prevention Knowledge (e.g., “If I saw someone being bullied, I would directly confront the person doing the bullying.”); three items that assessed Self-Image Knowledge (e.g., “Focusing on a bad experience we’ve had can keep us from trying new things.”); three items that assessed Decision-Making Knowledge (e.g., “It’s a good idea to make a decision first and then think about the consequences later.”); three items that assessed Advertising/Media Knowledge (e.g., “Advertisements have a stated message but also an implied or hidden message that is less obvious.”); three items that assessed Violence and the Media Knowledge (e.g., “There is not as much violence in real life as there is on TV.”); three items that assessed Coping with Anxiety Knowledge (e.g., “Mental rehearsal is a good relaxation technique.”); three items that assessed Coping with Anger Knowledge (e.g., “If you are very angry, it is best to confront a conflict right away.”); three items that assessed Communication Skills Knowledge (e.g., “I ask questions if I don’t understand someone.”); three items that assessed Social Skills Knowledge (e.g., “Practicing saying hello can make it easier to start new conversations.”); three items that assessed Assertiveness Knowledge (e.g., “Being assertive means calmly and firmly standing up for your rights without infringing on the rights of others.”); and three items that assessed Conflict Resolution Knowledge (e.g., “Conflicts are a natural part of relationships.”). An Overall Skills Knowledge score was determined based on the total number of the 36 items answered correctly.

2.8. Data Analysis

A series of GLM and multilevel analyses using MIXED models were conducted to examine the impact of the intervention on the post-test outcomes, controlling for the pretest score of the outcome, race/ethnicity, and gender. Including the baseline outcome score as a covariate has been found to be an efficient data analytic strategy for testing intervention effects using a two-arm randomized pre–post-test design [71]. Robust estimators were specified for the GLM analyses, which relax strict assumptions about the distribution of the dependent variable. Multilevel analyses with MIXED modeling were also conducted on the same set of outcomes to adjust for the potential impact of school-level clustering effects. One-tailed significance tests were used to determine significance levels for the analysis of intervention effects, as warranted by the unidirectional nature of hypothesized effects and the results of previous research testing this prevention approach [61]. A significance threshold of alpha = 0.05 was set a priori.

3. Results

Pre-Test Equivalence. Intervention and comparison group participants in the analysis sample were compared on relevant baseline demographic and behavioral variables (see

Table 4. Baseline Demographics by Condition.

	Intervention	Control	All
Sample Size	472	227	699
Age in Years: M (SD)	11.74 (0.61)	11.44 (0.61)	11.64 (0.66)
Race/Ethnicity:
Racial Minority (Non-White)	18.2%	26.4% *	20.9%
Hispanic/Latino	16.9%	27.0% *	20.0%
Sex (Male)	43.9%	44.1%	44.0%

Note: * p < 0.05.

). There were differences between the intervention and comparison groups at pre-test regarding race and ethnicity. A higher proportion of the comparison group reported they were from a racial minority group vs. White (χ2 (1) = 6.26, p < 0. 05). A higher proportion of the comparison group reported that they were Hispanic/Latino vs. not Hispanic/Latino (χ2 (1) = 7.22, p < 0. 03). There were no pre-test differences with regard to gender. Overall bullying involvement was similar across conditions at the pre-test assessment.

Bullying Outcomes. A series of GLM analyses was conducted to examine intervention effects on the frequency of bullying perpetration and victimization. Adjusted means for bullying perpetration at the post-test assessment are presented in

Table 5. Adjusted Means at Post-Test for Bullying Perpetration by Condition, GLM and MIXED Models.

	GLM		MIXED
	Intervention	Control	Intervention	Control
	Mean (SE)	Mean (SE)	Mean (SE)	Mean (SE)
Bullying
Physical	1.20 (0.04)	1.32 (0.08)	1.29 (0.10)	1.41 (0.11)
Verbal	1.73 (0.06)	1.69 (0.10)	1.74 (0.18)	1.71 (0.20)
Social	1.43 (0.05)	1.64 (0.10) *	1.43 (0.14)	1.73 (0.16)
Cyber	1.17 (0.04)	1.41 (0.11) *	1.17 (0.12)	1.50 (0.14) *

Note: * p < 0.05.

for each condition, after controlling for the pre-test values of each outcome, race/ethnicity, and gender. There were significant intervention effects on social bullying perpetration, with the mean lower in the intervention group relative to the comparison group (Wald χ2 (1) = 3.87, p < 0.049). There were also significant intervention effects on cyberbullying perpetration, with the mean lower in the intervention group relative to the comparison group (Wald χ2 (1) = 4.20, p < 0.041). There were no significant differences across groups for bullying victimization. A series of analyses was also conducted using MIXED modeling to account for the fact that students were clustered within schools. Schools were modeled as a random effect, and intervention group, race/ethnicity, gender, and the pre-test score of the outcome were modeled as fixed effects. Although not significant, the results of the MIXED model analyses were generally consistent with the results of the GLM analyses.

Skills Knowledge. Using the GLM approach, comparison of the intervention and comparison groups for knowledge regarding life skills taught in the intervention showed several significant intervention effects (see

Table 6. Adjusted Means at Post-Test for Skills Knowledge Variables by Condition, GLM and MIXED Models.

	GLM		MIXED
	Intervention	Control	Intervention	Control
	Mean (SE)	Mean (SE)	Mean (SE)	Mean (SE)
Bullying Prevention	4.38 (0.05)	4.04 (0.08) ***	4.27 (0.17)	4.16 (0.19)
Self-Image	2.22 (0.04)	2.07 (0.06) *	2.15 (0.08)	2.13 (0.09)
Decision Making	1.75 (0.03)	1.64 (0.04) *	1.72 (0.06)	1.68 (0.07)
Advertising/Media	2.34 (0.14)	2.26 (0.14)	1.75 (0.03)	1.85 (0.06)
Violence & the Media	1.75 (0.03)	1.84 (0.06)	1.67 (0.08)	1.84 (0.09)
Coping with Anxiety	2.40 (0.04)	2.22 (0.06) *	2.39 (0.12)	2.24 (0.14)
Coping with Anger	2.18 (0.04)	2.05 (0.05) *	2.15 (0.10)	2.13 (0.11)
Communication Skills	2.35 (0.04)	2.32 (0.05)	2.33 (0.11)	2.21 (0.12)
Social Skills	2.39 (0.04)	2.35 (0.06)	2.37 (0.07)	2.34 (0.08)
Assertiveness	2.56 (0.04)	2.40 (0.06) **	2.53 (0.11)	2.38 (0.13)
Conflict Resolution	2.36 (0.04)	2.28 (0.05)	2.32 (0.11)	2.29 (0.13)
Overall Skills Knowledge	25.94 (0.22)	24.48 (0.36) ***	25.51 (1.00)	24.95 (1.15)

Note: * p < 0.05, ** p < 0.01, *** p < 0.001.

). Overall Skills Knowledge was higher for the intervention group (M = 25.94) than the comparison group (M = 24.48; Wald χ2 (1) = 12.05, p < 0.001). Skills knowledge was further analyzed by type. Higher post-test knowledge means for the intervention relative to the comparison group were observed for Bullying Prevention (Wald χ2 (1) = 342.41, p < 0.001), Self-Image (Wald χ2 (1) = 4.72, p < 0.015), Decision Making (Wald χ2 (1) = 3.60, p < 0.031), Coping with Anxiety (Wald χ2 (1) = 6.38, p < 0.006), Coping with Anger (Wald χ2 (1) = 3.93, p < 0.023), and Assertiveness (Wald χ2 (1) = 5.63, p < 0.009). Although not significant, the MIXED model results were generally consistent with GLM results.

4. Discussion

A fruitful way to extend the benefits of evidence-based prevention programs is to adapt the most effective interventions and test them in new settings, with new populations, or with new behaviors. Informed by theory and research on the characteristics of effective programs for similar behaviors, the most effective intervention approaches can be tailored to address shared and unique risk and protective factors, while also maintaining their key components and active programmatic ingredients [2]. Furthermore, prevention researchers have been investigating ways to effectively incorporate digital, technology-based tools to enhance traditional, classroom-based teacher-led prevention models and create engaging multicomponent interventions. Over the past decade, e-learning and video games have been explored as multisensory, media-rich learning technologies that can enhance traditional educational approaches [43,44,45,46,47,48]. Given the increased role of multimedia technology in the learning and social environments of adolescents, along with the increasing availability of digital devices and internet access in and out of class [49], rigorous studies are needed to identify best practices for integrating digital health products into traditional classroom preventive interventions. Indeed, the potential of educational video games in multicomponent prevention programs should be examined given recent evidence that video games can improve pro-social skills and reduce antisocial behavior among school children [72].

The present study was designed to extend an evidence-based drug and violence prevention model to bullying through an intervention approach that incorporated e-learning modules for parents and educators and an interactive video game for students. The evidence-based Life Skills Training (LST) classroom program teaches youth personal self-management skills, social skills, drug refusal skills, and other life skills needed to successfully navigate developmental tasks, increase resilience, and facilitate healthy psychosocial development during adolescence [61]. Because LST addresses a wide variety of risk and protective factors via a positive youth development model, and bullying shares a similar set of risk and protective factors with substance use, it is well suited to be expanded to address the important public health problem of bullying, including cyberbullying. In the present study, bullying-specific content was added to the LST classroom program, digital e-learning modules on bullying prevention were provided to parents and school personnel, and an interactive educational video game was developed to provide opportunities for students to apply life skills to the prevention of bullying and other high-risk situations in a fun and engaging way. Results from the present study indicated that students who received the expanded intervention reported significantly less social bullying and cyberbullying perpetration and increased life skills (pro-health) knowledge relative to students attending the comparison schools. Thus, these results show that a multicomponent intervention consisting of a classroom-based curriculum, e-learning material for parents/caregivers and school personnel combined with an educational video game holds substantial potential for addressing bullying perpetration among middle school adolescents.

4.1. Implications for School-Based Bullying Prevention

A growing body of school-based research indicates that integrating prevention content with educational video games can be an effective approach for exposing youth to common, real-life difficulties and situations that allow prevention educators to teach important evidence-based content in a way that students find appealing and engaging [48,50,51]. A substantial portion of the adolescent US population plays video games, and as such games are important channels for delivering evidenced-based behavioral health interventions [49]. Features of video game technology (e.g., self-pacing and repeated use) support individualized learning while also offering teachers the potential for delivering program content in a standardized and time-efficient manner [53,54]. Gameplay combined with traditional class sessions can support important prevention goals.

The present study is the first examining the impact of a video game when used with the classroom-based LST program. The video game was designed to positively change social norms surrounding bullying and cyberbullying, challenge positive expectancies about bullying and cyberbullying, and enhance protective factors by building social, self-regulation, and relationship skills through interactive learning and behavioral rehearsal. Students perceived the Galaxia video game as immersive and evocative of interpersonal dynamics that commonly occur with peers in relationships and common school-like settings. Students found the visual aspects of the game, including the otherworldly characters in creative and colorful settings, to be engaging and fun. Teachers also found the video game appealing and remarked that its utility, interactivity, as well as the salience and effectiveness of the branched scenarios were important pedagogical devices. These findings represent an important contribution to the prevention literature, and add to a growing body of research showing that educational video games are effective not only in supporting regular educational objectives, but also effective in improving skills, promoting well-being, and reducing health risk behavior among elementary and secondary school youth [47,72,73,74].

Taken together, rigorously evaluated preventive interventions that combine evidence-based theoretically grounded content with modern sophisticated technology tools offer considerable potential for addressing the most problematic behavioral health concerns for young adolescents. The approach tested in the present study produced meaningful reductions in bullying behavior and important improvements in life skills. This approach capitalizes on the educational features of game technology and the advantages of teacher-led, in-person class sessions—both of which promote knowledge acquisition to provide opportunities for the application and practice of the knowledge and skills taught through the educational game. These findings underscore the value of multimedia preventive interventions and demonstrate that developmentally appropriate materials can be an effective means to deliver prevention programming to promote positive youth development and prevent bullying and cyberbullying as well drug abuse and violence. These findings align with previous research demonstrating that educational video games can effectively promote health-related behavioral change [46,53], reduce violence and aggression [47], and increase acquisition of pro-health related knowledge and skills [48,52]. The present study also provides additional empirical support for the positive youth development model. This study offers compelling new evidence that adding a video game component to an evidence-based classroom drug abuse and violence prevention program can effectively reduce various forms of bullying.

4.2. Strengths and Limitations

Strengths of the present study include a rigorous evaluation design; confidential self-report surveys using measures with well-established psychometric properties; standardized and well-tested protocols for recruiting schools, collecting data, and tracking participants over time; analysis of pre–post-test change at the individual level; an intervention approach building on a prevention model that has been extensively tested in multiple randomized trials; the application of theory and methods derived from the state-of-the science in the field of prevention; and inclusion of a diverse range (e.g., location, school size, and socioeconomic status of the community) of secondary schools. These findings have important public health relevance as they indicate that school-based programs that prevent adolescent violence and conduct problems, which if not addressed in early adolescence can lead to more intensive and expensive interventions (e.g., costly correctional and mental health services) in young adulthood [75]. Limitations include the possible underreporting of sensitive behaviors, the lack of follow-up after the initial post-test, limited statistical power due to the small number of schools, and a study design that did not permit the testing of individual components of the multicomponent intervention. Additionally, follow-up data are needed to determine the long-term durability of these prevention effects. Finally, because this was a school-based study that relied on students’ self-reports, the significant relationships among variables may partly reflect shared method variance (all data were obtained by self-report questionnaire).

4.3. Future Directions

Future research is needed to determine the long-term durability of this prevention approach and if the intervention effects are moderated by bullying severity or bullying role (bullies, victims, bully–victims, bystanders, defenders, and reinforcers). Longitudinal research is needed to further establish causal relationships between the intervention and reductions in various types of bullying (e.g., physical, social and cyberbullying) and other problem behaviors including general aggression and school violence in late adolescence. Research is also needed to investigate the extent to which the combination of bullying-specific content for students, parents/caregivers and school staff as well as the educational video game can be modified to further decrease bullying behavior. Another possible direction for future research would be to determine whether the prevention approach tested in this study would be strengthened by the addition of virtual reality or augmented reality components. Future studies should examine the impact of this intervention with other developmental levels including elementary and high school students.

5. Conclusions

The current study provides supporting evidence for the effectiveness of an enhanced classroom version of the Life Skills Training program when combined with an interactive educational video game. Findings from this study showed significant reductions in bullying and increases in health knowledge, skills knowledge, and life skills among students who received an enhanced version of LST combined with a video game compared to students who did not. This study extends previous research with LST by demonstrating prevention effects on bullying and aggressive behavior among young adolescents. The intervention materials tested were found to be appealing to students and teachers. These results are encouraging and suggest that an appealing gamified multimedia intervention can be effective in engaging young adolescents and support positive youth development. The findings of this study are timely and have important public health relevance. Moreover, because the intervention is innovative and engaging, it offers considerable promise for dissemination, adoption, and wide-scale implementation.