Next Article in Journal
Associations between Mental Health and COVID-19 Status among 18- and 19-Year-Old Adolescents: A Multi-Country Study
Previous Article in Journal
Migration as Cultural Phenomenon in a Globalized World: A Pilot Study on Lifestyle and Eating Behaviours of Adolescents Living in Rome
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Effectiveness of Digital Health Tools to Prevent Bullying among Middle School Students

by
Christopher Williams
1,2,*,
Kenneth W. Griffin
3,
Caroline M. Botvin
1,4,
Sandra Sousa
1 and
Gilbert J. Botvin
1,5
1
National Health Promotion Associates, White Plains, NY 10604, USA
2
Department of Psychology, Purchase College, State University of New York, Purchase, NY 10577, USA
3
Department of Global and Community Health, George Mason University, Fairfax, VA 22030, USA
4
Teachers College, Columbia University, New York, NY 10027, USA
5
Department of Population Health Sciences, Weill Cornell Medical College, New York, NY 10021, USA
*
Author to whom correspondence should be addressed.
Adolescents 2023, 3(1), 110-130; https://doi.org/10.3390/adolescents3010009
Submission received: 15 December 2022 / Revised: 17 January 2023 / Accepted: 23 January 2023 / Published: 26 January 2023

Abstract

:
Bullying is a widespread public health problem with significant behavioral and mental health consequences. The current study tested the effectiveness of combining interactive digital material for students, educators, and parents with class sessions to prevent bullying among middle school students. Fourteen middle schools were randomly assigned to intervention and comparison conditions. Both conditions received a classroom-based drug and violence prevention program that taught social skills, self-management skills, and social resistance skills. The intervention condition included class material on bullying and an educational video game for students that reinforced the classroom program; it also included digital material on bullying for parents and school staff. All students completed online pre- and post-test surveys to assess bullying-related behavior, knowledge, and life skills. Results indicated that students in the intervention schools reported significantly less bullying and cyberbullying perpetration and increased life skills knowledge relative to comparison schools. This study provides evidence that a school-based drug abuse and violence prevention program, when enhanced with a set of digital tools for students, parents, and school staff, holds considerable potential for addressing bullying among middle school adolescents.

1. Introduction

Aggression and violence during adolescence have been extensively examined in recent decades. A consistent finding from this body of research indicates that serious forms of aggression plateau from childhood through adulthood, but that less severe, more common forms of aggression may not begin until early or late adolescence [1]. Bullying is a pervasive subtype of adolescent aggression that has captured the attention of researchers, educators, mental health professionals, and policy makers [2]. Bullying involves persistent and intentional threatening and aggressive or verbally abusive behavior directed toward others at relative disadvantage, such as those who are younger, smaller, or weaker [3]. Targets who are perceived as “different” by their peers, including youth with obesity, chronic health problems, identified developmental disabilities, as well as lesbian, gay, bisexual and transgender youth are also at higher risk of bullying victimization [4,5,6]. Victims of bullying often feel unable to defend themselves because of a physical or social power imbalance [7]. Bullying can occur in almost any context although school-related incidents are most common for adolescents [8].
Bullying as a form of social aggression involves multiple roles including bullies, victims, or bully–victims (those who both perpetrate and are victimized), as well as bystanders, defenders, and reinforcers [9,10]. In traditional bullying, either physical or social in nature, offenders and victims are typically face to face or in close proximity. Cyberbullying involves harassing or threatening communication that is sent electronically from offender to victim via a variety of technologies including social media, text messages, mobile phone apps, internet forums, or online gaming. Cyberbullying is related to traditional school bullying, but there are important differences [11,12]. For example, in contrast to face-to-face bullying, where targets can leave a situation and escape to safety, those who are cyberbullied can feel continually victimized and unable to escape due in part to the ubiquitous, persistent, and permanent nature of online communications [13,14].
Bullying in all its forms is highly prevalent. Current estimates indicate that 30% of 6th, 24% of 7th, and 25% of 8th grade students report being bullied in the past 12 months and nearly 50% of these bullying experiences occur in school settings [15]. From 2007 to 2019, the percentage of individuals who have experienced cyberbullying at some point in their lifetimes has more than doubled, from 18% to 37% [16]. These prevalence estimates may not capture the true impact of bullying, as it is estimated that nearly two-thirds of bullying incidents, regardless of form, go unreported [17]. Furthermore, it is expected that cyberbullying among teens will continue to increase as new cohorts of youth obtain access to smartphones and the internet at younger ages [18,19].

1.1. Negative Behavioral and Mental Health Consequences of Bullying and Cyberbullying

Bullying and cyberbullying have significant short- and long-term behavioral and mental health consequences for victims and perpetrators that are pronounced, endure over time, and linked to a variety of problems. Victims are more likely to socially withdraw [20], perform poorly in school and have higher school dropout rates [21,22]. Victims are also more likely to report anxiety, depressive symptoms and suicidal ideation or attempts [23,24]. In addition to psychological problems, victims are also more likely to report physical symptoms including head and stomach aches, and sleeping problems [25,26]. Students who report frequently bullying others (perpetrators) are at increased risk of substance misuse, violence later in adolescence and adulthood, and suicide-related behavior [22]. For youth who both bully others and are bullied themselves, mental health problems are pervasive [27,28,29]. The negative effects of bullying are often persistent and can last into adulthood, contributing to a variety of problems including alcohol and drug dependence [30], violence, and offending later in life [31].
Although bullying and cyberbullying have a number of similar predictors and negative consequences [32] and can occur together [33], research shows that cyberbullies often exhibit risk profiles and characteristics that differ from those involved in traditional bullying [29,34]. For example, cyberbullies may engage in bullying behaviors online that, because of social norms and strictures, they would not necessarily exhibit in face-to-face encounters, reflective of online disinhibition—a lack of restraint one feels when communicating online in comparison to communicating in person. [35]. Cyberbullies also take advantage of the unique characteristics of digital communication technologies (e.g., anonymity and a potentially infinite audience) that are not applicable in face-to-face bullying contexts. Given the similarities and differences between in-person and cyberbullying along with the short- and long-term consequences of this aggressive behavior, it is imperative that effective preventive interventions are evaluated for their impact on this important public health concern.

1.2. Primary Prevention of Bullying and Cyberbullying

In the United States (US), preventing bullying has been recognized as an urgent priority for governmental action. Over the last decade, all 50 states and the District of Columbia have adopted or revised laws to address bullying. A landmark 2016 report by the National Academies of Science (NAS) [2] indicated that bullying is a public health crisis and recommended that rather than “adopting a different program to combat each new problem that emerges,” schools should integrate prevention efforts by using programs and services that address multiple skills to enhance resilience, promote positive behaviors, and prevent multiple risk behaviors with shared determinants. The NAS report [2] also indicated bullying prevention programs that promote punitive, zero-tolerance school policies (e.g., suspension or expulsion) are least effective, while those that promote social and emotional skill building produce more substantial reductions in bullying. Primary prevention research suggests that because bullying shares a common set of etiologic determinants with violence, substance use, and other externalizing behaviors, comprehensive school-based preventive interventions that address multiple problem behaviors within a social and emotional learning or positive youth development framework may be most efficient [36,37].
Several preventive interventions have been identified as promising or effective in reducing bullying behavior. A comprehensive meta-analysis examining 103 independent effect sizes of school-based anti-bullying programs reported reductions by approximately 20% for bullying perpetration and 16% for victimization [8], but that effect sizes varied widely across studies. Programs with the most substantial reductions were multicomponent interventions that targeted in-person bullying and cyberbullying, addressed various bullying roles, included active bystander intervention training, and provided relevant information and resources to parents [38]. Other promising and widely disseminated school-based bullying prevention programs include Positive Action [39], Steps to Respect [40], the Hazelden Cyberbullying program [41], and the Olweus Bullying Prevention Program [42]. Although these programs show effectiveness in preventing bullying among teens, they were not designed to comprehensively address multiple related risk behaviors—substance use, violence, and bullying—via a positive youth development framework that promotes resilience and addresses the shared risk and protective factors that are common across these outcomes.

1.3. The Promise of Educational Video Games

Bullying frequently occurs using online communications. Thus, using digital health tools to deliver bullying prevention content represents an alternative and innovative model for positive technology use [43]. In particular, interactive educational video games can be used to model developmentally appropriate online and in-person communication and engagement. Educational video gaming has been shown to be an effective means by which children and adolescents can learn to handle real-life difficulties and situations [43,44,45]. Video games have been used with success in health education and in preventing youth drug abuse, drunk driving, high-risk sexual behavior, and violence [46,47,48]. Approximately, ninety percent of teenagers in the US play video games [49], which presents an important opportunity for delivering pro-health behavioral interventions.
A growing body of research examining the effectiveness of educational video games indicates that multisensory, media-rich learning experiences make educational content more engaging and thereby increase student comprehension and retention, enhance intrinsic motivation, and facilitate behavior change [50,51]. The ability to commit to a goal and overcome challenges along with the capacity to be playful and have fun are common game elements that also motivate behavior change and promote a sense of wellbeing [52,53]. Motivational game features including immediate and continuous progress feedback (e.g., points scored, accumulation of badges, and level advancement), customizable avatars, and narratives with emotional and value-based rationales have been identified among the most effective video game interventions [54]. Games designed to be interactive and immersive can be suitable for communicating pro-health messages and promoting health behavior change because they not only entertain, but also increase intrinsic motivation to engage with and repeat health and behavioral lessons that might otherwise be less attractive if presented in more traditional, pedantic ways [55].

1.4. Life Skills Training

Several meta-analysis and systematic reviews suggest that cognitive-behavioral interventions that focus on changing internal thinking and affective processes to influence actual behavior are effective in improving adolescent problem behaviors [56,57], but are implemented infrequently [58,59]. Interventions based on the tenets of cognitive-behavioral therapy (CBT) hold significant promise for reducing aggression and bullying among adolescents [60]. A leading example of a drug and violence school-based prevention program based on cognitive-behavioral principles is Life Skills Training (LST). The LST model teaches youth personal self-management skills, social skills, drug refusal skills, resilience, and other life skills needed for healthy psychosocial development and successful navigation of key developmental tasks. LST targets both interpersonal and intrapersonal (such as knowledge, attitudes, and skills) processes through skills training methods designed to enhance social resistance skills, and a broad set of general skills that reduce problem behaviors [61]. Program content of the LST program is taught using interactive teaching methods including small group discussion with breakout sessions and skills practice through structured behavioral rehearsals.
LST has been tested in a series of randomized controlled trials reported in over 35 peer-reviewed publications [61]. These studies showed reductions of 50% or more relative to controls in cigarette smoking, alcohol use, marijuana use and use of illicit drugs among students receiving the LST program, as well as improvements in risk and protective factors associated with adolescent substance abuse. LST also has been shown to reduce violence, aggression, and delinquency among middle school youth [62]. Long-term follow-up data collected from students who participated in the LST program in middle school have shown that program effects lasted well into young adulthood [63]. Follow-up studies have also shown effects on behaviors not directly addressed by LST including risking driving [64], HIV risk behavior among young adults [65], and methamphetamine use [66]. Taken together, these findings indicate that LST can produce both immediate and long-term prevention effects that last into young adulthood, and that these effects can generalize to other risk behaviors not specially addressed in the program. The LST program combined with supplemental bullying prevention content offers considerable potential as a multimedia prevention tool for implementation in school-based settings.

1.5. Goal of Present Study

The present study was designed to test the effectiveness of the LST drug abuse and violence prevention program for middle school youth with and without a set of digital tools to provide students, parents, and school personnel with bullying-specific content and an educational video game for students that focused on the application of life skills to bullying and other high-risk situations. The bullying content added to the LST program was designed to help students identify and recognize bullying and its types (including physical, social, verbal, and cyberbullying); teach bystander intervention skills (e.g., distracting the bully); change normative beliefs and expectancies that support bullying; teach self-efficacy, self-control and coping skills as they relate to bullying; and build social, self-regulation, and healthy relationship skills. The main hypothesis examined whether LST plus the video game, supplemental bullying classroom materials, and anti-bullying informational e-learning modules for parents and school personnel would produce significant reductions in bullying relative to LST alone. It was also hypothesized that students who received the LST intervention with bullying content and the interactive video game would report improvements in health knowledge concerning the adverse effects of bullying, skills knowledge, and life skills, relative to students in the comparison group who received LST alone.

2. Materials and Methods

Using national lists of middle school principals, teachers, and district-level administrators, schools were randomly selected from different geographic areas across the U.S. and emailed recruitment fact sheets with a description of this study. The final roster of schools enrolled in this study were middle schools including students (ages 11–14) in grades 6–8 (42.9%), along with schools serving students from grades 7–8 (14.2%), grades 7–12th grade (7.1%), kindergarten to grade 8 (21.4%) or kindergarten to grade 12 (14.2%).

2.1. Sample

A total of 699 students from 14 middle schools voluntarily participated in the current study and completed the pre-test and post-test surveys. This sample was 44% male, 52.7% female, 1.1% other, and 2.2% preferred not to answer. The racial makeup of the sample was White (79.1%), Black (11.1%), American Indian or Alaskan Native (3.5%), Asian (4.3%), Native Hawaiian and Other Pacific Islander (1.3%), and 14.0% mixed or other races. Approximately 20.0% of participants reported that they were Latino/Hispanic. The mean age of the participants was 11.64 (SD = 0.63). Most participants were 11 (40.6%), 12 (42.3%), or 13 (11.7%) years old and in the 6th (60.9%) or 7th grade (32.6%).

2.2. Research Design

The effectiveness of the intervention was assessed through a cluster-randomized comparison group design in which schools were matched by geographical region and enrollment size prior to randomization. Schools were then randomly assigned to either receive the intervention (8 schools) or serve in the comparison group (6 schools). Students (n = 472) attending intervention schools received the standard LST prevention program with added bullying prevention content, an interactive video game, and e-learning modules containing anti-bullying content for parents/caregivers and school personnel. Students (n = 227) randomized to the comparison condition received the standard LST prevention program without bullying content or exposure to the video game.

2.3. Procedure

Study participants completed an online pre-test survey prior to the intervention and a post-test survey approximately 4 weeks after completion of the intervention. Data collection for students in the comparison group occurred at approximately the same time. The surveys assessed self-reported demographic information, bullying behavior, knowledge of adverse health and social effects of bullying, skills knowledge, and life skills. Unique identification codes were used to link pre-test and post-test surveys. To preserve confidentiality, student names were not included on any data collection materials. Students completed pre- and post-test surveys using computers, laptops, or other devices during a regular classroom period or computer lab session. All study procedures were similar to those used in previous prevention studies [67,68] and were approved by an Institutional Review Board. A referral protocol that provided a free and confidential 24 h hotline was available for any students who might have experienced emotional distress. No participating students reported distress and the protocol was not activated.

2.4. Intervention

Classroom Sessions. During class sessions (see Table 1), which were approximately 45 min in length, students participated in didactic lectures, small-group discussion with breakout sessions, and behavioral rehearsals (e.g., role playing). The LST program is conceptually organized into four major components. The first component, personal competence, includes class sessions that focus on self-image and self-improvement, assertiveness, coping with anxiety, and coping with anger. The second component, social competence, includes classroom sessions on social skills, assertiveness, communication skills, and resolving personal conflicts. The third component, drug resistance, includes smoking myths and realities, smoking and biofeedback, marijuana myths and realities, alcohol myths and realities, advertising, and violence and the media.
Bullying resistance skills comprise the fourth component, which includes newly developed bullying prevention classroom sessions that taught anti-bullying information, attitudes and norms, challenges to expectancies for the benefits of bullying, and skills designed to deter bullying and cyberbullying. During several class activities and skills training, students analyzed real-world scenarios and practiced bystander intervention skills in behavioral rehearsals that involved responding to bullying and other high-risk situations. In the first of two class sessions, students participated in interactive didactic activities to discuss different types of bullying (physical, social, verbal, and cyberbullying) and their physical, emotional, and social effects as well the four roles common in bullying incidents (i.e., bully, target, passive bystander, and active bystander). Students also discussed and dispelled common bullying myths to reinforce what bullying is and is not. In a second class session, students practiced ways to respond and report bullying to school personnel and other trusted adults. Students were also taught to respond to a bullying incident using various active bystander techniques (e.g., direct response, distract the bully, and delegate someone to seek help from an adult). In the same sessions, students were taught coping mechanisms for victims or bystanders.
Teacher’s Manual and Student Guide. Classroom materials included a comprehensive, step-by-step teacher’s manual with session goals, objectives, content, interactive activities, and instructions for implementing the program. The teacher manual was supplemented with a set of presentation slides containing information and skills to guide students during class sessions and the video game modules. Each student received a student guide that was structured to mirror the format and layout of the LST middle school program materials in order to standardize the intervention, enhance fidelity, and facilitate integration with other intervention materials.
Interactive Educational Video Game. Galaxia is an interactive educational video game designed to prevent bullying and cyberbullying among middle school youth. The game included a variety of animated characters and engaging scenarios that addressed individual and group processes related to in-person and online bullying victimization, perpetuation, and witnessing. The learner observed various scenarios unfold, watched characters interact with one another, and interacted with the game by selecting responses that the character could say in a particular exchange. Based on the educational content, one statement was considered the correct answer. In addition to skills training, Galaxia focused on the antecedents, responses, and consequences of bullying and cyberbullying.
Content for Galaxia was adapted from the LST program in a careful iterative process to ensure integrity to the model. Galaxia was named so to reflect outer space, where the game scenarios took place, and to appeal to a wide adolescent audience with characters having nonhuman physical characteristics and extraordinary facial features (see Figure 1). Galaxia was designed to be graphically appealing, with character animation to immerse students in real-life situations that commonly occur in middle school settings. Media-rich features including the capacity for immediate feedback, transmission of multiple cues (sound effects paired with correct responses, color variety to focus on perpetrators and victims, etc.), language variety (humorous and nonsensical words) and individualization (user selects from a library of images to build an avatar) were key elements to enhance player engagement. At the start of each game module, players were given a brief story setup about a situation involving their own avatar and other Galaxia characters and teachers. Storylines involved everyday challenges faced by young adolescent students (e.g., academic and extracurricular performance, peer acceptance, and bullying). Players navigated through each branched scenario until they reached a decision point. They were then presented with three possible dialogue options. One dialogue option correctly utilized a skill covered in the LST program, while the other two options were either wrong or not the most effective response. Only when players selected the correct life skill were they awarded points. For reinforcement, the correct skill name was displayed on the screen (e.g., “Be Direct as an Active Bystander” or “Tell a Trusted Adult”). Players earned a cumulative score, which was the sum of all points earned from each scenario. Students were granted access to each successive module based on mastery of the previous module. The game included a dashboard that informed students of their progress for each individual module and for modules combined. Both teachers and the research staff were able to view student exposure and mastery of game scenarios and this information provided critical dosage and fidelity information.
Galaxia included four modules (each approximately 30 min of play time) that reinforced cognitive-behavioral skills taught during classroom sessions. In Module 1, the player started the game to find student characters stressed about a number of situations including which classes to register for, not disappointing parents, and figuring out who damaged school property with graffiti. Mastery of self-improvement, decision-making, and resolving conflict skills was required to advance to the next module. The player began Module 2 to find characters focused on final exams and a school-wide talent show. In the scenario, some characters took their end-of-term group projects responsibilities seriously, while others procrastinated, creating a potentially anxiety-provoking situation. Effective use of communication skills, assertiveness, coping with anxiety, and social skills were required to advance to the next module. As outlined in Table 2, in Module 3, the player navigated several situations involving different forms of bullying. To accomplish the objectives of this module, the player needed to demonstrate mastery of anger management skills, skills for stopping bullying, as well as self-improvement and decision-making skills in the context of bullying incidents. In Module 4, the player faced several scenarios including being pressured by peers to share prescription medication to gain advantage on a test, carry out a school prank, sneak into an adult-rated movie, and try alcohol and drugs at a party. To receive the maximum number of points, the player was required to demonstrate mastery of resistance skills to avoid peer pressure to engage in prohibited behaviors and use prescription and illicit drugs.
E-Learning Modules for Parents/Caregivers and School Personnel. To ensure that students were exposed to consistent anti-bullying messages at school and at home, e-learning modules containing information about bullying were developed and given to parents and school staff. Both parent and educator modules provided content on the frequency and pervasiveness of bullying, types of bullying and cyberbullying, bully roles, consequences of bullying as well as healthy strategies for coping with bullying. Modules emphasized the importance of expressing compassion toward victims, tips for communicating with teens, strategies to handle bullying, and information on how to identify signs of a bully and a target. Modules geared toward professionals (teachers, support staff, bus drivers, coaches, tutors) provided content on how to detect bullying-related problems and how to intervene in bullying situations and emphasizing reparative disciplinary practices. Each module included knowledge checks and took approximately 30 min to complete.

2.5. Program Sequence

The curriculum intervention materials were implemented by regular school teachers. The provider (e.g., teachers and counselors) to student ratio was approximately 1:25. Class sessions took place in a defined sequential order starting with the Self-Image and Self-Improvement lesson and ending with two 45-min sessions on bullying. As shown in Table 2, there were 15 Life Skills Training topics implemented over 20 class sessions: (1) Self-Image and Self-Improvement; (2) Making Decisions; (3) Smoking: Myths and Realities; (4) Smoking and Biofeedback; (5) Alcohol: Myths and Realities; (6) Marijuana: Myths and Realities; (7) Advertising; (8) Violence and the Media; (9) Coping with Anxiety; (10) Coping with Anger; (11) Communication Skills; (12) Social Skills; (13) Assertiveness; (14) Resolving Conflicts; and, (15) Bullying and Cyberbullying. After the classroom sessions were completed, providers assigned students to play the video game. Providers were given the flexibility to assign the game modules in or out of class.

2.6. Online Provider Training Workshop

Prior to intervention implementation, program providers completed an asynchronous six-hour, self-paced online training workshop which offered instruction about principles and practices of the LST program, and strategies for facilitating discussions, conducting skills practice, and teaching bullying sessions. As shown in Table 3, providers learned to effectively implement the intervention, including how to structure classroom sessions and the supplemental video game. Providers reviewed the scope and sequence of the program including the goals, objectives, and learning activities of all intervention materials. Providers explored the conceptual model of LST and its underlying theory, methods, and effectiveness and practiced CBT skills training methods needed to successfully implement the classroom sessions of the program (e.g., facilitating classroom discussions, demonstrating new skills, coaching students through small-group behavioral rehearsal or skills practice, and providing positive feedback and reinforcement). Next, providers were given instruction on practical implementation issues including classroom management strategies, how to respond to disclosure of sensitive information by students, and how to establish ground rules for the classroom sessions (e.g., all students should be given the opportunity to participate and everyone’s contributions must be respected without criticism). Finally, providers were given instruction on how to integrate components of the in-class intervention with the video game modules. The workshop also clarified roles and responsibilities for participating in the randomized trial.

2.7. Measures

Online self-report surveys were administered to collect data prior to the intervention and approximately 4 weeks after completion of the intervention. Data collection for students in the comparison condition occurred on a similar schedule to the intervention condition to ensure that the interval between the pre-test and post-test was similar for both conditions. The surveys assessed several study variables including demographics; behavior and knowledge regarding physical, social, verbal, and cyberbullying victimization and perpetration; hypothesized risk and protective factors; and, the skills, knowledge, attitudes targeted by the intervention.

2.7.1. Demographic Information

Data concerning the demographic characteristics of the participants were collected using standard survey items assessing gender, age, and race/ethnicity.

2.7.2. Bullying

The student survey included 22 items selected from the Bully Survey [69], an established age-appropriate tool which assesses experiences with bullying victimization and perpetration. All bullying items had a 9-point frequency response scale anchored by 1 (never) and 9 (more than once a day). The McDonald’s ω coefficient for the 11 bullying victimization items was 0.92 and for the 11 bullying perpetration items was 0.92.
Bullying Perpetration was assessed in four domains: Physical, Verbal, Social, and Cyberbullying. Physical Bullying Perpetration was measured by three items: “About how often (if ever)”: (1) Have you pushed or shoved another student to make them feel bad?”; (2) “Have you beat up another student to make them feel bad?”; and (3) “Have you broken another student’s belongings on purpose to hurt them?” Verbal Bullying Perpetration was measured by two items: “About how often (if ever)”: (1) “Have you made fun of other students?”; and (2) “Have you said mean things behind their back?” Social Bullying Perpetration was measured by two items: “About how often (if ever)”: (1) “Have you excluded or ignored another student on purpose?”; and (2) “Have you spread rumors about another student to try to hurt their reputation?” Cyberbullying Perpetration was measured by four items: “About how often (if ever)”: (1) “Have you written or commented mean things about another student online?”; (2) “Have you sent unwanted messages to another student in order to hurt them?”; (3) “Have you threatened to post someone’s personal information, photos, or videos online in order to hurt them?”; and (4) “Have you posted someone’s personal information, photos, or videos online in order to hurt them?”
Bullying Victimization was assessed in four domains: Physical, Verbal, Social, and Cyberbullying. Physical Bullying Victimization was measured by three items: “About how often (if ever)”: (1) “Have you been pushed or shoved by another student on purpose?”; (2) “Have you been beat up by another student?”; and (3) “Have you had your belongings broken by another student on purpose?” Verbal Bullying Victimization was measured by two items: “About how often (if ever)”: (1) “Have other students made fun of you?”; and (2) “Have other students said mean things behind your back?” Social Bullying Victimization was measured by two items: “About how often (if ever)”: (1) “Have other students excluded you on purpose?”; and (2) “Have students spread rumors about you to try to ruin your reputation?” Cyberbullying Victimization was measured by four items: “About how often (if ever)”: (1) “Have other students written or commented mean things about you online?”; (2) “Have other students sent you unwanted messages online?; (3) “Have other students threatened to post your personal information, photos, or videos online in order to hurt you?”; and (4) “Have other students posted your personal information, photos, or videos online in order to hurt you?”

2.7.3. Skills Knowledge

Thirty-six “True/False” items used in previous prevention research assessed Skills Knowledge [70]. Knowledge items measured key intervention skills including Bullying Prevention, Self-Image, Decision Making, Advertising/Media, Violence and the Media, Coping with Anxiety, Coping with Anger, Communication Skills, Social Skills, Assertiveness, and Conflict Resolution. There were six items that measured Bullying Prevention Knowledge (e.g., “If I saw someone being bullied, I would directly confront the person doing the bullying.”); three items that assessed Self-Image Knowledge (e.g., “Focusing on a bad experience we’ve had can keep us from trying new things.”); three items that assessed Decision-Making Knowledge (e.g., “It’s a good idea to make a decision first and then think about the consequences later.”); three items that assessed Advertising/Media Knowledge (e.g., “Advertisements have a stated message but also an implied or hidden message that is less obvious.”); three items that assessed Violence and the Media Knowledge (e.g., “There is not as much violence in real life as there is on TV.”); three items that assessed Coping with Anxiety Knowledge (e.g., “Mental rehearsal is a good relaxation technique.”); three items that assessed Coping with Anger Knowledge (e.g., “If you are very angry, it is best to confront a conflict right away.”); three items that assessed Communication Skills Knowledge (e.g., “I ask questions if I don’t understand someone.”); three items that assessed Social Skills Knowledge (e.g., “Practicing saying hello can make it easier to start new conversations.”); three items that assessed Assertiveness Knowledge (e.g., “Being assertive means calmly and firmly standing up for your rights without infringing on the rights of others.”); and three items that assessed Conflict Resolution Knowledge (e.g., “Conflicts are a natural part of relationships.”). An Overall Skills Knowledge score was determined based on the total number of the 36 items answered correctly.

2.8. Data Analysis

A series of GLM and multilevel analyses using MIXED models were conducted to examine the impact of the intervention on the post-test outcomes, controlling for the pretest score of the outcome, race/ethnicity, and gender. Including the baseline outcome score as a covariate has been found to be an efficient data analytic strategy for testing intervention effects using a two-arm randomized pre–post-test design [71]. Robust estimators were specified for the GLM analyses, which relax strict assumptions about the distribution of the dependent variable. Multilevel analyses with MIXED modeling were also conducted on the same set of outcomes to adjust for the potential impact of school-level clustering effects. One-tailed significance tests were used to determine significance levels for the analysis of intervention effects, as warranted by the unidirectional nature of hypothesized effects and the results of previous research testing this prevention approach [61]. A significance threshold of alpha = 0.05 was set a priori.

3. Results

Pre-Test Equivalence. Intervention and comparison group participants in the analysis sample were compared on relevant baseline demographic and behavioral variables (see Table 4). There were differences between the intervention and comparison groups at pre-test regarding race and ethnicity. A higher proportion of the comparison group reported they were from a racial minority group vs. White (χ2 (1) = 6.26, p < 0. 05). A higher proportion of the comparison group reported that they were Hispanic/Latino vs. not Hispanic/Latino (χ2 (1) = 7.22, p < 0. 03). There were no pre-test differences with regard to gender. Overall bullying involvement was similar across conditions at the pre-test assessment.
Bullying Outcomes. A series of GLM analyses was conducted to examine intervention effects on the frequency of bullying perpetration and victimization. Adjusted means for bullying perpetration at the post-test assessment are presented in Table 5 for each condition, after controlling for the pre-test values of each outcome, race/ethnicity, and gender. There were significant intervention effects on social bullying perpetration, with the mean lower in the intervention group relative to the comparison group (Wald χ2 (1) = 3.87, p < 0.049). There were also significant intervention effects on cyberbullying perpetration, with the mean lower in the intervention group relative to the comparison group (Wald χ2 (1) = 4.20, p < 0.041). There were no significant differences across groups for bullying victimization. A series of analyses was also conducted using MIXED modeling to account for the fact that students were clustered within schools. Schools were modeled as a random effect, and intervention group, race/ethnicity, gender, and the pre-test score of the outcome were modeled as fixed effects. Although not significant, the results of the MIXED model analyses were generally consistent with the results of the GLM analyses.
Skills Knowledge. Using the GLM approach, comparison of the intervention and comparison groups for knowledge regarding life skills taught in the intervention showed several significant intervention effects (see Table 6). Overall Skills Knowledge was higher for the intervention group (M = 25.94) than the comparison group (M = 24.48; Wald χ2 (1) = 12.05, p < 0.001). Skills knowledge was further analyzed by type. Higher post-test knowledge means for the intervention relative to the comparison group were observed for Bullying Prevention (Wald χ2 (1) = 342.41, p < 0.001), Self-Image (Wald χ2 (1) = 4.72, p < 0.015), Decision Making (Wald χ2 (1) = 3.60, p < 0.031), Coping with Anxiety (Wald χ2 (1) = 6.38, p < 0.006), Coping with Anger (Wald χ2 (1) = 3.93, p < 0.023), and Assertiveness (Wald χ2 (1) = 5.63, p < 0.009). Although not significant, the MIXED model results were generally consistent with GLM results.

4. Discussion

A fruitful way to extend the benefits of evidence-based prevention programs is to adapt the most effective interventions and test them in new settings, with new populations, or with new behaviors. Informed by theory and research on the characteristics of effective programs for similar behaviors, the most effective intervention approaches can be tailored to address shared and unique risk and protective factors, while also maintaining their key components and active programmatic ingredients [2]. Furthermore, prevention researchers have been investigating ways to effectively incorporate digital, technology-based tools to enhance traditional, classroom-based teacher-led prevention models and create engaging multicomponent interventions. Over the past decade, e-learning and video games have been explored as multisensory, media-rich learning technologies that can enhance traditional educational approaches [43,44,45,46,47,48]. Given the increased role of multimedia technology in the learning and social environments of adolescents, along with the increasing availability of digital devices and internet access in and out of class [49], rigorous studies are needed to identify best practices for integrating digital health products into traditional classroom preventive interventions. Indeed, the potential of educational video games in multicomponent prevention programs should be examined given recent evidence that video games can improve pro-social skills and reduce antisocial behavior among school children [72].
The present study was designed to extend an evidence-based drug and violence prevention model to bullying through an intervention approach that incorporated e-learning modules for parents and educators and an interactive video game for students. The evidence-based Life Skills Training (LST) classroom program teaches youth personal self-management skills, social skills, drug refusal skills, and other life skills needed to successfully navigate developmental tasks, increase resilience, and facilitate healthy psychosocial development during adolescence [61]. Because LST addresses a wide variety of risk and protective factors via a positive youth development model, and bullying shares a similar set of risk and protective factors with substance use, it is well suited to be expanded to address the important public health problem of bullying, including cyberbullying. In the present study, bullying-specific content was added to the LST classroom program, digital e-learning modules on bullying prevention were provided to parents and school personnel, and an interactive educational video game was developed to provide opportunities for students to apply life skills to the prevention of bullying and other high-risk situations in a fun and engaging way. Results from the present study indicated that students who received the expanded intervention reported significantly less social bullying and cyberbullying perpetration and increased life skills (pro-health) knowledge relative to students attending the comparison schools. Thus, these results show that a multicomponent intervention consisting of a classroom-based curriculum, e-learning material for parents/caregivers and school personnel combined with an educational video game holds substantial potential for addressing bullying perpetration among middle school adolescents.

4.1. Implications for School-Based Bullying Prevention

A growing body of school-based research indicates that integrating prevention content with educational video games can be an effective approach for exposing youth to common, real-life difficulties and situations that allow prevention educators to teach important evidence-based content in a way that students find appealing and engaging [48,50,51]. A substantial portion of the adolescent US population plays video games, and as such games are important channels for delivering evidenced-based behavioral health interventions [49]. Features of video game technology (e.g., self-pacing and repeated use) support individualized learning while also offering teachers the potential for delivering program content in a standardized and time-efficient manner [53,54]. Gameplay combined with traditional class sessions can support important prevention goals.
The present study is the first examining the impact of a video game when used with the classroom-based LST program. The video game was designed to positively change social norms surrounding bullying and cyberbullying, challenge positive expectancies about bullying and cyberbullying, and enhance protective factors by building social, self-regulation, and relationship skills through interactive learning and behavioral rehearsal. Students perceived the Galaxia video game as immersive and evocative of interpersonal dynamics that commonly occur with peers in relationships and common school-like settings. Students found the visual aspects of the game, including the otherworldly characters in creative and colorful settings, to be engaging and fun. Teachers also found the video game appealing and remarked that its utility, interactivity, as well as the salience and effectiveness of the branched scenarios were important pedagogical devices. These findings represent an important contribution to the prevention literature, and add to a growing body of research showing that educational video games are effective not only in supporting regular educational objectives, but also effective in improving skills, promoting well-being, and reducing health risk behavior among elementary and secondary school youth [47,72,73,74].
Taken together, rigorously evaluated preventive interventions that combine evidence-based theoretically grounded content with modern sophisticated technology tools offer considerable potential for addressing the most problematic behavioral health concerns for young adolescents. The approach tested in the present study produced meaningful reductions in bullying behavior and important improvements in life skills. This approach capitalizes on the educational features of game technology and the advantages of teacher-led, in-person class sessions—both of which promote knowledge acquisition to provide opportunities for the application and practice of the knowledge and skills taught through the educational game. These findings underscore the value of multimedia preventive interventions and demonstrate that developmentally appropriate materials can be an effective means to deliver prevention programming to promote positive youth development and prevent bullying and cyberbullying as well drug abuse and violence. These findings align with previous research demonstrating that educational video games can effectively promote health-related behavioral change [46,53], reduce violence and aggression [47], and increase acquisition of pro-health related knowledge and skills [48,52]. The present study also provides additional empirical support for the positive youth development model. This study offers compelling new evidence that adding a video game component to an evidence-based classroom drug abuse and violence prevention program can effectively reduce various forms of bullying.

4.2. Strengths and Limitations

Strengths of the present study include a rigorous evaluation design; confidential self-report surveys using measures with well-established psychometric properties; standardized and well-tested protocols for recruiting schools, collecting data, and tracking participants over time; analysis of pre–post-test change at the individual level; an intervention approach building on a prevention model that has been extensively tested in multiple randomized trials; the application of theory and methods derived from the state-of-the science in the field of prevention; and inclusion of a diverse range (e.g., location, school size, and socioeconomic status of the community) of secondary schools. These findings have important public health relevance as they indicate that school-based programs that prevent adolescent violence and conduct problems, which if not addressed in early adolescence can lead to more intensive and expensive interventions (e.g., costly correctional and mental health services) in young adulthood [75]. Limitations include the possible underreporting of sensitive behaviors, the lack of follow-up after the initial post-test, limited statistical power due to the small number of schools, and a study design that did not permit the testing of individual components of the multicomponent intervention. Additionally, follow-up data are needed to determine the long-term durability of these prevention effects. Finally, because this was a school-based study that relied on students’ self-reports, the significant relationships among variables may partly reflect shared method variance (all data were obtained by self-report questionnaire).

4.3. Future Directions

Future research is needed to determine the long-term durability of this prevention approach and if the intervention effects are moderated by bullying severity or bullying role (bullies, victims, bully–victims, bystanders, defenders, and reinforcers). Longitudinal research is needed to further establish causal relationships between the intervention and reductions in various types of bullying (e.g., physical, social and cyberbullying) and other problem behaviors including general aggression and school violence in late adolescence. Research is also needed to investigate the extent to which the combination of bullying-specific content for students, parents/caregivers and school staff as well as the educational video game can be modified to further decrease bullying behavior. Another possible direction for future research would be to determine whether the prevention approach tested in this study would be strengthened by the addition of virtual reality or augmented reality components. Future studies should examine the impact of this intervention with other developmental levels including elementary and high school students.

5. Conclusions

The current study provides supporting evidence for the effectiveness of an enhanced classroom version of the Life Skills Training program when combined with an interactive educational video game. Findings from this study showed significant reductions in bullying and increases in health knowledge, skills knowledge, and life skills among students who received an enhanced version of LST combined with a video game compared to students who did not. This study extends previous research with LST by demonstrating prevention effects on bullying and aggressive behavior among young adolescents. The intervention materials tested were found to be appealing to students and teachers. These results are encouraging and suggest that an appealing gamified multimedia intervention can be effective in engaging young adolescents and support positive youth development. The findings of this study are timely and have important public health relevance. Moreover, because the intervention is innovative and engaging, it offers considerable promise for dissemination, adoption, and wide-scale implementation.

Author Contributions

C.W. supervised all study activities, including study design, data collection, interpretation of results, and wrote the first draft of the manuscript. K.W.G. was involved in study design and data collection, conducted the data analysis, and contributed significantly to the writing of the manuscript. C.M.B. contributed significantly to the intervention development and to the writing of the manuscript. S.S. was involved in data collection and contributed to the writing of the manuscript. G.J.B. provided oversight of this study and intervention development, and contributed significantly to the writing of the manuscript. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the National Institute on Child and Adolescent Development grant number R44HD074319.

Institutional Review Board Statement

This study was conducted in accordance with the Declaration of Helsinki, and approved by the Institutional Review Board of National Health Promotion Associates (R44HD074319) on 2 February 2017.

Informed Consent Statement

Written informed consent from the participants’ legal guardian/next of kin was not required to participate in this study in accordance with the National Institutes of Health (NIH) guidance and institutional requirements.

Data Availability Statement

The raw data used in this study are available from the corresponding author upon a reasonable request.

Acknowledgments

The authors would like to acknowledge Eric Kondel, Megan Wolff, Lynn Ibekwe, Madeline Liongson and Rachel Massar for their assistance with various phases of the trial.

Conflicts of Interest

Authors C.W., C.B., S.S. and G.B. are employees at National Health Promotion Associates (NHPA), which markets the program adapted and tested in this project. Author K.G. was an employee of NHPA before becoming a consultant with the firm.

References

  1. Espelage, D.L. Bullying in Early Adolescence: The Role of the Peer Group. ERIC Digest; ERIC Document Reproduction Service No. ED471912; ERIC Clearinghouse on Elementary and Early Childhood Education: Champaign, IL, USA, 2002. [Google Scholar]
  2. National Academies of Sciences, Engineering, and Medicine. Preventing Bullying through Science, Policy, and Practice; The National Academies Press: Washington, DC, USA, 2016. [Google Scholar] [CrossRef]
  3. American Psychological Association (APA). APA Dictionary of Psychology. American Psychological Association: Washington, DC, USA; Available online: https://dictionary.apa.org/bullying (accessed on 16 October 2022).
  4. Blake, J.J.; Kim, E.S.; Lund, E.M.; Zhou, Q.; Kwok, O.M.; Benz, M.R. Predictors of bully victimization in students with disabilities: A longitudinal examination using a national data set. J. Disabil. Policy Stud. 2016, 26, 199–208. [Google Scholar] [CrossRef]
  5. Kahle, L. Are sexual minorities more at risk? Bullying victimization among lesbian, gay, bisexual, and questioning youth. J. Interpers. Violence 2020, 35, 4960–4978. [Google Scholar] [CrossRef] [PubMed]
  6. Peguero, A.A.; Williams, L.M. Racial and ethnic stereotypes and bullying victimization. Youth Soc. 2013, 45, 545–564. [Google Scholar] [CrossRef]
  7. Salmon, S.; Turner, S.; Taillieu, T.; Fortier, J.; Afifi, T.O. Bullying victimization experiences among middle and high school adolescents: Traditional bullying, discriminatory harassment, and cybervictimization. J. Adolesc. 2018, 63, 29–40. [Google Scholar] [CrossRef]
  8. Gaffney, H.; Farrington, D.P.; Espelage, D.L.; Ttofi, M.M. Are cyberbullying intervention and prevention programs effective? A systematic and meta-analytical review. Aggress. Violent Behav. 2019, 45, 134–153. [Google Scholar] [CrossRef]
  9. Zych, I.; Farrington, D.P.; Llorent, V.J.; Ttofi, M.M. Protecting Children against Bullying and Its Consequences; Springer: Cham, Switzerland, 2017. [Google Scholar]
  10. Salmivalli, C.; Kaukiainen, A.; Kaistaniemi, L.; Lagerspetz, K.M. Self-evaluated self-esteem, peer-evaluated self-esteem, and defensive egotism as predictors of adolescents’ participation in bullying situations. Personal. Soc. Psychol. Bull. 1999, 25, 1268–1278. [Google Scholar] [CrossRef]
  11. Baldry, A.C.; Farrington, D.P.; Sorrentino, A. School bullying and cyberbullying among boys and girls: Roles and overlap. J. Aggress. Maltreatment Trauma 2017, 26, 937–951. [Google Scholar] [CrossRef]
  12. Chen, L.; Ho, S.S.; Lwin, M.O. A meta-analysis of factors predicting cyberbullying perpetration and victimization: From the social cognitive and media effects approach. New Media Soc. 2017, 19, 1194–1213. [Google Scholar] [CrossRef]
  13. Meldrum, R.C.; Patchin, J.W.; Young, J.T.; Hinduja, S. Bullying victimization, negative emotions, and digital self-harm: Testing a theoretical model of indirect effects. Deviant Behav. 2022, 43, 303–321. [Google Scholar] [CrossRef]
  14. Olweus, D.; Limber, S.P.; Breivik, K. Addressing specific forms of bullying: A large-scale evaluation of the Olweus bullying prevention program. Int. J. Bullying Prev. 2019, 1, 70–84. [Google Scholar] [CrossRef] [Green Version]
  15. National Center for Education Statistics. Student reports of bullying: Results from the 2017 School Crime Supplement to the National Victimization Survey. US Department of Education: Washington, DC, USA, 2019. Available online: http://nces.ed.gov/pubresearch/pubsino.asp?pubid=201506 (accessed on 15 November 2022).
  16. Patchin, J.W.; Hinduja, S. Cyberbullying among tweens in the United States: Prevalence, impact, and helping behaviors. J. Early Adolesc. 2022, 42, 414–430. [Google Scholar] [CrossRef]
  17. Petrosino, A.; Guckenburg, S.; DeVoe, J.; Hanson, T. What Characteristics of Bullying, Bullying Victims, and Schools are Associated with Increased Reporting of Bullying to School Officials? Issues & Answers; REL 2010-No. 092; Regional Educational Laboratory Northeast & Islands: Waltham, MA, USA, 2010. [Google Scholar]
  18. Alhajji, M.; Bass, S.; Dai, T. Cyberbullying, mental health, and violence in adolescents and associations with sex and race: Data from the 2015 Youth Risk Behavior Survey. Glob. Pediatr. Health 2019, 6, 2333794X19868887. [Google Scholar] [CrossRef] [Green Version]
  19. Hamm, M.P.; Newton, A.S.; Chisholm, A.; Shulhan, J.; Milne, A.; Sundar, P.; Ennis, M.A.; Scott, S.D.; Hartling, L. Prevalence and effect of cyberbullying on children and young people: A scoping review of social media studies. JAMA Pediatr. 2015, 169, 770–777. [Google Scholar] [CrossRef] [PubMed]
  20. Coelho, V.A.; Romão, A.M. The relation between social anxiety, social withdrawal and (cyber) bullying roles: A multilevel analysis. Comput. Hum. Behav. 2018, 86, 218–226. [Google Scholar] [CrossRef]
  21. Riffle, L.N.; Kelly, K.M.; Demaray, M.L.; Malecki, C.E.; Santuzzi, A.M.; Rodriguez-Harris, D.J.; Emmons, J.D. Associations among bullying role behaviors and academic performance over the course of an academic year for boys and girls. J. Sch. Psychol. 2021, 86, 49–63. [Google Scholar] [CrossRef] [PubMed]
  22. Centers for Disease Control and Prevention. Violence Prevention: Fast Fact: Preventing Bullying. Available online: https://www.cdc.gov/violenceprevention/youthviolence/bullyingresearch/fastfact.html (accessed on 15 November 2022).
  23. Moore, B.; Woodcock, S. Resilience, bullying, and mental health: Factors associated with improved outcomes. Psychol. Sch. 2017, 54, 689–702. [Google Scholar] [CrossRef]
  24. Holt, M.K.; Vivolo-Kantor, A.M.; Polanin, J.R.; Holland, K.M.; DeGue, S.; Matjasko, J.L.; Wolfe, M.; Reid, G. Bullying and suicidal ideation and behaviors: A meta-analysis. Pediatrics 2015, 135, e496–e509. [Google Scholar] [CrossRef] [Green Version]
  25. Gini, G.; Pozzoli, T. Association between bullying and psychosomatic problems: A meta-analysis. Pediatrics 2009, 123, 1059–1065. [Google Scholar] [CrossRef]
  26. Wolke, D.; Lereya, S.T. Long-term effects of bullying. Arch. Dis. Child. 2015, 100, 879–885. [Google Scholar] [CrossRef] [Green Version]
  27. Espelage, D.L.; Holt, M.K. Suicidal ideation and school bullying experiences after controlling for depression and delinquency. J. Adolesc. Health Off. Publ. Soc. Adolesc. Med. 2013, 53, S27–S31. [Google Scholar] [CrossRef] [Green Version]
  28. Haltigan, J.D.; Vaillancourt, T. Joint trajectories of bullying and peer victimization across elementary and middle school and associations with symptoms of psychopathology. Dev. Psychol. 2014, 50, 2426–2436. [Google Scholar] [CrossRef] [PubMed]
  29. Kowalski, R.M.; Limber, S.P. Psychological, physical, and academic correlates of cyberbullying and traditional bullying. J. Adolesc. Health 2013, 53, S13–S20. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  30. Ttofi, M.M.; Farrington, D.P.; Lösel, F.; Crago, R.V.; Theodorakis, N. School bullying and drug use later in life: A meta-analytic investigation. Sch. Psychol. Q. 2016, 31, 8. [Google Scholar] [CrossRef] [Green Version]
  31. Farrington, D.P.; Losel, F.; Ttofi, M.M.; Theodorakis, N. School Bullying, Depression and Offending Behavior Later in Life: An Updated Systematic Review of Longitudinal Studies; Swedish National Council for Crime Prevention: Stockholm, Sweden, 2012. [CrossRef]
  32. Williams, K.; Guerra, N. Prevalence and predictors of internet bullying. J. Adolesc. Health 2007, 41, S14–S21. [Google Scholar] [CrossRef] [PubMed]
  33. Waasdorp, T.E.; Bradshaw, C.P. The overlap between cyberbullying and traditional bullying. J. Adolesc. Health 2015, 56, 483–488. [Google Scholar] [CrossRef] [PubMed]
  34. Antoniadou, N.; Kokkinos, C.M. Cyber and school bullying: Same or different phenomena? Aggress. Violent Behav. 2015, 25, 363–372. [Google Scholar] [CrossRef]
  35. Suler, J. The online disinhibition effect. Cyberpsychology Behav. 2004, 7, 321–326. [Google Scholar] [CrossRef]
  36. Biglan, A.; Brennan, P.A.; Foster, S.L.; Holder, H.D. Helping Adolescents at Risk: Prevention of Multiple Problems of Youth; Guilford Press: New York, NY, USA, 2004. [Google Scholar]
  37. Guerra, N.; Bradshaw, C.P. Linking the prevention of problem behaviors and positive youth development: Core competencies for positive youth development. New Dir. Child Adolesc. Dev. 2008, 122, 1–17. [Google Scholar] [CrossRef]
  38. Gaffney, H.; Ttofi, M.M.; Farrington, D.P. What works in anti-bullying programs? Analysis of effective intervention components. J. Sch. Psychol. 2021, 85, 37–56. [Google Scholar] [CrossRef]
  39. Flay, B.R.; Allred, C.G. The positive action program: Improving academics, behavior, and character by teaching comprehensive skills for successful learning and living. In International Research Handbook on Values Education and Student Wellbeing; Springer: Dordrecht, The Netherlands, 2010; pp. 471–501. [Google Scholar] [CrossRef]
  40. Brown, E.C.; Low, S.; Smith, B.H.; Haggerty, K.P. Outcomes from a school-randomized controlled trial of STEPS TO RESPECT: A bullying prevention program. Sch. Psychol. Rev. 2011, 40, 423–443. [Google Scholar] [CrossRef]
  41. Limber, S.P.; Kowalski, R.M.; Agatston, P.W. Cyber Bullying: A Prevention Curriculum for Grades 6–12; Hazelden Publishing: Center City, MN, USA, 2008. [Google Scholar]
  42. Olweus, D.; Solberg, M.E.; Breivik, K. Long-term school-level effects of the Olweus Bullying Prevention Program (OBPP). Scand. J. Psychol. 2020, 61, 108–116. [Google Scholar] [CrossRef] [PubMed]
  43. Salonius-Pasternak, D.; Gelfond, H. The next level of research on electronic play: Potential benefits and contextual influences for children and adolescents. Hum. Technol. 2005, 1, 5–22. [Google Scholar] [CrossRef]
  44. Griffiths, M.D. Breaking the stereotype: The case of online gaming. Cyberpsychology Behav. 2003, 6, 81–91. [Google Scholar] [CrossRef] [PubMed]
  45. Griffiths, M.; Wood, R.T. Risk factors in adolescence: The case of gambling, video game playing, and the Internet. J. Gambl. Stud. 2000, 16, 199–225. [Google Scholar] [CrossRef] [PubMed]
  46. Baranowski, T.; Buday, R.; Thomson, D.; Baranowski, J. Playing for real video games and stories for health-related behavior change. Prev. Med. 2008, 34, 74–82. [Google Scholar] [CrossRef]
  47. Fontana, L.; Beckerman, A. Childhood violence prevention education using video games. Inf. Technol. Child. Educ. 2004, 16, 49–62. [Google Scholar]
  48. Lewis, M. Analysis of the roles of “serious games” in helping teach health-related knowledge and skills in changing behavior. J. Diabetes Sci. Technol. 2007, 1, 918–920. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  49. Vogels, E.A.; Gelles-Watnick, R.; Massarat, N. Teens, Social Media and Technology 2022; Pew Research Center: Washington, DC, USA, 2022; Available online: https://www.pewresearch.org/internet/2022/08/10/teens-social-media-and-technology-2022/ (accessed on 15 November 2022).
  50. Xu, J.; Lio, A.; Dhaliwal, H.; Andrei, S.; Balakrishnan, S.; Nagani, U.; Samadder, S. Psychological interventions of virtual gamification within academic intrinsic motivation: A systematic review. J. Affect. Disord. 2021, 293, 444–465. [Google Scholar] [CrossRef]
  51. Dichev, C.; Dicheva, D. Gamifying education: What is known, what is believed and what remains uncertain: A critical review. Int. J. Educ. Technol. High. Educ. 2017, 14, 9. [Google Scholar] [CrossRef] [Green Version]
  52. Cheek, C.; Fleming, T.; Lucassen, M.F.; Bridgman, H.; Stasiak, K.; Shepherd, M.; Orpin, P. Integrating health behavior theory and design elements in serious games. JMIR Ment. Health 2015, 2, e4133. [Google Scholar] [CrossRef] [PubMed]
  53. Cugelman, B. Gamification: What it is and why it matters to digital health behavior change developers. JMIR Serious Games 2013, 1, e3139. [Google Scholar] [CrossRef]
  54. Seaborn, K.; Fels, D.I. Gamification in theory and action: A survey. Int. J. Hum.-Comput. Stud. 2015, 74, 14–31. [Google Scholar] [CrossRef]
  55. Johnson, D.; Deterding, S.; Kuhn, K.A.; Staneva, A.; Stoyanov, S.; Hides, L. Gamification for health and wellbeing: A systematic review of the literature. Internet Interv. 2016, 6, 89–106. [Google Scholar] [CrossRef] [Green Version]
  56. Apsche, J.A.; Bass, C.K. A review and empirical comparison of three treatments for adolescent males with conduct and personality disorder: Mode Deactivation Therapy, Cognitive Behavior Therapy and Social Skills Training. Int. J. Behav. Consult. Ther. 2006, 2, 382. [Google Scholar] [CrossRef]
  57. Gerber, M.M.; Solari, E.J. Teaching effort and the future of cognitive-behavioral interventions. Behav. Disord. 2005, 30, 289–299. [Google Scholar] [CrossRef]
  58. Ttofi, M.M.; Farrington, D.P. Effectiveness of school-based programs to reduce bullying: A systematic and meta-analytic review. J. Exp. Criminol. 2011, 7, 27–56. [Google Scholar] [CrossRef]
  59. Bauman, S. The role of elementary school counselors in reducing school bullying. Elem. Sch. J. 2008, 108, 362–375. [Google Scholar] [CrossRef]
  60. Özabacı, N. Cognitive behavioural therapy for violent behaviour in children and adolescents: A meta-analysis. Child. Youth Serv. Rev. 2011, 33, 1989–1993. [Google Scholar] [CrossRef]
  61. Botvin, G.J.; Griffin, K.W. Life Skills Training: A competence enhancement approach to tobacco, alcohol, and drug abuse prevention. In Handbook of Adolescent Drug Use Prevention: Research, Intervention Strategies, and Practice; Scheier, L.M., Ed.; American Psychological Association: Washington, DC, USA, 2015; pp. 177–196. [Google Scholar] [CrossRef] [Green Version]
  62. Botvin, G.J.; Griffin, K.W.; Nichols, T.R. Preventing youth violence and delinquency through a universal school-based prevention approach. Prev. Sci. 2006, 7, 403–408. [Google Scholar] [CrossRef] [Green Version]
  63. Griffin, K.W.; Botvin, G.J.; Scheier, L.M.; Williams, C. Long-term behavioral effects of a school-based prevention program on illicit drug use among young adults. J. Public Health Res. 2023, 12, 1–5. [Google Scholar] [CrossRef]
  64. Griffin, K.W.; Botvin, G.J.; Nichols, T.R. Long-term follow-up effects of a school-based drug abuse prevention program on adolescent risky driving. Prev. Sci. 2004, 5, 207–212. [Google Scholar] [CrossRef] [PubMed]
  65. Griffin, K.W.; Botvin, G.J.; Nichols, T.R. Effects of a school-based drug abuse prevention program for adolescents on HIV risk behaviors in young adulthood. Prev. Sci. 2006, 7, 103–112. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  66. Spoth, R.L.; Clair, S.; Shin, C.; Redmond, C. Long-term effects of universal preventive interventions on methamphetamine use among adolescents. Arch. Pediatr. Adolesc. Med. 2006, 160, 876–882. [Google Scholar] [CrossRef] [Green Version]
  67. Botvin, G.J.; Griffin, K.W.; Diaz, T.; Ifill-Williams, M. Drug abuse prevention among minority adolescents: One-year follow-up of a school-based preventive intervention. Prev. Sci. 2001, 2, 1–13. [Google Scholar] [CrossRef]
  68. Griffin, K.W.; Williams, C.; Botvin, C.B.; Sousa, S.; Botvin, G.J. Effectiveness of a hybrid digital substance abuse prevention approach combining e-learning and in-person class sessions. Front. Digit. Health 2022, 4, 931276. [Google Scholar] [CrossRef] [PubMed]
  69. Swearer, S.M.; Turner, R.K.; Givens, J.E.; Pollack, W.S. “You’re so gay!”: Do different forms of bullying matter for adolescent males? Sch. Psychol. Rev. 2008, 37, 160–173. [Google Scholar] [CrossRef]
  70. Macaulay, A.P.; Griffin, K.W.; Botvin, G.J. Initial internal reliability and descriptive statistics for a brief assessment tool for the Life Skills Training drug-abuse prevention program. Psychol. Rep. 2002, 91, 459–462. [Google Scholar] [CrossRef] [PubMed]
  71. Wan, F. Statistical analysis of two-arm randomized pre-post designs with one post-treatment measurement. BMC Med. Res. Methodol. 2021, 21, 150. [Google Scholar] [CrossRef]
  72. Bjelajac, Ž.Đ.; Merdović, B. Influence of video games on prosocial and antisocial behavior. Kult. Polisa 2019, 16, 53–65. [Google Scholar]
  73. Fiellin, L.E.; Hieftje, K.D.; Pendergrass, T.M.; Kyriakides, T.C.; Duncan, L.R.; Dziura, J.D.; Fiellin, D.A. Video game intervention for sexual risk reduction in minority adolescents: Randomized controlled trial. J. Med. Internet Res. 2017, 19, e8148. [Google Scholar] [CrossRef]
  74. Halbrook, Y.J.; O’Donnell, A.T.; Msetfi, R.M. When and how video games can be good: A review of the positive effects of video games on well-being. Perspect. Psychol. Sci. 2019, 14, 1096–1104. [Google Scholar] [CrossRef] [PubMed]
  75. Foster, E.M.; Jones, D.; Conduct Problems Prevention Research Group. Can a costly intervention be cost-effective?: An analysis of violence prevention. Arch. Gen. Psychiatry 2006, 63, 1284–1291. [Google Scholar] [CrossRef] [PubMed] [Green Version]
Figure 1. Galaxia Screenshots for Gymnasium, Dorms, and Counselor’s Office.
Figure 1. Galaxia Screenshots for Gymnasium, Dorms, and Counselor’s Office.
Adolescents 03 00009 g001aAdolescents 03 00009 g001b
Table 1. Program Sequence for the LST Classroom Program with Bullying Content.
Table 1. Program Sequence for the LST Classroom Program with Bullying Content.
LessonGoals and ObjectivesKey SkillsSessions
Self-Image and Self-ImprovementTeach what self-image and self-improvement are
and how they impact one’s behavior.
Self-analysis, self-improvement, goal-setting, reframing
thoughts.
1
Making DecisionsTeach to use effective decision-making skills and
solve problems independently.
3 Cs of effective decision-making (clarify, consider, choose);
resisting group pressure.
2
Smoking: Myths and RealitiesDebunk common myths and misconceptions about
cigarettes and other forms of tobacco use.
Analyzing data; checking assumptions; considering pros/
cons.
1
Smoking and BiofeedbackTeach immediate physiological effects of smoking.Measuring heart rate; understanding scientific method.1
Alcohol: Myths and RealitiesDebunk common myths and misconceptions
about alcohol use.
Analyzing data; checking assumptions; considering pros/
cons; separating fact and fiction.
1
Marijuana: Myths and RealitiesDebunk common myths and misconceptions
about marijuana use.
Analyzing data; checking assumptions; considering pros/
cons; separating fact from fiction.
1
AdvertisingIncrease awareness of and teach resistance to
techniques used by advertisers to manipulate
consumer behavior.
Analyzing ads; recognizing persuasive techniques;
separating fact from fiction and want from needs.
1
Violence and the MediaIncrease awareness of media influences on
violence and how to check media presentations
against reality.
Analyzing perceptions about violence; comparing images in
media and reality; resistance to media distortions.
1
Coping with AnxietyDefine anxiety and common situations which
cause it.
Recognizing anxiety and its physical effects; healthy
techniques to manage anxiety; progressive relaxation skills,
mental rehearsal/ visualization, and deep breathing.
2
Coping with AngerTeach common situations that trigger anger and
how to cope with them.
Recognizing anger, its physical effects, and how to control
anger.
1
Communication SkillsTeach how to communicate effectively.Using verbal and non-verbal communication; techniques for
avoiding misunderstandings; clarifying; asking questions;
being specific; and paraphrasing.
1
Social SkillsTeach basic social skills to develop successful
interpersonal relationships.
Making social contacts; giving and receiving compliments;
effective listening; being persistent; having self-awareness;
considerate of others
2
AssertivenessTeach assertiveness and resist peer resistance
skills for drug use.
Reflecting on types of responses, consequences; awareness
of persuasive tactics; repertoire of refusal responses; verbal
and non-verbal assertiveness.
2
Resolving ConflictsTeach life skills for resolving conflicts.Analyzing conflict resolution choices; controlling anger;
building consensus; problem solving; negotiation and
compromise.
1
Bullying and CyberbullyingIdentify types of bullying, roles in bullying
situations, bystander intervention skills, and
techniques for coping.
3 Rs of Bullying (Recognize, Respond, Report), 3 Ds
Technique for Active Bystander Intervention (Direct,
Distract, Delegate), RISE-UP Against Bullying, techniques
for reporting bullying.
2
Total Class Periods 20
Table 2. Interactive Video Game Branched Scenarios for Module 3: Bullying and Cyberbullying.
Table 2. Interactive Video Game Branched Scenarios for Module 3: Bullying and Cyberbullying.
ScenarioObjectiveTriggerCorrect Avatar ResponsePoints Earned
Gymnasium
A character is verbally
aggressive toward other
characters and player
(through avatar) after
receiving failing grade
Managing AngerTeacher did not allow students to use most modern
calculator
Reframe Situation to Stay Calm
and Control Anger
75
Stop BullyingCharacter is verbally aggressive Be Direct as an Active Bystander150
Managing AngerCharacter is verbally aggressive Remind Yourself to Stay Calm225
Stop BullyingA character insults everyone because they are leaving
school for Winter break, but she has nowhere to go.
Express Yourself Using ‘I’
Statements
300
Dorms
A character has received
a number of negative
text messages
Stop BullyingAnonymous person threatens to send to the entire school
unflattering pictures of a main character
Consider Consequences before
Acting
375
Accusations and gossip about
who is sending threatening
text messages
Stop BullyingOne character is verbally abusive Be Direct as an Active Bystander450

Stop Bullying

Without adult supervision, characters plan to
investigate who’s sending anonymous texts

Tell a Trusted Adult or Ally

525
Counselor’s Office
Role playing with
counselor on how to
handle bullying
situations
Stop CyberbullyingTeacher offers to HelpMake a Specific Request600
Stop CyberbullyingBully ridicules other for asking for help from an
adult
Tell a Trusted Adult or Ally675
Stop BullyingThreat of physical aggressionLeave the Situation750
Dorms
One character breaks
controller on a game
console
Managing AngerCharacter broke game controller and demands that
the owner hands over spare controller
Take a Deep Breath and Silently Count to 10825
Character receives
another bad grade
Self-Image and
Self-Improvement
Character realizes she neglected to study which
caused her poor academic performance
Don’t Focus on Negative
Experiences
900
Character considers not
going home for Winter
break to avoid criticism
from parents
Make a DecisionCharacter faces the prospect of not going home
for Winter break
Consider Alternatives and
Consequences
975
Character thinks she knows
the identity of the cyberbully
Managing AngerCharacters considering retaliating against person
believed to be sending anonymous text messages
Stop and Calmly Count to 101050
Table 3. Online Program Provider Training Workshop.
Table 3. Online Program Provider Training Workshop.
ModuleTraining Activities for Providers
Module 1: Introductions
and Expectations
● Get acquainted through trainer-moderated discussion board to promote a cohesive and supportive learning community.
● Share workshop expectations and identify potential barriers to implementation for immediate feedback from the lead
trainer.
Module 2: Orientation● View archived presentation that describes study’s purpose.
● Ask lead trainer questions and concerns about provider roles and responsibilities.
Module 3: Principles and
Practices of the LifeSkills
Training Program
● View narrated video presentation about the theoretical framework and research supporting the LST program.
● Demonstrate understanding of how research principles and practices are incorporated into the LST instructional materials
and are reflected in key teaching strategies.
Module 4: Teaching Skills
for Facilitating Discussion
● Practice teaching skills, including facilitation, coaching, and behavioral rehearsal, to promote social emotional, cognitive
behavioral learning, and advance the acquisition of pro-health ideas, skills, and attitudes.
● View video presentations of the teaching skills demonstrated by a master trainer.
Module 5: Becoming
Familiar with LST
Instructional Material
and Resources
● Complete a virtual walk-through of a sample lesson noting key design elements to support fidelity delivery of the LST
lessons and facilitate preparation to implement the program.
● Take a virtual tour of the teaching resources on the LST website.
Module 6: Exploring the
LST Middle School
Program
● Providers view video presentations of each unit in the LST Middle School curriculum.
● Providers analyze two units of their choice and identify key objectives, skill sets, points to make and summaries by
completing and uploading a unit review worksheet for review and feedback by lead trainer.
● Discuss with other providers knowledge gained from each unit.
● Address specific challenges they foresee in delivering the lessons.
Module 7: Teaching
Bullying and
Cyberbullying Sessions
● Providers review key learning objectives and activities in bullying and cyberbullying units by completing check points and
engaging in peer-to-peer discussion posts.
● Reviewed video game dashboard to monitor student progress.
Module 8: Prevention
Education Resources
● Providers reviewed a virtual resource center with links to research articles and websites to enhance educator understanding
of evidence-based practices that address risk and protective factors impacting adolescent health and wellness.
● Reviewed resources and how they could be used to support high-quality program implementation.
Module 9: Bullying and
Cyberbullying Prevention
Feedback Forms
● Providers reviewed tools and resources essential for delivering live sessions and documenting their activities in the research
study.
● Examine fidelity, community events reports, and attendance forms to be submitted over the course of the program
implementation.
Module 10: Course
Feedback
● Providers gave feedback on the quality of the teacher training using an online survey instrument.
● Provider feedback used for continuous improvement of the online training.
Table 4. Baseline Demographics by Condition.
Table 4. Baseline Demographics by Condition.
InterventionControlAll
Sample Size472227699
Age in Years: M (SD)11.74 (0.61)11.44 (0.61)11.64 (0.66)
Race/Ethnicity:
 Racial Minority (Non-White)18.2%26.4% *20.9%
 Hispanic/Latino16.9%27.0% *20.0%
Sex (Male)43.9%44.1%44.0%
Note: * p < 0.05.
Table 5. Adjusted Means at Post-Test for Bullying Perpetration by Condition, GLM and MIXED Models.
Table 5. Adjusted Means at Post-Test for Bullying Perpetration by Condition, GLM and MIXED Models.
GLMMIXED
InterventionControlInterventionControl
Mean (SE)Mean (SE)Mean (SE)Mean (SE)
Bullying
 Physical1.20 (0.04)1.32 (0.08)1.29 (0.10)1.41 (0.11)
 Verbal1.73 (0.06)1.69 (0.10)1.74 (0.18)1.71 (0.20)
 Social1.43 (0.05)1.64 (0.10) *1.43 (0.14)1.73 (0.16)
 Cyber1.17 (0.04)1.41 (0.11) *1.17 (0.12)1.50 (0.14) *
Note: * p < 0.05.
Table 6. Adjusted Means at Post-Test for Skills Knowledge Variables by Condition, GLM and MIXED Models.
Table 6. Adjusted Means at Post-Test for Skills Knowledge Variables by Condition, GLM and MIXED Models.
GLMMIXED
InterventionControlInterventionControl
Mean (SE)Mean (SE)Mean (SE)Mean (SE)
Bullying Prevention 4.38 (0.05)4.04 (0.08) ***4.27 (0.17)4.16 (0.19)
Self-Image 2.22 (0.04)2.07 (0.06) *2.15 (0.08)2.13 (0.09)
Decision Making 1.75 (0.03)1.64 (0.04) *1.72 (0.06)1.68 (0.07)
Advertising/Media 2.34 (0.14)2.26 (0.14)1.75 (0.03)1.85 (0.06)
Violence & the Media 1.75 (0.03)1.84 (0.06)1.67 (0.08)1.84 (0.09)
Coping with Anxiety 2.40 (0.04)2.22 (0.06) *2.39 (0.12)2.24 (0.14)
Coping with Anger 2.18 (0.04)2.05 (0.05) *2.15 (0.10)2.13 (0.11)
Communication Skills 2.35 (0.04)2.32 (0.05)2.33 (0.11)2.21 (0.12)
Social Skills 2.39 (0.04)2.35 (0.06)2.37 (0.07)2.34 (0.08)
Assertiveness 2.56 (0.04)2.40 (0.06) **2.53 (0.11)2.38 (0.13)
Conflict Resolution 2.36 (0.04)2.28 (0.05)2.32 (0.11)2.29 (0.13)
Overall Skills Knowledge 25.94 (0.22)24.48 (0.36) ***25.51 (1.00)24.95 (1.15)
Note: * p < 0.05, ** p < 0.01, *** p < 0.001.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Williams, C.; Griffin, K.W.; Botvin, C.M.; Sousa, S.; Botvin, G.J. Effectiveness of Digital Health Tools to Prevent Bullying among Middle School Students. Adolescents 2023, 3, 110-130. https://doi.org/10.3390/adolescents3010009

AMA Style

Williams C, Griffin KW, Botvin CM, Sousa S, Botvin GJ. Effectiveness of Digital Health Tools to Prevent Bullying among Middle School Students. Adolescents. 2023; 3(1):110-130. https://doi.org/10.3390/adolescents3010009

Chicago/Turabian Style

Williams, Christopher, Kenneth W. Griffin, Caroline M. Botvin, Sandra Sousa, and Gilbert J. Botvin. 2023. "Effectiveness of Digital Health Tools to Prevent Bullying among Middle School Students" Adolescents 3, no. 1: 110-130. https://doi.org/10.3390/adolescents3010009

Article Metrics

Back to TopTop