Next Article in Journal
Researching Teacher Work Motivation in Ghana through the Lens of COVID-19
Next Article in Special Issue
Infection Prevention and Control in Public Hospitals and COVID-19 Temporary Treatment and Monitoring Facilities in the Philippines: Results of a Baseline Survey
Previous Article in Journal
Labour Mobility and Foreign Direct Investment: India during the COVID-19 Pandemic
Previous Article in Special Issue
Assessing the Vaccine Efficacy in Health Care Providers for Combating the COVID-19 Infection: Results from Tertiary Cancer Care Centre
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
COVID
Volume 3
Issue 2
10.3390/covid3020022
covid-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Behavioral Changes during the First Year of the COVID-19 Pandemic: A Longitudinal Comparison of Bullying, Cyberbullying, Externalizing Behavior Problems and Prosocial Behavior in Adolescents

by
Neele Bäker
* and
Jessica Schütz-Wilke
Department of Special Needs Education and Rehabilitation, Carl von Ossietzky University of Oldenburg, Ammerleander Heerstr. 114-118, 26129 Oldenburg, Germany
*
Author to whom correspondence should be addressed.
COVID 2023, 3(2), 289-300; https://doi.org/10.3390/covid3020022
Submission received: 19 January 2023 / Revised: 14 February 2023 / Accepted: 17 February 2023 / Published: 20 February 2023
(This article belongs to the Special Issue How COVID-19 and Long COVID Changed Individuals and Communities)
Versions Notes

Abstract

:
The COVID-19 pandemic has resulted in rapid, unprecedented changes in the lives of children and adolescents worldwide. During the first year in the COVID-19 pandemic German schools were partially closed. The restrictions to limit the pandemic can be viewed as incongruent with developmental tasks of children and adolescent, and this can harbor risks such as loss of education, well-being, and daily structure. Additionally, social skills could decrease. The current study analyzed behavioral changes in traditional bullying and cyberbullying, externalizing behavior problems and prosocial behavior from spring 2020 (pandemic outbreak) to spring 2021 (during the pandemic; a time when schools were closed and infection rates peaked). We addressed our research question with an online survey in a German sample. A total of 130 students (65 females and 65 males) with ages ranging from 10 to 17 (MT1 = 13.88; SDT1 = 1.26) participated. Our results revealed significant differences in cyberbullying and prosocial behavior and no significant differences in traditional bullying and externalizing behavior problems across one year. Cyberbullying increased and prosocial behavior decreased during the first year of pandemic.

1. Introduction

The COVID-19 pandemic has resulted in rapid, unprecedented changes in the lives of millions of children and adolescents. Although the course of the diseases in children and adolescents can be described as asymptomatic, or only mildly symptomatic, SARS-CoV-2 (Severe Acute Respiratory Syndrome Coronavirus 2) infections are significantly more common in children and adolescents than in adults [1,2]. The restrictions to contain the pandemic led to massive changes in the daily lives of everyone. Adolescence is a sensitive phase for social development with an increased need for social interactions [3]. Coping with the current situation and, moreover, adhering to the current restrictions, can be particularly difficult for children and adolescents. Additionally, these circumstances can be experienced as incongruent with their developmental tasks; for example, building new relationships with peers and socially responsible behavior [4,5,6]. Pandemic consequences relate to well-being, to everyday life, leisure activities and to the family situation. The well-being of adolescents can be constrained by loneliness, worries about contagion, worries about the future and increased stress levels [6,7,8]. This psychological stress that adolescents experience can be transferred to their behavior. A risk factor in the context of the restrictions and school closures is that teachers who have a control function in everyday school life can no longer observe the adolescents on a daily basis. The visibility in society for all types of bullying, especially indirect and social bullying, makes bullying very difficult to identify and address. This issue further increases with cyberbullying [9]. Even though schools were partially closed in order to limit the epidemic consequences of the COVID-19 pandemic [10], this might not break the cycle for victims of bullying.
The consequences of the Corona pandemic also cause major changes in the use of digital platforms. Not only the personal use of social media increased, but even the use of digital platforms for educational purposes increased [6,11,12]. Not all German schools were prepared for a digital switchover [13], and the digitization progress in German schools is heavily discussed and criticized in this context [14,15,16]. Furthermore, increased internet use and inexperience in dealing with digital media may lead to anti-social behaviors like cyberbullying [7,11,17].
Given results of previous research, negative correlations between bullying and prosocial behavior and positive correlations between bullying and externalizing behavior problems were expected [18,19,20,21].
Therefore, the current study examines the shifts in bullying during the COVID-19 pandemic. The primary goal of the study was to determine the alterations in both traditional bullying and cyberbullying, as well as the changes in prosocial behavior and externalizing behavioral problems from spring to summer in 2020 (when the pandemic first emerged) to spring to summer in 2021 (when schools were closed and the infection rates were at their highest). In addition to this broad inquiry, the research also explored the connections between bullying, prosocial behavior, and externalizing behavior problems.

2. Theoretical Background

2.1. Bullying among Adolescents in Times of COVID-19 Pandemic

Traditional bullying occurs when an individual is exposed to repetitive, deliberate, and negative actions of others, such as harassment due to a power imbalance [22,23,24]. In their review, Thomas et al. (2015) emphasized that the same criteria also apply to cyberbullying, and that additional elements such as anonymity and publicity could be added [23]. Further characteristics of cyberbullying are that it takes place in virtual spaces, has an easy accessibility and content is spreading quickly (e.g., by sending and posting pictures; [25]). Internet and social media can offer many advantages to adolescents, such as instant access to information and faster social communication. However, inadequate, excessive, and uncontrolled use can also be problematic for everyday life and social interaction. For example, cyberbullying was associated with problematic internet use and rule breaking as well as aggressive behavior [26]. The pandemic led to several restrictions, which resulted in a lack of contact with peers, limited opportunities for stress regulation, restructuring in schools to homeschooling, and consequently, an increase in daily screen time for adolescents [11,12]. According to Machimbarrena et al. (2018) the rising digitization and use of digital media create some risks like increasing cyberbullying and a decrease in adolescents’ well-being [17]. In general, recent studies identified decreasing tendencies in bullying [27,28,29,30]. Due to the COVID-19 lockdowns and social restrictions, the lives of adolescents seem to be increasingly digital. In addition, the pandemic is a heavy burden. Taking the aforementioned factors and influences of the pandemic into account, increased cyberbullying activity may be expected, but the effects of the pandemic on the psychosocial development of children and adolescents are still unclear [11,12].

2.2. Impact of the COVID-19 Pandemic on Adolescents’ Behavior Problems

Studies suggest that the psychological and social effects of the COVID-19 pandemic are insidious and could affect the mental health of young children and adolescents. In the course of reducing the incidence of infections, contact with peers in various countries was severely restricted and schools were partially closed. This can harbor risks such as loss of education, well-being, and daily structure. Additionally, communication skills could decrease [11,31] and adolescents could feel socially isolated and lonely [6,7,8]. It is important to note that interaction with peers is essential for the development and socialization of children and adolescents [4,5,6]. Studies highlighting the importance of social interaction argued that adolescents who were home alone during the pandemic reported higher levels of depression and anxiety [6,32]. Furthermore, studies highlighted the risk of loneliness and social exclusion on adolescent’s well-being and associated it with greater externalizing and internalizing behavior problems [33,34,35,36]. Fegert et al. (2020) highlighted that the pandemic hit many children and adolescents at a crucial moment in their social development, and that the effect of virtualization and distancing on the development should be further researched [11]. Taking the aforementioned aspects into account, the restrictions of the pandemic may also have had an impact on the prosocial behavior and behavioral problems of adolescents.

2.3. Current Study

The present study investigates changes in bullying during the Corona Pandemic. Our central research question was: How have traditional bullying and cyberbullying, prosocial behavior and externalizing behavior problems changed from spring to summer 2020 (pandemic outbreak) to spring to summer 2021 (during the pandemic; time when schools were closed, and infection rates peaked)? Next to this general question we considered specific correlations between bullying, prosocial behavior, and externalizing behavior problems. We addressed our research question with an online study in a German sample, from spring 2020 (T1) during the first pandemic wave of SARS-CoV-2 to spring 2021 (T2) after the second pandemic wave. To define the situation at the selected measurement times, descriptions of the RKI are used for orientation:
Situation Report (March, 2020; Robert Koch Institute [37]): “At the global and the national level, the situation is very dynamic and must be taken seriously. Severe and fatal courses occur in some cases. The number of cases, hospitalizations and fatalities in Germany continues to increase. The RKI currently assesses he risk to the health of the German population overall as high and as very high for risk groups. The probability of serious disease progression increases with increasing age and underlying illnesses. The risk of disease varies from region to region. The burden on the health care system depends on the geographical and age distribution of cases, health care capacity and initiation of containment measures (isolation, quarantine, social distancing etc.), and may be very high in some geographical regions. This assessment may change on short notice as a result of new findings”.
Situation Report (March, 2021; Robert Koch Institute, [38]): “In view of persistently high case numbers, the RKI (Robert Koch Institute) currently assesses the threat to the health of the general population to be very high. The revised version highlights the ongoing community transmission of SARS-CoV-2 as well as the occurrence of many outbreaks especially in households, day-care facilities for children and increasingly in schools as well as and occupational settings. Against the background of rising occurrence of variants of concern (VOC) with higher infectiousness, a rigorous reduction in physical contacts, usage of protective measures as well as intensive efforts to contain outbreaks and chains of infections are necessary to reduce the number of new infections and to protect vulnerable persons”.

3. Methods

3.1. Participants and Procedure

A total of 750 students attending fifth to eleventh grade from seventeen public secondary schools answered the questionnaire for T1. At both T1 and T2 130 students participated in the study. Due to the high dropout rate further analysis was conducted. To analyze if there are different characteristics between participants who took only the first survey (T1 only) and those who took both (T1 and T2; the final sample), a t-test was conducted but no significant differences were found. Additionally, an a priori power analysis with g*power revealed a minimal sample size of 54 to detect an effect of f2 = 0.05 with α = 0.05 and power of 0.95 [39,40]. The sample included 65 females and 65 males, with ages ranging from 10 to 17 (MT1 = 13.88; SDT1 = 1.26). The Gender distribution of the sample is homogeneous.
The study has received a positive vote from the Institutional Review Board of the responsible university. In addition, the approval of the responsible State Education Authority and the schools were obtained. For schools that were interested in participating organizational procedures were carried out, such as sending out the general participant information and obtaining the consent forms for the implementation of data collection by the school principals.
Due to the COVID-19 pandemic containment measures the survey of students could not take place in the schools; therefore, it was organized as an online survey. The study was reviewed by the Data Protection Authority of the responsible University. The schools were asked to provide the online study information to students and caregivers either via email or the school’s internal server. It was transmitted in two separated links: one for students and one for caregivers. All relevant information for the participants, as well as the consent forms, were included in the links and were provided or requested prior to the start of the survey. The questionnaire was accessible only after confirmation of consent. The students and schools did not receive any compensation for their participation.
The data were collected at two measurement times. The first test time was in spring to summer 2020. At this time the pandemic broke out in Germany and the Easter holidays were “extended”, it was not then clear that the next school year in Germany would consist largely of homeschooling. The second measurement time was in spring to summer 2021. In the last year, the students were almost exclusively in homeschooling or phased on-site in adapted learning groups. Schools still had severe restrictions and were only partially open.

3.2. Instruments

Bullying (KFN; Bergmann et al., 2017) [25]. Bullying perpetration was assessed with ten self-report items adapted from the Criminological Research Institute of Lower Saxony [25]. The first six items ask about the frequency of traditional bullying carried out in the last half of the school year, which can be classified into three areas: (1) physical violence in the form of blows, kicks or blackmail; (2) property damage; and (3) bullying through deliberate disregard by ignoring and encouraging others to also do so, as well as teasing. The second section of the scale consists of another four items focusing on cyberbullying. It is asked how often adolescents: (1) mock, insult, abuse or threaten; (2) spread rumors or bad-mouth someone; (3) publish or forward private messages or pictures of others on the internet for the purpose of exposure or the like; and (4) exclude others from a group via mobile phones or the internet. The participants are asked to state how often the behavior mentioned occurred in the last half of the school year: “How often have you mocked, insulted or threatened others via internet/smartphone?” The answer options are in the form of a six-point scale: (1) never; (2) once or twice; (3) three to six times; (4) several times per month; (5) once per week; and (6) several times a week (αT1 = 0.60; aT2 0.74). This means that adolescents rate their behavior before school closures (winter to spring 2020) and during school closures (winter to spring 2021).
Strengths and Difficulties Questionnaire (SDQ; Goodman, 1997) [41]. The Strengths and Difficulties Questionnaire [41] is a measuring instrument that is used to screen social behavior and for mapping prosocial behavior in social interactions. The self-assessment questionnaire for adolescents aged 11–17 years, which was translated into German, was used. The 25 items of the SDQ are evenly distributed over the five behavioral scales: (1) Emotional problems; (2) Externalizing behavioral problems; (3) Hyperactivity and attention problems; (4) Problems with peers; and (5) Prosocial behavior. Due to the question, however, only the prosocial behavior scale was used. This is made up of questions about willingness to help, sharing, comforting and consideration, such as “I try to be nice to other people”. A three-point Likert scale (1) not applicable, (2) partially applicable, (3) clearly applicable was used (αT1 = 0.61; aT2 = 0.77).
Externalizing Behavior Problems. To assess externalizing behavior problems, the adolescents completed the Youth Self-Report [42,43]. The YSR comprises eight problem subscales (anxious or withdrawn, somatic complaints, social, repetitive or attention problems, rule-breaking and aggressive behavior), three superordinate scales (overall problem scale, internalizing and externalizing problems) and six DSM-oriented subscales for symptoms (physical, affective, anxiety, inattention-hyperactivity, dissocial symptoms, and oppositional behavioral symptoms; [42]). In this study the superordinate scale externalizing behavior problems comprising of the subscales aggressive and rule-breaking behavior were used. On a scale ranging from (1) not true to (3) very or often true, adolescents indicated how often they displayed certain behaviors in the past six months, such as “I break rules at home, at school, or anywhere else” (αT1 = 0.88; aT2 = 0.88).

3.3. Data Analytic Procedures

To examine whether bullying, externalizing behavior problems and prosocial behavior have changed during the one year in the Corona Pandemic an ANOVA (Analysis of Variance) with repeated measurements was conducted to compare pre-pandemic states to pandemic states. Gender and age were controlled in the analyses. Calculation of descriptive analyses, Cronbach’s Alpha, correlation analysis, and t-tests for gender and age were conducted in SPSS version 27.

4. Results

4.1. Preliminary Analyses

Table 1 shows the descriptive statistics of for traditional bullying, cyberbullying, prosocial behavior, externalizing behavior problems and age for the two test times. The intercorrelations of the study variables are shown in Table 2. Most of the study variables correlated with one another. All bullying variables correlated positively with one another. Prosocial behavior at T1 correlated negatively with cyberbullying at both measurement times and traditional bullying at T1, but not at T2. Prosocial behavior at T2 correlated negatively with cyberbullying at T2 and externalizing behavior problems at T1 and T2. Externalizing behavior problems at T1 correlated positively with all bullying variables, except for traditional bullying at T2. Furthermore, externalizing behavior problems at T2 correlated positively with all bullying variables and negatively with prosocial behavior.
Table 3 shows the analyses of gender differences and Table 4 of age differences for all study variables. Gender differences only emerged for prosocial behavior at T2 (t(115) = 3.12; p = 0.002; d = 0.58) and externalizing behavior problems at T2 (t(116) = 2.12; p = 0.036; d = 0.39) with a small-to-medium effect [44]. Girls showed more prosocial behavior (M = 13.18; SD = 1.58) than boys (M = 12.23; SD = 1.73). Additionally, they also showed more externalizing behavior problems (Mgirls = 55.95; SDgirls = 9.83; Mboys = 52.28; SDboys = 8.89). Regarding age, there were only significant differences in cyberbullying at T1 (t(96.57) = 2.21; p = 0.005; d = 0.47) with a small to medium effect [44]. Older adolescents (M= 5.32; SD = 1.87) were more likely to bully online than younger adolescents (M= 4.58; SD = 0.75).

4.2. Main Analyses

The ANOVA with repeated measurements is presented in Table 5. Results indicate that there were differences in cyberbullying (F(1, 64) = 5.45, p = 0.023, η2p = 0.08) and prosocial behavior (F(1, 108) = 4.48, p = 0.037, η2p = 0.04) regarding pre-pandemic and pandemic states. There was no difference in traditional bullying (F(1, 61) = 1.92, p = 0.171) and externalizing behavior problems (F(1, 98) = 2.10, p = 0.151) before and during the pandemic. Cyberbullying (MT1 = 5.09; SDT1 = 1.76; MT2 = 5.91; SDT2 = 2.40) was significantly higher during the pandemic while prosocial behavior decreased (MT1 = 13.13; SDT1 = 1.97; MT2 = 12.69; SDT2 =1.71). The effects for prosocial behavior can be interpreted as small, and as medium regarding cyberbullying [44]. Gender and age were controlled in the analyses but revealed no significant interaction effects.

5. Discussion

This research examined whether there are differences in traditional bullying, cyberbullying, externalizing behavior problems and prosocial behavior in pre-pandemic and pandemic states. Our results revealed significant differences in cyberbullying and prosocial behavior, but no significant differences in traditional bullying and externalizing behavior problems across one year.
The results suggest that cyberbullying increased during the pandemic, but there were no significant differences in traditional bullying. However, it is interesting that traditional bullying has not changed. A decrease in bullying could have been expected due to a lack of opportunities as a consequence of the school closures to carry out bullying in school. An alternative could be the virtual space. This shows that bullying is not becoming less just because the perpetrators are deprived of the setting. The results indicate that the form of bullying is only changing and shifting to cyberbullying. Additionally, current studies indicate that adolescents do not clearly separate cyberbullying from traditional bullying [45,46]. It can be assumed that adolescents who become perpetrators or victims online do not distinguish between real situations and the virtual space. The online perpetrator or victim experience is transferred, for example, to everyday school life in which bullying is continued. On the other hand, bullying in everyday school can also be carried and continued online. This could imply that there is even less traditional bullying and more cyberbullying than the data show. Which in turn is a possible explanation that traditional bullying has not decreased in this sample.
The analysis indicates that prosocial behavior decreased during the pandemic. A possible explanation for this negative development could be the decrease in personal contact and interactions with peers [4,5,6]. Noteworthy is that prosocial behavior at the second test time rarely correlated with the other study variables. Previous research indicates that prosocial behavior is associated with bullying [20]. This result may be due to the low internal consistency of the scale at the second measurement point. Due to the contact restrictions it was hardly possible to interact with peers and to engage in prosocial behavior [6], so these contact restrictions may cause variability when answering the scale. Adolescents who generally share and help could not actually behave this way because there were no opportunities to do so.
The fact that the externalizing behavior problems have not changed significantly can be explained by the stability of the characteristic. Studies argue that individuals’ behavior problems are stable over time [47,48]. Our results do not correspond with the downward trend in Germany [49].
Gender and age did not show any significant interactions in the main analyses. However, the preliminary analyses revealed some differences, albeit inconsistent within the first and second measurement time. Girls showed more prosocial behavior than boys at T2, but also more externalizing behavior problems at T2. The results for prosocial behavior, but not for externalizing behavior problems, coincide with previous studies. Studies assume that girls show more internalizing behavior problems and boys more externalizing [49,50,51,52,53]. Maybe girls were struggling with the pandemic conditions more than boys [54]. There were no gender-specific differences regarding bullying. Previous studies described that girls more frequently engage in relational bullying, whereas boys engage in physical bullying [55,56]. It is possible that no differences were identified as these two forms of bullying were combined in the present study. Regarding age there were only differences in cyberbullying at T1. Older adolescents were more likely to bully online than younger adolescents. This result is in line with studies indicating an increase in cyberbullying with age [57,58]. However, there is still some disagreement about this [24,59].

6. Practical Implications

Research of the social and psychological consequences of the pandemic conditions and the intervention is urgently needed [7]. One goal that could be derived from this study is that the German school system still has the task to prepare students for an appropriate use of media. In current debates about the digitization of German schools it becomes clear that Germany has deficits in the pedagogical implementation of media in the classroom and is missing concepts to educate students on the safe usage of social media [14,15,16]. The aim of the Standing Conference is that by 2021 every student can access a digital learning environment at any time [60]. The formulated goals have not been achieved in the majority of German schools. The digitization gap in German schools poses additional challenges to school policies [61]. One lesson to be learned from the pandemic is that German schools should be better equipped in teaching with devices and need a curriculum regarding the goals of the usage of digital media. For prevention, schools are certainly part of the solution as they are an important part of all students’ social life [60,61]. The implications of this study mainly relate to interpersonal relationships in media as well as in school. Even after the corona pandemic, schools should attach great importance to creating a space for social interactions between students. The social exchange among the students should stimulate them to show prosocial behavior. Responsible use of media should also be given high priority in schools. Subjects should specifically focus on learning to learn online. Furthermore, a special focus of the lessons should be on highlighting the risks and benefits of digitization [26]. In conclusion, promoting prosocial behavior through social media, as well as promoting appropriate use of social media, can have a positive impact on the behavior of individuals. By leveraging the power of social media to encourage prosocial behavior individuals may be more likely to engage in such behavior, even in the midst of a global health crisis. Moreover, it is also important to promote the appropriate use of social media. By encouraging appropriate and responsible use of social media, such as limiting screen time and engaging in face-to-face interaction, individuals may be more likely to engage in prosocial behavior.

7. Limitations

This longitudinal study can serve to create new research questions or hypotheses about bullying and related factors. However, it has several limitations. The internal consistency for bullying is not satisfactory. A reason for the poor internal consistency can be the few items that make up this scale [62]. Furthermore, current studies describe that adolescents do not clearly separate cyberbullying from traditional bullying [45,46]. Nevertheless, the reliability is sufficient to carry out the analysis. The generalization of the results is limited due to the ad hoc sample. Furthermore, it should be noted that the data were collected online through self-reports by adolescents. In order to be able to detect more consequences of the restrictions caused by the pandemic and to counteract them, further studies of the effects are necessary. In addition, any consequences that have already been identified should be followed by interventions.

8. Conclusions

In conclusion, this study found that cyberbullying increased during the pandemic while prosocial behavior decreased. Despite the changes, traditional bullying and externalizing behavior problems remained relatively unchanged. These findings highlight the need for a better understanding of the impact of the pandemic on bullying and prosocial behavior, especially in regards to the shift towards cyberbullying. Additionally, it suggests that traditional bullying and externalizing behavior problems may have a level of stability over time. To further explore these findings, future research should aim to examine the connections between cyberbullying and traditional bullying, as well as the potential impact of school closures on prosocial behavior and bullying. Ultimately, the results of this study may inform the development of interventions to address the challenges faced by adolescents during the pandemic.

Author Contributions

Conceptualization, N.B. and J.S.-W.; methodology, N.B. and J.S.-W.; software, N.B. and J.S.-W.; validation, N.B. and J.S.-W.; formal analysis, N.B. and J.S.-W.; investigation, N.B. and J.S.-W.; resources, N.B. and J.S.-W.; data curation, N.B. and J.S.-W.; writing—original draft preparation, N.B. and J.S.-W.; writing—review and editing, N.B. and J.S.-W.; visualization, N.B. and J.S.-W.; supervision, N.B. and J.S.-W.; project administration, N.B. All authors have read and agreed to the published version of the manuscript.

Funding

The authors received no relevant financial support for the research, authorship, and/or publication of this article.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki, and approved by the Ethics Committee of the Carl von Ossietzky University of Oldenburg (protocol code Drs.EK/2019/054).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data that support the findings of this study are available from the corresponding author, upon reasonable request.

Conflicts of Interest

The authors declare that there is no conflict of interest.

References

  1. European Centre for Disease Prevention and Control. COVID-19 in Children and the Role of School Settings in Transmission—First Update. 2020. Available online: https://www.ecdc.europa.eu/en/publications-data/children-and-school-settings-covid-19-transmission (accessed on 16 January 2023).
  2. Riccardo, F.; Ajelli, M.; Andrianou, X.D.; Bella, A.; Del Manso, M.; Fabiani, M.; Bellino, S.; Boros, S.; Urdiales, A.M.; Marziano, V.; et al. Epidemiological characteristics of COVID-19 cases and estimates of the reproductive numbers 1 month into the epidemic, Italy, 28 January to 31 March 2020. Eurosurveillance 2020, 25, 2000790. [Google Scholar] [CrossRef] [PubMed]
  3. Turiel, E. Domain Specificity in Social Interactions, Social Thought, and Social Development. Child Dev. 2010, 81, 720–726. [Google Scholar] [CrossRef] [PubMed]
  4. Ellis, W.E.; Zarbatany, L. Understanding Processes of Peer Clique Influence in Late Childhood and Early Adolescence. Child Dev. Perspect. 2017, 11, 227–232. [Google Scholar] [CrossRef]
  5. Havighurst, R.J. Developmental Tasks and Education, 3rd ed.; David McKay Company, Inc.: Philadelphia, PA, USA, 1972. [Google Scholar]
  6. Magson, N.R.; Freeman, J.Y.A.; Rapee, R.M.; Richardson, C.E.; Oar, E.L.; Fardouly, J. Risk and Protective Factors for Prospective Changes in Adolescent Mental Health during the COVID-19 Pandemic. J. Youth Adolesc. 2021, 50, 44–57. [Google Scholar] [CrossRef] [PubMed]
  7. Holmes, E.A.; O’Connor, R.C.; Perry, V.H.; Tracey, I.; Wessely, S.; Arseneault, L.; Ballard, C.; Christensen, H.; Silver, R.C.; Everall, I.; et al. Multidisciplinary research priorities for the COVID-19 pandemic: A call for action for mental health science. Lancet Psychiatry 2020, 7, 547–560. [Google Scholar] [CrossRef]
  8. O’Connor, R.C.; Wetherall, K.; Cleare, S.; McClelland, H.; Melson, A.J.; Niedzwiedz, C.L.; O’Carroll, R.E.; O’Connor, D.B.; Platt, S.; Scowcroft, E.; et al. Mental health and well-being during the COVID-19 pandemic: Longitudinal analyses of adults in the UK COVID-19 Mental Health & Wellbeing study. Br. J. Psychiatry 2021, 218, 326–333. [Google Scholar] [CrossRef]
  9. Van Verseveld, M.D.A.; Fekkes, M.; Fukkink, R.G.; Oostdam, R.J. Teachers’ Experiences With Difficult Bullying Situations in the School: An Explorative Study. J. Early Adolesc. 2021, 41, 43–69. [Google Scholar] [CrossRef]
  10. United Nations Educational, Scientific and Cultural Organization [UNESCO]. COVID-19 Educational Disruption and Response. Available online: https://en.unesco.org/covid19/educationresponse (accessed on 16 January 2023).
  11. Fegert, J.M.; Vitiello, B.; Plener, P.L.; Clemens, V. Challenges and burden of the Coronavirus 2019 (COVID-19) pandemic for child and adolescent mental health: A narrative review to highlight clinical and research needs in the acute phase and the long return to normality. Child Adolesc. Psychiatry Ment. Health 2020, 14, 20. [Google Scholar] [CrossRef]
  12. Zhao, Y.; Guo, Y.; Xiao, Y.; Zhu, R.; Sun, W.; Huang, W.; Liang, D.; Tang, L.; Zhang, F.; Zhu, D.; et al. The Effects of Online Homeschooling on Children, Parents, and Teachers of Grades 1–9 During the COVID-19 Pandemic. Med. Sci. Monit. Int. Med. J. Exp. Clin. Res. 2020, 26, e925591-1. [Google Scholar] [CrossRef]
  13. Eickelmann, B.; Gerick, J. Lernen mit digitalen Medien. Zielsetzungen in Zeiten von Corona und unter besonderer Berücksichtigung von sozialen Ungleichheiten [Learning with digital media. Objectives in times of Corona and with special consideration of social inequalities]. Die Dtsch. Sch. Z. Für Erzieh. Bild. Und Pädagogische Prax. 2020, 16, 153–162. [Google Scholar] [CrossRef]
  14. Anger, C.; Plünnecke, A. Schulische Bildung zu Zeiten der Corona-Krise. [School education at the time of the corona crisis]. Perspekt. Wirtsch. 2020, 21, 353–360. [Google Scholar] [CrossRef]
  15. Engels, B. Corona: Stresstest für Die Digitalisierung in Deutschland [Corona: Stress Test for Digitization in Germany] (No. 23/2020); IW-Kurzbericht, Institut der deutschen Wirtschaft: Cologne, Germany, 2020. [Google Scholar]
  16. Wößmann, L.; Lergetporer, P.; Grewenig, E.; Kugler, F.; Werner, K. Fürchten sich die Deutschen vor der Digitalisierung? Ergebnisse des ifo Bildungsbarometers 2017 [Are the Germans afraid of digitization? Results of the Ifo Education Barometer 2017]. ifo Schnelldienst 2017, 70, 17–38. [Google Scholar]
  17. Machimbarrena, J.M.; Calvete, E.; Fernández-González, L.; Álvarez-Bardón, A.; Álvarez-Fernández, L.; González-Cabrera, J. Internet risks: An overview of victimization in cyberbullying, cyber dating abuse, sexting, online grooming and problematic internet use. Int. J. Environ. Res. Public Heal. 2018, 15, 2471. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Garaigordobil, M.; Machimbarrena, J.M. Victimization and Perpetration of Bullying/Cyberbullying: Connections with Emotional and Behavioral Problems and Childhood Stress. Psychosoc. Interv. 2019, 28, 67–73. [Google Scholar] [CrossRef] [Green Version]
  19. Gini, G. Associations between bullying behaviour, psychosomatic complaints, emotional and behavioural problems. J. Paediatr. Child Heal. 2008, 44, 492–497. [Google Scholar] [CrossRef]
  20. Marengo, D.; Jungert, T.; Iotti, N.O.; Settanni, M.; Thornberg, R.; Longobardi, C. Conflictual student–teacher relationship, emotional and behavioral problems, prosocial behavior, and their associations with bullies, victims, and bullies/victims. Educ. Psychol. 2018, 38, 1201–1217. [Google Scholar] [CrossRef]
  21. Oriol, X.; Miranda, R.; Amutio, A. Dispositional and situational moral emotions, bullying and prosocial behavior in adolescence. Curr. Psychol. 2021, 1–18. [Google Scholar] [CrossRef]
  22. Olweus, D. Bullying at School: What We Know and What We Can Do; Blackwell Publishing: Hoboken, NJ, USA, 1993. [Google Scholar]
  23. Thomas, H.J.; Connor, J.P.; Scott, J.G. Integrating Traditional Bullying and Cyberbullying: Challenges of Definition and Measurement in Adolescents—A Review. Educ. Psychol. Rev. 2015, 27, 135–152. [Google Scholar] [CrossRef]
  24. Zych, I.; Ortega-Ruiz, R.; Del Rey, R. Systematic review of theoretical studies on bullying and cyberbullying: Facts, knowledge, prevention, and intervention. Aggress. Violent Behav. 2015, 23, 1–21. [Google Scholar] [CrossRef]
  25. Bergmann, M.C.; Baier, D.; Rehbein, F.; Mößle, T. Jugendliche in Niedersachsen: Ergebnisse des Niedersachsensurveys 2013 und 2015 (KFN-Forschungsberichte No. 131) [Adolescents in Lower Saxony: Results of the Lower Saxony Survey 2013 and 2015 (KFN Research Reports No. 131)]; Kriminologisches Forschungsinstitut Niedersachsen eV (KFN): Hannover, Germany, 2017. [Google Scholar]
  26. Jung, Y.-E.; Leventhal, B.; Kim, Y.S.; Park, T.W.; Lee, S.-H.; Lee, M.; Park, S.H.; Yang, J.-C.; Chung, Y.-C.; Chung, S.-K.; et al. Cyberbullying, Problematic Internet Use, and Psychopathologic Symptoms among Korean Youth. Yonsei Med J. 2014, 55, 826–830. [Google Scholar] [CrossRef] [Green Version]
  27. Cosma, A.; Walsh, S.D.; Chester, K.L.; Callaghan, M.; Molcho, M.; Craig, W.; Pickett, W. Bullying victimization: Time trends and the overlap between traditional and cyberbullying across countries in Europe and North America. Int. J. Public Heal. 2020, 65, 75–85. [Google Scholar] [CrossRef] [PubMed]
  28. Due, P.; Damsgaard, M.T.; Rasmussen, M.; Holstein, B.E. Trends in Social Inequality in Exposure to Bullying at School 1994-2018. J. Educ. Soc. Behav. Sci. 2019, 32, 1–7. [Google Scholar] [CrossRef]
  29. Jadambaa, A.; Thomas, H.J.; Scott, J.G.; Graves, N.; Brain, D.; Pacella, R. Prevalence of traditional bullying and cyberbullying among children and adolescents in Australia: A systematic review and meta-analysis. Aust. New Zealand J. Psychiatry 2019, 53, 878–888. [Google Scholar] [CrossRef] [PubMed]
  30. Niproschke, S.; Oertel, L.; Schubarth, W.; Ulbricht, J.; Bilz, L. Mehr oder weniger Gewalt an Schulen? Eine Replikationsstudie 1996–2014 an sächsischen Schulen [More or less violence in schools? A replication study from 1996–2014 in Saxon schools]. Z. Soziologie Erzieh. Sozial. 2016, 36, 78–96. [Google Scholar] [CrossRef]
  31. BBC. Coronavirus: What Lockdown Is Like for Kids All around the World. 2020. Available online: https://www.bbc.co.uk/newsround/52106625 (accessed on 16 January 2023).
  32. Chen, F.; Zheng, D.; Liu, J.; Gong, Y.; Guan, Z.; Lou, D. Depression and anxiety among adolescents during COVID-19: A cross-sectional study. Brain Behav. Immun. 2020, 88, 36–38. [Google Scholar] [CrossRef]
  33. Arslan, G. School-Based Social Exclusion, Affective Wellbeing, and Mental Health Problems in Adolescents: A Study of Mediator and Moderator Role of Academic Self-Regulation. Child Indic. Res. 2018, 11, 963–980. [Google Scholar] [CrossRef]
  34. Arslan, G. School belongingness, well-being, and mental health among adolescents: Exploring the role of loneliness. Aust. J. Psychol. 2021, 73, 70–80. [Google Scholar] [CrossRef]
  35. Jones, A.C.; Schinka, K.C.; van Dulmen, M.H.M.; Bossarte, R.M.; Swahn, M.H. Changes in Loneliness during Middle Childhood Predict Risk for Adolescent Suicidality Indirectly through Mental Health Problems. J. Clin. Child Adolesc. Psychol. 2011, 40, 818–824. [Google Scholar] [CrossRef]
  36. Schinka, K.C.; van Dulmen, M.H.; Mata, A.D.; Bossarte, R.; Swahn, M. Psychosocial predictors and outcomes of loneliness trajectories from childhood to early adolescence. J. Adolesc. 2013, 36, 1251–1260. [Google Scholar] [CrossRef]
  37. RKI. Daily Situation Report of the Robert Koch Institute. 2020. Available online: https://www.rki.de/DE/Content/InfAZ/N/Neuartiges_Coronavirus/Situationsberichte/Gesamt.html (accessed on 16 January 2023).
  38. RKI. Daily Situation Report of the Robert Koch Institute. 2021. Available online: https://www.rki.de/DE/Content/InfAZ/N/Neuartiges_Coronavirus/Situationsberichte/Gesamt.html (accessed on 16 January 2023).
  39. Faul, F.; Erdfelder, E.; Lang, A.-G.; Buchner, A. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav. Res. Methods 2007, 39, 175–191. [Google Scholar] [CrossRef]
  40. Faul, F.; Erdfelder, E.; Buchner, A.; Lang, A.-G. Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behav. Res. Methods 2009, 41, 1149–1160. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  41. Goodman, R. The Strengths and Difficulties Questionnaire: A Research Note. J. Child Psychol. Psychiatry 1997, 38, 581–586. [Google Scholar] [CrossRef] [PubMed]
  42. Döpfner, M.; Achenbach, T.M.; Plück, J.; Kinnen, C.; Plück, J. CBCL/6-18R-TRF/6-18R-YSR/11-18R: Deutsche Schulalter-Formen der Child Behavior Checklist von Thomas M. Achenbach: Elternfragebogen über das Verhalten von Kindern und Jugendlichen (CBCL/6-18R), Lehrerfragebogen über das Verhalten von Kindern und Jugendlichen (TRF/6-18R), Fragebogen für Jugendliche (YSR/11-18R) [CBCL/6-18R-TRF/6-18R-YSR/11-18R: German school age forms of the Child Behavior Checklist by Thomas M. Achenbach: Parents Questionnaire about the Behavior of Children and Adolescents (CBCL/6-18R), Teacher Questionnaire about the Behavior of Children and Adolescents (TRF/6-18R), Questionnaire for Adolescents (YSR/11-18R)]; Hogrefe: Göttingen, Germany, 2014. [Google Scholar]
  43. Achenbach, T.M. Manual for the Child Behavior Checklist/4-18 and 1991 Profile; Department of Psychiatry, University of Vermont: Burlington, VT, USA, 1991. [Google Scholar]
  44. Cohen, J. Statistical Power Analysis for the Behavioral Sciences; Lawrence Erlbaum Associates: Hillsdale, NJ, USA, 1988. [Google Scholar] [CrossRef]
  45. Fischer, S.M.; John, N.; Melzer, W.; Kaman, A.; Winter, K.; Bilz, L. Traditional bullying and cyberbullying among children and adolescents in Germany–Cross-sectional results of the 2017/18 HBSC study and trends. J. Health Monit. 2020, 5, 56–72. [Google Scholar] [CrossRef]
  46. Leander, K.M.; McKim, K.K. Tracing the Everyday ‘Sitings’ of Adolescents on the Internet: A strategic adaptation of ethnography across online and offline spaces. Educ. Commun. Inf. 2003, 3, 211–240. [Google Scholar] [CrossRef]
  47. Dekovic, M.; Buist, K.L.; Reitz, E. Stability and Changes in Problem Behavior During Adolescence: Latent Growth Analysis. J. Youth Adolesc. 2004, 33, 1–12. [Google Scholar] [CrossRef]
  48. Reitz, E.; Deković, M.; Meijer, A.M. The Structure and Stability of Externalizing and Internalizing Problem Behavior During Early Adolescence. J. Youth Adolesc. 2005, 34, 577–588. [Google Scholar] [CrossRef]
  49. Klipker, K.; Baumgarten, F.; Göbel, K.; Lampert, T.; Hölling, H. Mental health problems in children and adolescents in germany. Results of the cross-sectional kiggs wave 2 study and trends. J. Health Monit. 2018, 3, 34–41. [Google Scholar] [CrossRef]
  50. Gershon, J. A Meta-Analytic Review of Gender Differences in ADHD. J. Atten. Disord. 2002, 5, 143–154. [Google Scholar] [CrossRef]
  51. Hölling, H.; Erhart, M.; Ravens-Sieberer, U.; Schlack, R. Verhaltensauffälligkeiten bei Kindern und Jugendlichen. Erste Ergebnisse aus dem Kinder- und Jugendgesundheitssurvey [Behavioral problems in children and adolescents. First results from the child and adolescent health survey] (KiGGS). Bundesgesundheitsblatt-Gesundh.-Gesundh. 2007, 50, 784–793. [Google Scholar] [CrossRef]
  52. Klein, A.M.; Otto, Y.; Fuchs, S.; Zenger, M.; von Klitzing, K. Psychometric Properties of the Parent-Rated SDQ in Preschoolers. Eur. J. Psychol. Assess. Off. Organ Eur. Assoc. Psychol. Assess. 2013, 29, 96–104. [Google Scholar] [CrossRef]
  53. Leadbeater, B.J.; Kuperminc, G.P.; Blatt, S.J.; Hertzog, C. A multivariate model of gender differences in adolescents’ internalizing and externalizing problems. Dev. Psychol. 1999, 35, 1268–1282. [Google Scholar] [CrossRef] [PubMed]
  54. Halldorsdottir, T.; Thorisdottir, I.E.; Meyers, C.C.A.; Asgeirsdottir, B.B.; Kristjansson, A.L.; Valdimarsdottir, H.B.; Allegrante, J.P.; Sigfusdottir, I.D. Adolescent well-being amid the COVID-19 pandemic: Are girls struggling more than boys? JCPP Adv. 2021, 1, e12027. [Google Scholar] [CrossRef] [PubMed]
  55. Dukes, R.L.; Stein, J.A.; Zane, J.I. Gender differences in the relative impact of physical and relational bullying on adolescent injury and weapon carrying. J. Sch. Psychol. 2010, 48, 511–532. [Google Scholar] [CrossRef] [PubMed]
  56. Fernández, M.V.C.; Castro, Y.R.; Garrido, J.M.F.; Otero, M.C. Bullying in Spanish Secondary Schools: Gender-Based Differences. Span. J. Psychol. 2013, 16, E21. [Google Scholar] [CrossRef]
  57. Pichel, R.; Foody, M.; Norman, J.O.; Feijóo, S.; Varela, J.; Rial, A. Bullying, Cyberbullying and the Overlap: What Does Age Have to Do with It? Sustainability 2021, 13, 8527. [Google Scholar] [CrossRef]
  58. Ryoo, J.H.; Wang, C.; Swearer, S.M. Examination of the change in latent statuses in bullying behaviors across time. Sch. Psychol. Q. 2015, 30, 105–122. [Google Scholar] [CrossRef]
  59. González-Cabrera, J.; León-Mejía, A.; Beranuy, M.; Gutiérrez-Ortega, M.; Alvarez-Bardón, A.; Machimbarrena, J.M. Relationship between cyberbullying and health-related quality of life in a sample of children and adolescents. Qual. Life Res. 2018, 27, 2609–2618. [Google Scholar] [CrossRef]
  60. KMK. Bildung in der digitalen Welt. Strategie der Kultusministerkonferenz [Education in the Digital World. Strategy of the Standing Conference of Education Ministers]. 2016. Available online: https://www.kmk.org/fileadmin/Dateien/veroeffentlichungen_beschluesse/2016/2016_12_08-Bildung-in-der-digitalen-Welt.pdf (accessed on 16 January 2023).
  61. Mußmann, F.; Hardwig, T.; Riethmüller, M.; Klötzer, S. Digitalisierung im Schulsystem 2021. Arbeitszeit, Arbeitsbedingungen, Rahmenbedingungen und Perspektiven von Lehrkräften in Deutschland. Ergebnisbericht [Digitization in the school system in 2021. Working Hours, Working Conditions, Framework and Per-spectives of Teachers in Germany. Results Report]; Georg-August-Universität Göttingen: Göttingen, Germany, 2021. [Google Scholar] [CrossRef]
  62. Pallant, J. SPSS Survival Manual: A Step by Step Guide to Data Analysis using SPSS for Windows, 3rd ed.; Open University Press/McGraw-Hill: Berkshire, UK, 2007. [Google Scholar]
Table
Table 1. Descriptive statistics for all study variables.
Table 1. Descriptive statistics for all study variables.
NMSDMDMinMaxα
Traditional Bullying T11008.072.4576180.63
Traditional Bullying T2828.442.4686200.67
Cyberbullying T11015.081.6344120.60
Cyberbullying T2835.752.2554120.66
Prosocial Behavior T112313.022.1136150.77
Prosocial Behavior T211712.731.71138150.61
Externalizing Behavior Problems T111151.859.435232890.88
Externalizing Behavior Problems T211854.189.535531880.88
Age T113013.881.26141017
Age T213014.821.29151118
Note. N = Number of valid cases, M = Mean; SD = Standard deviation; MD = median.
Table
Table 2. Intercorrelations between the study variables.
Table 2. Intercorrelations between the study variables.
1.2.3.4.5.6.7.
1. Traditional Bullying T11
2. Traditional Bullying T20.25 *1
3. Cyberbullying T10.53 ***0.27 *1
4. Cyberbullying T20.32 **0.46 ***0.46 ***1
5. Prosocial Behavior T1−0.34 **−0.13−0.23 *−0.27 *1
6. Prosocial Behavior T2−0.13−0.23 *−0.03−0.090.45 ***1
7. Externalizing behavior problems T10.47 ***0.180.45 ***0.33 **−0.37 ***−0.20 *1
8. Externalizing behavior problems T20.43 ***0.53 ***0.41***0.46 ***−0.24*−0.19 *0.62 ***
Note. * p < 0.05; ** p < 0.01; *** p < 0.001.
Table
Table 3. t-test for independent samples for gender differences.
Table 3. t-test for independent samples for gender differences.
GirlsBoys
MSDMSDtdfCohen’s d
Traditional Bullying T18.002.428.152.50−0.3098−0.06
Traditional Bullying T28.562.678.352.300.38800.08
Cyberbullying T15.191.444.961.830.70990.14
Cyberbullying T26.002.585.531.930.94810.21
Prosocial Behavior T113.352.1512.692.001.781210.32
Prosocial Behavior T213.181.5812.231.733.12 **1150.58
Externalizing behavior problems T152.6610.5551.028.150.921090.17
Externalizing behavior problems T255.959.8352.288.892.12 *1160.39
Note. M = Mean; SD = Standard deviation; * p < 0.05; ** p < 0.01.
Table
Table 4. t-test for independent samples for age differences.
Table 4. t-test for independent samples for age differences.
Younger AdolescentsOlder Adolescents
MSDMSDtdfCohen’s d
Traditional Bullying T17.592.358.292.471.34980.29
Traditional Bullying T28.231.978.542.660.52800.12
Cyberbullying T14.580.755.321.872.21 **96.570.47
Cyberbullying T25.541.945.842.380.57810.14
Prosocial Behavior T113.052.3713.011.97−0.10121−0.02
Prosocial Behavior T212.871.7612.661.69−0.62115−0.12
Externalizing behavior problems T149.949.3352.769.401.481090.30
Externalizing behavior problems T251.768.5955.339.781.921160.38
Note. M = Mean; SD = Standard deviation; ** p < 0.01 median split for Age T1 (14 years).
Table
Table 5. ANOVA with repeated measurements in pre-pandemic and pandemic states.
Table 5. ANOVA with repeated measurements in pre-pandemic and pandemic states.
T1 Pre-Pandemic StatesT2 Pandemic States Main EffectInteraction with GenderInteraction with Age
MSDMSDNdfFη2pFη2pFη2p
Traditional Bullying8.122.478.632.63651, 611.920.031.100.020.470.01
Cyberbullying5.091.765.912.40681, 645.45 *0.080.060.000.600.01
Prosocial Behavior13.131.9712.691.721121, 1084.48 *0.040.180.000.940.01
Externalizing behavior problems52.009.6554.259.451021, 982.100.020.030.000.000.00
Note. M = Mean; SD = Standard deviation; N = Number of valid cases, N decreased from first to second measurement time due to listwise deletion; * p < 0.05; F = F-Test, η2p = partial eta-square; controlling for gender, median split for Age T1 (14 years).
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Bäker, N.; Schütz-Wilke, J. Behavioral Changes during the First Year of the COVID-19 Pandemic: A Longitudinal Comparison of Bullying, Cyberbullying, Externalizing Behavior Problems and Prosocial Behavior in Adolescents. COVID 2023, 3, 289-300. https://doi.org/10.3390/covid3020022

AMA Style

Bäker N, Schütz-Wilke J. Behavioral Changes during the First Year of the COVID-19 Pandemic: A Longitudinal Comparison of Bullying, Cyberbullying, Externalizing Behavior Problems and Prosocial Behavior in Adolescents. COVID. 2023; 3(2):289-300. https://doi.org/10.3390/covid3020022

Chicago/Turabian Style

Bäker, Neele, and Jessica Schütz-Wilke. 2023. "Behavioral Changes during the First Year of the COVID-19 Pandemic: A Longitudinal Comparison of Bullying, Cyberbullying, Externalizing Behavior Problems and Prosocial Behavior in Adolescents" COVID 3, no. 2: 289-300. https://doi.org/10.3390/covid3020022

APA Style

Bäker, N., & Schütz-Wilke, J. (2023). Behavioral Changes during the First Year of the COVID-19 Pandemic: A Longitudinal Comparison of Bullying, Cyberbullying, Externalizing Behavior Problems and Prosocial Behavior in Adolescents. COVID, 3(2), 289-300. https://doi.org/10.3390/covid3020022

Article Metrics

Back to TopTop