Next Article in Journal
Consensus Panel Recommendations for the Pharmacological Management of Pregnant Women with Depressive Disorders
Previous Article in Journal
Nurses’ Experiences of Care in Portuguese Nursing Homes during the COVID-19 Pandemic: A Focus Group Study
Previous Article in Special Issue
Eldercare’s Turnover Intention and Human Resource Approach: A Systematic Review
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Beyond Bullying, Aggression, Discrimination, and Social Safety: Development of an Integrated Negative Work Behavior Questionnaire (INWBQ)

Cokkie Verschuren
Maria Tims
1 and
Annet H. De Lange
Department of Management and Organization, School of Business and Economics, Vrije Universiteit Amsterdam, 1081 HV Amsterdam, The Netherlands
The Faculty of Psychology, Open University, 6419 AT Heerlen, The Netherlands
The Department of Psychology, Universidade da Coruna, 15701 A Coruña, Spain
The Faculty of Psychology, Norwegian University of Science and Technology (NTNU), 7491 Trondheim, Norway
Norwegian School of Hotel Management, University of Stavanger, 4021 Stavanger, Norway
Department of Human Resource Studies, Faculty of Social and Behavioral Sciences, Tilburg University, 5037 AB Tilburg, The Netherlands
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2023, 20(16), 6564;
Submission received: 22 May 2023 / Revised: 21 July 2023 / Accepted: 24 July 2023 / Published: 11 August 2023
(This article belongs to the Special Issue Employee Wellbeing)


Negative work behavior (NWB) threatens employee well-being. There are numerous constructs that reflect NWBs, such as bullying, aggression, and discrimination, and they are often examined in isolation from each other, limiting scientific integration of these studies. We aim to contribute to this research field by developing a diagnostic tool with content validity on the full spectrum of NWBs. First, we provide a full description of how we tapped and organized content from 44 existing NWB measurement instruments and 48 studies. Second, we discussed our results with three experts in this research field to check for missing studies and to discuss our integration results. This two-stage process yielded a questionnaire measuring physical, material, psychological, sociocultural, and digital NWB. Furthermore, the questions include a range of potential actors of NWB, namely, internal (employees, managers) and external actors (clients, customers, public, and family members) at work and their roles (i.e., target, perpetrator, perpetrator’s assistant, target’s defender, outsider, and witness of NWBs). Finally, the questionnaire measures what type of harm is experienced (i.e., bodily, material, mental, and social harm).

1. Introduction

A broad range of negative work behaviors (NWBs) are currently being studied by clinicians and researchers, such as victimization [1], bullying [2], aggression [3], and deviance [4]. Although all these separate studies provide important knowledge about the prevalence and consequences of these negative work behaviors, the limitations of this narrow approach become irrefutably clear.
First, researchers have indicated that various types of NWB occur simultaneously and sequentially [5,6,7,8,9]. For instance, swearing (psychological verbal) and hitting (physical attack) often occur simultaneously. Another example of a combination of NWBs is found in #MeToo, where traditional types of negative (work) behaviors, such as physical (rape), material (no promotion), and psychological (gossiping) NWBs, alternate with digital NWBs, such as pornographic pictures via GSM (global system for mobile communications) [10]. This shows that the real and online worlds also merge or occur simultaneously and sequentially [11]. Further supporting the co-occurrence of NWBs, Privitera and Campbell [12] found that 5.8% of their respondents reported experiencing only one type of negative act, while 83.5% reported experiencing two or more types of real/online negative acts.
Second, next to the often isolated focus on negative work behaviors instead of their co-occurrence, existing measurement instruments usually focus on one actor, such as organizational actors (e.g., Negative Acts Questionnaire: NAQ-R) [13] or clients (e.g., Healthcare-worker’s Aggressive Behaviour Scale-Users) [14], whereas multiple actors are often involved in NWBs. These actors vary between strangers/public, workers/managers, clients/pupils/customers, and relatives of those as an individual or as a group [15].
A disadvantage of measuring actor types separately is that it ignores the increasingly blurring boundaries between internal and external actor types in modern workplaces. Examples of these workplaces are the healthcare [16], the police [17], education [18], or the public sector in general [19]. In these workplaces, external actors were found to play a key role in employees’ interactions with each other, with customers, and with actor relatives [20]. Therefore, the dynamics with external actor types, such as the public, clients, students, customers, providers, and their relatives, should be included in research on NWBs to have a complete view of NWBs.
A third limitation of the focus on measuring separate NWBs is that limited insights are gained on the specific nature of their inflicted harm on different actors and their roles. Previous research has shown that NWB inflicts harm on targets [21], perpetrators [22], bystanders as witness [23], companies [24], and society [25]. However, the current questionnaires often limit their focus to the harm of targets, thus only collecting data on one source [26]. Another aspect is that scientists have indicated that multiple types of harm, such as material (e.g., job loss) and psychological harm (e.g., PTSD), occur in combination [27,28]. From these insights, it is important to measure harm among various actors and their roles.
As a final limitation, as these constructs of NWB overlap, separate measurements offer no discriminating validity. This may create the problem that bullying or sexual harassment also have features of interpersonal conflict or aggression, which are considered different types of NWB. Based on the above observed challenges in measuring NWBs, and to further scientific knowledge on this important topic, the purpose of this study is to develop a full-spectrum diagnostic instrument to assess NWBs. This developmental breadth was searched for in the coverage of NWB types with co-occurrence patterns, types of harm, actor types, and actor roles in accordance with the model of Verschuren et al. [29]. These authors reviewed the NWB field and identified the overlapping and unique aspects of the operationalizations of NWBs to specify a new integrative definition of NWB. The key elements in their definition are that NWBs are (1) distinguished in five natures (i.e., physical, material, psychological, sociocultural, and/or digital); (2) are associated with different types of harm (i.e., physical, psychological, material, and social); (3) are engaged in by different actors (i.e., criminal/stranger, customer/client/pupil, coworker/manager, or relative); (4) can be performed by individuals, dyads, triads, or groups; and (5) are often taking place over time.
Thus, using this model as the guiding framework of this study, fully examining the concept of NWB requires measurement of the combination of the different NWB types; they can be physical (e.g., kicking), material (e.g., littering), psychological (e.g., isolating), sociocultural (e.g., discriminating on gender), or digital (e.g., hacking), each occurring in different patterns. These patterns of negative work behaviors tend to be systematic (e.g., not a one-time event but a repeated process [30]); tend to be ongoing (e.g., occurrence of incidents during a year) [31,32]; tend to escalate into more serious NWB (e.g., worsens from verbal into physical acts) [33]; and vary from being overtly visible, such as publicly criticizing [34], to covert and less visible forms as a silent treatment [35].
Furthermore, a full measure of NWB should include the different types of harm, namely, physical (e.g., cardiovascular disease) [36], material (e.g., replacement costs by employee turnover) [37], psychological (e.g., post-traumatic stress disorder) [38], and social harm (e.g., family consequences) [39]. Additionally, the NWB measure should include the measurement of different actor types involved in NWBs, i.e., strangers/public (e.g., visitor) [14], workers/managers (e.g., teacher) [40], clients/pupils/customers (e.g., supplier) [41], and relatives (e.g., friend) [14]. Finally, the actor roles need to be assessed, which requires the measurement of the roles of the target, perpetrator, and bystander as witness, assistant, defender, and outsider. This means that a respondent to the NWB questionnaire can have different roles (e.g., target/perpetrator).
To achieve our goal of developing a full-spectrum diagnostic instrument to assess NWBs in an integrative way, we thoroughly screened and analyzed existing instruments and studies on digital NWB and its inflicted harm to build subcategories and select measurement items that fit an integrated NWB questionnaire. We also report adjustments that were made to existing items or creation of new items. To validate our efforts, we next report the results of an expert panel who thoroughly evaluated the development of the integrated NWB questionnaire (INWBQ).
This article is organized as follows. First, we explain our development method for the INWBQ on dimension and item selection. Then, we present our results for the dimensions of NWB, occurrence patterns, harm, actor types, and actor roles. Next, we explain which adjustments were made and why. Thereafter, we indicate how we processed the contributions from the expert panel. We end with a discussion about scientific and practical implications, limitations, and future research.

2. Method

To derive an item pool for an integrated NWB instrument, as indicated above, we used the definition of Verschuren et al. [29] as a starting point. Using these aspects of NWBs, we examined the literature to identify measurement instruments that can serve as the basis for our INWBQ. Below, we outline the different steps we took to identify these instruments.

2.1. Selection of Instruments

Inclusion of construct instruments and studies:
We looked for questionnaires to fill all aspects from the definition of NWB, i.e., type of NWB, harm of NWB, types of actors, actor roles, and occurrence patterns. We searched for these instruments in the review of Verschuren et al. [29] that identified the 16 most frequently cited NWB constructs as keywords [29]:
Workplace aggression, i.e., “efforts by individuals to harm others with whom they work, or have worked, or the organizations in which they are currently, or were previously, employed. This harm-doing is intentional and includes psychological as well as physical injury” [42] (p. 38).
Bullying, i.e., “harassing, offending, socially excluding someone or negatively affecting someone’s work. It has to occur repeatedly and regularly (e.g., weekly) and over a period of time (e.g., about six months). Bullying is an escalating process in the course of which the person confronted ends up in an inferior position becoming the target of systematic negative social acts. A conflict cannot be called bullying if the incident is an isolated event or if two parties of approximately equal ‘strength’ are in conflict” [43] (p. 26).
Mobbing, i.e., “situations where a worker, supervisor, or manager is systematically and repeatedly mistreated and victimized by fellow workers, subordinates, or superiors. The term is widely used in situations where repeated aggressive and even violent behavior is directed against an individual over some period of time” [44] (p. 379).
Harassment/discrimination, i.e., “interpersonal behavior aimed at intentionally harming another employee in the workplace” [45] (p. 998).
Workplace deviance, i.e., “voluntary behavior that violates significant organizational norms and, in so doing, threatens the well-being of the organization or its members, or both” [46] (p. 555).
Counterproductive work behavior, i.e., “volitional acts that harm or are intended to harm organizations or people in organizations” [47] (p. 151).
Workplace violence, i.e., “the act or threat of violence, ranging from verbal abuse to physical assaults directed toward persons at work or on duty. The impact of workplace violence can range from psychological issues to physical injury, or even death” [48] (p. 1).
Abuse, i.e., “interactions between organizational members that are repeated hostile verbal and nonverbal, often nonphysical behaviors directed at a person(s) such that the target’s sense of him/herself as a competent worker and person is negatively affected” [49] (p. 212).
Terror, i.e., “hostile and unethical communication which is directed in a systematic way by one or a number of persons mainly toward one individual. These actions take place often (almost every day) and over a long period (at least for six months) and, because of this frequency and duration, result in considerable psychic, psychosomatic and social misery” [50] (p. 120).
Injustice, i.e., violating distributive, procedural, and interpersonal rules [51]. Distributive injustice, i.e., “ dissatisfaction and low morale related to a person’s suffering injustice in social exchanges… felt injustice is a response to a discrepancy between what is perceived to be and what is perceived should be (e.g., effort and reward, allocation of scarce rewards between different people)” [52] (p. 272); Procedural justice, i.e., “members’ sense of the moral propriety of how they are treated—is the “glue” that allows people to work together effectively. Justice defines the very essence of individuals’ relationship to employers” [53] (p. 34). In contrast, injustice (e.g., inconsistent treatment, discrimination or ill-treatment, imprecision, ethical flaw, or prejudice), is like “a corrosive solvent that can dissolve bonds within the community. Injustice is hurtful to individuals and harmful to organizations” [53] (p. 36); Interpersonal justice, i.e., “treating an employee with dignity, courtesy, and respect”. This includes “informational justice: sharing relevant information with employees.” [53] (p. 36).
Interpersonal conflict, i.e., “range from minor disagreements between coworkers to physical assaults on others. The conflict may be overt (e.g., being rude to a coworker) or may be covert (e.g., spreading rumors about a coworker)” [54] (p. 357).
victimization i.e., “an employee’s perception of having been the target, either momentarily or over time, of emotionally, psychologically, or physically injurious actions by another organizational member with whom the target has an ongoing relationship” [55] (p. 1023);
scapegoating: i.e., “an extreme form of prejudice in which an outgroup is unfairly blamed for having intentionally caused an ingroup’s misfortunes” [56] (p. 244).
Micropolitics, i.e.,” referring to employees’ perceptions in organizations, they often describe political behaviors in negative terms and associate these with self-serving behaviors, usually at the expense of others” [57] (p. 139).
Ostracism i.e., “the exclusion, rejection, or ignoring of an individual (or group) by another individual (or group) that hinders one’s ability to establish or maintain positive interpersonal relationships, work-related success, or favorable reputation within one’s place of work” [58] (p. 217).
Incivility, i.e., “low-intensity deviant behavior with ambiguous intent to harm the target, in violation of workplace norms for mutual respect. Uncivil behaviors are characteristically rude and discourteous, displaying a lack of regard for others” [59] (p. 457).
Social safety, i.e., “in a systemic or socio-ecological approach [60,61]), the dyad (victim and perpetrator), the triad (dyad and bystander) and the group are placed in larger systems around them such as the school, neighborhood and society. Therefore, power relations are influenced by the cultural and social context” [62] (p. 109), [63].
We focused on the existing published measurement instruments of these constructs, preferably validated ones. This search yielded a total of 34 construct instruments. In the expert round we added 10 validated instruments, mainly unnamed by their author(s). These instruments were abbreviated with the author names.
We selected the following construct instruments:
  • Aggression: Baron Neuman Geddes Scale (BNGS) [64]; Indirect Aggression Scale—Target (IAS-T) [65]; Fox & Stalworth Scale (FSS) [66]; Cyber aggression, scale [67]; Hospital Aggressive Behavior Scale for coworkers/superiors (HABS-CS) [68]; Healthcare-worker’s Aggressive Behavior Scale-Users (HABS-U) [14].
  • Bullying: Jóhannsdóttir Ólafsson Scale (JOS) [69]; Workplace Bullying Questionnaire—Bullied by Others (WBQ-BO) [70]; Negative Acts Questionnaire: Revised NAQ-R [13]; Cyber NAQ [12]; Escala de Abuso Psicológico Aplicado en el Lugar de Trabajo (EAPA-T) [71]; Patchin & Hinduja Cyber bullying scale (PHCS) [72]; Workplace Cyberbullying Measure (WCM) [73]; Bullying Participants Behavior Questionnaire (BPBQ) [74]; CBQ and CBQ-S (short) [75]; ICA-W [76].
  • Mobbing: Mobbing with effects on the victim [77]; Leidse Mobbing Schaal-ll. (LEMS-II) [78]; (LIPT-60 scale) [79].
  • Harassment/discrimination: Work Harassment Scale (WHS) [80]; Ethnic Harassment Experiences scale (EHE) [81]; Generalized Workplace Harassment Questionnaire (GWHQ) [82]; Cyber harassment [83]; Gender Experience Questionnaire (SEQ) [84]
  • Deviance: Interpersonal and Organizational Deviance Scale (IODS) [31]; Cyber loafing [85,86]; Work related Social Media Questionnaire (WSMQ) [87].
  • CWB: Counterproductive Work Behavior Checklist—long version (CWB-C) [88]
  • Violence: Violence Research health care: in Brazil, Bulgaria, Lebanon, Portugal, South Africa, Thailand, Australia [89].
  • Abuse: Abusive Supervision Scale [90]
  • Terror: Leymann Inventory of Psychological Terror (LIPT) [50]; Cyberstalking [91,92,93].
  • Injustice: Combined effect of perceived organizational injustice and perceived politics on deviant behaviors [94].
  • Interpersonal conflict: Interpersonal Workplace events Scale (IWES) [95]; Interpersonal Conflict at Work Scale (ICAWS) [54].
  • Victimization/scapegoating: Perceived Victimization Scale (PVS) [96]; Juvenile Victimization Questionnaire Interview (JVQ) [97]; Cyberstalking victimization [98].
  • Micropolitics: Perceptions of Organizational Politics Scale (POPS) [99]
  • Ostracism: Workplace Ostracism Scale (WOS) [100]
  • Incivility: Workplace Incivility Scale (WIS) [101]; Uncivil Workplace Behavior Questionnaire (UWBQ) [102]; Cyber incivility [103].
  • Social safety: Social Safety Index at work (SVI) [104]; Monitor of social safety in primary and secondary education [41,105].
This resulted in a total of 44 instruments, of which 39 were validated. The non-validated instruments were: Mobbing with effects on the victim [77]; Violence Research health care in Brazil, Bulgaria, Lebanon, Portugal, South Africa, Thailand, Australia [89]; Cyberstalking [91]; Cyberstalking victimization [98]; Monitoring of social safety in primary and secondary education [41,105]. We still included them because few or no measurement instruments of the constructs in question were found.
However, we also acknowledged that existing measures of NWB might lack aspects that are becoming increasingly important in modern organizations (i.e., cyber-enabled and/or cyber-dependent digital NWB) [106], or that are part of our definition of NWB but not present in existing measures of NWB (i.e., material organizational damage) [107]. Inclusion of this missing information yielded a total of 16 studies on digital NWB, and 32 studies on harm that provided input to the development of the items for the INWBQ.
We selected the following studies on digital NWB: Internet bullying [108]; Cyber-mobbing [109]; Blackmail [110]: Discriminating systems: Gender, race, and power in AI [111]; Cyber fraud [112]; Mail and wire fraud [113]; Technology facilitated violence [114]; Cyber violence on Twitter [40]; Spyware [115]; Problematic Internet Use [116]; Impact of social media on Millennials [117]; Internet politics [118]; Internet ostracism [119]; Cyber incivility [120]; Building digital safety for journalism [121]; Cybercrime, Vandalizing the information society [106].
We selected the following studies on harm: cardiovascular disease and depression [122]; costs [37,107]; psychological stress reactions and physiological stress response [123]; depression of bystanders [124]; sleep difficulties [125]; mental health [21]; job satisfaction and psychological wellbeing [126]; disability retirement [25]; type 2 diabetes [127]; health complaints, social support, personality [128]; psychiatric symptoms [38]; burnout [129]; pain [130]; mental and physical health [131]; job burnout [132]; infectious disease [133]; sickness absence [134]; revanche porn [135]; displaced aggression to family [39]; safety perception [136]; health [137,138]; coronary heart disease [139]; cardiovascular disease [36]; emotional exhaustion in work and family domains [140]; implications for work and family [141]; shame responses [142]; employers & co-workers respond to workplace bullying [143]; employee adiposity [144]; mental health [145].
We listed a total of 44 instruments and 48 studies ordered by construct, including the goal, number of cyber-enabled/-dependent NWB items, questions to actor types and actor roles, number of items on harm, rating scale and anchors, duration, total number of items, and validity. The added instruments in the expert round are indicated with an asterisk; see Supplementary Table S1, Supplementary Materials. We further unpack the content of this table in the Section 3.
Several measurement instruments were excluded from our effort to integrate NWB instruments because their items were derived from instruments already included in this study (i.e., Luxembourg Workplace Mobbing Scale (LWMS) [146], Gutenburg Health Study (GHS) [147], Escala de Abuso Psicológico Aplicado en el Lugar de Trabajo Revised (EAPT-T-R), [148], Short Negative Acts Questionnaire (SNAQ) [149]), or because they were not work-related (i.e., Contextual Victimization by Community Violence [150]), or because they addressed coping behaviors rather than NWBs (i.e., De-escalating Aggressive Behavior Scale [151,152]).

2.2. Description of Steps in Developing the INWBQ

Coding Decisions

From the definition of NWB presented above, we coded the dimensions. To distinguish the NWBs and harm dimensions, we formulated physical harm into bodily harm, material harm into material damage, and psychological harm into mental harm. Because we wanted to develop a questionnaire that is broadly applicable to sectors, we chose the broad group of actor types and coded these dimensions after the model of Merchant and Lundell [15] for stranger, coworker, customer/client/pupil, and relative. The dimensions for actor roles were defined as perpetrator, target, assistant, outsider, and defender, as in the Bullying Participants Behavior Questionnaire (BPBQ) [74] (See Table 1).
We deleted the dimension content that was too general, unclearly worded, or not work-related, combined similar ones, and divided the remaining content into subcategories. We then summed the total number of (highly) similar items in each subcategory. Those NWBs that were included in many instruments (i.e., had the highest number of items) were deemed more important to be included in our instrument and were ranked 1, followed by those with fewer items (ranked 2 or lower). Based on this ranking, we selected the number of items needed to represent each subcategory.
To include the content of two instruments, the Injustice Scale [94] and the Perception of Organizational Politics Scale [99], we reformulated their 24 reverse-scored items into NWBs. This is because these reverse-scored items measure positive rather than negative work behavior.
NWB is a systematically repeating pattern, over a longer period, that can escalate from mild to serious forms [33,81], and be covert without being visible, to overt and visible [153,154], for several months to a year [38,155]. We analyzed the measurement of these patterns (i.e., systematic frequency, duration, escalation, visibility) in the instruments and merged these modes for adoption in our questionnaire.
Since one author (CV) assessed the items in dimensions and subcategories, we examined the inter-rater agreement of all authors in this process. The other authors categorized a random sample of twenty items in dimensions and nine items in subcategories. The inter-rater reliability for both was 100% after discussing some inconsistencies.
Finally, after the coding process was finished, we adjusted the questionnaire on five points (i.e., removing double-barreled parts of selected items, collecting anonymous digital NWB under the covert NWB, asking concrete behaviors, questioning harm, asking mixed participant roles, developing items on escalation and visibility, and adjusting items to external actors). Next, we presented our INWBQ to a panel of experts in this research field. Based on their comments and recommendations, final adjustments were made (see Figure 1).

3. Results

3.1. Dimensions and Item Selection for NWB, Harm, Actor Types, and Roles

3.1.1. Dimensions of NWB

In Supplementary Table S1 we list the selected instruments and studies. The abbreviations of the unnamed instruments were taken from the study of Escartín et al. [156]. The goals of the questionnaires could be divided into three types: development (e.g., valid and reliable measure to assess cyberbullying) [73], prevalence (e.g., identify and address workplace violence in the health sector of nine countries) [89], and research on a causal relationship of coping or harm with NWB. Found separate scales for measuring this relation in our studies were the Impact of Event Scale (IES and IES-R (revised) [157], Hospital Anxiety and Depression Scale (HADS) [158], Job Control Scale (QPS) [159], Trauma Evaluation Form (TIF), Posttraumatic Stress Diagnostic Scale (PDS), Maslach Burnout Inventory (MBI) [160], Brief Job Stress Questionnaire(BJSQ) [161], Brief Trauma Questionnaire [162], Violent Incident Form [163] Goldberg scales for anxiety and depression [164], the Demand Control/Support Questionnaire for Stress [165], Occupational Stress Questionnaire (OSQ,) [166], Positive and Negative Affect Scale (PANAS) [167], Outcome Questionnaire (OQ-45.2) [168], Maslach Burnout Inventory (MBI) [169], family undermining survey [39]. Found NWB instruments with scales measuring harm were 11 items of the Turkish version of the LIPT [170], items from the Workplace Sexual Harassment and assault scale [171].
The NWB dimensions were physical, material, psychological, sociocultural, and digital. We filled these dimensions with NWBs from the instrument items and formed subcategories. For instance, for the material NWB dimension, these subcategories were property, resources, job decisions, sabotage, and displays of NWB content. Then, we reduced the number of NWBs further by combining similar ones (e.g., waste and litter). (Due to limited space in this article, full subcategories and their reduction are not included in this article. Here, we suffice with two subcategories of the material NWB dimension and one subcategory of the mental damage dimension to illustrate this process. The full material can be requested from the author). Double-barreled items were distributed separately among the dimensions and subcategories (e.g., causing damage with financial costs was divided among material NWB and material harm). Next, we selected items from the instruments for each final subcategory.
For example, for the dimension material, we selected an item from the NWB “theft” for the subcategory material property, which had the highest rank with a total of 14 highly similar items: “Took items from your desk without prior permission” (UWBQ); and, from the NWB “destruction of property”, the second rank with 12 highly similar items: “Have your personal property or workplace property damaged by someone”(cyber aggression) (see Supplementary Table S2 from the Supplementary Materials, with NWBs of double-barreled and multiple-choice items separately included in red).
In the digital NWB dimension, we made two subcategories reflecting both cyber-enabled NWB and cyber-dependent NWB because they differ. Cyber-enabled NWB concerned traditional NWBs also using a digital medium such as a GSM. Cyber-dependent NWB cannot occur without a digital medium such as identity theft. Cyber, in these two subcategories, is another term for digital, derived from the study of Furnell [106].

3.1.2. Harm Dimensions

The dimensions of harm that we wanted to cover with the measure were bodily harm, material damage, mental harm, and social harm [29]. NWB has been conceived as behavior that harms or threatens to harm others at work (e.g., in workplace aggression [42], bullying [172], workplace harassment [45], counterproductive work behavior [173], and workplace victimization [55]). The NWB instruments ask about several types of inflicted harm due to NWB, such as bodily harm (e.g., physical complaints, SVI) and mental harm (e.g., fear, cyber aggression). On the other hand, various studies used separate questionnaires to show that NWB is associated with bodily harm, i.e., sleep difficulties [125], health impairment [137], cardio-vascular disease [122], and pain conditions [130] (see Supplementary Table S1 for studies on harm).
These studies also showed that NWB temporarily inflicts mental harm, i.e., psychological distress and stress reactions [123], lower job satisfaction [126], persistent mental complaints, i.e., depression, anxiety, burnout, and PTSD (see review of Verkuil et al. [21]), and turnover intention [174]. Moreover, it has been shown that these outcomes result in material damage, i.e., sickness absence [137], and social harm, i.e., harm to family members [39,97,98] (see Table S1 for studies on harm).
Because four dimensions of harm were part of our integrated NWB definition and some types of harm were not found in the instruments, we included items on physical harm for illness (e.g., diabetes type 2 [127]), material societal damage (e.g., reduced productivity [107]), mental harm in a combination of complaints (e.g., PTSD symptoms, [16,19]), and social harm (e.g., premature retirement [107]) from the studies to fill these dimensions and form subcategories. We reduced the number of harms by removing harms that were too general (e.g., negative psychological outcome), and by combining similar harms (e.g., fear, scared, anxiety). Then we selected items for each subcategory based on the rank number. For example, we selected the item “Did the NWB cause any of the following temporarily mental complaints” with the multiple-choice answers “felt anxious (rank 1), embarrassed (rank 2), stressed out (rank 2), felt angry (rank 3), affect social contacts (rank 4), had difficulties concentrating (rank 5)”. (See Supplementary Table S3 from the Supplementary Materials)

3.1.3. Dimension Actor Types

The subcategories for the dimension actor types were criminal/stranger, customer/client/pupil, coworker/manager, or relative [15]. We gathered the actor types of the instruments for each subcategory, such as Internet user for stranger (cyber fraud). We added a subcategory for actor group of people based on the content of the instruments such as citizens, shared interest group, and organizational group (POPS). After deleting content in subcategories that was too general (e.g., stranger: individual), and combining different names for the same actor types (e.g., worker, subordinate, employee, and colleague were combined into coworker), we obtained our dimension with subcategories for actor types (see Table 2).
Although we found several items to capture actor types, prior to the study we chose to select which actor types are relevant for sampling. Of importance here is to delimit what actor types are considered part of an organization’s network. For instance, this can be done through a realist approach, in which organizations themselves determine the social boundaries, or a nominal approach, in which the boundary is conceptually imposed from the analysis goal [175]. Prior to a survey, a project group can determine the external actors with the help of Table 2.

3.1.4. Dimension Actor Roles

Most of the instruments (n = 37) question actors about the role of the target (e.g., being given tasks with unreasonable deadlines, NAQ-R). Other instruments (n = 11) question actors as the perpetrator (e.g., acted rudely toward someone at work, Interpersonal and Organizational Deviance scale). A minority (n = 5) question bystanders as a witness (e.g., people in this organization attempt to build themselves up by tearing others down, POPS).
The Bullying Participant Behavior Questionnaire (BPBQ) of Demaray [74] questions about three more bystander roles: assistant, defender, and outsider. The assistant is willing to encourage, join in, or aid a bully in bullying others (e.g., “I have made fun of someone when they were pushed, punched, or slapped”). The defender participates in behaviors related to defending on behalf of a victim (e.g., “I tried to make people stop spreading rumors about others”). The outsider acknowledges that bullying occurs but chooses to actively ignore it (e.g., “I pretended not to notice when things were taken or stolen from another student”).
We followed the example of the BPBQ and added the role of witness. In our questionnaire, we combined subcategories referring to bully, aggressor, attacker, perpetrator into perpetrator, and combined subcategories of victim and target into target (see Table 2). In Section 3.1.6 we describe how we adjusted our items to the actor roles.

3.1.5. Occurrence Patterns

The NWB dimension that reflects occurrence patterns was subcategorized as systematic frequency, duration, escalation, and visibility based on the definition of Verschuren et al. [29]. Below we discuss each pattern with our choice for measurement.


Most NWB instruments (n = 23) measure the systematic repetitive pattern of NWB as a frequency of occurrence using a 5-point Likert scale. We chose this 5-point scale to leave a center point as an option for a neutral answer [176]. The 3- and 7-point scales also offer this possibility. However, the 3-point scale has too few options, which can make the choice feel forced, while the 7-point scale offers so many options that nuances fall away, and disengagement behavior of respondents increases [177]. We deleted the scales without frequency (e.g., strongly disagree and strongly agree). We combined the 5-point anchors for the INWBQ into the following: 1never, 2rarely, 3at least once a month, 4at least once a week, 5daily.


The measurement instruments use different terms for duration. We summarized this as last year at work. Reporting periods greater than 12 months can lead to potentially serious biases in self-reports due to memory distortions [178,179].
However, when we only ask about the frequency of digital NWB (e.g., weekly), we do not collect information about the weekly amount of use on a medium. By charting the hours of use per day, indicating weekly digital NWB, we have more information on this.
The inventory of place and time of digital use are what Alhabash and colleagues [180] called the viral reach of users. This refers to the interactivity of users on digital media with messages distributed and shared online. We adopted this term for the medium use subcategory with multiple-choice questions about the type of medium (e.g., cloud storage) and duration of medium use (e.g., ½–1 h a day) [108]. We preferred to ask about hours of use, as a percentage usage per day seemed too abstract and difficult to answer.


As escalation is an essential pattern of all NWB labels, it is remarkable that we found only 2 items about escalation: “Not reporting a problem so it would get worse”(CWB-C) [181] and “escalation from online NWB to offline NWBs” (cyberstalking) [91]. Because these two items were quite specific on reporting and online NWB, we developed two general extra items; see Section 3.1.6 under Adjustments.


It is important to include items on visibility because tactics used to remain invisible have a function in disguising the identity of the perpetrator [34,42], and in maximizing harm while minimizing danger to perpetrators, as also indicated with the “effect-danger ratio” [182]. An example of this is the invisible role of the instigator in digital NWB [121]. Another more positive reason for invisible NWB is to keep an option for restoration [183].
However, because most items in the instruments were covert, overt NWB could not be adequately measured. Therefore, following the example in the LIPT-60 by Rivera and Abuín [79], we strived for about one-third of the items to be overt/visible NWB. We also formulated two extra questions on visibility. See Section 3.1.6 for this adjustment.

3.1.6. Adjustments

After establishing dimensions with subcategories for the INWBQ, and selecting items for each of these subcategories, we adjusted some of these items. Our first adjustment involved removing double-barreled parts in the selected items. These are confusing to the respondent and cannot be scored separately. For example, we removed ethnic in the item “Used racial or ethnic slurs to describe you” (FSS).
Another adjustment concerned the anonymous nature of digital NWB. Although this anonymous behavior may be visible, we collect it under the visibility pattern of covert NWB. This is because we consider an unknown perpetrator to be covert NWB.
We decided not to ask about construct names in the items (i.e., “Verbally abused someone at work”, CWB-C). A prior explanation may not preclude interpretations by the respondent. Therefore, we selected concrete and objective behaviors, and we deleted all construct names in our reduction process of items. In this line of reasoning, our third adjustment concerned our way of asking about harm with the construct names as depression, burnout, and PTSD.
These inflicted types of harm by NWB have frequently been examined with separate questionnaires, such as the BDI for depression [184] and IES for PTSD [185]. These questionnaires contain questions on symptoms of dimensions within the relevant constructs. Several NWB questionnaires from this study (e.g., violence research) ask about burn-out, depression, or PTSS inflicted by NWB with one single question. We considered it inappropriate for respondents to self-assess their mental illness. Instead, in three items, we ask them about a combination of symptoms of depression, PTSD, and burnout without naming this a mental illness.
More specifically, for the NWB-inflicted harm depression, we ask about a combination of negative mood [21,98,112], loss of interest (SVI), withdrawal [144], feeling worthless (cyber aggression), feeling embarrassed (GWHQ), and sleep problems (WCM). In doing so, we align with the measurement of depression symptoms [184]. For the NWB-inflicted harm burnout, we ask about a combination of emotional exhaustion [21,114], changed work attitudes (e.g., cynical, less professional) [37,71], low self-esteem (JOS), and alienation (SVI). To make respondents understand this last symptom, we reworded alienation in “not as the one I used to be”. For the NWB-inflicted harm PTSD, we ask about a combination of repeated thinking (SVI), avoiding (FSS), dreams and images of past events (cyber aggression), blame myself (cyber harassment), and emotional reactions as anger (cyber aggression) to align with the measurement of PTSD symptoms [185]. To make respondents understand the question, we reworded repeated thinking as flashbacks, and avoiding as avoid places, people, and activities.
The fourth adjustment concerned asking about mixed actor roles in NWB. To this end, we rewrote all NWB items for the most neutral role of witness (changes in items for this role adjustment were underlined). Escartín et al. [186] showed that different degrees of severity can be questioned, regardless of the participant’s role as a target or witness of NWB. This makes the witness role suitable for interrogating NWB frequencies. In addition, this role provides a good starting point to ask about other roles the respondent fulfilled. For this, we ask the respondent what behavior they performed in the situation in question, with multiple options as a possibility to be ticked; for instance, “someone received unreasonable work demands”, first answer: frequency: 1. never–5. daily, second answer: tick all that apply: I looked away, I encouraged it, I openly disapproved it, etc.
For harm, we also asked the respondent about their role as a witness to the complaints experienced by colleagues and the organization inflicted by NWB. Because this method of questioning could not be applied to the symptoms of depression, PTSD, or burnout, we made the only exception here. For these specific types of harm, we ask whether the respondent experienced these combinations of symptoms themselves.
The fifth adjustment concerned the escalation and visibility occurrence patterns. As we found only two items on escalation in the instruments, we formulated two additional questions for this pattern; for example: “Did the behavior escalate from mild into more serious or physical types over time?” As we found one item on visibility, we formulated two extra items for visibility; for example: “Did the negative behavior change from covert into a more overt and visible nature?” Furthermore, we selected one-third overt items for visibility, that is, a total of 25 overt (indicated in italics) and 42 covert items. Although the covert items are observable by many potential witnesses in the work setting, mental elements such as “trivial”, “purposely”, “unfair”, or “necessary” are not observable, and therefore we considered them as covert NWB.
Our final adjustment concerned the formulation for external actor types. In this way, these questions could be answered by external actor types; for instance, we changed “salary” into “payment”.

3.1.7. Expert Panel

We invited scholars with specific expertise of research on NWB, actor roles, and the development of a questionnaire about actor roles in NWB to take part. We approached eight experts, of which five dropped out: two were out of the office, three had too much work in the planned period, and one of them did not react. Finally, three experts in the field provided answers to our questions.
We asked the expert panel if we were missing important instruments and studies, if they would add additional steps to our method, to consider with us the way of asking about actor roles, and their opinion on the results. Their valuable additions in the literature and studies allowed us to add 10 important instruments (see Supplementary Table S1). Some suggested short scales, used in a variety of national cohort studies, but, in the end, these were not included because they contained items from instruments we already included.
Another point of feedback concerned the need to clearly articulate the INWBQ’s practical use in the introduction. How do we want the scale to be used? The purpose of our questionnaire was two-fold: to measure NWB according to the integral definition of NWB and to provide organizations with tools to undertake interventions.
One of the experts pointed out that recreating an essentially longer scale of all NWBs and roles may be a barrier to researchers. If researchers are supposed to pick relevant subscales, why wouldn’t they just pick an established scale such as that in Supplementary Table S1? That is of course always possible, but the overlap between instruments weakens the results of such research. In other words, the question remains whether, for example, withdrawal is only a consequence of incivility, as measured with the WIS [144]. When this instrument overlaps with other construct instruments, these results remain obscure. During categorization into subcategories, we indeed encountered a large amount of item overlap between the instruments. Furthermore, to substantiate the item selection by dimension, we ranked the NWBs by summing the total number of items per subcategory. Items that were found among many instruments received the highest rank and were included in the INWBQ. This substantiation can help researchers when considering their choice of an instrument. These two additions also respond to another expert’s request to make a clearer case for why the INWBQ with this breadth, including digital/online behavior, is necessary.
Our steps in the method to develop data, subcategories, design, and items seemed thorough and clearly delineated to the experts. The empirical and theoretical arguments for this method were found to be adequate. Our rationales were both empirical, using this study, and theoretical based on our review and integrated definition.
Furthermore, the experts really challenged us on our way of asking about the different actor roles. For instance, as each subcategory should have items for each role type, this would create a very long list to complete. It was suggested to lessen the number of roles to perpetrator, victim, and bystander, and include within these roles various types of NWB (e.g., physical, verbal). Another expert rightly pointed out that these roles are not fixed but mixed [187]. For instance, research indicated that more than one-half of the bullies also reported being victimized [188]. Because we were determined to stick to our original definition, we formulated the questions in the witness role. Choosing the witness role prevents respondents from viewing themselves in a favorable light, despite not wanting to disclose active involvement as a bully or exaggerating their involvement as a target [189]. Subsequently, for each item, we also ask about which of the five roles the respondent fulfilled. We question these roles regarding concrete behavior per role.
One expert explained to us that there are also “uninvolved” youth in bullying. Therefore, there is not much difference between witness and outsider because some children say they do not see it [188]. As this could also be the case for adults, we chose two very clear behaviors for the outsider role as “I shut my door”, and “I looked away”. Furthermore, it was unclear what instructions would be presented to the person completing the measure. We included an introduction for the INWBQ.
Another important expert addition was about measuring behaviors and consequences in the same questionnaire. In other fields, questionnaires about only consequences have been created independently. As harm is an essential part of the most frequently cited NWB definitions (see our method for these), we included it in the INWBQ. Moreover, combining NWB and its potential harm can help organizations understand the relationship between the risk and its damage, enabling effective risk management, resource allocation, decision making, and compliance. By prioritizing risks based on their potential impact, organizations can allocate resources more efficiently and comply with legal or regulatory requirements [190].
However, properly measuring harm inflicted by NWB has proven to be difficult in past years. Some scholars suggested that NWB could be characterized by the degree of harm it causes. From the target perspective, severity could be measured by the sum of the number of negative actions [191]. This implies that, as the degree of NWB increases, so do the negative outcomes. Other scholars emphasized that it is not the objective behavior but how victims perceive and evaluate this behavior that determines the degree of harm [185,192]. Therefore, Agervold [193] advocated to identify certain types of behaviors, independent of the target perspective.
One application of the above insights is that we inventory harm with a separate Part II, independent of the NWB types identified. This part does not ask about the harm from the target perspective. We ask the respondent to provide an observation of the impact on their environment. This also allows us to inventory the harm for organizations and society that many studies on NWB underline such as sick leave, damaged reputation, and premature retirement.

3.1.8. INWBQ

In Supplementary Table S4 we present the INWBQ Part I with 69 items on NWBs from existing scales, with in-depth multiple-choice questions about five actor roles and seven items about NWB patterns. Harm is assessed with 11 items (including 50 multiple-choice questions) in Part II of this questionnaire. For the transparency of this development study, in Supplementary Table S4 we indicate the source of each item and the ranking number of this NWB or harm in the subcategory of its dimension. (See Supplementary Table S4 of the Supplementary Materials).

4. Discussion

The purpose of this study was to develop a questionnaire that integrates the content of measurement instruments of 16 most-cited NWB constructs such as bullying, aggression, and discrimination. This was needed since the plurality of different conceptualizations and operationalizations of NWB overlap, threatening their discriminant validity of measurement [194]. In addition, research showed the 83.5% of the different types of NWBs occur simultaneous and sequentially [12], and involve multiple internal and external actor types [15], with more roles than perpetrator and target [195]. Finally, measuring separate NWBs from one source limits our insights into the specific nature of NWB and its inflicted harm to different actor types and roles. It became highly necessary to develop a measurement tool integrating these factors. Our result is the Integrative NWB Questionnaire (INWBQ), which covers five dimensions of NWB in four occurrence patterns, four actor types, six actor roles, and four harms.

4.1. Scientific Implications

The vast majority of current NWB measurement instruments focus on physical, psychological, and material NWB, whereas the measurement of sociocultural and digital dimensions in NWB has remained separate from these dimensions. That is, different measurement instruments exist that look at, for example, gender or digital bullying, whereas these NWBs are likely to overlap or co-occur with each other. By recognizing these characteristics in NWB, we contribute to the literature on NWB by illustrating that NWB is a broad construct in which many aspects overlapping with each other. Focusing on only one of these NWB types will create an incomplete picture of reality in an organization.
Another important theoretical contribution concerns our inclusion of four actor types (internal and external to the organization) and six actor roles (target, perpetrator, assistant, defender, outsider, and witness) in addition to the specific type of NWB. Scholars have been urging for several years that data should not only be collected from employees in the target role [26], whereas the majority of information is still collected from this source. The INWBQ identifies external actors in modern workplaces as part of NWB, such as the public, clients, students, and their relatives. These actors are questioned regarding six mixed roles. This will yield new scientific insights into actor types and their roles in NWB. It also illustrates that the influence of actors is not restricted to people internal to the organization, but that increasingly actors outside of the organization influence NWB inside the organization.
Finally, asking about bodily harm, material damage, and mental and social harm for the above-mentioned four actor types and six actor roles is another important scientific implication. It is known that bystanders in the organization develop complaints [196,197]. However, how this varies by actor role and externality provides new insights and research avenues to better address the current challenges that organizations may face. In other words, people other than the target and perpetrator can experience negative consequences based on observing NWB in an organization. The INWBQ allows researchers to also take them into account.
Taken together, this paper advances a much broader picture of NWB that takes into account the overlap between different NWBs, the different actors and their roles involved in NWB, and the harm involved, which together will hopefully facilitate more knowledge that can stimulate interventions to prevent NWB.

4.2. Practical Implications

The developed INWBQ consists of two parts. Part I investigates four actor types in six actor roles on physical, material, psychological, sociocultural, and digital NWB. Part II investigates the inflicted bodily harm, material damage, and mental and social harm of these actor types and roles.
For Part I, we recommend to include all modules in organizational research. Individual modules from Part I of the INWBQ do not give a complete picture of an organizational NWB problem, as the probability of combinations of NWB is 83.5% [12]. Taking all modules together will bring these combinations into focus.
If necessary, usage of the separate dimensions of Part I is possible in different research contexts, such as the LWMS [146] for psychological research. For national surveys, such as the GHS [147], separate dimensions are less suitable and further research could develop an abbreviated INWBQ considering the individual types of NWB.
Likewise, we do not recommend using individual modules of Part II of the INWBQ. Porath and Pearson [37] did an excellent job of describing how one type of harm, in time, causes other types. Various health and mental complaints turn into persistent complaints that negatively affect the production process and the reputation of the organization. This also causes material damage through, for example, increased staff turnover and intervention costs.
It is, of course, possible to take Part II separately from Part I when only a picture of the harm and damage is needed and the NWB is known. This can provide additional information for other methods that map harm, such as sick leave analyses, customer complaints, or turnover rates.
Both parts of the INWBQ extract much more information that is valid for organizational consultants, psychologists, and scientists to work with. Part 1 unravels NWBs in different roles and provides new insights to implement specific interventions for bystanders; for example, to increase positive behavior such as defending a target while reducing NWB by assisting a perpetrator. Part II, with the analysis of four types of harm, provides new insights into organizational risks, total costs, and care required. This provides opportunities for system-based interventions involving multiple stakeholders.

4.3. Limitations and Future Research

We encountered several limitations in this study. First, in the questionnaires, we could only draw on one example from Demaray [74] on bystander research. This questionnaire did not cover changing roles, as required by our starting point. Moreover, it was not about workers but about youth. Longitudinal research is needed to further investigate changing roles of workers with the INWBQ.
Another limitation of this study is that we could not build on measuring some types of harm from existing NWB instruments (33 Bodily harms, 18 Material damage, 107 Mental harms, 7 Social harms). We added these from additional studies on harm of NWB. As a consequence, we assume, but do not know, whether, for instance, sleep difficulties caused by bullying [125] are also caused by other types of NBW. Integrally including harm in the measurement of various types of NWB, as with the INWBQ, may provide new ways for future research.
A limitation is also that our questionnaire has 76 items in Part I and 11 (50 multiple-choice) items in Part II, creating the risk of non-response. Therefore, attention to procedures to increase the response is necessary. In general, web survey design requires extra attention to procedures to manage the risk of non-response, coverage, and sampling errors [198]. On the other hand, this Internet context provides additional opportunities to make the survey attractive, such as providing context and incremental sampling to reduce dropouts [199]. By grouping the harm items we reduced the time taken to complete the survey, which influences the response rate [200]. Furthermore, we suggest stepwise administration of the questionnaire by sending it in parts to the respondent over a period.
Finally, a limitation was the exclusion of coping because this behavior was not within our baseline definition. Nevertheless, in the NWB instruments we found several items regarding 7 different coping styles: active problem solving (8 items), social support seeking (8 items), avoidance behavior (8 items), palliative reaction (7 items), depressive reaction (11 items), expression of emotions (10 items), and comforting cognitions (1 item) (coping styles after Schreurs et al. [201]). Recent research suggests that these coping styles may mitigate harm, but do not include an effective style for de-escalation of NWB [202]. Therefore, for the inclusion of coping styles within NWB instruments, we believe future research could focus on inclusion of de-escalating styles such as appreciating, reducing anxiety, providing guidance, and maintaining appropriate distance from a person, as measured with the EDABS [151].
Despite these limitations, we believe we have taken a substantial step forward with the INWBQ to provide a complete picture of the facets involved in NWB.

5. Conclusions

To the best of our knowledge, this study presents the first attempt to develop an integrated NWB measurement. We succeeded in shaping the content of the INWBQ from 44 NWB measurement instruments and 48 additional studies. With this method we wanted to provide optimal content validity. In this paper we provide a transparent description of the ways we investigated data, built subcategories, and selected and adjusted items.
We developed a full-spectrum diagnostic NWB tool because current tools have shortcomings. They mainly measure one source, i.e., organizational actors in the role of the target. Furthermore, the analyzed instruments show overlapping content, threatening discriminant validity. They do not measure escalation as a key element of all constructs. Meanwhile, new scientific insights for interventions call for measuring different co-occurring types of NWBs, analyzing the dynamics between internal and external actors, and mapping the mixed roles of perpetrator, target, and bystanders.
Although further analytical research on refinement and validation is needed, the INWBQ fills an important research gap relating to these shortcomings. Specifically, Part I allows investigation of four actor types in six actor roles in terms of physical, material, psychological, sociocultural, and digital NWB. In addition, Part II offers insight for organizations about the inflicted bodily harm, material damage, and mental and social harm of these actor types and roles.

Supplementary Materials

The following Supplementary materials can be downloaded at: Supplementary Table S1: 44 Construct instruments 16 studies on digital NWB, and 32 studies on harm; Supplementary Table S2: Dimension Material NWB, Subcategories Property and Displays, Ranking; Supplementary Table S3: Dimension Mental Harm, Subcategory Temporarily Mental Complaints, Ranking; Supplementary Table S4: INWBQ [203,204].

Author Contributions

C.V.: study conceptualization, data collection, and drafting of the manuscript. M.T.: continuous feed-back, critical revisions during drafting, supervision, and substantial revision suggestions. A.H.D.L.: final editing and substantial revisions suggestions. M.T. and A.H.D.L.: independent rating of the categorization of a random sample of twenty items in dimensions and nine items in subcategories. All authors have read and agreed to the published version of the manuscript.


The authors received no external funding for the submitted work.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author/s.


We are very grateful for the contribution the three experts (Kara Ng, Organisational Psychology, Alliance Manchester Business School, The University of Manchester, Manchester, UK; Michelle Demaray, School Psychology Program, Psychology Department, Northern Illinois University, DeKalb, Illinois, USA; and Jorge Escartín Solanelles, Department of Social Psychology, Universitat de Barcelona, Barcelona, Spain; Organisational Psychology, Institute of Psychiatry, Psychology and Neuroscience, King´s College London, London, UK), made to our study. They provided us with outstanding insights and additional sources.

Conflicts of Interest

The authors declare no conflict of interest.


  1. Codina, M.; Pereda, N.; Guilera, G. Lifetime Victimization and Poly-Victimization in a Sample of Adults with Intellectual Disabilities. J. Interpers. Violence 2020, 37, 2062–2082. [Google Scholar] [CrossRef] [PubMed]
  2. Rosta, J.; Aasland, O.G. Perceived Bullying among Norwegian Doctors in 1993, 2004 and 2014–2015: A Study Based on Cross-Sectional and Repeated Surveys. BMJ Open 2018, 8, e018161. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Quigley, E.; Ruggiero, J.; McEvoy, T.; Hollister, S.; Schwoebel, A.; Inacker, P. Addressing Workplace Violence and Aggression in Health Care Settings: One Unit’s Journey. Nurse Lead. 2020, in press. [Google Scholar]
  4. Tabor, J.; Griep, Y.; Collins, R.; Mychasiuk, R. Investigating the Neurological Correlates of Workplace Deviance Using a Rodent Model of Extinction. Sci. Rep. 2018, 8, 17316. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Einarsen, S.; Hoel, H.; Zapf, D.; Cooper, C.L. Bullying and Harassment in the Workplace: Developments in Theory, Research, and Practice, 2nd ed.; CRC Press: Boca Raton, NY, USA; London, UK, 2011. [Google Scholar]
  6. Gruys, M.L.; Sackett, P.R. Investigating the Dimensionality of Counterproductive Work Behavior. Int. J. Sel. Assess. 2003, 11, 30–42. [Google Scholar] [CrossRef]
  7. O’Connor, M.; Gutek, B.A.; Stockdale, M.; Geer, T.M.; Melançon, R. Explaining Sexual Harassment Judgments: Looking beyond Gender of the Rater. Law Hum. Behav. 2004, 28, 69–95. [Google Scholar] [CrossRef]
  8. Richman, J.A.; Rospenda, K.M.; Nawyn, S.J.; Flaherty, J.A.; Fendrich, M.; Drum, M.L.; Johnson, T.P. Sexual Harassment and Generalized Workplace Abuse among University Employees: Prevalence and Mental Health Correlates. Am. J. Public Health 1999, 89, 358–363. [Google Scholar] [CrossRef] [Green Version]
  9. Rospenda, K.M.; Richman, J.A.; Shannon, C.A. Prevalence and Mental Health Correlates of Harassment and Discrimination in the Workplace. J. Interpers. Violence 2009, 24, 819–843. [Google Scholar] [CrossRef] [Green Version]
  10. Holroyd-Leduc, J.M.; Straus, S.E. #MeToo and the Medical Profession. Can. Med. Assoc. J. 2018, 190, E972–E973. [Google Scholar] [CrossRef] [Green Version]
  11. Black, S.M.; Creese, S.; Guest, R.M.; Pike, B.; Saxby, S.J.; Fraser, D.S.; Stevenage, S.V.; Whitty, M.T. Superidentity: Fusion of Identity across Real and Cyber Domains. In Global Forum on SuperIdentity; University of Southampton: Hampshire, UK, 2012; pp. 1–10. [Google Scholar]
  12. Privitera, C.; Campbell, M. Cyberbullying: The New Face of Workplace Bullying? Cyber Psychol. Behav. 2009, 12, 395–400. [Google Scholar] [CrossRef] [PubMed]
  13. Einarsen, S.; Hoel, H.; Notelaers, G. Measuring Exposure to Bullying and Harassment at Work: Validity, Factor Structure and Psychometric Properties of the Negative Acts Questionnaire-Revised. Work Stress 2009, 23, 24–44. [Google Scholar] [CrossRef]
  14. Ruiz-Hernández, J.A.; López-García, C.; Llor-Esteban, B.; Galián-Muñoz, I.; Benavente-Reche, A.P. Evaluation of the Users Violence in Primary Health Care: Adaptation of an Instrument. Int. J. Clin. Health Psychol. 2016, 16, 295–305. [Google Scholar] [CrossRef] [Green Version]
  15. Merchant, J.A.; Lundell, J.A. Workplace Violence Intervention Research Workshop, April 5–7, Washington, DC. Am. J. Prev. Med. 2001, 20, 135–140. [Google Scholar] [CrossRef] [PubMed]
  16. Cheung, T.; Lee, P.H.; Yip, P.S.F. The Association between Workplace Violence and Physicians’ and Nurses’ Job Satisfaction in Macau. PLoS ONE 2018, 13, e0207577. [Google Scholar] [CrossRef] [PubMed]
  17. Cele, B. Addendum to the Saps Annual Report; South African Police Service: Pretoria, South Africa, 2018. [Google Scholar]
  18. DeSouza, E.R. Frequency Rates and Correlates of Contrapower Harassment in Higher Education. J. Interpers. Violence 2011, 26, 158–188. [Google Scholar] [CrossRef] [PubMed]
  19. Cannavò, M.; La Torre, F.; Sestili, C.; La Torre, G.; Fioravanti, M. Work Related Violence as a Predictor of Stress and Correlated Disorders in Emergency Department Healthcare Professionals. Clin. Ter. 2019, 170, E110–E123. [Google Scholar] [CrossRef] [PubMed]
  20. Levine, M.; Taylor, P.J.; Best, R. Third Parties, Violence, and Conflict Resolution. Psychol. Sci. 2011, 22, 406–412. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  21. Verkuil, B.; Atasayi, S.; Molendijk, M.L. Workplace Bullying and Mental Health: A Meta- Analysis on Cross-Sectional and Longitudinal Data. PLoS ONE 2015, 10, e0135225. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  22. Heerde, J.A.; Hemphill, S.A. Are Bullying Perpetration and Victimization Associated with Adolescent Deliberate Self-Harm? A Meta-Analysis. Arch. Suicide Res. 2019, 23, 353–381. [Google Scholar] [CrossRef]
  23. Sprigg, C.A.; Niven, K.; Dawson, J.; Farley, S.; Armitage, C.J. Witnessing Workplace Bullying and Employee Well-Being: A Two-Wave Field Study. J. Occup. Health Psychol. 2019, 24, 286–296. [Google Scholar] [CrossRef] [Green Version]
  24. Porath, C.L.; Pearson, C.M. Emotional and Behavioral Responses to Workplace Incivility and the Impact of Hierarchical Status. J. Appl. Soc. Psychol. 2012, 42 (Suppl. S1), E326–E357. [Google Scholar] [CrossRef]
  25. Nielsen, M.; Emberland, J.S.; Knardahl, S. Workplace Bullying as a Predictor of Disability Retirement. J. Occup. Environ. Med. 2017, 59, 609–614. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. Neall, A.M.; Tuckey, M.R. A Methodological Review of Research on the Antecedents and Consequences of Workplace Harassment. J. Occup. Organ. Psychol. 2014, 87, 225–257. [Google Scholar] [CrossRef]
  27. Campo, V.R.; Klijn, T.P. Verbal Abuse and Mobbing in Pre-Hospital Care Services in Chile. Rev. Lat. Am. Enferm. 2018, 25, e2956. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  28. Fredericksen, E.D.; McCorkle, S. Explaining Organizational Responses to Workplace Aggression. Public Pers. Manag. 2013, 42, 223–238. [Google Scholar] [CrossRef]
  29. Verschuren, C.M.; Tims, M.; de Lange, A.H. A Systematic Review of Negative Work Behavior: Toward an Integrated Definition. Front. Psychol. 2021, 12, 21. [Google Scholar] [CrossRef]
  30. Bayramoğlu, M.M.; Toksoy, D. Leadership and Bullying in the Forestry Organization of Turkey. Biomed Res. Int. 2017, 2017, 1–11. [Google Scholar] [CrossRef]
  31. Bennett, R.J.; Robinson, S.L. Development of a Measure of Workplace Deviance. J. Appl. Psychol. 2000, 85, 349–360. [Google Scholar] [CrossRef] [Green Version]
  32. Marcus, B.; Schuler, H.; Quell, P.; Humpfner, G. Measuring Counterproductivity: Development and Initial Validation of a German Self-Report Questionnaire. Int. J. Sel. Assess. 2002, 10, 18–35. [Google Scholar] [CrossRef]
  33. Nielsen, M.B.; Knardahl, S. Is Workplace Bullying Related to the Personality Traits of Victims? A Two-Year Prospective Study. Work Stress 2015, 29, 128–149. [Google Scholar] [CrossRef]
  34. Kaukiainen, A.; Salmivalli, C.; Björkqvist, K.; Österman, K.; Lahtinen, A.; Kostamo, A.; Lagerspetz, K. Overt and Covert Aggression in Work Settings in Relation to the Subjective Well-being of Employees. Aggress. Behav. 2001, 27, 360–371. [Google Scholar] [CrossRef]
  35. Thomas, S.P.; Burk, R. Junior Nursing Students’ Experiences of Vertical Violence during Clinical Rotations. Nurs. Outlook 2009, 57, 226–231. [Google Scholar] [CrossRef] [PubMed]
  36. Jacob, L.; Kostev, K. Conflicts at Work Are Associated with a Higher Risk of Cardiovascular Disease. GMS Ger. Med. Sci. 2017, 15, 1–8. [Google Scholar] [CrossRef]
  37. Porath, C.L.; Pearson, C.M. The Cost of Bad Behavior. Organ. Dyn. 2010, 39, 64–71. [Google Scholar] [CrossRef]
  38. Baran Tatar, Z.; Yuksel, S. Mobbing at Workplace Psychological Trauma and Documentation of Psychiatric Symptoms. Arch. Neuropsychiatry 2018, 56, 57–62. [Google Scholar] [CrossRef] [PubMed]
  39. Hoobler, J.M.; Brass, D.J. Abusive Supervision and Family Undermining as Displaced Aggression. J. Appl. Psychol. 2006, 91, 1125–1133. [Google Scholar] [CrossRef]
  40. Nagle, J. Twitter, Cyber-Violence, and the Need for a Critical Social Media Literacy in Teacher Education: A Review of the Literature. Teach. Teach. Educ. 2018, 76, 86–94. [Google Scholar] [CrossRef]
  41. Scholte, R.; Nelen, W.; De Wit, W.; Kroes, G. Sociale Veiligheid in en Rond Scholen [Social Safety in and around Schools] 2010–2016; Praktikon, B.V., Ed.; Commissioned by the Ministry of Education, Culture and Science (OCW) Netherlands: Nijmegen, The Netherlands, 2016. [Google Scholar]
  42. Baron, R.A.; Neuman, J.H. Workplace Violence and Workplace Aggression: Evidence on Their Relative Frequency and Potential Causes. Aggress. Behav. 1996, 22, 161–173. [Google Scholar] [CrossRef]
  43. Einarsen, S.V.; Hoel, H.; Zapf, D.; Cooper, C.L. The Concept of Bullying and Harassment at Work: The European Tradition. In Bullying and Harassment in the Workplace: Theory, Research and Practice; Einarsen, S.V., Hoel, H., Zapf, D., Cooper, C.L., Eds.; CRC Press, Taylor & Francis Group: Boca Raton, NY, USA, 2020; pp. 3–41. [Google Scholar]
  44. Einarsen, S. Harassment and Bullying at Work. Aggress. Violent Behav. 2000, 5, 379–401. [Google Scholar] [CrossRef]
  45. Bowling, N.A.; Beehr, T.A. Workplace Harassment from the Victim’s Perspective: A Theoretical Model and Meta-Analysis. J. Appl. Psychol. 2006, 91, 998–1012. [Google Scholar] [CrossRef]
  46. Robinson, S.L.; Bennett, R.J. A Typology of Deviant Workplace Behaviors: A Multidimensional Scaling Study. Acad. Manag. J. 1995, 38, 555–572. [Google Scholar] [CrossRef]
  47. Spector, P.E.; Fox, S. The Stressor-Emotion Model of Counterproductive Work Behavior. In Counterproductive Work Behavior: Investigations of Actors and Targets; Fox, S., Spector, P.E., Eds.; American Psychological Association: Washington, DC, USA, 2005; pp. 151–174. [Google Scholar] [CrossRef]
  48. NIOSH. Occupational Violence. Available online: (accessed on 10 January 2023).
  49. Keashly, L. Interpersonal and Systemic Aspects of Emotional Abuse at Work: The Target’s Perspective. Violence Vict. 2001, 16, 233–268. [Google Scholar] [CrossRef]
  50. Leymann, H. Mobbing and Psychological Terror at Workplaces. Violence Vict. 1990, 5, 119–126. [Google Scholar] [CrossRef] [PubMed]
  51. Colquitt, J.A.; Rodell, J.B. Measuring Justice and Fairness. In The Oxford Handbook of Justice in the Workplace; Cropanzano, R.S., Ambrose, M.L., Eds.; Oxford University Press: London, UK, 2015; pp. 1–30. [Google Scholar] [CrossRef] [Green Version]
  52. Adams, J.S. Inequity in Social Exchange. In Advances in Experimental Social Psychology; Behavioral Research Service General Electric Company Crotonville: New York, NY, USA, 1965; Volume 2, pp. 267–299. [Google Scholar] [CrossRef]
  53. Cropanzano, R.; Bowen, D.E.; Gilliland, S.W. The Management of Organizational Justice. Acad. Manag. Perspect. 2007, 21, 34–48. [Google Scholar] [CrossRef]
  54. Spector, P.E.; Jex, S.M. Development of Four Self-Report Measures of Job Stressors and Strain: Interpersonal Conflict at Work Scale, Organizational Constraints Scale, Quantitative Workload Inventory, and Physical Symptoms Inventory. J. Occup. Health Psychol. 1998, 3, 356–367. [Google Scholar] [CrossRef] [PubMed]
  55. Aquino, K.; Lamertz, K. A Relational Model of Workplace Victimization: Social Roles and Patterns of Victimization in Dyadic Relationships. J. Appl. Psychol. 2004, 89, 1023–1034. [Google Scholar] [CrossRef] [PubMed]
  56. Glick, P. Choice of Scapegoats. In On the Nature of Prejudice: Fifty Years after Allport; Dovidio, J.F., Glick, P., Rudman, L., Eds.; Blackwell Publishing: Hoboken, NJ, USA, 2005; pp. 244–261. [Google Scholar] [CrossRef]
  57. Poon, J.M.L. Situational Antecedents and Outcomes of Organizational Politics Perceptions. J. Manag. Psychol. 2003, 18, 138–155. [Google Scholar] [CrossRef]
  58. Hitlan, R.T.; Cliffton, R.J.; DeSoto, C.M. Perceived Exclusion in the Workplace: The Moderating Effects of Gender on Workrelated Attitudes and Psychological Health. N. Am. J. Psychol. 2006, 8, 217–236. [Google Scholar]
  59. Andersson, L.M.; Pearson, C.M. Tit for Tat? The Spiraling Effect of Incivility in the Workplace. Acad. Manag. Rev. 1999, 24, 452–471. [Google Scholar] [CrossRef]
  60. Barboza, G.E.; Schiamberg, L.B.; Oehmke, J.; Korzeniewski, S.J.; Post, L.A.; Heraux, C.G. Individual Characteristics and the Multiple Contexts of Adolescent Bullying: An Ecological Perspective. J. Youth Adolesc. 2009, 38, 101–121. [Google Scholar] [CrossRef]
  61. Ferrer, B.M.; Ruiz, D.M.; Amador, L.V.; Orford, J. School Victimization among Adolescents. An Analysis from an Ecological Perspective. Psychosoc. Interv. 2011, 20, 149–160. [Google Scholar] [CrossRef]
  62. Carrera, M.V.; DePalma, R.; Lameiras, M. Toward a More Comprehensive Understanding of Bullying in School Settings. Educ. Psychol. Rev. 2011, 23, 479–499. [Google Scholar] [CrossRef]
  63. Broersen, A.; Ossenblok, A.; Montesano Montessori, N. Pesten En Sociale Veiligheid Op Scholen: Definities En Keuzes Op Micro-, Meso- En Macroniveau [Bullying and Social Safety in Schools: Definitions and Choices at Micro, Meso and Macro Level]. In Tijdschrift voor Orthopedagogiek; Instondo: Dordrecht, The Netherlands, 2015; Volume 54, pp. 108–119. [Google Scholar]
  64. Baron, R.A.; Neuman, J.H.; Geddes, D. Social and Personal Determinants of Workplace Aggression: Evidence for the Impact of Perceived Injustice and the Type A Behavior Pattern. Aggress. Behav. 1999, 25, 281–296. [Google Scholar] [CrossRef]
  65. Forrest, S.; Eatough, V.; Shevlin, M. Measuring Adult Indirect Aggression: The Development and Psychometric Assessment of the Indirect Aggression Scales. Aggress. Behav. 2005, 31, 84–97. [Google Scholar] [CrossRef]
  66. Fox, S.; Stallworth, L.E. Racial/Ethnic Bullying: Exploring Links between Bullying and Racism in the US Workplace. J. Vocat. Behav. 2005, 66, 438–456. [Google Scholar] [CrossRef]
  67. Weatherbee, T.G. Cyberaggression in the Workplace: Construct Development, Operationalization, and Measurement. Master’s Thesis, Saint Mary’s University, Halifax, NS, Canada, December 2007. [Google Scholar]
  68. Waschgler, K.; Ruiz-Hernández, J.A.; Llor-Esteban, B.; Jiménez-Barbero, J.A. Vertical and Lateral Workplace Bullying in Nursing. J. Interpers. Violence 2013, 28, 2389–2412. [Google Scholar] [CrossRef]
  69. Jóhannsdóttir, H.L.; Ólafsson, R.F. Coping with Bullying in the Workplace: The Effect of Gender, Age and Type of Bullying. Br. J. Guid. Counc. 2004, 32, 319–333. [Google Scholar] [CrossRef]
  70. Lee, R.T.; Brotheridge, C.M. When Prey Turns Predatory: Workplace Bullying as a Predictor of Counteraggression/Bullying, Coping, and Well-Being. Eur. J. Work Organ. Psychol. 2006, 15, 352–377. [Google Scholar] [CrossRef]
  71. Escartín, J.; Rodríguez-Carballeira, Á.; Gómez-Benito, J.; Zapf, D. Development and Validation of the Workplace Bullying Scale EAPA-T. Int. J. Clin. Health Psychol. 2010, 10, 519–539. [Google Scholar]
  72. Patchin, J.W.; Hinduja, S. Measuring Cyberbullying: Implications for Research. Aggress. Violent Behav. 2015, 23, 69–74. [Google Scholar] [CrossRef]
  73. Farley, S.; Coyne, I.; Axtell, C.; Sprigg, C. Design, Development and Validation of a Workplace Cyberbullying Measure, the WCM. Work Stress 2016, 30, 293–317. [Google Scholar] [CrossRef]
  74. Demaray, M.K.; Summers, K.H.; Jenkins, L.N.; Becker, L.D. Bullying Participant Behaviors Questionnaire (BPBQ): Establishing a Reliable and Valid Measure. J. Sch. Violence 2016, 15, 158–188. [Google Scholar] [CrossRef]
  75. Jönsson, S.; Muhonen, T.; Forssell, R.C.; Bäckström, M. Assessing Exposure to Bullying through Digital Devices in Working Life: Two Versions of a Cyberbullying Questionnaire (CBQ). Psychology 2017, 8, 477–494. [Google Scholar] [CrossRef] [Green Version]
  76. Vranjes, I.; Baillien, E.; Vandebosch, H.; Erreygers, S.; De Witte, H. When Workplace Bullying Goes Online: Construction and Validation of the Inventory of Cyberbullying Acts at Work (ICA-W). Eur. J. Work Organ. Psychol. 2018, 27, 28–39. [Google Scholar] [CrossRef]
  77. Leymann, H. The Content and Development of Mobbing at Work. Eur. J. Work Organ. Psychol. 1996, 5, 165–184. [Google Scholar] [CrossRef]
  78. Hubert, A.B.; Furda, J. Leidse Mobbing Schaal-II (LEMS-II; Geïndividualiseerde Versie (Leiden Mobbing Scale-II (LEMS-II Individualised Version); Leiden University: Leiden, The Netherlands, 1996. [Google Scholar]
  79. Rivera, J.; Abuín, M. Cuestionario de Estrategias de Acoso En El Trabajo: El LIPT-60 En Version Española [Questionnaire of Etrategies of Harrassment at the Workplace: The LIPT-60 Spanisch Version). Psiquis 2003, 24, 59–69. [Google Scholar]
  80. Björkqvist, K.; Österman, K.; Hjelt-Bäck, M. Aggression among University Employees. Aggress. Behav. 1994, 20, 173–184. [Google Scholar] [CrossRef]
  81. Schneider, K.T.; Hitlan, R.T.; Radhakrishnan, P. An Examination of the Nature and Correlates of Ethnic Harassment Experiences in Multiple Contexts. J. Appl. Psychol. 2000, 85, 3–12. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  82. Rospenda, K.M.; Richman, J.A. The Factor Structure of Generalized Workplace Harassment. Violence Vict. 2004, 19, 221–238. [Google Scholar] [CrossRef]
  83. Beran, T.; Li, Q. Cyber-Harassment: A Study of a New Method for an Old Behavior. J. Educ. Comput. Res. 2005, 32, 265–277. [Google Scholar] [CrossRef] [Green Version]
  84. Leskinen, E.A.; Cortina, L.M. Dimensions of Disrespect: Mapping and Measuring Gender Harassment in Organizations. Psychol. Women Q. 2014, 38, 107–123. [Google Scholar] [CrossRef]
  85. Lim, V.K. The IT Way of Loafing on the Job: Cyberloafing, Neutralizing and Organizational Justice. J. Organ. Behav. 2002, 23, 675–694. [Google Scholar] [CrossRef]
  86. Weatherbee, T.G.; Kelloway, E. A Case of Cyberdeviancy: Cyberaggression in the Workplace. In Handbook of Workplace Violence; Kelloway, K.E., Barling, J., Hurrell, J., Eds.; SAGE Publications, Inc.: Thousand Oaks, CA, USA, 2006; pp. 445–488. [Google Scholar] [CrossRef]
  87. Landers, R.N.; Callan, R.C. Validation of the Beneficial and Harmful Work-Related Social Media Behavioral Taxonomies. Soc. Sci. Comput. Rev. 2014, 32, 628–646. [Google Scholar] [CrossRef]
  88. Spector, P.E.; Fox, S.; Penney, L.M.; Bruursema, K.; Goh, A.; Kessler, S. The Dimensionality of Counterproductivity: Are All Counterproductive Behaviors Created Equal? J. Vocat. Behav. 2006, 68, 446–460. [Google Scholar] [CrossRef]
  89. Di Martino, V. Workplace Violence in the Health Sector Country Case Studies Brazil, Bulgaria, Lebanon, Portugal, South Africa, Thailand and an Additional Australian Study. Synthesis Report; ILO/ICN/WHO/PSI Joint Programme on Workplace Violence in the Health Sector: Geneva, Switzerland, 2009. [Google Scholar]
  90. Tepper, B.J. Consequences of Abusive Supervision. Acad. Manag. J. 2000, 43, 178–190. [Google Scholar] [CrossRef]
  91. Every-Palmer, S.; Barry-Walsh, J.; Pathé, M. Harassment, Stalking, Threats and Attacks Targeting New Zealand Politicians: A Mental Health Issue. Aust. N. Z. J. Psychiatry 2015, 49, 634–641. [Google Scholar] [CrossRef] [Green Version]
  92. James, D.V.; Farnham, F.R.; Wilson, S.P. The Fixated Threat Assessment Centre. In International Handbook of Threat Assessmen; Meloy, J., Hoffmann, J., Eds.; Oxford University Press: New York, NY, USA, 2013; pp. 299–320. [Google Scholar]
  93. Pathé, M.; Phillips, J.; Perdacher, E.; Heffernan, E. The Harassment of Queensland Members of Parliament: A Mental Health Concern. Psychiatry Psychol. Law 2014, 21, 577–584. [Google Scholar] [CrossRef]
  94. Khattak, M.N.; Zolin, R.; Muhammad, N. The Combined Effect of Perceived Organizational Injustice and Perceived Politics on Deviant Behaviors. Int. J. Confl. Manag. 2021, 32, 62–87. [Google Scholar] [CrossRef]
  95. Keashly, L.; Trott, V.; MacLean, L.M. Abusive Behavior In the Workplace: A Preliminary Investigation. Violence Vict. 1994, 9, 341–357. [Google Scholar] [CrossRef]
  96. Aquino, K.; Bradfield, M. Perceived Victimization in the Workplace: The Role of Situational Factors and Victim Characteristics. Organ. Sci. 2000, 11, 525–537. [Google Scholar] [CrossRef]
  97. Hamby, S.L.; Finkelhor, D.; Ormrod, R.; Turner, H.; Hamby, S.L.; Finkelhor, D.; Ormrod, R.; Turner, H. The Juvenile Victimization Questionnaire (JVQ): Administration and Scoring Maual; Crimes Against Children Research Centre: Durham, NH, USA, 2004. [Google Scholar]
  98. Reyns, B.W.; Henson, B.; Fisher, B.S. Stalking in the Twilight Zone: Extent of Cyberstalking Victimization and Offending among College Students. Deviant Behav. 2012, 33, 1–25. [Google Scholar] [CrossRef]
  99. Kacmar, K.M.; Carlson, D.S. Further Validation of the Perceptions of Politics Scale (POPS): A Multiple Sample Investigation. J. Manag. 1997, 23, 627–658. [Google Scholar] [CrossRef]
  100. Ferris, D.L.; Brown, D.J.; Berry, J.W.; Lian, H. The Development and Validation of the Workplace Ostracism Scale. J. Appl. Psychol. 2008, 93, 1348–1366. [Google Scholar] [CrossRef] [PubMed]
  101. Cortina, L.M.; Magley, V.J.; Williams, J.H.; Langhout, R.D. Incivility in the Workplace: Incidence and Impact. J. Occup. Health Psychol. 2001, 6, 64–80. [Google Scholar] [CrossRef]
  102. Martin, R.J.; Hine, D.W. Development and Validation of the Uncivil Workplace Behavior Questionnaire. J. Occup. Health Psychol. 2005, 10, 477–490. [Google Scholar] [CrossRef]
  103. Lim, V.K.G.; Teo, T.S.H. Mind Your E-Manners: Impact of Cyber Incivility on Employees’ Work Attitude and Behavior. Inf. Manag. 2009, 46, 419–425. [Google Scholar] [CrossRef]
  104. Verschuren, C.M. Handleiding Sociale Veiligheidsindex: Sociale Veiligheid Binnen de Werkomgeving (Manual Social Safetyindex: Social Safety in the Working Environment), 2nd ed.; Kerckebosch: Zeist, The Netherlands, 2012. [Google Scholar]
  105. Sijbers, R.; De Wit, W.; Fettelaar, D.; Mooij, T. Sociale Veiligheid in En Rond Scholen. Primair (Speciaal) Onderwijs 2010–2014; Voortgezet (Speciaal) Onderwijs 2006-2014. [Social Safety in and around Schools. Primary (Special) Education 2010–2014; Secundairy (Special) Education 2006–2014]; ITS, Radboud Universiteit: Nijmegen, The Netherlands, 2014. [Google Scholar]
  106. Furnell, S. Cybercrime: Vandalizing the Information Society; Addison-Wesley: London, UK, 2002. [Google Scholar]
  107. Giga, S.I.; Hoel, H.; Lewis, D. The Costs of Workplace Bullying; Department for Business, Enterprise and Regulatory Reform: London, UK, 2008. [Google Scholar]
  108. Gofin, R.; Avitzour, M. Traditional Versus Internet Bullying in Junior High School Students. Matern. Child Health J. 2012, 16, 1625–1635. [Google Scholar] [CrossRef]
  109. Fawzi, N. Cyber-Mobbing: Ursachen und Auswirkungen von Mobbing im Internet (Cyber-Mobbing: Causes and Effects of Mobbing on the Internet), Band 37; Beck, K., Höflich, J., Kamps, K., Krotz, F., Schweiger, W., Wirth, W.B., Eds.; Edition Reinhard Fischer: Baden-Baden, Germany, 2009. [Google Scholar]
  110. Da Silva João, A.L.; Saldanha Portelada, A.F. Mobbing and Its Impact on Interpersonal Relationships at the Workplace. J. Interpers. Violence 2019, 34, 2797–2812. [Google Scholar] [CrossRef]
  111. West, S.M.; Crawford, K.; Whittaker, M. Discriminationg Systems: Gender, Race, and Power in AI; AI Now Institute: New York, NY, USA, 2019. [Google Scholar]
  112. Trembly, A.C. E-Mails Nail Wrongdoers. Natl. Underwrit. P&C 2004, 108, 5. [Google Scholar] [CrossRef]
  113. Neese, W.T.; Ferrell, L.; Ferrell, O.C. An Analysis of Federal Mail and Wire Fraud Cases Related to Marketing. J. Bus. Res. 2005, 58, 910–918. [Google Scholar] [CrossRef]
  114. Henry, N.; Powell, A. Sexual Violence in the Digital Age. Soc. Leg. Stud. 2016, 25, 397–418. [Google Scholar] [CrossRef]
  115. Stafford, T.F.; Urbaczewski, A. Spyware: The Ghost in the Machine. Commun. Assoc. Inf. Syst. 2004, 14, 291–306. [Google Scholar] [CrossRef]
  116. Caplan, S.E. Relations among Loneliness, Social Anxiety, and Problematic Internet Use. CyberPsychology Behav. 2007, 10, 234–242. [Google Scholar] [CrossRef]
  117. Kavitha, S.; Bhuvaneswari, R. Impact of Social Media on Millenials- a Conceptual Study. J. Manag. Sci. Technol. 2016, 4, 80–86. [Google Scholar]
  118. Fung, A.; Russon Gilman, H.; Shkabatur, J. Six Models for the Internet + Politics. Int. Stud. Rev. 2013, 15, 30–47. [Google Scholar] [CrossRef]
  119. Zadro, L.; Williams, K.D.; Richardson, R. How Low Can You Go? Ostracism by a Computer Is Sufficient to Lower Self-Reported Levels of Belonging, Control, Self-Esteem, and Meaningful Existence. J. Exp. Soc. Psychol. 2004, 40, 560–567. [Google Scholar] [CrossRef]
  120. Park, Y.; Fritz, C.; Jex, S. Daily Cyber Incivility and Distress: The Moderating Roles of Resources at Work and Home. J. Manag. 2018, 44, 2535–2557. [Google Scholar] [CrossRef]
  121. Henrichsen, J.R.; Betz, M.; Lisosky, J.M. Building Digital Safety for Journalism: A Survey of Selected Issues; UNESCO Publishing: Paris, France, 2015. [Google Scholar]
  122. Kivimaki, M.; Virtanen, M.; Vartia, M.; Elovainio, M.; Vahtera, J.; Keltikangas-Järvinen, L. Workplace Bullying and the Risk of Cardiovascular Disease and Depression. Occup. Environ. Med. 2003, 60, 779–783. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  123. Hogh, A.; Hansen, Å.M.; Mikkelsen, E.G.; Persson, R. Exposure to Negative Acts at Work, Psychological Stress Reactions and Physiological Stress Response. J. Psychosom. Res. 2012, 73, 47–52. [Google Scholar] [CrossRef] [PubMed]
  124. Emdad, R.; Alipour, A.; Hagberg, J.; Jensen, I.B. The Impact of Bystanding to Workplace Bullying on Symptoms of Depression among Women and Men in Industry in Sweden: An Empirical and Theoretical Longitudinal Study. Int. Arch. Occup. Environ. Health 2013, 86, 709–716. [Google Scholar] [CrossRef] [Green Version]
  125. Hansen, Å.M.; Hogh, A.; Garde, A.H.; Persson, R. Workplace Bullying and Sleep Difficulties: A 2-Year Follow-up Study. Int. Arch. Occup. Environ. Health 2014, 87, 285–294. [Google Scholar] [CrossRef]
  126. Arenas, A.; Giorgi, G.; Montani, F.; Mancuso, S.; Perez, J.F.; Mucci, N.; Arcangeli, G. Workplace Bullying in a Sample of Italian and Spanish Employees and Its Relationship with Job Satisfaction, and Psychological Well-Being. Front. Psychol. 2015, 6, 1–10. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  127. Xu, T.; Magnusson Hanson, L.L.; Lange, T.; Starkopf, L.; Westerlund, H.; Madsen, I.E.H.; Rugulies, R.; Pentti, J.; Stenholm, S.; Vahtera, J.; et al. Workplace Bullying and Violence as Risk Factors for Type 2 Diabetes: A Multicohort Study and Meta-Analysis. Diabetologia 2018, 61, 75–83. [Google Scholar] [CrossRef] [Green Version]
  128. Einarsen, S.; Raknes, B.I.; Matthiesen, S.B.; Hellesøy, O.H. Helsemessige Aspekter Ved Mobbing i Arbeidslivet: Modererende Effekter Av Sosial Støtte Og Personlighet (Health Aspects of Bullying in the Workplace: Moderating Effects of Social Support and Personality). Nord. Psykol. 1996, 48, 116–137. [Google Scholar] [CrossRef]
  129. Savicki, V.; Cooley, E.; Gjesvold, J. Harassment as a Predictor of Job Burnout in Correctional Officers. Crim. Justice Behav. 2003, 30, 602–619. [Google Scholar] [CrossRef]
  130. Takaki, J.; Taniguchi, T.; Hirokawa, K. Associations of Workplace Bullying and Harassment with Pain. Int. J. Environ. Res. Public Health 2013, 10, 4560–4570. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  131. Thurston, R.C.; Chang, Y.; Matthews, K.A.; von Känel, R.; Koenen, K. Association of Sexual Harassment and Sexual Assault with Midlife Women’s Mental and Physical Health. JAMA Intern. Med. 2019, 179, 48. [Google Scholar] [CrossRef]
  132. Smoktunowicz, E.; Baka, L.; Cieslak, R.; Nichols, C.F.; Benight, C.C.; Luszczynska, A. Explaining Counterproductive Work Behaviors among Police Officers: The Indirect Effects of Job Demands Are Mediated by Job Burnout and Moderated by Job Control and Social Support. Hum. Perform. 2015, 28, 332–350. [Google Scholar] [CrossRef]
  133. Magnavita, N. The Exploding Spark: Workplace Violence in an Infectious Disease Hospital- Longitudinal Study. Biomed Res. Int. 2013, 2013, 316358. [Google Scholar] [CrossRef] [Green Version]
  134. Slany, C.; Schütte, S.; Chastang, J.-F.; Parent-Thirion, A.; Vermeylen, G.; Niedhammer, I. Psychosocial Work Factors and Long Sickness Absence in Europe. Int. J. Occup. Environ. Health 2014, 20, 16–25. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  135. Citron, D.K.; Franks, M.A. Criminalizing Revenge Porn. Wake For. Law Rev. 2014, 345, 1–39. [Google Scholar]
  136. Nissen, A.; Hansen, M.B.; Nielsen, M.; Knardahl, S.; Heir, T. Employee Safety Perception Following Workplace Terrorism: A Longitudinal Study. Eur. J. Psychotraumatol. 2019, 10, 1478584. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  137. Elovainio, M.; Kivimaki, M.; Vahtera, J. Organizational Justice: Evidence of a New Psychosocial Predictor of Health. Am. J. Public Health 2002, 92, 105–108. [Google Scholar] [CrossRef] [PubMed]
  138. Hyde, M.; Jappinen, P.; Theorell, T.; Oxenstierna, G. Workplace Conflict Resolution and the Health of Employees in the Swedish and Finnish Units of an Industrial Company. Soc. Sci. Med. 2006, 63, 2218–2227. [Google Scholar] [CrossRef] [PubMed]
  139. Kivimäki, M.; Ferrie, J.E.; Brunner, E.; Head, J.; Shipley, M.J.; Vahtera, J.; Marmot, M.G. Justice at Work and Reduced Risk of Coronary Heart Disease Among Employees. Arch. Intern. Med. 2005, 165, 2245. [Google Scholar] [CrossRef] [Green Version]
  140. Thompson, M.J.; Carlson, D.S.; Kacmar, K.M.; Vogel, R.M. The Cost of Being Ignored: Emotional Exhaustion in the Work and Family Domains. J. Appl. Psychol. 2020, 105, 186–195. [Google Scholar] [CrossRef]
  141. Zivnuska, S.L.; Carlson, D.S.; Carlson, J.R.; Harris, K.J.; Harris, R.B.; Valle, M. Information and Communication Technology Incivility Aggression in the Workplace: Implications for Work and Family. Inf. Process. Manag. 2020, 57, 102222. [Google Scholar] [CrossRef]
  142. Felblinger, D.M. Incivility and Bullying in the Workplace and Nurses’ Shame Responses. J. Obstet. Gynecol. Neonatal Nurs. 2008, 37, 234–242. [Google Scholar] [CrossRef]
  143. Namie, G. How Employers Pay for Bullying; Workplace Bullying Institute: Washington, DC, USA, 2008. [Google Scholar]
  144. Sliter, K.A.; Sliter, M.T.; Withrow, S.A.; Jex, S.M. Employee Adiposity and Incivility: Establishing a Link and Identifying Demographic Moderators and Negative Consequences. J. Occup. Health Psychol. 2012, 17, 409–424. [Google Scholar] [CrossRef]
  145. Sabbath, E.L.; Williams, J.A.R.; Boden, L.I.; Tempesti, T.; Wagner, G.R.; Hopcia, K.; Hashimoto, D.; Sorensen, G. Mental Health Expenditures: Association with Workplace Incivility and Bullying among Hospital Patient Care Workers. J. Occup. Environ. Med. 2018, 60, 737–742. [Google Scholar] [CrossRef]
  146. Steffgen, G.; Sischka, P.; Schmidt, A.F.; Kohl, D.; Happ, C. The Luxembourg Workplace Mobbing Scale. Eur. J. Psychol. Assess. 2019, 35, 164–171. [Google Scholar] [CrossRef] [Green Version]
  147. Garthus-Niegel, S.; Nübling, M.; Letzel, S.; Hegewald, J.; Wagner, M.; Wild, P.S.; Blettner, M.; Zwiener, I.; Latza, U.; Jankowiak, S.; et al. Development of a Mobbing Short Scale in the Gutenberg Health Study. Int. Arch. Occup. Environ. Health 2016, 89, 137–146. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  148. Escartín, J.; Monzani, L.; Leong, F.; Rodríguez-Carballeira, Á. A Reduced Form of the Workplace Bullying Scale the EAPA-T-R: A Useful Instrument for Daily Diary and Experience Sampling Studies. Work Stress 2017, 31, 42–62. [Google Scholar] [CrossRef]
  149. Notelaers, G.; Van der Heijden, B.; Hoel, H.; Einarsen, S. Measuring Bullying at Work with the Short-Negative Acts Questionnaire: Identification of Targets and Criterion Validity. Work Stress 2019, 33, 58–75. [Google Scholar] [CrossRef] [Green Version]
  150. Gurrola-Pena, G.M.; Balcazar-Nava, P.E.; Esparza del Villar, O.; Lozano-Razo, G.; Zavala-Rayas, J. Construction and Validation of the Contextual Victimization Questionnaire (CVCV) with Mexican Young Adults. Int. J. Emerg. Ment. Health Hum. Resil. 2018, 20, 1–7. [Google Scholar] [CrossRef]
  151. Mavandadi, V.; Bieling, P.J.; Madsen, V. Effective Ingredients of Verbal De-Escalation: Validating an English Modified Version of the ‘De-Escalating Aggressive Behaviour Scale’. J. Psychiatr. Ment. Health Nurs. 2016, 23, 357–368. [Google Scholar] [CrossRef]
  152. Leymann, H.; Gustafsson, A. Mobbing at Work and the Development of Post-Traumatic Stress Disorders. Eur. J. Work Organ. Psychol. 1996, 5, 251–275. [Google Scholar] [CrossRef]
  153. Khan, A.K.; Quratulain, S.; Bell, C.M. Episodic Envy and Counterproductive Work Behaviors: Is More Justice Always Good? J. Organ. Behav. 2014, 35, 128–144. [Google Scholar] [CrossRef]
  154. Neuman, J.H.; Baron, R.A. Workplace Violence and Workplace Aggression: Evidence Concerning Specific Forms, Potential Causes, and Preferred Targets. J. Manag. 1998, 24, 391–419. [Google Scholar] [CrossRef]
  155. Einarsen, S.; Skogstad, A. Bullying at Work: Epidemiological Findings in Public and Private Organizations. Eur. J. Work Organ. Psychol. 1996, 5, 185–201. [Google Scholar] [CrossRef]
  156. Escartín, J.; Vranjes, I.; Baillien, E.; Notelaers, G. Workplace Bullying and Cyberbullying Scales: An Overview. In Concepts, Approaches and Methods. Handbooks of Workplace Bullying, Emotional Abuse and Harassment; D’Cruz, P., Noronha, E., Notelaers, G.R.C., Eds.; Springer: Singapore, 2021; pp. 325–368. [Google Scholar] [CrossRef]
  157. Weiss, D.; Marmar, C.; Wilson, J.; Keane, T. Assessing Psychological Trauma and PTSD; Guilford Press: New York, NY, USA, 1997. [Google Scholar]
  158. Bjelland, I.; Dahl, A.A.; Haug, T.T.; Neckelmann, D. The Validity of the Hospital Anxiety and Depression Scale. J. Psychosom. Res. 2002, 52, 69–77. [Google Scholar] [CrossRef] [PubMed]
  159. Dallner, M.; Elo, A.; Gamberale, L.F.; Al, E. Validation of the General Nordic Questionnaire (QPSNordic) for Psychological and Social Factors at Work; Nordic Council of Ministers: Copenhagen, Denmark, 2000. [Google Scholar]
  160. Maslach, C.; Jackson, S.E.; Leiter, M.P. Maslach Burnout Inventory: In Evaluating Stress: A Book of Resources; Zalaquett, C.P., Wood, R.J., Eds.; Scarecrow Education: Lanham, MD, USA, 1997; pp. 191–218. [Google Scholar]
  161. Shimomitsu, T.; Haratani, T.; Iwata, N.; Nakamura, K. The Final Development of the Brief Job Stress Questionnaire Mainly Used for Assessment of the Individuals. In A Research Report Relating to Stress in the Workplace and Its Impact on Workers’ Health; Kato, M., Ed.; Ministry of Labour: Tokyo, Japan, 2000; pp. 126–164. [Google Scholar]
  162. Koenen, K.C.; De Vivo, I.; Rich-Edwards, J.; Smoller, J.W.; Wright, R.J.; Purcell, S.M. Protocol for Investigating Genetic Determinants of Posttraumatic Stress Disorder in Women from the Nurses’ Health Study II. BMC Psychiatry 2009, 9, 29. [Google Scholar] [CrossRef] [Green Version]
  163. Arnetz, J.E. The Violent Incident Form (VIF): A Practical Instrument for the Registration of Violent Incidents in the Health Care Workplace. Work Stress 1998, 12, 17–28. [Google Scholar] [CrossRef]
  164. Goldberg, D.; Bridges, K.; Duncan-Jones, P.; Grayson, D. Detecting Anxiety and Depression in General Medical Settings. BMJ 1988, 297, 897–899. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  165. Karasek, R.A. Job Demands, Job Decision Latitude, and Mental Strain. Implication for Job Redesign. Adm. Sci. Q. 1979, 24, 285–308. [Google Scholar] [CrossRef]
  166. Elo, A.-L.; Leppänen, A.; Jahkola, A. Validity of a Single-Item Measure of Stress Symptoms. Scand. J. Work Environ. Health 2003, 29, 444–451. [Google Scholar] [CrossRef] [Green Version]
  167. Thompson, E.R. Development and Validation of an Internationally Reliable Short-Form of the Positive and Negative Affect Schedule (PANAS). J. Cross Cult. Psychol. 2007, 38, 227–242. [Google Scholar] [CrossRef]
  168. Lambert, M.J.; Finch, A.M.; Okiishi, J.; Burlingame, G.M.; McKelvey, C.; Reisinger, C.W. Administration and Scoring Manual for the OQ-10.2: An Adult Outcome Questionnaire for Screening Individuals and Population Outcome Monitoring; American Professional Credentialing Services: Stevenson, MD, USA, 1998. [Google Scholar]
  169. Maslach, C.; Jackson, S.E.; Head, M.P. Maslach Burnout Inventory; Consulting Psychologists Press: Palo Alto, CA, USA, 1986. [Google Scholar]
  170. Körükcü, Ö.; Bulut, O.; Tuzcu, A.; Bayram, Z.; Öztürk Türkmen, H. An Adaptation of Leymann Inventory of Psychological Terror to Health Sciences Programs in Turkey. Anatol. J. Psychiatry 2014, 15, 335–343. [Google Scholar] [CrossRef] [Green Version]
  171. Breiding, M.J.; Smith, S.G.; Basile, K.C.; Walters, M.L.; Chen, J.; Merrick, M.T. Prevalence and Characteristics of Sexual Violence, Stalking, and Intimate Partner Violence Victimization—National Intimate Partner and Sexual Violence Survey, United States, 2011. MMWR Surveill. Summ. 2014, 63, 1–18. [Google Scholar]
  172. Namie, G.; Namie, R. Workplace Bullying: The Silent Epidemic. Empl. Rights Q. 2000, 1, 1–12. [Google Scholar]
  173. Sackett, P.; DeVore, C. Counterproductive Behaviours at Work. In Handbook of Industrial Work and Organizational Psychology, Personal Psychology; Anderson, N., Ones, D.S., Sinangil, H.K., Viswesvaran, C., Eds.; Sage Publications, Inc.: London, UK, 2001; pp. 145–164. [Google Scholar]
  174. Mathisen, G.E.; Einarsen, S.; Myketlun, R. The Occurrences and Correlates of Bullying and Harassment in the Restaurant Sector. Scand. J. Psychol. 2008, 49, 59–68. [Google Scholar] [CrossRef] [PubMed]
  175. Laumann, E.; Marsden, P.V.; Prenski, D. The Boundary Specification Problem in Network Analysis (Applied for Conference Methods in Social Network Analysis); Burt, R., Minor, M., Eds.; Sage Publications, Inc.: Beverly Hills, CA, USA, 1983. [Google Scholar]
  176. Robinson, M.A. Using Multi-Item Psychometric Scales for Research and Practice in Human Resource Management. Hum. Resour. Manag. 2018, 57, 739–750. [Google Scholar] [CrossRef] [Green Version]
  177. Bougie, R.; Sekaran, U. Research Methods for Business: A Skill Building Approach, 8th ed.; John Wiley & Sons Inc.: Hoboken, NJ, USA, 2020. [Google Scholar]
  178. Arvey, R.D.; Cavanaugh, M.A. Using Surveys to Assess the Prevalence of Sexual Harassment: Some Methodological Problems. J. Soc. Issues 1995, 51, 39–52. [Google Scholar] [CrossRef]
  179. Dalal, R.S. A Meta-Analysis of the Relationship between Organizational Citizenship Behavior and Counterproductive Work Behavior. J. Appl. Psychol. 2005, 90, 1241–1255. [Google Scholar] [CrossRef]
  180. Alhabash, S.; McAlister, A.R.; Hagerstrom, A.; Quilliam, E.T.; Rifon, N.J.; Richards, J.I. Between Likes and Shares: Effects of Emotional Appeal and Virality on the Persuasiveness of Anticyberbullying Messages on Facebook. Cyberpsychol. Behav. Soc. Netw. 2013, 16, 175–182. [Google Scholar] [CrossRef] [PubMed]
  181. Fox, S.; Spector, P.E.; Miles, D. Counterproductive Work Behavior (CWB) in Response to Job Stressors and Organizational Justice: Some Mediator and Moderator Tests for Autonomy and Emotions. J. Vocat. Behav. 2001, 59, 291–309. [Google Scholar] [CrossRef] [Green Version]
  182. Björkqvist, K.; Österman, K.; Lagerspetz, K.M.J. Sex Differences in Covert Aggression among Adults. Aggress. Behav. 1994, 20, 27–33. [Google Scholar] [CrossRef]
  183. Skarlicki, D.P.; Folger, R. Retaliation in the Workplace: The Roles of Distributive, Procedural, and Interactional Justice. J. Appl. Psychol. 1997, 82, 434–443. [Google Scholar] [CrossRef]
  184. Beck, A.T.; Steer, R.A.; Brown, G. Beck Depression Inventory–II (BDI-II); APA Publishing: Washington, DC, USA, 1996. [Google Scholar] [CrossRef]
  185. Lazarus, R.S.; Folkman, S. Cognitive Theories of Stress and the Issue of Circularity. In Dynamics of Stress.The Plenum Series on Stress and Coping; Appley, M.H., Trumbull, R., Eds.; Springer: Boston, MA, USA, 1986; pp. 63–80. [Google Scholar] [CrossRef]
  186. Escartín, J.; Rodríguez-Carballeira, A.; Zapf, D.; Porrúa, C.; Martín-Peña, J. Perceived Severity of Various Bullying Behaviours at Work and the Relevance of Exposure to Bullying. Work Stress 2009, 23, 191–205. [Google Scholar] [CrossRef]
  187. Ireland, J. Bullying among Prisoners: Innovations in Theory and Research. In Bullying among Prisoners: The Need for Innovation; Ireland, J.L., Ed.; Willan Publishing: Cullompton, UK, 2013; pp. 1–30. [Google Scholar] [CrossRef]
  188. Haynie, D.L.; Nansel, T.; Eitel, P.; Crump, A.D.; Saylor, K.; Yu, K.; Simons-Morton, B. Bullies, Victims, and Bully/Victims. J. Early Adolesc. 2001, 21, 29–49. [Google Scholar] [CrossRef]
  189. Juvonen, J.; Graham, S. Peer Harassment in School, 1st ed.; Juvonen, J., Graham, S., Eds.; The Guilford Press: New York, NY, USA; London, UK, 2016. [Google Scholar]
  190. Koller, G.; O’Connor, M.K. Risk Assessment and Decision Making in Business and Industry: A Practical Guid, 2nd ed.; CRC Press (Chapman & Hall): Boca Raton, FL, USA, 2005. [Google Scholar]
  191. Lutgen-Sandvik, P.; Tracy, S.J.; Alberts, J.K. Burned by Bullying in the American Workplace: Prevalence, Perception, Degree and Impact. J. Manag. Stud. 2007, 44, 837–862. [Google Scholar] [CrossRef]
  192. Frese, M.; Zapf, D. Methodological Issues in the Study of Work Stress: Objective vs. Subjective Measurement of Work Stress and the Question of Longitudinal Studies. In Causes, Coping and Consequences of Stress at Work; Cooper, C.L., Payne, R., Eds.; Wiley: Chichester, UK, 1988; pp. 375–410. [Google Scholar]
  193. Agervold, M. Bullying at Work: A Discussion of Definitions and Prevalence, Based on an Empirical Study. Scand. J. Psychol. 2007, 48, 161–172. [Google Scholar] [CrossRef] [PubMed]
  194. Shaffer, J.A.; DeGeest, D.; Li, A. Tackling the Problem of Construct Proliferation. Organ. Res. Methods 2016, 19, 80–110. [Google Scholar] [CrossRef]
  195. Paull, M.; Omari, M.; Standen, P. When Is a Bystander Not a Bystander? A Typology of the Roles of Bystanders in Workplace Bullying. Asia Pac. J. Hum. Resour. 2012, 50, 351–366. [Google Scholar] [CrossRef]
  196. Mayhew, C.; McCarthy, P.; Chappell, D.; Quinlan, M.; Barker, M.; Sheehan, M. Measuring the Extent of Impact from Occupational Violence and Bullying on Traumatised Workers. Empl. Responsib. Rights J. 2004, 16, 117–134. [Google Scholar] [CrossRef]
  197. Lutgen-Sandvik, P. Take This Job and…: Quitting and Other Forms of Resistance to Workplace Bullying. Commun. Monogr. 2006, 73, 406–433. [Google Scholar] [CrossRef]
  198. Vicente, P.; Reis, E. Using Questionnaire Design to Fight Nonresponse Bias in Web Surveys. Soc. Sci. Comput. Rev. 2010, 28, 251–267. [Google Scholar] [CrossRef]
  199. Couper, M.P.; Traugott, M.W.; Lamias, M.J. Web Survey Design and Administration. Public Opin. Q. 2001, 65, 230–253. [Google Scholar] [CrossRef]
  200. Fuchs, M.; Couper, M.P.; Hansen, S.E. Technology Effects: Do Capi or Papi Interviews Take Longer? J. Off. Stat. 2000, 16, 273–286. [Google Scholar]
  201. Schreurs, P.J.G.; van de Willige, J.G.; Brosschot, F.; Graus, G.M.H.; Tellegen, B. De Utrechtse Coping Lijst (UCL): Omgaan Met Problemen En Gebeurtenissen (The Utrecht Coping List: Coping with Problems and Events), revised ed.; Pearson: Amsterdam, The Netherlands, 1993. [Google Scholar]
  202. Hallett, N.; Dickens, G.L. De-Escalation: A Survey of Clinical Staff in a Secure Mental Health Inpatient Service. Int. J. Ment. Health Nurs. 2015, 24, 324–333. [Google Scholar] [CrossRef]
  203. Nielsen, M.; Gjerstad, J.; Jacobsen, D.P.; Einarsen, S.V. Does Ability to Defend Moderate the Association between Exposure to Bullying and Symptoms of Anxiety? Front. Psychol. 2017, 8, 1–11. [Google Scholar] [CrossRef] [Green Version]
  204. Magnavita, N.; Di Stasio, E.; Capitanelli, I.; Lops, E.A.; Chirico, F.; Garbarino, S. Sleep Problems and Workplace Violence: A Systematic Review and Meta-Analysis. Front. Neurosci. 2019, 13, 997. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Development INWBQ. * Verschuren et al., 2021 [29].
Figure 1. Development INWBQ. * Verschuren et al., 2021 [29].
Ijerph 20 06564 g001
Table 1. INWBQ dimensions.
Table 1. INWBQ dimensions.
Actor typesStrangerWorkerCostumer/client/pupilRelative--
Actor rolesPerpetratorTargetAssistantDefenderOutsiderWitness
Occurrence PatternSystematic
NWB = negative work behavior.
Table 2. Dimensions Actor Types & Roles, Subcategories.
Table 2. Dimensions Actor Types & Roles, Subcategories.
Dimension Actor Types, Subcategories, TypesDimension Actor Roles, Subcategories, Roles
Subcategories stranger
  • Public, visitors, and users of social media (based on e.g.,
    deviance scale, UWBQ, HABS-U, digital safety)
Subcategory coworker/manager
  • Co-worker (based on e.g., HABS-CS, cyber aggression,
  • Professional e.g., teacher, politician, marketer, journalist,
    health care worker (based on e.g., HABS-CS,
    cyberstalking victimization)
  • Manager (based on e.g., UWBQ, WIS,
    Abusive supervision scale)
Subcategory client/customer/pupil,
  • Client, customer, students, supplier (based on e.g.,
    cyber incivility, monitor social safety)
Subcategory relative,
  • Family, acquaintance, (ex) friend, intimate, source
    (based on e.g., HABS-U, JVQ)
Subcategory actor group
  • Citizens, shared interest group, organizational group
    (based on e.g., violence research, POPS)
Subcategory role perpetrator
  • Bully, attacker (based e.g., HABS-CS, CWB-C,
Subcategory role target
  • Victim, target (based on e.g., ICA-W, EHE,
Subcategory bystander role witness
  • See, observe, meaning (based on e.g., Injustice
    scale, POPS)
Subcategory bystander role assistant
  • Willingness to encourage, join in, or aid a bully
    in bullying others (based on BPBQ, SVI)
Subcategory bystander role defender
  • Participation in behaviors related to defending
    on behalf of a victim (BPBQ)
Subcategory bystander role outsider
  • Chooses to actively ignore it (based on BPBQ,
    SVI, monitor social safety)
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Verschuren, C.; Tims, M.; De Lange, A.H. Beyond Bullying, Aggression, Discrimination, and Social Safety: Development of an Integrated Negative Work Behavior Questionnaire (INWBQ). Int. J. Environ. Res. Public Health 2023, 20, 6564.

AMA Style

Verschuren C, Tims M, De Lange AH. Beyond Bullying, Aggression, Discrimination, and Social Safety: Development of an Integrated Negative Work Behavior Questionnaire (INWBQ). International Journal of Environmental Research and Public Health. 2023; 20(16):6564.

Chicago/Turabian Style

Verschuren, Cokkie, Maria Tims, and Annet H. De Lange. 2023. "Beyond Bullying, Aggression, Discrimination, and Social Safety: Development of an Integrated Negative Work Behavior Questionnaire (INWBQ)" International Journal of Environmental Research and Public Health 20, no. 16: 6564.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop