Next Article in Journal
Self-Reported Sleep Disturbance from Road, Rail and Aircraft Noise: Exposure-Response Relationships and Effect Modifiers in the SiRENE Study
Next Article in Special Issue
Risk Factors for Non-Communicable Diseases at Baseline and Their Short-Term Changes in a Workplace Cohort in Singapore
Previous Article in Journal
Effects of Aromatherapy Combined with Music Therapy on Anxiety, Stress, and Fundamental Nursing Skills in Nursing Students: A Randomized Controlled Trial
Previous Article in Special Issue
“This Isn’t Just about Things, It’s about People and Their Future”: A Qualitative Analysis of the Working Conditions and Strains of Social Workers in Refugee and Homeless Aid
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

How Abusive Supervision Affects Employees’ Unethical Behaviors: A Moderated Mediation Examination of Turnover Intentions and Caring Climate

1
Business School, Central University of Finance and Economics, Beijing 100081, China
2
School of Economics and Management, Tongji University, Shanghai 200092, China
*
Authors to whom correspondence should be addressed.
These authors contributed equally to this work.
Int. J. Environ. Res. Public Health 2019, 16(21), 4187; https://doi.org/10.3390/ijerph16214187
Submission received: 18 September 2019 / Revised: 17 October 2019 / Accepted: 25 October 2019 / Published: 29 October 2019
(This article belongs to the Special Issue Workplace Health and Wellbeing 2019)

Abstract

:
Drawing on psychological contract theory, this research contributes to the unethical behavior literature by exploring employees’ turnover intentions as a mediator of the relationship between abusive supervision and employees’ unethical behavior and the moderating role of the caring climate in the relationship between turnover intentions and unethical behavior. The results from a sample of 679 reveal that turnover intentions mediate relationship between abusive supervision and subordinates’ unethical behavior, and caring climate moderates the positive relationship between turnover intentions and subordinates’ unethical behavior. We also find that the indirect effect is moderated by the caring climate. The theoretical and practical implications are discussed.

1. Introduction

Over the years, several attempts have been made in the behavioral ethics literature to explain unethical decision making and behaviors [1,2,3,4]. In accordance with this trend, the vast majority of research has focused on identifying the precursors of unethical behaviors. Among these scholarly efforts, there are three main perspectives: The Bad Apple Perspective, the Bad Barrel Perspective, and the Interactionist Perspective [2,3]. The “bad apples” argument attributes unethical behavior in the organization to a few unsavory individuals [5] who lack some personal quality, such as moral character. This approach prioritizes individual differences and treats measures such as individuals’ economic value orientation, political value orientation, Machiavellianism, and cognitive moral development as significantly related to ethical decision-making behavior [6,7,8]. In comparison, the “bad barrel” argument assumes that something in the organizational environment poisons otherwise good “apples” and attributes unethical behavior to competition, management’s results orientation, a lack of reinforcement of ethical behavior, requests from authority figures to behave unethically, peer behavior, etc. [3]. The interactionist model of unethical behavior in organizations uses a multiple-influences perspective. According to the interactionist perspective, the individual reacts to an ethical dilemma, and the individual’s cognitive moral development stage determines how an individual think about the dilemma [2]. This model assumes that it is not enough to explain or predict ethical decision-making behavior based only on individuals’ cognition of right and wrong or contextual determinants. Instead, additional individual and situational variables interact with the cognitive component to determine how an individual is likely to behave in response to an ethical dilemma.
Informed by the above theoretical models, recent studies have begun to focus on how the role of supervisors or leaders affects subordinates’ unethical behavior, such as theft and other unlawful conduct at the workplace, failure to honor commitments, deliberate deception, and violation of conscience [9,10,11]. However, these studies focus mainly on the role of ethical leadership in inhibiting the unethical behavior of subordinates. That is, existing knowledge on the role of leadership in employee unethical behavior remains incomplete because little is known as to whether the dark side of leadership in general may affect unethical behavior of employees. Thus, further studies are needed to find the triggering or activating effects of negative leadership on employees’ unethical behavior.
Abusive supervision is one such trigger or activator. It was defined by Tepper [12] as “subordinates’ perceptions of the extent to which supervisors engage in the sustained display of hostile verbal and nonverbal behaviors, excluding physical contact.” Given the prevalence and far-reaching impact of abusive supervision [13], illuminating the influence of abusive supervision on subordinates’ unethical behavior is of theoretical and empirical significance. However, to our knowledge, only a small number of studies have focused directly on the associations between abusive supervision and subordinate unethical behavior. For example, research that draw from social cognitive theory demonstrates that abusive supervision can be associated with subordinates’ unethical behavior [14], and based on trait activation theory, Greenbaum et al. [8] verified the person—(i.e., Machiavellianism) situation (i.e., abusive supervision) interactionist model to predict unethical behavior in organizations. Though these studies have made some contributions to the abusive and unethical behavior literatures, the processes underlying this perception-to-behavior link and the boundary conditions of these processes remain unclear. This is an unfortunate oversight both practically and theoretically. Practically, individual unethical behavior hinders the development of organization [15], and leader behaviors also play an important role in the growth and prohibition of unethical behavior. Theoretically, in view of extant abusive supervision and unethical behavior research, we expect abusive supervision to increase subordinates’ unethical behavior.
In order to explain the processes underlying abusive supervision and unethical behavior, we draw from psychological contract theory [16] to theorize that subordinates’ turnover intentions mediate the indirect relationship between abusive supervision and unethical behavior. According to psychological contract theory [16], psychological contract refers to “employees’ perceptions of what they owe to their employers and what their employers owe to them” [17]. Employees with intentions to leave are likely to psychologically detach themselves from the organization [18], and the weak psychological contract results in negative employee outcomes such as unethical behavior [19]. Researchers have also argued that abused employees are more inclined to show turnover intentions [20] and that these employees are unlikely to feel obligated to stay in the organization. Such detachments may drive employees to break situational norms. Thus, the first purpose of this research is to examine the indirect effect of abusive supervision on unethical behavior via turnover intentions.
Moreover, another insight of psychological contract theory highly related to our model is that individuals’ response to weak psychological contract in different ways depending on organizational context, such organizational context can modify the impact of turnover intentions on unethical behavior. One important factor that may impact coping style is organizational caring climate, unethical behavior researchers have found that “everyone for himself” climates are more likely to increase unethical behavior. In comparison, the reverse relationship is found where there is a climate that focuses on everyone’s interests [21]. Following these, the second purpose of this study is to investigate the moderating role of caring climate on the relationship between turnover intentions and unethical behavior.
In sum, the current research tests a theoretical model of the relationship between abusive supervision and unethical behavior and is designed to make three unique contributions. First, we draw from psychological contract theory to make a novel theoretical contribution that explains why relational aspects of supervisor–subordinate interactions affect subordinates’ behavioral responses to abusive supervision. An examination of this effect is likely to advance current understanding of the consequences of the dark side of leadership in organizations, and it also extends the limited understanding of the relationship between abusive supervision and employees’ unethical behavior. Second, by adopting a perspective based on psychological contract theory, we explore how turnover intentions mediate the relationship between abusive supervision and unethical behavior. This investigation advances the research by identifying a new underlying mechanism that explains the relationship between abusive supervision and unethical behavior. Finally, by illustrating the moderating role of caring climate, we recognize the boundary conditions which abusive supervision induces unethical behavior. This examination contributes to both the leadership literatures and the burgeoning studies on unethical behavior by demonstrating the ways in which organizational context can affect leadership processes and consequences. Figure 1 outlines the theoretical model tested in this study.

2. Theoretical Background and Hypotheses

2.1. The Mediating Role of Turnover Intentions

Abusive supervision refers to a destructive leadership style where employees perceive treatment from their supervisor as hostile verbal and/or non-verbal expressions [12], such as ridiculing, yelling at, and intimidating subordinates; taking credit for subordinates’ achievements; and attributing undesirable outcomes to subordinates’ personal factors [22,23]. Based on previous studies, we argue that psychological contract theory provides a new perspective in explaining subordinates who display unethical behavior in abusive organizational situations.
Psychological contract theory is deeply influenced by the social exchange theory [24]. Further, it has been proposed that reciprocity is central to explaining the relationships between the evaluation of the psychological contract and employee attitudes and behaviors [24]. According to psychological contract theory [16], employees hold a specific set of beliefs regarding their relationship with the organization, which includes the reciprocal obligations of each party. As noted by Robinson [17], “psychological contracts refer to employees’ perceptions of what they owe to their employers and what their employers owe to them”. Researchers have shown that psychological contracts include both transactional and relational components [25,26]. The transactional component involves the more static aspects of the exchange relationship, such as pay and benefits. The relational component involves the more flexible aspects of the exchange relationship that presume a long-term relationship with the organization, such as training and development opportunities, promotion opportunities, and support from supervisors [27].
Employee turnover is defined as the cessation of membership in an organization by an individual who has received monetary compensation from the organization [28]. The turnover research has paid attention to the intentions and reasons behind turnover and to its consequences. For example, Mobley [29] provided a comprehensive explanation of the psychological process underlying turnover. In the process of making a withdrawal decision, individuals first evaluate their current jobs and experience satisfaction or dissatisfaction based on these jobs. A turnover intention can be formed if psychological attachment is decreased.
Previous studies have clearly established that leadership can play an important role in construction and maintenance of employees’ psychological contract [30]. For example, when supervisors show abusive supervision behaviors to subordinates, subordinates will regard it as disrespect, contempt and humiliation to themselves. This is contrary to the expectation or belief that one should be treated fairly. It violates the exchange principle and ethics of “equality and reciprocity” between the two parties in the relationship, damages the positive image of the supervisors in the eyes of subordinates. That is, abusive supervision creates a low-quality relationship between supervisors and subordinates, which decreases the subordinates’ attachment and loyalty to the supervisor and organization. In this case, subordinates who do not feel that their supervisors care about them are unlikely to feel obliged to remain within the organization [13]. Moreover, subordinates are concerned about whether supervisors can provide them with sufficient learning and development opportunities and a just climate [31]. As abusive supervisors struggle to meet subordinates’ expectations, the relational contract between subordinates and supervisors or organization will be difficult to maintain. Similarly, a number of empirical studies have confirmed the positive relationship between abusive supervision and the turnover intentions of subordinates [32] Therefore, we hypothesize
H1: 
Abusive supervision will be positively related to subordinates’ turnover intentions.
Turnover intentions signal that the relational contract between subordinates and the organization has been breached [33]. It stands to reason that, if an individual feels that this relationship has been breached, then they will develop strong intentions to quit. A subordinate with strong turnover intentions has effectively quit before officially leaving the organization and is emotionally disconnected from other employees and the organization, but such individuals should be less likely to be concerned about what their employer owes them or what they owe their employer in a relational sense [33]. Not only will such subordinates be less likely to help others, we argue that they will be more prone to behaving in ways that are counter to organizational policies, practices, and norms. Recent research reveals that turnover intentions result in increased incidents of negative work behavior [34]. We accordingly propose that turnover intentions impact subsequent unethical behavior and ultimately serve as a critical mediating mechanism explaining the relationship between abusive supervision and unethical behavior. Therefore, we hypothesize
H2: 
Turnover intentions is positively related to subordinates’ unethical behavior.
We hypothesize a positive association between abusive supervision and subordinates’ turnover intentions. We also hypothesize that there is a positive relationship between turnover intentions and subordinates’ unethical behavior. Together, these hypotheses suggest that turnover intentions mediate the relationships between abusive supervision and subordinates’ unethical behavior. These mediated relationships are in line with research on psychological contract theory which has support the relationship between leadership and subordinates’ behavior. Therefore, we hypothesize
H3: 
Turnover intentions mediate the positive relationship between abusive supervision and unethical behavior.

2.2. The Moderating Role of Caring Climate

There is reason to believe that the mechanism proposed in the above hypotheses may vary in its strength. Psychological contract theory [16] argued that organizational characteristics (e.g., organizational climate) affect how individuals cope with decreased psychological contract. Here, we propose that organizational climate will play critical roles in shaping the violation-behavior relationship.
In organizations, employees work within a system of rules, regulations, procedures, and hierarchy [35]. The moral content of the organizational climate can have an impact on the moral development of the individual, and appropriate behavior can be learned by employees within their organization through climate perceptions. Rusaw [36] posited organizational climate is one of the organizational characteristics that influence individuals’ ethical decision. Researches also illustrate that the organizational climate serves as a behavioral norm for ethical judgments [37] and managerial moral decisions [38]. Ethical climate, which is one specific variable in the dimensions of climate, is a type of work climate that guides ethical behavior within an organization [39]. Ethical climate helps employees decide what is right and wrong. Victor and Cullen’s [40,41] initial work identified different types of ethical climate: professionalism, caring, rules, instrumental, efficiency, and independence. Fu and Deshpande [39] found that among these ethical climates, the caring climate had the strongest correlation with ethical behavior. Thus, we focus on this climate type in our study. Previous studies have also found that the caring climate has a significantly positive impact on various organizational outcomes, such as job satisfaction [42,43], supervisor satisfaction [44], job performance [39], and organizational commitment [45]. It is important to determine how organizations can deal with abusive supervision and the consequent unethical behaviors of employees. Therefore, this study tests the moderating effects of the caring climate.
We propose that a caring climate can mitigate the positive effects of turnover intentions on unethical behavior. Within a caring climate, employees’ major consideration is the impact of utilitarian decisions on others rather than their self-interest [46]. Employees in a caring climate are also more likely to make decisions that are beneficial for the majority of people involved in the decisions [41]. Thus, we believe that a caring climate can offer cues for the expression of unethical behaviors and motivate employees to behave well. There has been indirect empirical support for our arguments. Research has found that employees’ negative affect may lead to deviant behavior, an effect that was significantly weakened when the caring climate was high [47]. Hence, for employees within a higher perceived caring climate, turnover intentions should have a constrained positive effect on unethical behavior, whereas employees within a lower perceived caring climate should be more likely to engage in unethical behavior. As a result, we suggest that the caring climate moderates the positive relationship between turnover intentions and unethical behavior.
The prior arguments represent an integrated framework in which turnover intentions mediate the positive relationship between abusive supervision and unethical behavior and caring climate moderates the relationship between turnover intentions and unethical behavior. According to the notion that the caring climate moderates the relationship between turnover intentions and unethical behavior, and considering that abusive supervision is positively related to turnover intentions, it is logical to propose that the caring climate also moderates the mediating mechanism for turnover intentions in the relationship between abusive supervision and unethical behavior. As discussed above, a weaker relationship between turnover intentions and unethical behavior emerges in high-caring climate organizations, and the indirect effect of abusive supervision on unethical behavior through turnover intentions may also be weaker in such organizations. Therefore, we hypothesize
H4: 
Caring climate moderates the indirect effect between abusive supervision and unethical behaviors through turnover intentions. Specifically, the indirect effect is stronger when the organizational caring climate is low versus high, and the moderation effect occurs between turnover intentions and unethical behaviors.

3. Methods

3.1. Procedure and Sample

The participants in this study were subordinates who worked at three high-tech enterprises based in Shanghai, China. The participants worked in a number of job domains and business functions related to the sales, product development, and after-sale services. Incomplete or inconsistent participant responses were seen as invalid or missing data, and thus were excluded in the survey. The final sample consists of 679 participants (estimated response rate of 79.9%); 58.8% of the respondents were male. Most of the respondents were aged 20–30 (73.6%) and 31–60 (26.4%), which gave us an average respondent age of 28.9 years old (SD = 5.1). Over half of the respondents had an undergraduate degree (67.5%). The participants had an average job tenure of 5.5 years (SD = 3.7) and an average organizational tenure of 3.8 years (SD = 2.6).

3.2. Measures

Abusive Supervision. Abusive supervision was assessed with Tepper’s [12] 15-item scale. The items were rated on a five-point scale ranging from 1 (extremely uncharacteristic of me) to 5 (extremely characteristic of me). Sample items ask whether one’s supervisor “… is rude to me,” “… makes negative comments about me to others,” “… reminds me of my past mistakes and failures,” and “… tells me I’m incompetent.” The coefficient alpha was 0.94 in the current study.
Turnover Intentions. Employees’ turnover intention was assessed with a five-item scale from Bozeman and Perrewe [48]. These five items were rated on a five-point Likert scale (5 = mostly true; 1 = mostly false). Sample items include “I will probably look for a new job in the near future,” “At the present time, I am actively searching for another job in a different organization,” and “It is unlikely that I will actively look for a different organization to work for in the next year (reverse scored).” The coefficient alpha was 0.89 in the current study.
Unethical Behavior. Unethical behavior was assessed with the 18-item scale developed and validated by Chen and Tang [49]. Each item was rated on a five-point frequency scale from 1 (never) to 5 (always). Sample items include “waste company time surfing on the Internet, playing computer games, and socializing,” “take no action against the fraudulent charges made by one’s company,” “Abuse the company expense accounts and falsify accounting records,” and “Overcharge customers to increase sales and to earn a higher bonus.” The coefficient alpha for the unethical behavior scale was 0.95 in the current study.
Caring Climate. The caring climate was assessed with the five-item scale developed by Victor and Cullen [40]. A five-point Likert scale (with 5 representing “strongly agree” and 1 representing “strongly disagree”) was used to measure the caring climate. Sample items include “The most important concern is the good of all of the people in the company as a whole,” “Our major concern is always what is best for the other person,” and “What is best for everyone in the company is the major consideration here.” The coefficient alpha was 0.89 in the current study.

3.3. Analytical Strategy

Prior to hypotheses testing, we used Mplus 7 to conducted a series of confirmatory factor analyses and examined multiple fit indices including Comparative Fit Index (CFI), Tucker–Lewis Index (TLI), Root Mean Square Error of Approximation (RMSEA), and Standardized Root Mean Square Residual (SRMR). Then, we used SPSS 21.0 to examine means, standard deviations, and inter- correlations for the study variables. Finally, we tested the hypothesized moderated mediation model using hierarchical regression analyses via PROCESS 2.16 for SPSS (IBM, New York, NY, USA) [50], bootstrapping was set to 5000 resamples. Additionally, because unstandardized coefficients are the preferred metric in causal modeling [50,51], all of the path coefficients are reported as unstandardized OLS regression coefficients.

4. Results

4.1. Confirmatory Factor Analysis (CFA)

Before testing our hypotheses, we first used Mplus 7 (Informer Technologies, New York, NY, USA,) to conduct a CFA and assess the distinctiveness of our key variables (abusive supervision, turnover intentions, unethical behavior, and caring climate). As shown in Table 1, the CFA results suggested that our four-factor baseline model fit the data substantially better than any of the alternative models, confirming discriminant validity (χ2 = 2220.40, df = 854, CFI = 0.92, TLI = 0.91, RMSEA = 0.05, SRMR = 0.05). We next applied Fornell and Larcker’s [52] method check for convergent and discriminant validity. Table 2 shows the square roots of the average variance extracted for each latent construct. All the estimates exceeded the correlation between the factors comprising each pair. In addition, all parcels significantly loaded on the intended latent constructs and had standardized loadings above 0.5. Therefore, all the proposed constructs have good convergent and discriminant validity.

4.2. Common Method Bias

To check the problem of common method bias, Harman’s single-factor test [53] was conducted. The analysis returned seven factors with eigenvalues greater than 1, with the first factor explaining less than 50% of the variance (37.27% of 71.09%). In addition, a follow-up factor analysis with 4fourseparate factors showed that all of the factor loadings exceeded 0.50, which indicated a good inter-item reliability. In addition, we conducted a CFA by controlling for the effects of a single unmeasured latent method factor [53], the results showed that the five-factor model does not have a better data fitting than four-factor model. Thus, the findings provided no indications of common method variance.

4.3. Descriptive Statistics and Correlations

The means, standard deviations, correlations, and coefficient alphas for all of the study variables are presented in Table 2. Abusive supervision was positively related to turnover intentions (r = 0.40, p < 0.01), and turnover intentions were positively related to unethical behavior (r = 0.33, p < 0.001). Abusive supervision also exhibited a significant relationship to unethical behavior (r = 0.52, p < 0.001).

4.4. Hypothesis Testing

Hypothesis 1 predicted the relationship between abusive supervision and turnover intentions, Hypothesis 2 predicted the relationship between turnover intentions and unethical behavior, and Hypothesis 3 predicted an indirect effect between abusive supervision and unethical behavior through turnover intentions. These hypotheses were supported with significant direct effects and an indirect effect (see Table 3). There was a significant abusive supervision to turnover intentions direct effect (a effect = 0.39, p < 0.001), a turnover intentions to unethical behavior direct effect (b effect = 0.14, p < 0.001), and an indirect effect through turnover intentions (indirect effect = 0.055, CI.95 = 0.013, 0.098). An examination of the remaining direct effect of abusive supervision on unethical behavior (c’ effect = 0.47, p < 0.001) indicated partial mediation, supporting Hypothesis 1, Hypothesis 2, and Hypothesis 3.
To test Hypothesis 4, which proposes that the indirect effect of abusive supervision on unethical behaviors through turnover intentions is conditioned by the level of caring climate, we conducted a moderated mediation analysis with caring climate as a moderator of the relationship between turnover intentions and unethical behavior. Support was found for this hypothesis with significant interaction terms in the moderated mediation models (see Table 3). There was a significant negative interaction term on the turnover intentions to unethical behavior path (effect= −0.03, p < 0.05). The interaction is graphically displayed in Figure 2. Additionally, the index of moderated mediation indicates that any two conditional indirect effects defined by different values of caring climate are statistically different (index = −0.011, CI.95 = −0.025, 0.001).
To illustrate the presence of moderated mediation, we report the indirect effect at different levels of the moderator. Table 3 illustrates these moderated indirect effects through changes in the level of caring climate for unethical climate. Specifically, low levels of caring climate have a significant indirect effect (indirect effect = 0.022, CI.95 = 0.004, 0.040), while mean and high levels of caring climate have non-significant indirect effects (indirect effect = 0.010, CI.95 = −0.001, 0.030; indirect effect = −0.001, CI.95 = −0.030, 0.025). This means that abusive supervision has the strongest association with unethical behavior through turnover intentions when the level of the caring climate is low.

5. Discussion

An understanding of ethical decision making in organizations is important for the development of organization science. It is thus meaningful to capture the important interfaces among individual and situational variables that together shape this complex phenomenon. However, research that uses an interactionist model of ethical decision making is limited. To address the above concerns, and in response to previous researchers’ [2,21] call for a person-situation interactionist model for understanding behavioral ethics, we developed the current study by investigating the relationship between abusive supervision and employees’ unethical behavior. Furthermore, we focused on two potential underlying mechanisms, predicting that employees’ turnover intentions would mediate this relationship and the caring climate to moderate the relationship between turnover intentions and unethical behavior. Overall, we found support for our hypothesized model.

5.1. Theoretical Implications

Our research contributes to streams of literature on behavioral ethics and ethical decision making in several ways. First, by clarifying the relationship between abusive supervision and employees’ unethical behavior, we add a new perspective on unethical behavior and shed new light on the relationship between abusive supervision and unethical behavior. Reviewing the literature, we found that the previous research focuses on the inhibition of positive leadership on employees’ unethical behaviors [54]. However, few empirical studies have explored the direct influence of dark side of leadership on employees’ unethical behavior, thus leading us to pay attention to the effect of “praising virtue” but ignore the effect of “punishing vice.” Therefore, by drawing extensively from psychological contract theory, we focused on abusive supervision as an essential antecedent of employees’ unethical behavior. Previous research demonstrates that abusive supervision positively affects employees’ negative outcomes, such as counterproductive behaviors, supervisor-directed deviance, work-family conflict, psychological distress, and somatic health complaints [12,55,56,57,58,59,60]. However, research from an ethical perspective remains incomplete. We found evidence for a positive relationship between abusive supervision and employees’ unethical behavior. This result provides a wider perspective on the impact mechanism of abusive supervision and also provides support for the idea that the quality of the interpersonal treatment that employees receive from leaders is an important way in which leaders can influence followers’ unethical behavior [61]. Specifically, abusive supervision represents low- quality leader- employee relationships, it is contrary to the expectation or belief that one should be treated fairly, and it also violates the norms of reciprocity. In order to maintain the balance of relationship between the individuals and organization, employees may engage in covert and broader directed retaliation [62], such as unethical behavior, including behaviors such as falsifying expense reports, abusing sick time, and misusing company resources.
Second, our findings contribute to psychological contract theory by demonstrating that dark side of leadership provide a critical context that can induce subordinates’ unethical behavior via weakening subordinates’ psychological attachment towards organization. Previous studies focus on the psychological process of abused subordinates, for example, drawing on social cognitive theory, scholars indicated how social referents (i.e., abusive supervision) may influence subordinates’ unethical behavior through distinct process of moral agency [14]. It emphasized the important role of internalized self-standards and capacities for self-regulation on the mechanism between abusive supervision and unethical behavior. In addition, based on trait activation theory, the study indicated that abusive supervision can serve as a situational factor that activates the role of personality in predicting unethical behavior [8]. Unlike the prior researches, we used psychological contract theory, which emphasizes the reciprocal relationship between leaders or organizations and subordinates to illustrate that turnover intentions may be a crucial underlying mechanism that explain the positive association between abusive supervision and unethical behavior. Thus, we improve our understanding of which subordinates are likely to engage in unethical behavior in order to meaningfully balance relational processes between leaders and subordinates.
Finally, by focusing on organizational characteristics—caring climate—as a moderator, this study builds a better understanding of the conditions under which unethical behavior is likely to occur. Unethical behavior in organizations is viewed as a consequence of both organizational and individual influences. However, empirical studies investigating the causes of unethical behavior are relatively few and have little to say about the interaction between the individual and organizational perspectives [3]. To fill this void, this research studies the moderating effect of the caring climate. As predicted, the results show that the caring climate plays a role in overriding the indirect influence of abusive supervision on unethical behavior. Specifically, the indirect effect between abusive supervision and unethical behavior was stronger for those in a lower caring climate. This result clarifies the findings of Hershcovis et al. [63] and suggests that the overarching role of the organization may not be sufficient to override the more immediate influence of the supervisor.

5.2. Practical Implications

In practical terms, unethical behavior is costly for both managers and organizations in the Chinese context [64]. Our findings suggest two paths by which managers and organizations can minimize employees’ unethical behavior. The first is to take measures to reduce abusive supervision. Organizations should create a zero-tolerance culture regarding abusive behavior and offer abuse-prevention training for supervisors [65]. Moreover, research has revealed that relationship conflict leads supervisors to show abusive supervision [23]. Thus, organizations should encourage supervisors and subordinates to use suitable behavioral strategies to cope with relationship conflict. The second path to diminishing unethical behavior is to create a caring climate. A caring climate in an organization may prevent abusive supervision from resulting in unethical behavior on the part of employees. To increase the level of the caring climate, certain procedural designs and institutional arrangements should be set up in organizations, such as control over who is hired and promoted to a supervisory position. Employees who have displayed abusive traits toward coworkers need to be not only identified and stopped, but also prevented from rising to supervisory positions.

5.3. Limitations and Future Directions

Although this study contributes to the literature in several ways, it also has a few limitations. The first design limitation is our use of self-reported measures, which may make our findings vulnerable to the effects of common method variance. We argue that given our research goal, a same-source data design was acceptable. First, a meta-analysis shows that self-reports of sensitive data (such as unethical behavior) are more accurate than data collected from other- reports [66]. Further, in view of employees’ deviant behaviors are concealed and difficult to be detected by others are concealed and difficult to be detected by others, self-reports are more accurate [67]. Second, scholars [68,69] have concluded that, if a moderating effect is found, it should be taken as strong evidence that CMV did not present a bias. Notably, our results show significant moderating effects and that the indirect effects varied over levels of the moderator variable. Though common method variance is not a serious problem in this study, this limitation should nonetheless be noted. Future research could examine these relationships with self and other reports to provide greater confidence in the study’s inferences. Second, we collected data at the same time in an effort to establish the temporal precedence of abusive supervision and unethical behavior. We also took steps to enhance the nature of causality in our study. We ordered the variables in the data collection efforts in such a way to ensure the causal chain was accurately represented. The ordering of variables in the model was based on strong theory, as well as experimental research. A future study could use a longitudinal design to confirm the casual relationship between abusive supervision and unethical behavior. Third, our data supported the importance of the caring climate in understanding the effect of abusive supervision on unethical behavior. However, it is also possible that the other boundary conditions may be used to explore the above relationship. Thus, we encourage future studies to contribute to the unethical behavior literature by examining the moderating impact of other variables. Finally, this study has provided insight into abusive supervision in the Chinese cultural context and responded to Wei and Si’s [70] call for further studies of the abusive supervision of Chinese employees. Future research may focus on national and cultural differences in abusive supervision [13], as cross-cultural research could establish whether large power distances and other cultural and institutional factors render Asian employees especially vulnerable to factors leading to abusive supervision [71].

6. Conclusions

Why do employees engage in unethical behavior when they are subjected to abusive supervision? As predicted, this research found that (1) abusive supervision has a significant direct impact on the unethical behavior of Chinese employees, (2) turnover intentions mediate the relationship between abusive supervision and unethical behavior, and (3) this indirect effect is conditional upon the caring climate. The testing of these mediation and moderated mediation mechanisms helped to answer the question above. Moreover, our findings extend the unethical behavior literature and suggest methods that managers can follow to reduce their employees’ unethical behavior.

Author Contributions

Conceptualization, S.L.; Data curation, Q.Z.; Formal analysis, F.W.; Software, Q.Z.; Supervision, F.W.; Writing—original draft, S.L.; Writing—review and editing, Q.Z. and F.W.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Jones, T.M. Ethical decision making by individuals in organizations: An issue-contingent model. Acad. Manag. Rev. 1991, 16, 366–395. [Google Scholar]
  2. Treviño, L.K. Ethical decision making in organizations: A person-situation interactionist model. Acad. Manag. Rev. 1986, 11, 601–617. [Google Scholar]
  3. Treviño, L.K.; Youngblood, S.A. Bad apples in bad barrels: A causal analysis of ethical decision-making behavior. J. Appl. Psychol. 1990, 75, 378–385. [Google Scholar]
  4. Treviño, L.K.; Weaver, G.R.; Reynolds, S.J. Behavioral ethics in organizations: A review. J. Manag. 2006, 32, 951–990. [Google Scholar]
  5. Simpson, J.C. Wall Street’s courting of MBAs proceeds apace despite scandals. Wall Str. J. 1987, 2, 1. [Google Scholar]
  6. Hegarty, W.H.; Sims, H.P. Some determinants of unethical decision behavior: An experiment. J. Appl. Psychol. 1978, 63, 451–457. [Google Scholar]
  7. Hegarty, W.H.; Sims, H.P. Organizational philosophy, policies, and objectives related to unethical decision behavior: A laboratory experiment. J. Appl. Psychol. 1979, 64, 331–338. [Google Scholar]
  8. Greenbaum, R.L.; Hill, A.; Mawritz, M.B.; Quade, M.J. Employee Machiavellianism to unethical behavior: The role of abusive supervision as a trait activator. J. Manag. 2017, 43, 585–609. [Google Scholar]
  9. Mayer, D.M.; Kuenzi, M.; Greenbaum, R.; Bardes, M.; Salvador, R.B. How low does ethical leadership flow? Test of a trickle-down model. Organ. Behav. Hum. Decis. Processes 2009, 108, 1–13. [Google Scholar]
  10. Brown, M.E.; Mitchell, M.S. Ethical and unethical leadership: Exploring new avenues for future research. Bus. Ethics Q. 2010, 20, 583–616. [Google Scholar]
  11. Mo, S.; Shi, J. Linking ethical leadership to employees’ organizational citizenship behavior: Testing the multilevel mediation role of organizational concern. J. Bus. Ethics 2015, 21, 1–12. [Google Scholar]
  12. Tepper, B.J. Consequences of abusive supervision. Acad. Manag. J. 2000, 43, 178–190. [Google Scholar]
  13. Tepper, B.J. Abusive supervision in work organizations: Review, synthesis, and research agenda. J. Manag. 2007, 33, 261–289. [Google Scholar]
  14. Hannah, S.T.; Schaubroeck, J.M.; Peng, A.C.; Lord, R.G.; Trevino, L.K.; Kozlowski, S.W.; Doty, J. Joint influences of individual and work unit abusive supervision on ethical intentions and behaviors: A moderated mediation model. J. Appl. Psychol. 2013, 98, 579–592. [Google Scholar]
  15. Motro, D.; Ordóñez, L.D.; Pittarello, A.; Welsh, D.T. Investigating the effects of anger and guilt on unethical behavior: A dual-process approach. J. Bus. Ethics 2018, 152, 133–148. [Google Scholar]
  16. Rousseau, D. Psychological Contracts in Organizations: Understanding Written and Unwritten Agreements; Sage Publications: Thousand Oaks, CA, USA, 1995. [Google Scholar]
  17. Robinson, S.L. Trust and breach of the psychological contract. Adm. Sci. Q. 1996, 41, 574–599. [Google Scholar]
  18. Haque, A.; Fernando, M.; Caputi, P. The relationship between responsible leadership and organisational commitment and the mediating effect of employee turnover intentions: An empirical study with Australian employees. J. Bus. Ethics 2019, 156, 759–774. [Google Scholar]
  19. Kutaula, S.; Gillani, A.; Budhwar, P.S. An analysis of employment relationships in Asia using psychological contract theory: A review and research agenda. Hum. Resour. Manag. Rev. 2019. [Google Scholar] [CrossRef]
  20. Harvey, P.; Stoner, J.; Hochwarter, W.; Kacmar, C. Coping with abusive supervision: The neutralizing effects of ingratiation and positive affect on negative employee outcomes. Leadersh. Q. 2007, 18, 264–280. [Google Scholar]
  21. Kish-Gephart, J.J.; Harrison, D.A.; Treviño, L.K. Bad apples, bad cases, and bad barrels: Meta-analytic evidence about sources of unethical decisions at work. J. Appl. Psychol. 2010, 95, 1–31. [Google Scholar]
  22. Bies, R.J. Interactional (in) justice: The sacred and the profane. In Advances in Organizational Behavior; Greenberg, J., Cropanzano, R., Eds.; Stanford University Press: Stanford, CA, USA, 2001; forthcoming. [Google Scholar]
  23. Tepper, B.J.; Moss, S.E.; Duffy, M.K. Predictors of abusive supervision: Supervisor perceptions of deep-level dissimilarity, relationship conflict, and subordinate performance. Acad. Manag. J. 2011, 54, 279–294. [Google Scholar]
  24. Doden, W.; Grote, G.; Rigotti, T. Does leader–member exchange buffer or intensify detrimental reactions to psychological contract breach? The role of employees’ career orientation. J. Vocat. Behav. 2018, 106, 192–208. [Google Scholar]
  25. Restubog, S.L.D.; Bordia, P.; Tang, R.L. Effects of psychological contract breach on performance of IT employees: The mediating role of affective commitment. J. Occup. Organ. Psychol. 2006, 79, 299–306. [Google Scholar]
  26. Turnley, W.H.; Bolino, M.C.; Lester, S.W.; Bloodgood, J.M. The impact of psychological contract fulfillment on the performance of in-role and organizational citizenship behaviors. J. Manag. 2003, 29, 187–206. [Google Scholar]
  27. Robinson, S.L.; Rousseau, D.M. Violating the psychological contract: Not the exception but the norm. J. Organ. Behav. 1994, 15, 245–259. [Google Scholar]
  28. Mobley, W.H. Employee Turnover: Causes, Consequences, and Control; Addison-Wesley: Reading, MA, USA, 1982. [Google Scholar]
  29. Mobley, W.H. Intermediate linkages in the relationship between job satisfaction and employee turnover. J. Appl. Psychol. 1977, 62, 237–240. [Google Scholar]
  30. Tekleab, A.G.; Taylor, M.S. Aren’t there two parties in an employment relationship? Antecedents and consequences of organization-employee agreement on contract obligations and violations. J. Organ. Behav. 2003, 24, 585–608. [Google Scholar]
  31. Huang, X.; Wright, R.P.; Chiu, W.C.; Wang, C. Relational schemas as sources of evaluation and misevaluation of leader–member exchanges: Some initial evidence. Leadersh. Q. 2008, 19, 266–282. [Google Scholar]
  32. Schyns, B.; Schilling, J. How bad are the effects of bad leaders? A meta-analysis of destructive leadership and its outcomes. Leadersh. Q. 2013, 24, 138–158. [Google Scholar]
  33. Burris, E.R.; Detert, J.R.; Chiaburu, D.S. Quitting before leaving: The mediating effects of psychological attachment and detachment on voice. J. Appl. Psychol. 2008, 93, 912–922. [Google Scholar]
  34. Mai, K.M.; Ellis, A.P.; Christian, J.S.; Porter, C.O. Examining the effects of turnover intentions on organizational citizen- ship behaviors and deviance behaviors: A psychological contract approach. J. Appl. Psychol. 2016, 101, 1067–1081. [Google Scholar]
  35. Wood, R.; Bandura, A. Social cognitive theory of organizational management. Acad. Manag. Rev. 1989, 14, 361–384. [Google Scholar]
  36. Rusaw, A.C. Leading Public Organizations an Interactive Approach; Harcourt College Publishers: New York, NY, USA, 2001. [Google Scholar]
  37. Barnett, T.; Vaicys, C. The moderating effect of individuals’ perceptions of ethical work climate on ethical judgments and behavioral intentions. J. Bus. Ethics 2000, 27, 351–362. [Google Scholar]
  38. Elm, D.R.; Nichols, M.L. An investigation of the moral reasoning of managers. J. Bus. Ethics 1993, 12, 817–833. [Google Scholar]
  39. Fu, W.; Deshpande, S.P. The impact of caring climate, job satisfaction, and organizational commitment on job performance of employees in a China’s insurance company. J. Bus. Ethics 2014, 124, 339–349. [Google Scholar]
  40. Victor, B.; Cullen, J.B. The organizational bases of ethical work climates. Adm. Sci. Q. 1988, 101–125. [Google Scholar]
  41. Cullen, J.B.; Parboteeah, K.P.; Victor, B. The effects of ethical climates on organizational commitment: A two-study analysis. J. Bus. Ethics 2003, 46, 127–141. [Google Scholar]
  42. Okpara, J.O.; Wynn, P. The impact of ethical climate on job satisfaction, and commitment in Nigeria: Implications for management development. J. Manag. Dev. 2008, 27, 935–950. [Google Scholar]
  43. Goldman, A.; Tabak, N. Perception of ethical climate and its relationship to nurses’ demographic characteristics and job satisfaction. Nurs. Ethics 2010, 17, 233–246. [Google Scholar]
  44. Deshpande, S.P. The impact of ethical climate types on facets of job satisfaction: An empirical investigation. J. Bus. Ethics 1996, 15, 655–660. [Google Scholar]
  45. Filipova, A.A. Relationships among ethical climates, perceived organizational support, and intent-to-leave for licensed nurses in skilled nursing facilities. J. Appl. Gerontol. 2011, 30, 44–66. [Google Scholar]
  46. Wimbush, J.C.; Shepard, J.M. Toward an understanding of ethical climate: Its relationship to ethical behavior and supervisory influence. J. Bus. Ethics 1994, 13, 637–647. [Google Scholar]
  47. Chen, C.C.; Chen, M.Y.C.; Liu, Y.C. Negative affectivity and workplace deviance: The moderating role of ethical climate. Int. J. Hum. Resour. Manag. 2013, 24, 2894–2910. [Google Scholar]
  48. Bozeman, D.P.; Perrewé, P.L. The effect of item content overlap on organizational commitment questionnaire–turnover cognitions relationships. J. Appl. Psychol. 2001, 86, 161–173. [Google Scholar]
  49. Chen, Y.J.; Tang, T.L.P. Attitude toward and propensity to engage in unethical behavior: Measurement invariance across major among university students. J. Bus. Ethics 2006, 69, 77–93. [Google Scholar]
  50. Hayes, A.F. Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression-Based Approach; The Guilford Press: New York, NY, USA, 2013. [Google Scholar]
  51. James, L.R.; Mulaik, S.A.; Brett, J. Causal Analysis: Assumptions, Models, and Data; Sage Publications: Beverly Hills, CA, USA, 1982. [Google Scholar]
  52. Fornell, C.; Larcker, D.F. Evaluating structural equation models with unobservable variables and measurement error. J. Mark. Res. 1981, 18, 39–50. [Google Scholar]
  53. Podsakoff, P.M.; MacKenzie, S.B.; Lee, J.Y.; Podsakoff, N.P. Common method biases in behavioral research: A critical review of the literature and recommended remedies. J. Appl. Psychol. 2003, 88, 879–903. [Google Scholar]
  54. Effelsberg, D.; Solga, M.; Gurt, J. Transformational leadership and follower’s unethical behavior for the benefit of the company: A two-study investigation. J. Bus. Ethics 2014, 120, 81–93. [Google Scholar]
  55. Carlson, D.; Ferguson, M.; Hunter, E.; Whitten, D. Abusive supervision and work–family conflict: The path through emotional labor and burnout. Leadersh. Q. 2012, 23, 849–859. [Google Scholar]
  56. Duffy, M.K.; Ganster, D.C.; Pagon, M. Social undermining in the workplace. Acad. Manag. J. 2002, 45, 331–351. [Google Scholar]
  57. Detert, J.R.; Treviño, L.K.; Burris, E.R.; Andiappan, M. Managerial modes of influence and counterproductivity in organizations: A longitudinal business-unit-level investigation. J. Appl. Psychol. 2007, 92, 993–1005. [Google Scholar]
  58. Mitchell, M.S.; Ambrose, M.L. Abusive supervision and workplace deviance and the moderating effects of negative reciprocity beliefs. J. Appl. Psychol. 2007, 92, 1159–1168. [Google Scholar]
  59. Restubog, S.L.D.; Scott, K.L.; Zagenczyk, T.J. When distress hits home: The role of contextual factors and psychological distress in predicting employees’ responses to abusive supervision. J. Appl. Psychol. 2011, 96, 713. [Google Scholar]
  60. Shoss, M.K.; Eisenberger, R.; Restubog, S.L.D.; Zagenczyk, T.J. Blaming the organization for abusive supervision: The roles of perceived organizational support and supervisor’s organizational embodiment. J. Appl. Psychol. 2013, 98, 158–168. [Google Scholar]
  61. Treviño, L.K.; Brown, M.E. The role of leaders in influencing unethical behavior in the workplace. In Managing Organizational Deviance; Kidwell, R.E., Martin, C.L., Eds.; Sage: London, UK, 2005; pp. 69–87. [Google Scholar]
  62. Ambrose, M.L.; Seabright, M.A.; Schminke, M. Sabotage in the workplace: The role of organizational injustice. Organ. Behav. Hum. Decis. Processes 2002, 89, 947–965. [Google Scholar]
  63. Hershcovis, M.S.; Turner, N.; Barling, J.; Arnold, K.A.; Dupré, K.E.; Inness, M.; LeBlanc, M.M.; Sivanathan, N. Predicting workplace aggression: A meta-analysis. J. Appl. Psychol. 2007, 92, 228–238. [Google Scholar]
  64. Liu, S.; Lin, X.; Hu, W. How followers’ unethical behavior is triggered by leader-member exchange: The mediating effect of job satisfaction. Soc. Behav. Personal. Int. J. 2013, 41, 357–366. [Google Scholar]
  65. Liu, J.; Kwan, K.H.; Wu, L.Z.; Wu, W. Abusive supervision and subordinate supervisor-directed deviance: The moderating role of traditional values and the mediating role of revenge cognitions. J. Occup. Organ. Psychol. 2010, 83, 835–856. [Google Scholar]
  66. Berry, C.M.; Carpenter, N.C.; Barratt, C.L. Do other-reports of counterproductive work behavior provide an incremental contribution over self-reports? A meta-analytic comparison. J. Appl. Psychol. 2012, 97, 613–636. [Google Scholar]
  67. Lee, R.M. Doing Research on Sensitive Topics; Sage: London, UK, 1993. [Google Scholar]
  68. Podsakoff, P.M.; MacKenzie, S.B.; Podsakoff, N.P. Sources of method bias in social science research and recommendations on how to control it. Annu. Rev. Psychol. 2012, 63, 539–569. [Google Scholar]
  69. Siemsen, E.; Roth, A.; Oliveira, P. Common method bias in regression models with linear, quadratic, and interaction effects. Organ. Res. Methods 2010, 13, 456–476. [Google Scholar]
  70. Wei, F.; Si, S. Psychological contract breach, negative reciprocity, and abusive supervision: The mediated effect of organizational identification. Manag. Organ. Rev. 2013, 9, 541–561. [Google Scholar]
  71. Snell, R.S.; Yi, Z.; Chak, A.M. Representational predicaments for employees: Their impact on perceptions of supervisors’ individualized consideration and on employee job satisfaction. Int. J. Hum. Resour. Manag. 2013, 24, 1646–1670. [Google Scholar]
Figure 1. Theoretical model tested in this study.
Figure 1. Theoretical model tested in this study.
Ijerph 16 04187 g001
Figure 2. Interaction between turnover intentions and caring climate with regard to unethical behavior.
Figure 2. Interaction between turnover intentions and caring climate with regard to unethical behavior.
Ijerph 16 04187 g002
Table 1. Results of confirmatory factor analysis.
Table 1. Results of confirmatory factor analysis.
Model χ2dfCFITLIRMSEASRMR
Four-factor model2220.408540.920.910.050.05
Three-factor model a2747.308570.880.860.080.09
Two-factor model b3536.628590.830.810.100.12
One-factor model c4139.719030.780.760.130.14
a This model combines turnover intentions and unethical behavior into one factor. b This model combines turnover intentions, caring climate, and unethical behavior into one factor. c This model combines all measurement items into one grand factor. Both three-factor model and two-factor model reported in Table 1 are optimal. CFI: Comparative Fit Index. TLI: Tucker–Lewis Index. RMSEA: Root Mean Square Error of Approximation. SRMR: Standardized Root Mean Square Residual.
Table 2. Descriptive statistics, correlations, and reliabilities.
Table 2. Descriptive statistics, correlations, and reliabilities.
VariablesMSD1234Square Root AVE
1. Abusive supervision1.360.440.94 (0.81)
2. Turnover intentions2.100.760.40 **0.89 (0.78)
3. Caring climate4.130.77−0.43 **−0.42 **0.89 (0.75)
4. Unethical behavior1.210.390.52 **0.33 **−0.39 **0.95(0.83)
N = 679. Cronbach’s alpha are in italics and appear on the diagonal; * p < 0.05 (two-tailed test); ** p < 0.01 (two-tailed test). AVE: Average Variance Extracted.
Table 3. Mediation and moderated mediation estimates.
Table 3. Mediation and moderated mediation estimates.
Mediation Model
Direct EffectsCoefficientSEtpModel R2
Turnover intentions as DV
Constant0.910.234.000.0000.18 ***
Abusive supervision0.390.0411.110.000
Unethical behavior as DV
Constant0.200.220.930.3510.29 ***
Abusive supervision0.470.0413.040.000
Turnover intentions0.140.043.920.000
Indirect effectEffectBoot SEBoot LLCIBoot ULCI
Abusive supervision on unethical behavior0.0550.0220.0130.098
Moderated Mediation
Direct EffectsCoefficientSEtpModel R2
Turnover intentions as DV
Constant0.910.234.000.0000.18 ***
Abusive supervision0.390.0411.110.000
Unethical behavior as DV
Constant1.310.0815.740.0000.32 ***
Abusive supervision0.160.0111.060.000
Turnover intentions0.030.011.800.072
Caring climate−0.070.01−4.640.000
Turnover intentions *
Caring climate
−0.030.01−2.550.011
Conditional indirect effectsEffectBoot SEBoot LLCIBoot ULCI
Low caring climate0.0220.0090.0040.040
Mean caring climate0.0100.010−0.0010.030
High caring climate−0.0010.014−0.0300.025
Index of moderated mediationIndexBoot SEBoot LLCIBoot ULCI
Turnover intentions *
Caring climate
−0.0110.006−0.025−0.001
N = 679. Effect size estimates are unstandardized coefficients. Boot 5000 bootstrap samples, LLCI bias corrected lower limit confidence interval ULCI bias corrected upper limit confidence interval. *** p < 0.001. Turnover intentions * Caring climate is Turnover intentions × Caring climate, when testing the moderating role of caring climate, it means the interaction between Turnover intentions and Caring climate.

Share and Cite

MDPI and ACS Style

Liu, S.; ZHU, Q.; Wei, F. How Abusive Supervision Affects Employees’ Unethical Behaviors: A Moderated Mediation Examination of Turnover Intentions and Caring Climate. Int. J. Environ. Res. Public Health 2019, 16, 4187. https://doi.org/10.3390/ijerph16214187

AMA Style

Liu S, ZHU Q, Wei F. How Abusive Supervision Affects Employees’ Unethical Behaviors: A Moderated Mediation Examination of Turnover Intentions and Caring Climate. International Journal of Environmental Research and Public Health. 2019; 16(21):4187. https://doi.org/10.3390/ijerph16214187

Chicago/Turabian Style

Liu, Shubo, Qianlin ZHU, and Feng Wei. 2019. "How Abusive Supervision Affects Employees’ Unethical Behaviors: A Moderated Mediation Examination of Turnover Intentions and Caring Climate" International Journal of Environmental Research and Public Health 16, no. 21: 4187. https://doi.org/10.3390/ijerph16214187

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop