Next Article in Journal
Assessing the Visualization-Based Decision Support System for Environmental Impact Assessments
Next Article in Special Issue
The Robust Italian Validation of the Coping Humor Scale (RI-CHS) for Adult Health Care Workers
Previous Article in Journal
Cochlear Implant Results in Older Adults with Post-Lingual Deafness: The Role of “Top-Down” Neurocognitive Mechanisms
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJERPH
Volume 19
Issue 3
10.3390/ijerph19031344
ijerph-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Burnout Assessment Tool (BAT): Validity Evidence from Brazil and Portugal

by
Jorge Sinval
1,2,3,4,*,
Ana Claudia S. Vazquez
5,
Claudio Simon Hutz
6,
Wilmar B. Schaufeli
7,8 and
Sílvia Silva
1
1
Business Research Unit (BRU-IUL), Instituto Universitário de Lisboa (ISCTE-IUL), 1649-026 Lisbon, Portugal
2
William James Center for Research, ISPA—Instituto Universitário, 1149-041 Lisbon, Portugal
3
Faculty of Philosophy, Sciences and Languages of Ribeirão Preto, University of São Paulo, Ribeirão Preto 14040-901, SP, Brazil
4
Faculdade de Medicina, Universidade de Lisboa, 1649-028 Lisbon, Portugal
5
Department of Psychology, Universidade Federal de Ciências da Saúde de Porto Alegre, Porto Alegre 90050-170, RS, Brazil
6
Department of Psychology, Universidade Federal do Rio Grande do Sul, Porto Alegre 90035-002, RS, Brazil
7
Research Unit Occupational & Organizational Psychology and Professional Learning KU Leuven, 3000 Leuven, Belgium
8
Department of Psychology, Utrecht University, 3584 CS Utrecht, The Netherlands
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2022, 19(3), 1344; https://doi.org/10.3390/ijerph19031344
Submission received: 23 December 2021 / Revised: 17 January 2022 / Accepted: 21 January 2022 / Published: 25 January 2022
(This article belongs to the Special Issue The Assessment of Wellbeing and Mental Health: A Global Public Health Issue)
Browse Figures
Versions Notes

Abstract

:
The Burnout Assessment Tool (BAT) has been gaining increased attention as a sound and innovative instrument in its conceptualization of burnout. BAT has been adapted for several countries, revealing promising validity evidence. This paper aims to present the psychometric properties of the Brazilian and Portuguese versions of the BAT in both the 23-item and 12-item versions. BAT’s validity evidence based on the internal structure (dimensionality, reliability, and measurement invariance) and validity evidence based on the relations to other variables are the focus of research. A cross-sectional study was conducted with two non-probabilistic convenience samples from two countries (N = 3103) one from Brazil (nBrazil = 2217) and one from Portugal (nPortugal = 886). BAT’s original structure was confirmed, and it achieved measurement invariance across countries. Using both classic test theory and item response theory as frameworks, the BAT presented good validity evidence based on the internal structure. Furthermore, the BAT showed good convergent evidence (i.e., work engagement, co-worker support, role clarity, work overload, and negative change). In conclusion, the psychometric properties of the BAT make this freely available instrument a promising way to measure and compare burnout levels of Portuguese and Brazilian workers.

1. Introduction

Although the burnout syndrome appeared in the 1970s, it is still a global issue such that the 11th revision of the International Classification of Diseases of World Health Organization (ICD-11) defines it as an occupational phenomenon with risk of harming health [1]. The adopted definition of burnout in the ICD-11 comprises three factors (exhaustion, cynicism, and reduced professional efficacy) as the framework proposed by Maslach et al. [2]. However, the conceptualization of burnout is somewhat controversial [3]; for example, a meta-analytical study on the physicians’ burnout found 142 unique definitions of burnout with at least 47 unique definitions using MBI. Some constructs, such as depression and fatigue, are conceptually linked to job burnout [4,5]. These phenomena are potentially part of the process of long-term sick leave. At the core of burnout lies severe fatigue (i.e., exhaustion); however persistently fatigued workers are not necessarily (by definition) in burnout, nor must burned-out workers necessarily report fatigue as the main complaint [5]. Occupational fatigue has been linked to an imbalance between the intensity and duration and timing of work with recovery time [6]. Studies over decades have shown evidence that burnout syndrome predicts various negative consequences to individuals and organizations, such as cardiovascular diseases, hypercholesterolemia, type 2 diabetes, coronary heart disease, musculoskeletal disorders, prolonged fatigue, headaches, gastrointestinal issues, mood disturbance, depressive symptoms, absenteeism, poor performance, insomnia, depressive symptoms, and life and job dissatisfaction [7,8,9,10,11,12,13,14,15,16].
Research shows that job demands (e.g., work overload) are more associated with job burnout, while job resources (e.g., co-worker support) are more related to job burnout’s antipode, i.e., work engagement [17]. Nowadays, researchers claim the COVID-19 pandemic has posed strain and increased workload and job stress, particularly in healthcare workers, who have presented a higher risk of burnout than other occupations [18,19,20]. Going beyond the individual consequences of burnout, recent research has also investigated burnout in a large range of occupations, organizations, and countries [21,22,23,24,25,26]. The literature has firmly established that burnout is not only detrimental for workers’ health but also has negative effects at the organizational level.
The most widely used instrument to assess burnout is the Maslach Burnout Inventory (MBI) [27]. Despite MBI’s early contribution to enlighten burnout as an important psychological state to be deeply studied, researchers are still discussing its theoretical framework and psychometric basis and the practical applicability of this instrument [21,28]. Schaufeli et al. [29] summarize these criticisms of MBI as including the following as the most important: (a) the questioning of the validity evidence of the constituting dimensions of burnout, (b) the lack of clinically established cut-off values, (c) the lack of representative and national samples to ground its statistical norms, (d) the limitations of its practical usability, and (e) the inconsistent dimensionality also in the cross-national studies on MBI [30]. Finally, with similar problems and weaknesses, there are other burnout measures, such as the Copenhagen Burnout Inventory [31], the Oldenburg Burnout Inventory [32], and, recently, the COVID-19 Burnout Scale [33].

1.1. Burnout Assessment Tool (BAT)

Taken together, these criticisms call for an alternative instrument to assess burnout and overcome these flaws using a novel conceptualization of the matter, which has been addressed by the development of the Burnout Assessment Tool (BAT) [34]. The BAT instrument assumes that burnout is a syndrome assessed by core symptoms (exhaustion, mental distance, emotional impairment, and cognitive impairment) and secondary symptoms (psychological distress and psychosomatic complaints), which could be associated with depressed mood and other comorbidities. Therefore, BAT considers burnout a second-order factor that acts as a syndrome, meaning that all four components are connected and belong to the same higher-order construct, i.e., burnout [21]. Based on the Job Demands–Resources Model (JD-R) [17], the key components constituting the burnout process are the draining energy that leads to feeling exhausted and extremely tired at the same time that the distancing mentally manifests itself as a lack of interest and aversion to work [35]. In addition, in-depth interviews with experts brought two significant dimensions of burnout, which were not known until then, which are emotional impairment and cognitive impairment. These dimensions affect one’s self-regulation to deal adequately with the daily working activities and to recover self-energy linked to the motivational process [36].
Meanwhile, BAT has largely been investigated [37,38,39,40,41] and has demonstrated measurement invariance between seven countries in Europe and Japan [21]. As in the works of De Beer et al. [21] and Sakakibara et al. [28], the current study is based on the BAT reconceptualization of burnout as a work-related state of exhaustion, extreme tiredness with reduced ability to regulate cognitive and emotional processes, and mental distancing. It can develop depressed mood as well as non-specific psychological and psychosomatic complaints [34]. Despite using the raw scores of only one item from MBI (i.e., “I feel exhausted at the end of the working day”), Schaufeli [24] found medium levels of burnout in Portugal, in comparison with a random sample of workers from thirty-five European countries (n = 43,675), using data from the 6th European Working Conditions Survey [42]. While in Brazil there is no publication reporting burnout scores using a survey (conducted with a representative sample) at the national level.
This current study is focused on the psychometric properties of BAT from a cross-national perspective (i.e., Brazil and Portugal). The main goal is to assess BAT’s validity evidence based on the internal structure and based on the relations to other variables.

1.2. Research Hypotheses

Following the recommendations of the Standards for Educational and Psychological Testing [43], this study aims to evaluate two types of validity evidence for both for BAT’s Portuguese and Brazilian version: one related to the internal structure, and one based on the relations to other variables (i.e., work engagement, role clarity, co-worker support, work overload, and negative change). BAT’s original structure was successfully confirmed in several countries in a study by De Beer et al. [21] with data from Austria, Belgium (Flanders), Finland, Germany, Ireland, Japan, and the Netherlands. The Japanese version of BAT was also confirmed in a different study [28], while the South Korean version maintained the hierarchical structure with four first-order dimensions, albeit with the removal of one item from the mental distance factor [44]. The Russian version also provided evidence indicative of the stability of the hierarchical structure [45]. Altogether, it is expected that the hypothesized hierarchical structure for BAT-23 (one second-order latent variable with four first-order factors, 23 indicators) and BAT-12 (second-order factor with four first-order dimensions, 12 items) hold with a satisfactory fit to the data in both Brazil and Portugal (H1). The reliability of the scores is one of the key components of the internal structure of any psychometric instrument [46]. It can be analyzed using four different types of approaches: internal consistency, test–retest, parallel forms, and interrater agreement. Previous research showed good evidence of internal consistency estimates using the ordinal α [47] for both second-order factor and first-order dimensions [21]. BAT’s also presented satisfactory evidence in terms of test–retest evidence [48]. The second hypothesis (H2) states that BAT presents satisfactory internal consistency estimates (≥0.80) [46]. Measurement invariance is another component of the validity evidence based on the internal structure; it is an essential feature that is necessary before any substantive group comparisons (e.g., countries, sex) can be established. BAT has shown measurement invariance between seven countries Austria, Belgium (Flanders), Finland, Germany, Ireland, Japan, and the Netherlands [21]. Regarding sex, there is no single study investigating measurement invariance among sex. However, it is known that sex might be an important factor regarding burnout [49], it is expected that females present higher levels of burnout [50] namely in terms of exhaustion [51,52]. While others did not reach definite conclusions [25].
Other instruments measuring burnout and related constructs have previously shown measurement invariance among workers from Brazil and Portugal [53] and among sex within the two mentioned countries [54,55]. Reinforcing the similarity of the measurement structure of psychometric instruments among workers from the two countries. It is hypothesized (H3) that BAT holds measurement invariance among countries (Brazil and Portugal), and sex.
Another important source of validity evidence is provided by the relationship of instrument scores to external variables to the instrument [43]. This source of evidence allows understanding if the interpretation of the scores can be done as expected by the nomological network of constructs [56]. The JD-R model identifies possible antecedents of job burnout [57]. The central idea of the JD-R model is that working conditions, which are specific to every occupation, can generally be classified as either job demands or job resources, and those job characteristics will contribute to job burnout and work engagement [58]. The JD-R model suggests that work engagement is negatively related to burnout, since high job demands lead to a health impairment process (i.e., job burnout) and high resources will lead to a motivational process, i.e., work engagement [59]. Several meta-analyses have supported the relationship between job demands and resources and burnout [60,61,62]. In these studies, several job demands and resources were identified. For instance, social support, workload, and role clarity have been found as relevant demands and resources. As such, it is expected to observe a positive association between burnout and job demands and a negative relation between burnout and job resources [28,39,63]. It is anticipated that BAT’s scores are negatively correlated with work engagement, role clarity, and co-worker support and positively correlated with work overload, and negative change (H4).

2. Methods

2.1. Sampling, and Data Collection

In this cross-sectional survey, a non-probabilistic convenience sample was collected. The inclusion criteria consisted of participants being able to read Portuguese and having easy access to a smartphone, PC, or tablet where they could open a digital questionnaire. The authors invited workers from Brazil and Portugal to participate. Considering BAT’s second-order factor and four first-order dimensions with 23 manifest variables, it results in a total of 226 degrees of freedom [64], assuming that the population RMSEA should be not higher than 0.08 (i.e., ε₀ = 0.08; H0: ε ≥ 0.08), since rejecting this hypothesis will lead to the conclusion that the model fit is better than 0.08, the recommended cutoff for a reasonable fit [65]. Additionally, the true population RMSEA was considered to be ε = 0.064 based on the findings from de Beer’s et al. [21] study using a sample of 10,138 participants. Altogether, for an α = 0.05, β = 0.20 (i.e., power = 0.80) resulted in a required sample size of n = 171 [66].

2.2. Constructs and Psychometric Instruments

All used measures were used in their adapted version to Brazilian and Portuguese contexts.

2.2.1. Job Burnout

The BAT was used to assess burnout [29] through the development of two transculturally adapted versions: one for Brazil and one for Portugal (Table 1). The BAT-23 is a self-report psychometric instrument that comprises 23 items to be answered using a five-point rating scale (1—“Never”; 2—“Rarely”; 3—“Sometimes”; 4—“Often”; 5—“Always”). The BAT-23 version measures burnout’s core symptoms, and another version is also available that also includes items to assess burnout’s secondary symptoms. To develop the Portuguese version (Table 1), the BAT’s English version was used [34] following the ITC Guidelines for Translating and Adapting Tests [67]. BAT-23 operationalizes burnout as a second-order construct with four first-order factors: exhaustion (eight items), mental distance (five items), cognitive impairment (five items), and emotional impairment (five items). From BAT-23’s items, it is possible to extract a short version (BAT-12) with three items per each first-order latent construct (Table 1).

2.2.2. Work Engagement

Work engagement refers to a positive motivational state and is composed of vigor, dedication, and absorption [68]. This construct was measured with the ultra-short version of the Utrecht Work Engagement Scale (UWES-3) [69], which used items from the short-version (i.e., UWES-9). The used UWES-3 items have been previously adapted with success to Portugal and Brazil [70,71]. It uses a seven-point ordinal scale (0—“Never”; 1—“Almost never”; 2—“Rarely”; 3—“Sometimes”; 4—“Often”; 5—“Very often”; 6—“Always”), with one item pertaining to each of the three dimensions. The UWES has shown good convergent evidence with the burnout scores since work engagement and burnout are moderately and negatively related [29,72]. The UWES already presented measurement invariance among Brazil and Portugal in the 9-item version [70]. One example of an item is: “At my work, I feel bursting with energy.”

2.2.3. Co-Worker Support

Co-worker support refers to the function and quality of social relationships at work, such as perceived availability of help from coworkers or support actually received [73]. To assess the perceptions of co-worker support, the co-worker support sub-scale (3 items) of the Energy Compass psychometric instrument [74] was used. The items were answered using an ordinal five-point scale (1—“Never”; 2—“Seldom”; 3—“Sometimes”; 4—“Often”; 5—“Always”). One example of an item is: “Can you count on your colleagues for help and support when needed?”

2.2.4. Role Clarity

The clarity of the role assesses the extent to which the tasks to be performed are clearly defined and the expectations and responsibilities for the employee are clear [75]. This construct was assessed using the sub-scale role clarity of the Energy Compass psychometric instrument [74]. The three items of the sub-scale were answered using a five-point ordinal scale (1—“Never”; 2—“Seldom”; 3—“Sometimes”; 4—“Often”; 5—“Always”). One example of an item is: “Is it sufficiently clear what you need to do in your job?”

2.2.5. Work Overload

Work overload can be defined as the extent to which the employee has to deal with changes in job content, ICT systems, and leadership, as well as in the organization as a whole. Four items answered using a five-point ordinal scale (1—“Never”; 2—“Seldom”; 3—“Sometimes”; 4—“Often”; 5—“Always”) were used, as suggested in Schaufeli et al. [76]. One example of an item is, “Do you have too much work to do?”

2.2.6. Negative Change

Negative change refers to the pessimistic views produced by the introduction of modifications at work, e.g., pace of work, interpersonal conflict, work-home conflict, and use of skills [77]. The negative change construct was assessed using the corresponding subscale (three items) from the Energy Compass psychometric instrument [74] answered in a five-point scale of frequency (1—“Never”; 2—“Seldom”; 3—“Sometimes”; 4—“Often”; 5—“Always”). An example of an item is, “Do changes cause turmoil in your company?”

2.3. Procedure

Data were collected simultaneously in Portugal and Brazil. The workers were invited to participate through social networks or e-mail. Firstly, the participants were presented with the electronic informed consent, which they had to accept to participate in the study. The digital survey was deployed using LimeSurvey [78] and SurveyMonkey [79], which contained a group of psychometric instruments together with a group of sociodemographic and job questions. To check how likely the research process would work, a pilot study was conducted with 15 workers, which provided feedback (e.g., potential issues with the digital platforms where the survey was deployed, clarity of the questions/items, and mean time of fulfillment).
All the subjects gave their informed consent for inclusion before they participated in the study. The study was conducted in accordance with the Declaration of Helsinki, and the study was approved by the Ethics Committee of the Federal University of Health Sciences of Porto Alegre Brazil, (CAAE 78617617.8.0000.5345; 25 October 2017).

2.4. Data Analysis

To conduct the statistical analysis the statistical programming language R [80] through the integrated development environment, RStudio [81] was used. To estimate the adequate sample size for the confirmatory factor analysis, the MBESS package [82] was used. The skimr package [83] and the table1 package [84] were utilized to produce the descriptive statistics. The skewness (sk) using the “sample” method (i.e., sample skewness of the distribution) and the kurtosis (ku) using the “sample excess” method (i.e., sample kurtosis of the distribution with a value of 3 being subtracted) were calculated using the PerformanceAnalytics package [85]. The coefficient of variation (CV) was estimated with the sjstats package [86], the standard error of the mean (SEM) was calculated with the plotrix package [87]. The mode was computed by the modeest package [88]. Absolute values of |sk| > 3 and |ku| > 7 were considered as severe univariate normality violations [89,90]. To evaluate the multivariate normality the psych package [91] was used to calculate Mardia’s multivariate kurtosis [92].
To obtain evidence about the originally proposed dimensionality of the measurement models, the confirmatory factor analysis (CFA) was used. The following goodness-of-fit indices were used: NFI (Normed Fit Index), TLI (Tucker–Lewis Index), CFI (Comparative Fit Index), RMSEA (Root Mean Square Error of Approximation), and SRMR (Standardized Root Mean Square Residual). Estimates above 0.95 are considered good for NFI, TLI, and CFI [93]. While values below 0.08 were considered good for SRMR and RMSEA [94]. The package lavaan [95] was used to run the CFA analysis using the Weighted Least Squares Means and Variances (WLSMV) estimator [96]. The WLSMV was chosen because it does not require multivariate normality as an assumption, and because all items of the used psychometric instruments have an ordinal response scale.
The Average Variance Extracted (AVE) was estimated to test the evidence for convergent validity [97]. Satisfactory convergent validity evidence in terms of the internal structure was assumed for AVE ≥ 0.5 [98].
Item response theory analysis was conducted using a multidimensional polytomous Rasch model [99] as a particular case of the multidimensional random coefficients multinomial logit model (MRCMLM) [100]. The TAM package [101] was used to conduct the multidimensional polytomous Rasch analysis. Wright maps (also known as item-person maps or item maps) were used to present the location of both items and respondents on the same scale [102,103]. The WrightMap package [104] was used to produce the Wright Maps. Two mean square fit statistics (i.e., infit and outfit) were used to assess how well the data fit the model [105]. Considering the ordinal nature of the rating scale (i.e., 1—“Never” to 5—“Always”), the interval (0.6; 1.4) was considered as reasonable for the item mean square ranges for infit and outfit statistics [106]. Values above 1 suggest an increasing quantity of answers diverging from model’s predictions, while values below 1 indicate answers with less heterogeneity than expected [107].
To assess the evidence of reliability of the first-order factors, the following estimators of internal consistency were used: composite reliability (CR) [97], the αordinal [47], and ω [108]. Values of ≥ 0.8 on the different mentioned estimators are considered indicative of acceptable reliability evidence [46,98]. The second-order latent factor also had estimates of internal consistency: the proportion of variance among first-order common factors that is attributable to the second-order factor (ωL2), the proportion of variance of a composite score calculated from the observed indicators that is attributable to the second-order factor (ωL1), and the proportion of observed variance explained by the second-order factor after partialling out the uniqueness from the first-order factors (ωpartial L1). Both second-order and first-order internal consistency estimates were calculated using the semTools package [109]. In the item response theory framework, the MRCMLM provided the expected a posteriori (EAP) reliability index for each latent factor. The EAP reliability is defined as the ratio of the variance of the EAPs and the variance of the plausible values [110]. Values of EAP reliability ≥ 0.8 are preferable.
Using the theta-parameterization for categorical items through the semTools package [109], measurement invariance was evaluated comparing a group of eight different models [111]: (I) configural invariance, (II) thresholds of the indicators, (III) first-order factor loadings, (IV) structural weights, (V) intercepts of the first-order factors, (VI) latent means, (VII) disturbances of the first-order factors, and (VIII) residual variances of observed variables. The differences between the nested models were compared using two criteria. The ΔCFI ≤ −0.010 criterion [112], which advocates the non-rejection of the null hypothesis of invariance if the ΔCFI is smaller or equal to −0.010, and the Δχ2 criterion [113], which does not reject the null hypothesis of invariance if a significant χ2 robust difference test is obtained.
The structural models were tested using the lavaan package [95] to test validity evidence based on relations to other variables. In the latent score means comparison, Cohen’s d [114] was used as an effect size measure. The doBy package [115] was used to compute the raw score percentiles. A significance level of 5% was used (α = 0.05).

3. Results

3.1. Descriptive Statistics of Study Participants

A merged sample of 3103 workers was collected (nBrazil = 2217; nPortugal = 886) 74.2% female, with an average of 37.2 (11.1) years old. More than half of the workers (53.4%) were professionals according to the International Standard Classification of Occupations (ISCO-08) [116], and 72.5% had graduation or a higher academic level. Table 2 presents the descriptive statistics for each country, and for the merged sample.

3.2. Validity Evidence Base on the Internal Structure

This source of validity evidence investigates the dimensionality, reliability of the scores, and measurement invariance.

3.2.1. Dimensionality

The distributional properties of BAT’s 23 items are presented in Table 3; these were used to judge distributional properties and psychometric sensitivity. None of the items in both countries presented severe univariate normality violations [89,90]. Mardia’s multivariate kurtosis [92] for the data from Brazil was 101.637 (p < 0.001), while for the data from Portugal it was 60.063 (p < 0.001). All items in both countries had the maximum range of possible answers, and no outliers were removed. These items’ distributional properties are indicative of appropriate psychometric sensitivity, as it would be expected that these items would follow an approximately normal distribution in the population under study. Despite these univariate normality indicators, the weighted least squares means and variances (WLSMV) [96] estimation method was used, taking into consideration the ordinal level of measurement of the items.
The BAT-23’s second-order latent factor model presented an acceptable fit (H1) to merged samples data (χ2(226) = 4884.023; p < 0.001; n = 3103; CFI = 0.988; NFI = 0.988; TLI = 0.987; SRMR = 0.057; RMSEA = 0.082; P(RMSEA ≤ 0.05) < 0.001; 90% CI ]0.080; 0.084[). Both the fit the to the data from Brazil (Figure 1; χ2(226) = 3398.828; p < 0.001; n = 2217; CFI = 0.988; NFI = 0.987; TLI = 0.987; SRMR = 0.058; RMSEA = 0.080; P(RMSEA ≤ 0.05) < 0.001; 90% CI ]0.077; 0.082[) and the fit to the data from Portugal (Figure 1; χ2(226) = 1841.686; p < 0.001; n = 886; CFI = 0.987; NFI = 0.986; TLI = 0.986; SRMR = 0.064; RMSEA = 0.090; P(RMSEA ≤ 0.05) < 0.001; 90% CI ]0.086; 0.094[) was satisfactory. All factor loadings (λi ≥ 0.57) and all structural weights (γi ≥ 0.81) were satisfactory for the joint data.
Regarding BAT-12, the fit to the data of the merged samples was satisfactory (H1; χ2(50) = 521.809; p < 0.001; n = 3103; CFI = 0.996; NFI = 0.995; TLI = 0.994; SRMR = 0.037; RMSEA = 0.055; P(RMSEA ≤ 0.05) = 0.023; 90% CI ]0.051; 0.059[), as was the fit to the data of each country model, i.e., Brazil (Figure 2; χ2(50) = 331.365; p < 0.001; n = 2217; CFI = 0.996; NFI = 0.995; TLI = 0.995; SRMR = 0.036; RMSEA = 0.050; P(RMSEA ≤ 0.05) = 0.439; 90% CI ]0.045; 0.056[) and Portugal (Figure 2; χ2(50) = 300.631; p < 0.001; n = 886; CFI = 0.992; NFI = 0.991; TLI = 0.990; SRMR = 0.052; RMSEA = 0.075; P(RMSEA ≤ 0.05) < 0.001; 90% CI ]0.067; 0.084[).
All of BAT-23’s dimensions presented satisfactory AVE values both for the joint data (AVEExhaustion = 0.63; AVEMental Distance = 0.67; AVECognitive Impairment = 0.64; AVEEmotional Impairment = 0.67), and individually for Brazil (AVEExhaustion = 0.62; AVEMental Distance = 0.67; AVECognitive Impairment = 0.65; AVEEmotional Impairment = 0.68) and Portugal (AVEExhaustion = 0.68; AVEMental Distance = 0.65; AVECognitive Impairment = 0.63; AVEEmotional Impairment = 0.65). BAT-12 also showed satisfactory AVE estimates for the joint sample (AVEExhaustion = 0.72; AVEMental Distance = 0.57; AVECognitive Impairment = 0.72; AVEEmotional Impairment = 0.72), as it did for each individual countries’ sample: exhaustion (AVEBR = 0.71; AVEPT = 0.75), mental distance (AVEBR = 0.58; AVEPT = 0.52), cognitive impairment (AVEBR = 0.72; AVEPT = 0.72), and emotional impairment (AVEBR = 0.73; AVEPT = 0.67). The obtained AVE values are indicative of convergent evidence in terms of internal structure: on average the dimensions explain 62% or more (AVEi ≥ 0.62) of the variance of its indicators for BAT-23, and 52% or more of the variance of its indicators for BAT-12, i.e., AVEi ≥ 0.52 [98].
From a Rasch perspective, the items match the workers’ sample, since BAT’s items garnered information about workers at all ranges of the burnout distribution. Figure 3 displays both items’ scale values (in terms of location) and persons’ burnout levels (in terms of their location) spaced along a common vertical axis marked with a logits scale [103,117].

3.2.2. Reliability of the Scores: Internal Consistency

The reliability values of the second-order factor were good for Portugal (ωL2 = 0.90; ωL1 = 0.86; ωpartial L1 = 0.97) and Brazil (ωL2 = 0.90; ωL1 = 0.87; ωpartial L1 = 0.97), as they were for the joint sample (ωL2 = 0.90; ωL1 = 0.87; ωpartial L1 = 0.97). BAT-12 presented similar values for Portugal (ωL2 = 0.96; ωL1 = 0.84; ωpartial L1 = 0.94), Brazil (ωL2 = 0.92; ωL1 = 0.84; ωpartial L1 = 0.94) and for the joint sample (ωL2 = 0.93; ωL1 = 0.84; ωpartial L1 = 0.94). The internal consistency evidence of the second-order construct was very satisfactory when using the individual data from each country, as when using the joint data (H2). In the BAT-23 the EAP reliability estimate for the burnout latent score was 0.89 for Portugal, 0.74 for Brazil, and 0.88 for the joint sample. The BAT-12 model presented an EAP reliability estimate of 0.85 for Portugal, 0.84 for Brazil, and 0.85 for the joint sample. Regarding the internal consistency estimates of the first-order factors, all used estimators presented values, which were indicative of satisfactory reliability evidence for both BAT versions (H2; αord i ≥ 0.76; ωi ≥ 0.71; CRi ≥ 0.76; Table 4). For both BAT-23 and BAT-12, the EAP reliability estimates for the four latent first-order factors were satisfactory (EAPi ≥ 0.82; Table 4).

3.2.3. Measurement Invariance

The measurement invariance among countries and sex was tested through a group of nested models with increasing constraints (Table 5). Full uniqueness measurement invariance (i.e., strict invariance) was achieved among countries and sex for BAT-23 (H3) considering the ΔCFI ≤ −0.010 [112]. Using the Δχ2 criterion [113] thresholds, invariance among countries was achieved, and first-order factor loadings invariance was obtained among sex. However, the Δχ2 criterion is too restrictive [90]; consequently, the ΔCFI criterion was preferred. The fit of the data to the model was acceptable among countries and sex, as seen in the dimensionality analysis. The measurement of burnout using BAT works in a similar manner across countries and sex, allowing comparisons of scores to be established between the different groups.
BAT-12 presented scalar measurement invariance among workers from Brazil and Portugal (H3). In order to avoid negative disturbance (it is not theoretically possible) of the mental distance latent variable among the Portuguese sample, the disturbance of the mental distance first-order factor was constrained to 0.1 for three models (i.e., 4, 5, and, 6). Full uniqueness measurement invariance was achieved among sex using the ΔCFI criterion (H3).
The Pearson’s correlations between the raw BAT-12 and BAT-23 scores were very strong and statistically significant for the total burnout score (r = 0.979, t(3,101) = 267.096; p < 0.001) and for all first-order dimensions, i.e., exhaustion (r = 0.932, t(3,101) = 143.472; p < 0.001), mental distance (r = 0.960, t(3,101) = 191.928; p < 0.001), cognitive impairment (r = 0.948, t(3,101) = 165.722; p < 0.001), and emotional impairment (r = 0.948, t(3,101) = 166.541; p < 0.001). The raw mean scores per dimension and for the total burnout score are presented (Table 6 and Table 7).

3.3. Validity Evidence Based on the Relations to Other Variables

The latent correlations among BAT’s burnout latent factor with other related latent variables were investigated. Regarding the sample from Brazil, the measurement model with burnout-related constructs provided a satisfactory fit to the data when using either BAT-23 (χ2(683) = 5501.555; p < 0.001; n = 1006; CFI = 0.978; NFI = 0.975; TLI = 0.976; SRMR = 0.074; RMSEA = 0.084; P(RMSEA ≤ 0.05) < 0.001; 90% CI ]0.082; 0.086[) or BAT-12 (χ2(331) = 2454.727; p < 0.001; n = 1006; CFI = 0.978; NFI = 0.975; TLI = 0.976; SRMR = 0.073; RMSEA = 0.080; P(RMSEA ≤ 0.05) < 0.001; 90% CI ]0.077; 0.083[). The data of the workers’ sample from Portugal also presented a satisfactory fit to the data when using either BAT-12 (χ2(331) = 1150.641; p < 0.001; n = 355; CFI = 0.987; NFI = 0.982; TLI = 0.985; SRMR = 0.076; RMSEA = 0.084; P(RMSEA ≤ 0.05) < 0.001; 90% CI ]0.078; 0.089[) or BAT-23 (χ2(683) = 2691.560; p < 0.001; n = 355; CFI = 0.988; NFI = 0.983; TLI = 0.986; SRMR = 0.076; RMSEA = 0.091; P(RMSEA ≤ 0.05) < 0.001; 90% CI ]0.088; 0.095[). In terms of reliability, the internal consistency estimates obtained ranged from acceptable to good: work engagement (ωBrazil = 0.86; ωPortugal = 0.74), co-worker support (ωBrazil = 0.84; ωPortugal = 0.86), role clarity (ωBrazil = 0.83; ωPortugal = 0.89), negative change (ωBrazil = 0.68; ωPortugal = 0.79), and work overload (ωBrazil = 0.85; ωPortugal = 0.84).
The models’ latent correlations were virtually the same whether using BAT-23 or BAT-12 within each country (Table 8). The sample from Brazil presented BAT’s latent burnout score correlations ranging from −0.81 (work engagement) to 0.59 (work overload) when using BAT-23, and they ranged from −0.80 (work engagement) to 0.61 (work overload) when using BAT-12. For the Portuguese sample, the latent burnout scores’ correlations ranged from −0.75 with work engagement to 0.54 with negative change for BAT-23. For the same sample, BAT-12’s burnout latent correlations varied from −0.73 (work engagement) to 0.54 (negative change). All latent correlations with BAT’s burnout score were statistically significant (Table 8) and presented a moderate to strong effect size (H4) [114].

4. Discussion

The data from Brazil and Portugal provided robust validity evidence for the BAT-23, and the BAT-12 using item response theory (i.e., multidimensional Rasch model) and classical test theory (i.e., confirmatory factor analysis) in conjunction. Satisfactory evidence was obtained based on both the internal structure and the relations to other variables. The current study adds to the already available evidence about BAT’s psychometric properties using the classical test theory, e.g., [21], and item response theory [37]. The present study intends to take advantage of the benefits of the two measurement theories in conjunction while bringing some novelties, such as the second-order estimates of internal consistency (i.e., ωL2, ωL1, and ωpartial L1) and the EAP reliability index. In terms of the Rasch model, the MRCMLM was used in contrast with the unidimensional approach used in BAT’s previous research [37]. The multidimensional measurement model is both substantively advantageous and technically appropriate in cases where the unidimensionality is not expected [99]. The multidimensional approach considered BAT’s four first-order dimensions, and a second-order latent variable. This is also the first study to provide infit and outfit estimates for BAT’s items, these two mean square statistics are useful to understand how well the data fit the model [107].
The originally proposed dimensionality for BAT-23 and BAT-12 presented a satisfactory fit to the data for both countries without removing items (H1). Such findings are corroborated by samples from other American and European countries, Ecuador using BAT-23 and BAT-12 [40], and Italy using BAT-23 [38]. Currently, the cumulated evidence of BAT’s dimensionality is consistent across countries from Asia, America, and Europe [21].
Globally, the evidence of the reliability of the scores in terms of internal consistency obtained by both samples was satisfactory both for second- and first-order dimensions (H2). In fact, only the mental distance dimension of the BAT-12 version with the Portuguese data presented estimates slightly below the desirable the Portuguese workers; nevertheless, those values were acceptable (i.e., ≥0.71). BAT’s mental distance was the first-order dimension that had the lowest α, and ω in the Ecuadorian version [40], as did in the Italian version [38]. However, samples from other countries showed that mental distance did not present the lowest internal consistency estimates of all first-order dimensions [21]. As expected, BAT-12’s first-order internal consistency estimates were lower than BAT-23’s ones. Notwithstanding, BAT-12’s internal consistency estimates were globally satisfactory [46], with both classical test theory (i.e., αord, ω, and CR) and item theory response estimators (i.e., EAP).
Both versions of BAT had measurement invariance (i.e., at least scalar) for countries and sex (H3), allowing mean comparisons for BAT among countries, and sex. BAT-23 presented measurement invariance among seven countries in a previous study by De beer et al. [21]. One of the novelties of the current study is the measurement invariance of BAT-23 among sex and the test of measurement invariance among countries and sex of BAT’s short version (i.e., BAT-12).
The BAT’s scores’ relation to other variables presented convergent evidence (H4) since all latent correlations’ paths were statistically significant (moderate to strong effect sizes) with the theoretically expected direction for each correlation pair. BAT’s burnout latent scores were negatively correlated with work engagement, role clarity, and co-workers’ support. Positive latent correlations were found among BAT’s burnout latent scores, work overload, and negative change. The latent correlations’ effect sizes were similar among countries. Burnout’s correlation with work overload and burnout’s correlation with role clarity presented the largest difference among countries. The observed latent associations between BAT’s scores and job demands and job resources are in accordance with the findings from research reading other BAT’s versions; for example, the Romanian [39] and the Japanese [28]. BAT’s burnout latent scores’ correlation with work engagement was the strongest negative correlation for both countries. Strong negative correlations between burnout and work engagement are in accordance with what is theoretically expected from these two constructs [58,118]. Regarding the data from Brazil, the strongest positive correlation with BAT’s burnout scores was achieved with work overload. While for Portugal, the strongest correlation for BAT’s burnout latent scores was observed with negative change. However, the absolutes values were smaller than the ones observed for burnout and work engagement correlation. The data provided validity evidence based on the relations to other variables, allowing to consistently build up on the existing burnout’s nomological network of constructs [56] reinforcing BAT’s psychometric properties.
Both BAT-23 and BAT-12 presented good validity evidence and thus both can be used to measure burnout levels among workers from Brazil and from Portugal. The advantage of using BAT-23 concerns its finer-grained assessment of burnout (i.e., more items lead to capturing more content of the construct); however, if time is a constraint and other measures are being collected, BAT-12 can be a more parsimonious alternative. As the results showed, the obtained raw scores for BAT-12 and BAT-23 presented an almost perfect correlation (i.e., 0.98). Moreover, short-versions of psychometric instruments are preferable since their validity evidence is not compromised with its shorting from the full-length version. The main goal of using a short-version is to reduce the time burden of assessment. Usually, short-versions have lower estimates of reliability than full versions. The BAT-12 results were (in some of the first-order dimensions) slightly lower than the BAT-23 ones, although with no meaningful losses in terms of its satisfactory validity evidence. Practitioners and researchers opting between BAT-23 and BAT-12 will have to balance between time-saving of brevity versus construct content coverage and validity evidence [119]. The BAT-12 option seems to be the most balanced, since its validity evidence is equivalent to its longer counterpart, and longer instruments can present several problems, for example, boredom, fatigue, increasing dropout rates, and lack of attention [120].

4.1. Weaknesses, Strengths, and Suggestions for Further Research

The obtained non-probabilistic convenience sample introduces some degree of selection bias. However, probabilistic sampling (i.e., all units in the population have known and positive probabilities of inclusion) is only possible when there is a complete and up-to-date list of the member of the population being investigated [121,122], which was not the case. Even with large samples, the representativeness of the samples cannot be assumed if the sampling method is not probabilistic. However, many valid conclusions can still be taken from the current study. Future research should be conducted with samples from occupational groups with few elements in the current paper (e.g., craft and related trades workers or elementary occupations). The sex proportions and academic level should also be more similar to each workers’ population parameters. The current correlational study has a cross-sectional design. Longitudinal designs can strengthen the validity evidence of BAT, namely allowing longitudinal measurement invariance to be tested, which will allow BAT’s structure stability through time to be studied. The current paper only investigated two of the five sources of validity evidence [43]. One of them (i.e., validity evidence based on the relations to other variables) was only analyzed from a correlational perspective with five related constructs. Further research on the BAT’s scores’ relations to other variables should expand to other conceptually linked constructs such as fatigue using, for example, the Portuguese adaptation of the Occupational Fatigue Exhaustion/Recovery [6] or the Brazilian adaptation of the Feeling of Fatigue scale [123]. Test-criterion relationships should be analyzed in future studies using a predictive or concurrent design. Future studies should also investigate other sources of validity evidence, e.g., the validity evidence based on the response processes.
The findings of the current study are based on large samples from two different countries. The obtained findings are promising in terms of the measurement of burnout’s core symptoms. Future research should investigate the version of BAT including the secondary symptoms items for both countries, so as to also compare BAT’s psychometric properties directly with the Brazilian and Portuguese adaptations of other psychometric instruments that measure burnout (e.g., MBI, OLBI, CBI). It will also be convenient to obtain cut-off values for different levels of burnout; for such a purpose, clinical samples will have to be investigated. Using the receiver operating characteristic (ROC) curve [124] will allow sensitivity (true positive rate) and specificity (true negative rate) to be estimated. Although it should be taken into consideration that BAT’s score by themselves will not be enough, a full thorough clinical interview and complementary information will be required [29]. Another call that should be made for future studies is the incorporation of the increasing evidence (prior knowledge) about BAT’s dimensionality to take advantage of the Bayesian approach [125], which particularly useful with small samples and allows some frequentist approach potential problems to be avoided (e.g., non-convergence, negative variances).

4.2. Practical Implications

BAT-12 was shown to be virtually equal to BAT-23 in terms of scores, as in terms of validity evidence, representing an equally robust alternative to measure burnout. The decision to use BAT-23 or BAT-12 will be related to the level of detail that one intends to obtain regarding the burnout measurement. BAT’s Brazilian and Portuguese versions are invariant in terms of measurement, allowing for comparisons of means among countries, and between males and females. BAT’s scores presented the expected associations with related measures. The quartiles and mean scores are also provided as the first reference in terms of burnout at the country and sex levels. BAT is a promising instrument and is a viable alternative to measuring burnout in workers from Brazil and Portugal.

5. Conclusions

The data of multi-occupational workers from Brazil and Portugal presented good validity evidence for both BAT-23 and BAT-12, supporting its use to measure and to compare burnout levels among sex and countries. BAT’s scores provided support for the theoretical nomological network of constructs. Both samples’ data fitted well the original structure of BAT-23 and BAT-12 with good reliability evidence. This leads to the conclusion that BAT is a good instrument for practitioners and researchers to measure burnout among different occupations.

Author Contributions

Conceptualization, J.S., A.C.S.V., C.S.H., W.B.S. and S.S.; methodology, J.S.; software, J.S.; formal analysis, J.S.; investigation, J.S., A.C.S.V., C.S.H. and S.S.; resources, J.S., A.C.S.V., C.S.H. and S.S.; data curation, J.S.; writing—original draft preparation, J.S.; writing—review and editing, J.S., A.C.S.V., C.S.H., W.B.S. and S.S.; visualization, J.S.; project administration, J.S. and S.S.; funding acquisition, S.S. All authors have read and agreed to the published version of the manuscript.

Funding

Jorge Sinval: This work was produced with the support of INCD, and it was funded by FCT I.P. under the project Advanced Computing Project CPCA/A0/7417/2020, platform Stratus. Ana Claudia S. Vazquez: This work was supported by the National Council for Scientific and Technological Development—CNPq. Jorge Sinval, and Sílvia Silvia: This work was supported by Fundação para a Ciência e a Tecnologia, grant UIDB/00315/2020.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and approved by the Ethics Committee of the Federal University of Health Sciences of Porto Alegre (Comitê de Ética em Pesquisa da Fundação Faculdade Federal de Ciências Médicas de Porto Alegre; CEP-UFCSPA) (protocol code: CAE 78617617.8.0000.5345; date of approval: 25 October 2017).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on reasonable request from the corresponding author.

Acknowledgments

The authors would like to acknowledge all the international researchers in the broader context of the Burnout Assessment Tool (BAT) consortium for their continued support and contribution to this important area of research.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

References

  1. World Health Organization. International Statistical Classification of Diseases and Related Health Problems, 11th ed.; World Health Organization: Geneva, Switzerland, 2020. [Google Scholar]
  2. Maslach, C.; Schaufeli, W.B.; Leiter, M.P. Job burnout. Annu. Rev. Psychol. 2001, 52, 397–422. [Google Scholar] [CrossRef] [Green Version]
  3. Durand-Moreau, Q.V. Is burn-out finally a disease or not? Occup. Environ. Med. 2019, 76, 938. [Google Scholar] [CrossRef]
  4. Koutsimani, P.; Montgomery, A.; Georganta, K. The relationship between burnout, depression, and anxiety: A systematic review and meta-analysis. Front. Psychol. 2019, 10, 1–19. [Google Scholar] [CrossRef] [Green Version]
  5. Huibers, M.J.H.; Beurskens, A.J.H.M.; Prins, J.B.; Kant, I.J.; Bazelmans, E.; Van Schayck, C.P.; Knottnerus, J.A.; Bleijenberg, G. Fatigue, burnout, and chronic fatigue syndrome among employees on sick leave: Do attributions make the difference? Occup. Environ. Med. 2003, 60, i26–i31. [Google Scholar] [CrossRef] [Green Version]
  6. Santos, J.; Ramos, C.; Silva, A.; Amorim, S.; Winwood, P.; Carvalhais, C. Psychometric properties of the portuguese version of the occupational fatigue exhaustion/recovery (OFER) scale among industrial shift workers. Arch. Environ. Occup. Health 2021, 1–7. [Google Scholar] [CrossRef] [PubMed]
  7. Alessandri, G.; Perinelli, E.; De Longis, E.; Schaufeli, W.B.; Theodorou, A.; Borgogni, L.; Caprara, G.V.; Cinque, L. Job burnout: The contribution of emotional stability and emotional self-efficacy beliefs. J. Occup. Organ. Psychol. 2018, 91, 823–851. [Google Scholar] [CrossRef]
  8. Brand, S.; Beck, J.; Hatzinger, M.; Harbaugh, A.; Ruch, W.; Holsboer-Trachsler, E. Associations between satisfaction with life, burnout-related emotional and physical exhaustion, and sleep complaints. World J. Biol. Psychiatry 2010, 11, 744–754. [Google Scholar] [CrossRef] [PubMed]
  9. Ahola, K.; Hakanen, J.J. Job strain, burnout, and depressive symptoms: A prospective study among dentists. J. Affect. Disord. 2007, 104, 103–110. [Google Scholar] [CrossRef]
  10. Engelbrecht, G.J.; Beer, L.T.; Schaufeli, W.B. The relationships between work intensity, workaholism, burnout, and self-reported musculoskeletal complaints. Hum. Factors Ergon. Manuf. Serv. Ind. 2020, 30, 59–70. [Google Scholar] [CrossRef]
  11. Hakanen, J.J.; Schaufeli, W.B. Do burnout and work engagement predict depressive symptoms and life satisfaction? A three-wave seven-year prospective study. J. Affect. Disord. 2012, 141, 415–424. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  12. Hallsten, L.; Voss, M.; Stark, S.; Josephson, M. Job burnout and job wornout as risk factors for long-term sickness absence. Work 2011, 38, 181–192. [Google Scholar] [CrossRef] [PubMed]
  13. Hillhouse, J.J.; Adler, C.M.; Walters, D.N. A simple model of stress, burnout and symptomatology in medical residents: A longitudinal study. Psychol. Health Med. 2000, 5, 63–73. [Google Scholar] [CrossRef]
  14. Lizano, E.L.; Mor Barak, M. Job burnout and affective wellbeing: A longitudinal study of burnout and job satisfaction among public child welfare workers. Child. Youth Serv. Rev. 2015, 55, 18–28. [Google Scholar] [CrossRef]
  15. Salvagioni, D.A.J.; Melanda, F.N.; Mesas, A.E.; González, A.D.; Gabani, F.L.; De Andrade, S.M. Physical, psychological and occupational consequences of job burnout: A systematic review of prospective studies. PLoS ONE 2017, 12, e0185781. [Google Scholar] [CrossRef] [PubMed]
  16. Toker, S.; Melamed, S.; Berliner, S.; Zeltser, D.; Shapira, I. Burnout and risk of coronary heart disease: A prospective study of 8838 employees. Psychosom. Med. 2012, 74, 840–847. [Google Scholar] [CrossRef] [PubMed]
  17. Bakker, A.B.; Demerouti, E.; Sanz-Vergel, A.I. Burnout and work engagement: The JD–R approach. Annu. Rev. Organ. Psychol. Organ. Behav. 2014, 1, 389–411. [Google Scholar] [CrossRef]
  18. Shah, K.; Chaudhari, G.; Kamrai, D.; Lail, A.; Patel, R.S. How essential is to focus on physician’s health and burnout in coronavirus (COVID-19) pandemic? Cureus 2020, 12, e7538. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  19. Morgantini, L.A.; Naha, U.; Wang, H.; Francavilla, S.; Acar, Ö.; Flores, J.M.; Crivellaro, S.; Moreira, D.; Abern, M.; Eklund, M.; et al. Factors contributing to healthcare professional burnout during the COVID-19 pandemic: A rapid turnaround global survey. PLoS ONE 2020, 15, e0238217. [Google Scholar] [CrossRef]
  20. Cabarkapa, S.; Nadjidai, S.E.; Murgier, J.; Ng, C.H. The psychological impact of COVID-19 and other viral epidemics on frontline healthcare workers and ways to address it: A rapid systematic review. Brain Behav. Immun.-Health 2020, 8, 100144. [Google Scholar] [CrossRef]
  21. De Beer, L.T.; Schaufeli, W.B.; De Witte, H.; Hakanen, J.J.; Shimazu, A.; Glaser, J.; Seubert, C.; Bosak, J.; Sinval, J.; Rudnev, M. Measurement invariance of the Burnout Assessment Tool (BAT) across seven cross-national representative samples. Int. J. Environ. Res. Public Health 2020, 17, 5604. [Google Scholar] [CrossRef]
  22. Hu, Q.; Schaufeli, W.B.; Taris, T.W. How are changes in exposure to job demands and job resources related to burnout and engagement? A longitudinal study among Chinese nurses and police officers. Stress Health 2017, 33, 631–644. [Google Scholar] [CrossRef] [PubMed]
  23. Schaufeli, W.B.; Bakker, A.B.; van Rhenen, W. How changes in job demands and resources predict burnout, work engagement, and sickness absenteeism. J. Organ. Behav. 2009, 30, 893–917. [Google Scholar] [CrossRef] [Green Version]
  24. Schaufeli, W.B. Burnout in Europe: Relations with National Economy, Governance and Culture; KU Leuven: Leuven, Belgium, 2018. [Google Scholar]
  25. Seidler, A.; Thinschmidt, M.; Deckert, S.; Then, F.; Hegewald, J.; Nieuwenhuijsen, K.; Riedel-Heller, S.G. The role of psychosocial working conditions on burnout and its core component emotional exhaustion—A systematic review. J. Occup. Med. Toxicol. 2014, 9, 10. [Google Scholar] [CrossRef] [Green Version]
  26. Vîrgă, D.; Schaufeli, W.B.; Taris, T.W.; van Beek, I.; Sulea, C. Attachment styles and employee performance: The mediating role of burnout. J. Psychol. 2019, 153, 383–401. [Google Scholar] [CrossRef]
  27. Maslach, C.; Jackson, S.E.; Leiter, M.P. Maslach Burnout Inventory Manual, 4th ed.; Mind Garden: Menlo Park, CA, USA, 2016. [Google Scholar]
  28. Sakakibara, K.; Shimazu, A.; Toyama, H.; Schaufeli, W.B. Validation of the Japanese version of the burnout assessment tool. Front. Psychol. 2020, 11, 1–15. [Google Scholar] [CrossRef]
  29. Schaufeli, W.B.; Desart, S.; De Witte, H. Burnout Assessment Tool (BAT)—Development, validity, and reliability. Int. J. Environ. Res. Public Health 2020, 17, 9495. [Google Scholar] [CrossRef] [PubMed]
  30. Squires, A.; Finlayson, C.; Gerchow, L.; Cimiotti, J.P.; Matthews, A.; Schwendimann, R.; Griffiths, P.; Busse, R.; Heinen, M.; Brzostek, T.; et al. Methodological considerations when translating “burnout”. Burn. Res. 2014, 1, 59–68. [Google Scholar] [CrossRef] [Green Version]
  31. Kristensen, T.S.; Borritz, M.; Villadsen, E.; Christensen, K.B. The copenhagen burnout inventory: A new tool for the assessment of burnout. Work Stress 2005, 19, 192–207. [Google Scholar] [CrossRef]
  32. Demerouti, E.; Bakker, A.B.; Vardakou, I.; Kantas, A. The convergent validity of two burnout instruments: A multitrait-multimethod analysis. Eur. J. Psychol. Assess. 2003, 19, 12–23. [Google Scholar] [CrossRef]
  33. Yıldırım, M.; Solmaz, F. COVID-19 burnout, COVID-19 stress and resilience: Initial psychometric properties of COVID-19 Burnout Scale. Death Stud. 2020, 1–9. [Google Scholar] [CrossRef]
  34. Schaufeli, W.B.; De Witte, H.; Desart, S. Manual Burnout Assessment Tool (BAT); Unpublished internal report; KU Leuven: Leuven, Belgium, 2019. [Google Scholar]
  35. Schaufeli, W.B.; Taris, T.W. The conceptualization and measurement of burnout: Common ground and worlds apart the views expressed in Work & Stress Commentaries are those of the author(s), and do not necessarily represent those of any other person or organization, or of the journal. Work Stress 2005, 19, 256–262. [Google Scholar] [CrossRef]
  36. Sinval, J.; van Veldhoven, M.; Oksanen, T.; Azevedo, L.F.; Atallah, Á.N.; Melnik, T.; Marôco, J. Interventions for improving recovery from work. Cochrane Database Syst. Rev. 2021, 12, CD014518. [Google Scholar] [CrossRef]
  37. Hadžibajramović, E.; Schaufeli, W.B.; De Witte, H. A rasch analysis of the Burnout Assessment Tool (BAT). PLoS ONE 2020, 15, e0242241. [Google Scholar] [CrossRef] [PubMed]
  38. Angelini, G.; Buonomo, I.; Benevene, P.; Consiglio, P.; Romano, L.; Fiorilli, C. The Burnout Assessment Tool (BAT): A contribution to Italian validation with teachers’. Sustainability 2021, 13, 9065. [Google Scholar] [CrossRef]
  39. Oprea, B.; Iliescu, D.; De Witte, H. Romanian short version of the Burnout Assessment Tool: Psychometric properties. Eval. Health Prof. 2021, 44, 406–415. [Google Scholar] [CrossRef]
  40. Vinueza-Solórzano, A.M.; Portalanza-Chavarría, C.A.; de Freitas, C.P.P.; Schaufeli, W.B.; De Witte, H.; Hutz, C.S.; Souza Vazquez, A.C. The Ecuadorian version of the Burnout Assessment Tool (BAT): Adaptation and validation. Int. J. Environ. Res. Public Health 2021, 18, 7121. [Google Scholar] [CrossRef] [PubMed]
  41. Spagnoli, P.; Buono, C.; Kovalchuk, L.S.; Cordasco, G.; Esposito, A. Perfectionism and burnout during the COVID-19 crisis: A two-wave cross-lagged study. Front. Psychol. 2021, 11, 1–7. [Google Scholar] [CrossRef] [PubMed]
  42. Eurofound. Sixth European Working Conditions Survey—Overview Report (2017 Update); Publications Office of the European Union: Luxembourg, 2017; ISBN 9789289715966. [Google Scholar] [CrossRef]
  43. American Educational Research Association; American Psychological Association; National Council on Measurement in Education. Standards for Educational and Psychological Testing; American Psychological Association: Washington, DC, USA, 2014; ISBN 9780935302356. [Google Scholar]
  44. Soohyun, C. A preliminary validation study for the Korean version of the Burnout Assessment Tool (K-BAT). Korean J. Ind. Organ. Psychol. 2020, 33, 461–499. [Google Scholar] [CrossRef]
  45. Kolachev, N.; Osin, E.; Schaufeli, W.B.; Desart, S. Personal resources and burnout: Evidence from a study among librarians of Moscow region. Organ. Psychol. 2019, 9, 129–147. [Google Scholar]
  46. Nunnally, J.C.; Bernstein, I.H. Psychometric Theory, 3rd ed.; McGraw-Hill: New York, NY, USA, 1994; ISBN 0-07-047849-X. [Google Scholar]
  47. Zumbo, B.D.; Gadermann, A.M.; Zeisser, C. Ordinal versions of coefficients alpha and theta for Likert rating scales. J. Mod. Appl. Stat. Methods 2007, 6, 21–29. [Google Scholar] [CrossRef]
  48. Schaufeli, W.B.; De Witte, H.; Desart, S. De Burnout Assessment Tool (BAT): Een nieuw instrument voor het meten van burn-out. Tijdschr. Klin. Psychol. 2020, 50, 267–283. [Google Scholar]
  49. Adriaenssens, J.; De Gucht, V.; Maes, S. Determinants and prevalence of burnout in emergency nurses: A systematic review of 25 years of research. Int. J. Nurs. Stud. 2015, 52, 649–661. [Google Scholar] [CrossRef]
  50. Dimou, F.M.; Eckelbarger, D.; Riall, T.S. Surgeon burnout: A systematic review. J. Am. Coll. Surg. 2016, 222, 1230–1239. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  51. Purvanova, R.K.; Muros, J.P. Gender differences in burnout: A meta-analysis. J. Vocat. Behav. 2010, 77, 168–185. [Google Scholar] [CrossRef]
  52. Innstrand, S.T.; Langballe, E.M.; Falkum, E.; Aasland, O.G. Exploring within- and between-gender differences in burnout: 8 different occupational groups. Int. Arch. Occup. Environ. Health 2011, 84, 813–824. [Google Scholar] [CrossRef] [PubMed]
  53. Sinval, J.; Marôco, J. Short index of job satisfaction: Validity evidence from Portugal and Brazil. PLoS ONE 2020, 15, e0231474. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  54. Sinval, J.; Miller, V.; Marôco, J. Openness toward organizational change scale (OTOCS): Validity evidence from Brazil and Portugal. PLoS ONE 2021, 16, e0249986. [Google Scholar] [CrossRef]
  55. Sinval, J.; Sirgy, M.J.; Lee, D.-J.; Marôco, J. The quality of work life scale: Validity evidence from Brazil and Portugal. Appl. Res. Qual. Life 2020, 15, 1323–1351. [Google Scholar] [CrossRef]
  56. Cronbach, L.J.; Meehl, P.E. Construct validity in psychological tests. Psychol. Bull. 1955, 52, 281–302. [Google Scholar] [CrossRef] [Green Version]
  57. Demerouti, E.; Bakker, A.B.; Nachreiner, F.; Schaufeli, W.B. The job demands-resources model of burnout. J. Appl. Psychol. 2001, 86, 499–512. [Google Scholar] [CrossRef]
  58. Bakker, A.B.; Demerouti, E. Job demands–resources theory: Taking stock and looking forward. J. Occup. Health Psychol. 2017, 22, 273–285. [Google Scholar] [CrossRef]
  59. Schaufeli, W.B.; Taris, T.W. A critical review of the job demands-resources model: Implications for improving work and health. In Bridging Occupational, Organizational and Public Health: A Transdisciplinary Approach; Bauer, G.F., Hämmig, O., Eds.; Springer: Dordrecht, The Netherlands, 2014. [Google Scholar] [CrossRef]
  60. Alarcon, G.M. A meta-analysis of burnout with job demands, resources, and attitudes. J. Vocat. Behav. 2011, 79, 549–562. [Google Scholar] [CrossRef]
  61. Halbesleben, J.R.B. Sources of social support and burnout: A meta-analytic test of the conservation of resources model. J. Appl. Psychol. 2006, 91, 1134–1145. [Google Scholar] [CrossRef]
  62. Lesener, T.; Gusy, B.; Wolter, C. The job demands-resources model: A meta-analytic review of longitudinal studies. Work Stress 2019, 33, 76–103. [Google Scholar] [CrossRef]
  63. Van den Broeck, A.; Elst, T.V.; Baillien, E.; Sercu, M.; Schouteden, M.; De Witte, H.; Godderis, L. Job demands, job resources, burnout, work engagement, and their relationships: An analysis across sectors. J. Occup. Environ. Med. 2017, 59, 369–376. [Google Scholar] [CrossRef] [PubMed]
  64. Rigdon, E.E. Calculating degrees of freedom for a structural equation model. Struct. Equ. Model. A Multidiscip. J. 1994, 1, 274–278. [Google Scholar] [CrossRef]
  65. Jak, S.; Jorgensen, T.D.; Verdam, M.G.E.; Oort, F.J.; Elffers, L. Analytical power calculations for structural equation modeling: A tutorial and shiny app. Behav. Res. Methods 2021, 53, 1385–1406. [Google Scholar] [CrossRef] [PubMed]
  66. Kelley, K.; Lai, K. Sample size planning for confirmatory factor models: Power and accuracy for effects of interest. In The Wiley Handbook of Psychometric Testing: A Multidisciplinary Reference on Survey, Scale and Test Development; Irwing, P., Booth, T., Hughes, D., Eds.; John Wiley & Sons: Hoboken, NJ, USA, 2018; pp. 113–138. [Google Scholar] [CrossRef]
  67. International Test Commission. ITC Guidelines for Translating and Adapting Tests, 2nd ed.; International Test Commission: Quebec, QC, Canada, 2018; Volume 18, pp. 101–134. [Google Scholar] [CrossRef]
  68. Schaufeli, W.B.; Bakker, A.B. Utrecht Work Engagement Scale; Occupational Health Psychology Unit, Utrecht University: Utrecht, The Netherlands, 2003. [Google Scholar]
  69. Schaufeli, W.B.; Shimazu, A.; Hakanen, J.J.; Salanova, M.; de Witte, H. An ultra-short measure for work engagement. Eur. J. Psychol. Assess. 2017, 1–15. [Google Scholar] [CrossRef]
  70. Sinval, J.; Pasian, S.R.; Queirós, C.; Marôco, J. Brazil-Portugal transcultural adaptation of the UWES-9: Internal consistency, dimensionality, and measurement invariance. Front. Psychol. 2018, 9, 1–18. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  71. Vazquez, A.C.S.; Magnan, E.d.S.; Pacico, J.C.; Hutz, C.S.; Schaufeli, W.B. Adaptation and validation of the Brazilian version of the Utrecht Work Engagement Scale. Psico-USF 2015, 20, 207–217. [Google Scholar] [CrossRef] [Green Version]
  72. Sinval, J.; Queirós, C.; Pasian, S.R.; Marôco, J. Transcultural adaptation of the Oldenburg Burnout Inventory (OLBI) for Brazil and Portugal. Front. Psychol. 2019, 10, 1–28. [Google Scholar] [CrossRef]
  73. Schwarzer, R.; Knoll, N. Functional roles of social support within the stress and coping process: A theoretical and empirical overview. Int. J. Psychol. 2007, 42, 243–252. [Google Scholar] [CrossRef]
  74. Schaufeli, W.B. Applying the job demands-resources model. Organ. Dyn. 2017, 46, 120–132. [Google Scholar] [CrossRef]
  75. Rizzo, J.R.; House, R.J.; Lirtzman, S.I. Role conflict and ambiguity in complex organizations. Adm. Sci. Q. 1970, 15, 150. [Google Scholar] [CrossRef]
  76. Schaufeli, W.B.; Bakker, A.B.; van der Heijden, F.M.M.A.; Prins, J.T. Workaholism among medical residents: It is the combination of working excessively and compulsively that counts. Int. J. Stress Manag. 2009, 16, 249–272. [Google Scholar] [CrossRef] [Green Version]
  77. Greubel, J.; Kecklund, G. The impact of organizational changes on work stress, sleep, recovery and health. Ind. Health 2011, 49, 353–364. [Google Scholar] [CrossRef] [Green Version]
  78. LimeSurvey, An Open Source Survey Tool [Computer Software]; GmbH LimeSurvey: Hamburg, Germany, 2021.
  79. SurveyMonkey, [Computer Software]; SurveyMonkey: San Mateo, CA, USA, 2021.
  80. R Core Team. R: A Language and Environment for Statistical Computing (Version 4.1.1) [Computer Software]. 2021. Available online: https://www.r-project.org/ (accessed on 22 December 2021).
  81. RStudio Team. RStudio: Integrated Development for R (Version 1.4.1717) [Computer Software]. 2021. Available online: https://www.rstudio.com/ (accessed on 22 December 2021).
  82. Kelley, K. MBESS: The MBESS R Package (R Package Version 4.6.0) [Computer Software]. 2019. Available online: https://cran.r-project.org/package=MBESS (accessed on 22 December 2021).
  83. McNamara, A.; Arino de la Rubia, E.; Zhu, H.; Ellis, S.; Quinn, M. skimr: Compact and Flexible Summaries of Data (R Package Version 2.1.3) [Computer Software]. 2021. Available online: https://cran.r-project.org/package=skimr (accessed on 22 December 2021).
  84. Rich, B. table1: Tables of Descriptive Statistics in HTML (R Package Version 1.4.2) [Computer Software]. 2021. Available online: https://cran.r-project.org/package=table1 (accessed on 22 December 2021).
  85. Peterson, B.G.; Carl, P. PerformanceAnalytics, Econometric Tools for Performance and Risk Analysis (R Package Version 2.0.4) [Computer Software]. 2020. Available online: https://cran.r-project.org/package=PerformanceAnalytics (accessed on 22 December 2021).
  86. Lüdecke, D. sjstats: Statistical Functions for Regression Models (R Package Version 0.18.1) [Computer Software]. 2021. Available online: https://cran.r-project.org/package=sjstats (accessed on 22 December 2021). [CrossRef]
  87. Lemon, J. plotrix: A Package in the Red Light District of R. R-News 2006, 6, 8–12. [Google Scholar]
  88. Poncet, P. modeest: Mode Estimation (R Package Version 2.4.0) [Computer Software]. 2019. Available online: https://cran.r-project.org/package=modeest (accessed on 22 December 2021).
  89. Finney, S.J.; DiStefano, C. Non-normal and categorical data in structural equation modeling. In Structural Equation Modeling: A Second Course; Hancock, G.R., Mueller, R.O., Eds.; Information Age Publishing: Charlotte, NC, USA, 2013; pp. 439–492. [Google Scholar]
  90. Marôco, J. Análise de Equações Estruturais: Fundamentos Teóricos, Software & Aplicações, 3rd ed.; ReportNumber: Pêro Pinheiro, Portugal, 2021; ISBN 978-989-96763-6-7. [Google Scholar]
  91. Revelle, W. psych: Procedures for Psychological, Psychometric, and Personality Research (R package Version 1.9.4) [Computer Software]. 2019. Available online: https://cran.r-project.org/package=psych (accessed on 22 December 2021).
  92. Mardia, K.V. Measures of multivariate skewness and kurtosis with applications. Biometrika 1970, 57, 519–530. [Google Scholar] [CrossRef]
  93. Hu, L.; Bentler, P.M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct. Equ. Model. A Multidiscip. J. 1999, 6, 1–55. [Google Scholar] [CrossRef]
  94. Byrne, B.M. Structural Equation Modeling with AMOS, 3rd ed.; Multivariate applications series; Routledge: New York, NY, USA, 2010; ISBN 9780805863727. [Google Scholar] [CrossRef]
  95. Rosseel, Y. lavaan: An R package for structural equation modeling. J. Stat. Softw. 2012, 48, 1–21. [Google Scholar] [CrossRef] [Green Version]
  96. Muthén, B.O. Latent variable structural equation modeling with categorical data. J. Econom. 1983, 22, 43–65. [Google Scholar] [CrossRef]
  97. Fornell, C.; Larcker, D.F. Evaluating structural equation models with unobservable variables and measurement error. J. Mark. Res. 1981, 18, 39–50. [Google Scholar] [CrossRef]
  98. Hair, J.F.; Babin, B.J.; Anderson, R.E.; Black, W.C. Multivariate Data Analysis, 8th ed.; Cengage Learning: Andover, UK, 2019; ISBN 9781473756540. [Google Scholar]
  99. Briggs, D.C.; Wilson, M. An introduction to multidimensional measurement using Rasch models. J. Appl. Meas. 2003, 4, 87–100. [Google Scholar]
  100. Adams, R.J.; Wilson, M.; Wang, W.-C. The multidimensional random coefficients multinomial logit model. Appl. Psychol. Meas. 1997, 21, 1–23. [Google Scholar] [CrossRef]
  101. Robitzsch, A.; Kiefer, T.; Wu, M. TAM, Test Analysis Modules (R Package Version 3.7-16) [Computer Software]. 2021. Available online: https://rdrr.io/cran/TAM/ (accessed on 22 December 2021).
  102. Wright, B.D.; Stone, M.H. Best Test Design; Mesa Press: Chicago, IL, USA, 1979; ISBN 79-88489. [Google Scholar]
  103. Callingham, R.; Bond, T. Some notes on the term: “Wright map”. Rasch Meas. Trans. 2006, 18, 85–104. [Google Scholar]
  104. Irribarra, D.T.; Freund, R. WrightMap: IRT Item-Person Map with “ConQuest” Integration (R Package Version 1.2.3) [Computer Software]. 2020. Available online: https://cran.r-project.org/package=WrightMap (accessed on 22 December 2021).
  105. Mair, P. Modern Psychometrics with R; Use R! Springer: Cham, Switzerland, 2018; ISBN 978-3-319-93175-3. [Google Scholar] [CrossRef]
  106. Bond, T.G.; Fox, C.M. Applying the rasch model. In Applying the Rasch Model: Fundamental Measurement in the Human Sciences, 3rd ed.; Routledge: New York, NY, USA, 2015; ISBN 9780805854619. [Google Scholar] [CrossRef]
  107. Linacre, J.M. What do infit and outfit mean-square and standardized mean? Rasch Meas. Trans. 2002, 16, 878. [Google Scholar]
  108. McDonald, R.P. Test Theory: A Unified Treatment; Routledge: Mahwah, NJ, USA, 1999; ISBN 9781410601087. [Google Scholar] [CrossRef]
  109. Jorgensen, T.D.; Pornprasertmanit, S.; Schoemann, A.M.; Rosseel, Y. semTools: Useful Tools for Structural Equation Modeling (R Package Version 0.5–5) [Computer Software]. 2021. Available online: https://cran.r-project.org/package=semTools (accessed on 22 December 2021).
  110. Adams, R.J. Reliability as a measurement design effect. Stud. Educ. Eval. 2005, 31, 162–172. [Google Scholar] [CrossRef]
  111. Millsap, R.E.; Yun-Tein, J. Assessing factorial invariance in ordered-categorical measures. Multivar. Behav. Res. 2004, 39, 479–515. [Google Scholar] [CrossRef] [Green Version]
  112. Cheung, G.W.; Rensvold, R.B. Evaluating goodness-of-fit indexes for testing measurement invariance. Struct. Equ. Model. A Multidiscip. J. 2002, 9, 233–255. [Google Scholar] [CrossRef]
  113. Satorra, A.; Bentler, P.M. A scaled difference chi-square test statistic for moment structure analysis. Psychometrika 2001, 66, 507–514. [Google Scholar] [CrossRef] [Green Version]
  114. Cohen, J. Statistical Power Analysis for the Behavioral Sciences, 2nd ed.; Lawrence Erlbaum Associates: Hillsdale, NJ, USA, 1988; ISBN 0805802835. [Google Scholar]
  115. Højsgaard, S.; Halekoh, U. doBy: Groupwise Statistics, LSmeans, Linear Contrasts, Utilities (R Package Version 4.6-2) [Computer Software]. 2018. Available online: https://cran.r-project.org/package=doBy (accessed on 22 December 2021).
  116. International Labour Office. International Standard Classification of Occupations; International Labour Organization: Geneva, Switzerland, 2012; Volume I, ISBN 978-92-2-125953-4. [Google Scholar] [CrossRef]
  117. Wilson, M. Constructing Measures: An Item Response Moeling Approach; Routledge: Mahwah, NJ, USA, 2004; ISBN 9781410611697. [Google Scholar] [CrossRef]
  118. Halbesleben, J.R.B. A meta-analysis of work engagement: Relationships with burnout, demands, resources, and consequences. In Work Engagement: A Handbook of Essential Theory and Research; Bakker, A.B., Leiter, M.P., Eds.; Routledge: Hove, UK, 2010; pp. 102–117. [Google Scholar] [CrossRef]
  119. Greer, F.; Liu, J. Creating short forms and screening measures. In Principles and Methods of Test Construction: Standards and Recent Advances; Schweizer, K., DiStefano, C., Eds.; Psychological assessment—science and practice; Hogrefe Publishing: Boston, MA, USA, 2016; pp. 272–287. ISBN 978-0-88937-449-2. [Google Scholar]
  120. Rammstedt, B.; Beierlein, C. Can’t we make it any shorter? The limits of personality assessment and ways to overcome them. J. Individ. Differ. 2014, 35, 212–220. [Google Scholar] [CrossRef]
  121. Galloway, A. Non-probability sampling. In Encyclopedia of Social Measurement; Kempf-Leonard, K., Ed.; Elsevier: Amsterdam, The Netherlands, 2005; Volume 2, pp. 859–864. [Google Scholar] [CrossRef]
  122. Vehovar, V.; Toepoel, V.; Steinmetz, S. Nonprobability sampling. In The SAGE Handbook of Survey Methodology; Wolf, C., Joye, D., Smith, T.W., Fu, Y., Eds.; Sage: Thousand Oaks, CA, USA, 2016; p. 329. [Google Scholar] [CrossRef]
  123. Celestino, V.R.R.; Bucher-Maluschke, J.S.N.F.; Marqueze, E.C. Fatigue at work: Scale validation with airline pilots. BAR-Brazilian Adm. Rev. 2020, 17, 1–24. [Google Scholar] [CrossRef]
  124. Krzanowski, W.J.; Hand, D.J. ROC Curves for Continuous Data; Routledge: Boca Raton, FL, USA, 2009; ISBN 978-1-4398-0021-8. [Google Scholar] [CrossRef]
  125. Merkle, E.C.; Rosseel, Y. blavaan: Bayesian structural equation models via parameter expansion. J. Stat. Softw. 2018, 85, 1–30. [Google Scholar] [CrossRef] [Green Version]
Ijerph 19 01344 g001 550
Figure 1. BAT-23’s factor loadings (λ) and structural weights (γ) for Brazil (left) and Portugal (right). * Indicates a fixed parameter. All paths were statistically significant (p < 0.001).
Figure 1. BAT-23’s factor loadings (λ) and structural weights (γ) for Brazil (left) and Portugal (right). * Indicates a fixed parameter. All paths were statistically significant (p < 0.001).
Ijerph 19 01344 g001
Ijerph 19 01344 g002 550
Figure 2. BAT-12’s factor loadings (λ) and structural weights (γ) for Brazil (top) and Portugal (bottom). * Indicates a fixed parameter. All paths were statistically significant (p < 0.001).
Figure 2. BAT-12’s factor loadings (λ) and structural weights (γ) for Brazil (top) and Portugal (bottom). * Indicates a fixed parameter. All paths were statistically significant (p < 0.001).
Ijerph 19 01344 g002
Ijerph 19 01344 g003 550
Figure 3. Wright Map for BAT-23 for both samples. Ex—Exhaustion; MD—Mental Distance; CI—Cognitive Impairment; CI—Emotional Impairment. Each item has its four thresholds identified (i.e., t1, t2, t3, t4).
Figure 3. Wright Map for BAT-23 for both samples. Ex—Exhaustion; MD—Mental Distance; CI—Cognitive Impairment; CI—Emotional Impairment. Each item has its four thresholds identified (i.e., t1, t2, t3, t4).
Ijerph 19 01344 g003
Table
Table 1. BAT English, Brazilian, and Portuguese adaptations.
Table 1. BAT English, Brazilian, and Portuguese adaptations.
ItemEnglishBrazilPortugal
NeverRarelySometimesOftenAlwaysNuncaRaramenteAlgumas vezesFrequentementeSempre
1234512345
ExhaustionExaustão
1 SAt work, I feel mentally exhaustedNo trabalho, sinto-me mentalmente exaustoNo trabalho, sinto-me mentalmente exausto(a)
2Everything I do at work requires a great deal of effortTudo o que faço no trabalho exige muito esforçoTudo o que faço no trabalho exige muito esforço
3 SAfter a day at work, I find it hard to recover my energyAcho difícil recuperar minha energia depois de um dia de trabalhoDepois de um dia no trabalho, acho difícil recuperar a minha energia
4 SAt work, I feel physically exhaustedNo trabalho, sinto-me fisicamente exaustoNo trabalho, sinto-me fisicamente exausto(a)
5When I get up in the morning, I lack the energy to start a new day at workAo levantar pela manhã, me falta energia para começar um novo dia no trabalhoQuando me levanto de manhã, falta-me a energia para começar um novo dia no trabalho
6I want to be active at work, but somehow I am unable to manageQuero ser ativo no trabalho, mas de alguma forma não consigoQuero estar ativo(a) no trabalho, mas de alguma forma sou incapaz de o fazer
7When I exert myself at work, I get tired quicker than normalQuando eu me esforço no trabalho, me canso mais rápido do que o normalQuando me esforço no trabalho, fico rapidamente cansado(a)
8At the end of my working day, I feel mentally exhausted and drainedNo final do meu dia de trabalho, eu me sinto mentalmente exausto e esgotadoNo final de um dia de trabalho, sinto-me mentalmente exausto(a) e esgotado(a)
Mental distanceDistância mental
9 SI struggle to find any enthusiasm for my workEu luto para encontrar algum entusiasmo pelo meu trabalhoTenho dificuldade em encontrar algum entusiasmo pelo meu trabalho
10 SAt work, I do not think what I am doing and I function on autopilotNão penso no que estou fazendo no meu trabalho, eu funciono em piloto automáticoNo trabalho, não penso muito no que estou a fazer e funciono em piloto automático
11I feel a strong aversion towards my jobSinto forte aversão pelo meu trabalhoSinto uma forte aversão em relação ao meu trabalho
12I feel indifferent about my jobSinto-me indiferente em relação ao meu trabalhoSinto-me indiferente em relação ao meu trabalho
13 SI am cynical about what my work means to othersSou pessimista sobre o que meu trabalho significa para os outrosSou cínico(a) sobre o que o meu trabalho significa para os outros
Cognitive impairmentIncapacidade no Controlo Cognitivo
14 SAt work, I have trouble staying focusedEm meu trabalho, tenho dificuldade em manter o focoNo trabalho, tenho dificuldade em manter-me focado(a)
15At work I struggle to think clearlyNo trabalho, eu me esforço para pensar claramenteNo trabalho, luto para pensar claramente
16I am forgetful and distracted at work Sou esquecido e distraído no trabalhoSou esquecido(a) e distraído(a) no trabalho
17 SWhen I’m working, I have trouble concentratingTenho dificuldade em me concentrar quando estou trabalhandoQuando estou a trabalhar, tenho dificuldade em me concentrar
18 SI make mistakes in my work because I have my mind on other thingsCometo erros no trabalho porque minha mente está em outras coisasFaço erros no meu trabalho porque tenho a cabeça sobrecarregada com outras coisas
Emotional impairmentIncapacidade no Controlo Emocional
19 SAt work, I feel unable to control my emotionsNo trabalho, sinto-me incapaz de controlar as minhas emoçõesNo trabalho, sinto-me incapaz de controlar as minhas emoções
20 SI do not recognize myself in the way I react emotionally at workEu não me reconheço na maneira como reajo emocionalmente no trabalhoNão me reconheço na maneira como reajo emocionalmente no trabalho
21During my work I become irritable when things do not go the way I want Durante o trabalho, fico irritado quando as coisas não são do jeito que eu queroDurante o trabalho, fico irritadiço(a) quando as coisas não são como eu quero
22I get upset and sad at work without knowing whyFico insatisfeito e triste no trabalho sem saber o porquêFico perturbado(a) e triste no trabalho sem saber porquê
23 SAt work I may overreact unintentionallyNo trabalho, eu posso ter reações exageradas sem quererPode acontecer que no trabalho eu reaja exageradamente sem querer
Note. S—short version (BAT-12).
Table
Table 2. Descriptive statistics of the samples’ sociodemographic, and occupational group variables.
Table 2. Descriptive statistics of the samples’ sociodemographic, and occupational group variables.
BrazilPortugalTotal
(n = 2217)(n = 886)(N = 3103)
Age (Years)
M (SD)36.9 (11.1)38.9 (11.4)37.2 (11.1)
Mdn [Min, Max]36.0 [17.0, 90.0]41.0 [18.0, 68.0]36.0 [17.0, 90.0]
Sex
Female1653 (74.8%)537 (72.5%)2190 (74.2%)
Male558 (25.2%)204 (27.5%)762 (25.8%)
Academic Level
High school, vocational education, or lower554 (25.0%)258 (34.6%)812 (27.4%)
Graduation547 (24.7%)204 (27.3%)751 (25.3%)
Post-graduation1116 (50.3%)284 (38.1%)1400 (47.2%)
Occupational Group (ISCO-08)
Armed Forces Occupations6 (0.3%)0 (0.0%)6 (0.2%)
Clerical Support Workers1 (0.1%)133 (20.2%)134 (5.2%)
Craft and Related Trades Workers1 (0.1%)5 (0.8%)6 (0.2%)
Elementary Occupations15 (0.8%)4 (0.6%)19 (0.7%)
Managers149 (7.8%)63 (9.6%)212 (8.3%)
Professionals1071 (56.2%)300 (45.5%)1371 (53.4%)
Services and Sales Workers75 (3.9%)83 (12.6%)158 (6.2%)
Technicians and Associate Professionals589 (30.9%)65 (9.9%)654 (25.5%)
Plant and Machine Operators and Assemblers0 (0.0%)7 (1.1%)7 (0.3%)
Note. Because of rounding, some results may appear inconsistent.
Table
Table 3. BAT items’ infit, outfit statistics, and distributional properties for Portugal and Brazil.
Table 3. BAT items’ infit, outfit statistics, and distributional properties for Portugal and Brazil.
ItemMSDMinP25MdnP75MaxHistogramSEMCVModeskkuInfitOutfit
Brazil
Item 1 S3.310.9913345▁▃▇▆▂0.020.303−0.23−0.220.9810.977
Item 2 3.190.9713345▁▃▇▅▂0.020.303−0.05−0.311.4121.434
Item 3 S2.941.1212345▂▆▇▅▂0.020.3830.09−0.710.9240.920
Item 4 S2.821.0612335▂▆▇▅▂0.020.3830.13−0.510.8970.884
Item 5 2.671.1412335▅▇▇▅▂0.020.4320.32−0.631.0751.087
Item 6 2.321.0612235▆▇▆▂▁0.020.4620.56−0.301.0991.109
Item 7 2.561.1012235▅▇▇▃▂0.020.4320.37−0.501.0431.033
Item 8 3.201.1112345▂▅▇▆▃0.020.353−0.09−0.710.8710.857
Item 9 S2.471.2711235▇▇▆▅▂0.030.5120.49−0.841.1151.108
Item 10 S2.051.0711235▇▆▅▂▁0.020.5210.84−0.011.1391.132
Item 11 1.751.0511125▇▃▂▁▁0.020.6011.381.140.9751.028
Item 12 1.711.0011125▇▃▂▁▁0.020.5811.391.280.9940.935
Item 13 S1.691.0411125▇▂▂▁▁0.020.6111.501.501.2851.422
Item 14 S2.271.0112235▅▇▆▂▁0.020.4420.56−0.150.9190.902
Item 15 2.471.2411235▆▇▆▃▂0.030.5020.58−0.631.5621.685
Item 16 2.050.9311235▅▇▃▁▁0.020.4520.870.660.9850.983
Item 17 S2.190.9612235▅▇▅▁▁0.020.4420.660.180.8850.869
Item 18 S1.950.8511225▆▇▃▁▁0.020.4320.860.920.9940.981
Item 19 S1.950.9111225▇▇▃▁▁0.020.4720.920.711.0611.075
Item 20 S1.710.9311125▇▅▂▁▁0.020.5511.401.690.9560.889
Item 21 2.351.0412235▅▇▆▂▁0.020.4420.54−0.231.1851.190
Item 22 1.991.0811235▇▆▃▂▁0.020.5410.930.061.0471.019
Item 23 S1.860.9611225▇▆▃▁▁0.020.5211.120.890.9750.980
Portugal
Item 1 S3.210.8613345▁▂▇▅▁0.030.273−0.090.220.9270.930
Item 2 3.180.8613345▁▂▇▅▁0.030.273−0.040.121.3131.312
Item 3 S3.070.9212345▁▅▇▅▁0.030.3030.03−0.240.9000.907
Item 4 S2.840.9112335▁▆▇▃▁0.030.3230.17−0.220.9120.905
Item 5 2.821.0012335▂▆▇▃▁0.030.3630.26−0.290.9790.979
Item 6 2.450.9112235▂▇▆▂▁0.030.3720.460.040.9880.989
Item 7 2.490.9012235▂▇▆▂▁0.030.3620.41−0.020.8880.879
Item 8 3.120.9313345▁▃▇▅▁0.030.3030.02−0.160.9130.903
Item 9 S2.750.9912335▂▆▇▃▁0.030.3630.24−0.250.9790.986
Item 10 S2.290.9912235▅▇▅▂▁0.030.4320.54−0.241.1521.148
Item 11 2.021.0311235▇▇▅▁▁0.030.5110.840.150.8590.878
Item 12 1.850.9611225▇▆▃▁▁0.030.5211.080.720.8760.772
Item 13 S1.750.9511125▇▅▂▁▁0.030.5511.210.911.2571.339
Item 14 S2.420.9412235▃▇▆▂▁0.030.3920.40−0.060.8160.808
Item 15 2.761.1212345▃▇▇▅▂0.040.4120.30−0.671.3101.349
Item 16 2.160.8412235▃▇▃▁▁0.030.3920.560.290.9700.961
Item 17 S2.450.8812235▂▇▆▂▁0.030.3620.430.290.7710.776
Item 18 S2.230.8112235▂▇▅▁▁0.030.3620.560.530.9600.944
Item 19 S2.080.8112235▃▇▃▁▁0.030.3920.721.011.1501.161
Item 20 S1.870.8711225▇▇▃▁▁0.030.4720.960.820.8550.821
Item 21 2.340.8912235▃▇▆▂▁0.030.3820.33−0.171.0761.085
Item 22 1.990.9411235▇▇▅▁▁0.030.4720.770.080.9810.929
Item 23 S2.070.8112235▃▇▃▁▁0.030.3920.540.250.9790.987
Note. S—short version (BAT-12).
Table
Table 4. Reliability Estimates for the Sample from Brazil, Portugal, and Joint Data.
Table 4. Reliability Estimates for the Sample from Brazil, Portugal, and Joint Data.
BAT-23
BAT DimensionsBrazilPortugalTotal
αordωCREAPαordωCREAPαordωCREAP
Exhaustion0.920.910.950.920.920.910.960.920.930.920.960.92
Mental Distance0.910.880.930.880.910.880.930.890.900.880.930.88
Cognitive Impairment0.890.860.920.860.890.860.910.870.890.860.920.87
Emotional Impairment0.910.880.950.870.910.880.940.880.900.880.940.88
BAT-12
BAT DimensionsBrazilPortugalTotal
αordωCREAPαordωCREAPαordωCREAP
Exhaustion0.880.850.880.850.900.860.900.850.880.850.880.85
Mental Distance0.810.760.810.840.760.710.760.830.800.750.800.84
Cognitive Impairment0.870.850.880.830.860.840.880.840.870.850.880.84
Emotional Impairment0.890.850.890.820.860.810.860.820.880.840.880.82
Table
Table 5. Measurement Invariance (for BAT-23 and BAT-12) Between Countries, and Sex.
Table 5. Measurement Invariance (for BAT-23 and BAT-12) Between Countries, and Sex.
BAT-23
Countries (nBrazil = 2217, and nPortugal = 886)
Model Invarianceχ2scaleddfCFIscaledΔχ2ΔCFIscaled
1—Configural6635.9284520.934--
2—Thresholds6725.0564940.93450.517 ns−0.000
3—Factor loadings 6677.4115130.93476.445 ***0.000
4—Structural weights6303.6095160.9388.782 *0.004
5—Intercepts (first-order)6558.4805390.936239.490 ***−0.002
6—Latent means6572.8095430.93694.758 ***0.000
7—Disturbances7064.5325470.93015.470 **−0.006
8—Residuals6371.6455700.938204.445 ***−0.008
Sex (nFemale = 2190, and nMale = 762)
1—Configural6325.5364520.934--
2—Thresholds6458.0354940.93353.446 ns−0.001
3—Factor loadings 6319.2525130.93521.808 ns0.002
4—Structural weights6034.9405160.9389.471 *0.003
5—Intercepts (first-order)6076.0325390.93878.902 ***0.000
6—Latent means5163.3965430.94816.141 **0.010
7—Disturbances5973.7355470.93959.214 ***−0.009
8—Residuals5407.5225700.946147.978 ***0.007
BAT-12
Countries (nBrazil = 2217, and nPortugal = 886)
Model Invarianceχ2scaleddfCFIscaledΔχ2ΔCFIscaled
1—Configural1043.6781000.977--
2—Thresholds1080.6611200.97726.138 ns0.000
3—Factor loadings 1081.7931280.97721.379 **0.000
4—Structural weights1032.0241320.97816.411 **0.001
5—Intercepts (first-order)1170.4091440.975113.210 ***−0.003
6—Latent means1750.0031480.961127.530 ***−0.014
7—Disturbances1690.9571510.963-132.455 ns0.002
8—Residuals1763.4871630.961118.999 ***−0.002
Sex (nFemale = 2190, and nMale = 762)
1—Configural930.2821000.979--
2—Thresholds970.2101200.97923.217 ns0.000
3—Factor loadings 958.4241280.9799.269 ns0.000
4—Structural weights911.3281310.9809.925 *0.001
5—Intercepts (first-order)973.7601430.97944.065 ***−0.001
6—Latent means845.6881470.98215.721 **0.003
7—Disturbances1172.3541510.97446.429 ***−0.008
8—Residuals1173.8081630.97557.478 ***0.001
Note. ns p > 0.05; * p ≤ 0.05; ** p ≤ 0.01; *** p < 0.001. The χ2scaled column contains the robust test. The Δχ2 column contains the robust difference test which is a function of two standard (not robust) statistics.
Table
Table 6. BAT’s raw mean scores distributional properties (workers from Brazil).
Table 6. BAT’s raw mean scores distributional properties (workers from Brazil).
BAT−23
Brazil Total Sample (n = 2217)
DimensionMSDMinP25MdnP75MaxHistogramskku
Exhaustion2.870.831.002.252.883.385.00▂▅▇▃▂0.18−0.27
Mental distance1.930.871.001.201.802.405.00▇▃▂▁▁1.090.78
Cognitive impairment2.190.791.001.602.202.605.00▇▇▃▁▁0.620.43
Emotional impairment1.970.811.001.401.802.405.00▇▅▂▁▁1.081.17
Burnout2.320.691.001.832.262.745.00▅▇▅▁▁0.680.58
Brazil Female Sample (n = 1653)
Exhaustion2.900.831.002.382.883.385.00▂▅▇▃▂0.15−0.30
Mental distance1.900.861.001.201.602.405.00▇▃▂▁▁1.110.79
Cognitive impairment2.190.781.001.602.002.605.00▇▇▃▁▁0.590.31
Emotional impairment1.990.801.001.401.802.405.00▇▅▂▁▁1.010.95
Burnout2.330.691.001.832.262.745.00▅▇▅▁▁0.650.37
Brazil Male Sample (n = 558)
Exhaustion2.780.831.002.252.753.385.00▂▆▇▃▁0.26−0.13
Mental distance2.020.901.001.401.802.605.00▇▃▂▁▁1.060.79
Cognitive impairment2.170.791.001.602.202.605.00▇▇▃▁▁0.690.81
Emotional impairment1.910.821.001.201.602.405.00▇▅▁▁▁1.291.88
Burnout2.290.691.041.782.222.745.00▆▇▅▁▁0.761.29
BAT−12
Brazil Total Sample (n = 2217)
Exhaustion3.020.931.002.333.003.675.00▂▃▇▃▂0.07−0.35
Mental distance2.070.921.001.332.002.675.00▇▅▃▁▁0.840.19
Cognitive Impairment2.140.821.001.672.002.675.00▇▇▅▁▁0.690.46
Emotional impairment1.840.821.001.001.672.335.00▇▃▂▁▁1.211.54
Burnout2.270.691.001.752.172.675.00▅▇▃▁▁0.720.78
Brazil Female Sample (n = 1653)
Exhaustion3.060.911.002.333.003.675.00▂▃▇▃▂0.06−0.35
Mental distance2.040.911.001.332.002.675.00▇▃▃▁▁0.860.16
Cognitive impairment2.150.811.001.672.002.675.00▇▇▆▁▁0.640.35
Emotional impairment1.860.811.001.331.672.335.00▇▅▂▁▁1.151.33
Burnout2.280.691.001.752.172.675.00▅▇▃▁▁0.700.55
Brazil Male Sample (n = 558)
Exhaustion2.910.941.002.333.003.675▂▅▇▃▂0.11−0.34
Mental distance2.140.931.001.332.002.675▇▅▅▁▁0.810.36
Cognitive impairment2.100.841.001.332.002.675▇▇▅▁▁0.840.78
Emotional impairment1.770.841.001.001.672.005▇▃▂▁▁1.422.24
Burnout2.230.691.001.672.172.675▅▇▃▁▁0.801.52
Table
Table 7. BAT’s raw mean scores distributional properties (workers from Portugal).
Table 7. BAT’s raw mean scores distributional properties (workers from Portugal).
BAT-23
Portugal Total Sample (n = 886)
DimensionMSDMinP25MdnP75MaxHistogramskku
Exhaustion2.910.741.002.382.883.385.00▁▅▇▃▁0.300.26
Mental distance2.140.801.001.402.002.605.00▇▆▃▁▁0.700.09
Cognitive impairment2.410.721.002.002.402.805.00▃▇▅▁▁0.410.54
Emotional impairment2.080.701.001.602.002.405.00▇▇▃▁▁0.700.80
Burnout2.450.621.002.042.392.835.00▂▇▅▁▁0.540.56
Portugal Female Sample (n = 537)
Exhaustion2.950.761.122.502.883.385.00▂▇▇▅▂0.310.05
Mental distance2.100.791.001.402.002.605.00▇▆▃▁▁0.68-0.04
Cognitive impairment2.390.721.002.002.402.805.00▃▇▅▁▁0.330.51
Emotional impairment2.080.691.001.602.002.405.00▇▇▂▁▁0.761.00
Burnout2.450.631.042.002.392.835.00▂▇▅▁▁0.500.41
Portugal Male Sample (n = 204)
Exhaustion2.850.731.002.382.753.385.00▁▆▇▃▁0.300.37
Mental distance2.250.851.001.602.202.805.00▇▇▃▁▁0.780.35
Cognitive impairment2.490.761.002.002.402.855.00▃▇▃▂▁0.570.55
Emotional impairment2.090.771.001.402.002.605.00▇▇▃▁▁0.720.68
Burnout2.480.661.002.002.392.844.91▂▇▅▂▁0.700.77
BAT-12
Portugal Total Sample (n = 886)
Exhaustion3.050.801.002.673.003.675.00▁▃▇▃▁0.120.05
Mental distance2.270.781.001.672.002.675.00▇▇▆▂▁0.590.11
Cognitive Impairment2.380.761.002.002.332.675.00▅▇▇▁▁0.480.59
Emotional impairment2.020.711.001.332.002.335.00▇▇▃▁▁0.710.70
Burnout2.430.611.002.002.332.755.00▂▇▃▁▁0.520.65
Portugal Female Sample (n = 537)
Exhaustion3.090.821.002.673.003.675.00▁▃▇▃▂0.12-0.01
Mental distance2.240.771.001.672.002.675.00▇▇▇▁▁0.55-0.01
Cognitive impairment2.370.751.002.002.332.675.00▅▇▇▁▁0.420.56
Emotional impairment2.020.701.001.332.002.335.00▇▇▃▁▁0.750.83
Burnout2.430.611.002.002.332.835.00▂▇▅▁▁0.460.53
Portugal Male Sample (n = 204)
Exhaustion2.960.801.002.333.003.335.00▁▅▇▃▁0.180.02
Mental distance2.370.831.001.672.333.005.00▆▇▆▂▁0.650.21
Cognitive impairment2.430.811.002.002.332.755.00▃▇▆▂▁0.760.72
Emotional impairment2.020.771.001.332.002.335.00▇▆▃▁▁0.780.75
Burnout2.440.661.002.002.332.834.92▂▇▅▂▁0.740.83
Table
Table 8. Measurement model standardized latent correlations.
Table 8. Measurement model standardized latent correlations.
BAT-23
BurnoutWork EngagementNegative ChangeWork OverloadRole ClarityCo-Workers Support
Burnout −0.81 ***0.52 ***0.59 ***−0.62 ***−0.45 ***
Work Engagement−0.75 *** −0.61 ***−0.32 ***0.66 ***0.41 ***
Negative Change0.54 ***−0.57 *** 0.36 ***−0.56 ***−0.28 ***
Work Overload0.35 ***0.08 *0.16 *** −0.38 ***−0.21 ***
Role Clarity−0.42 ***0.49 ***−0.55 ***0.02 ns 0.47 ***
Co-workers Support−0.45 ***0.49 ***−0.52 ***−0.12 ***0.53 ***
BAT-12
BurnoutWork EngagementNegative ChangeWork OverloadRole ClarityCo-Workers Support
Burnout −0.80 ***0.53 ***0.61 ***−0.62 ***−0.44 ***
Work Engagement−0.73 *** −0.61 ***−0.31 ***0.66 ***0.41 ***
Negative Change0.54 ***−0.57 *** 0.36 ***−0.56 ***−0.28 ***
Work Overload0.35 ***0.09 **0.15 *** −0.38 ***−0.21 ***
Role Clarity−0.44 ***0.48 ***−0.56 ***0.02 ns 0.47 ***
Co-workers Support−0.46 ***0.49 ***−0.52 ***−0.12 ***0.53 ***
Note. The upper triangle values are relative to the sample from Brazil, while the lower triangle values are relative to the sample from Portugal. ns p > 0.05; * p ≤ 0.05; ** p ≤ 0.01; *** p < 0.001.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Sinval, J.; Vazquez, A.C.S.; Hutz, C.S.; Schaufeli, W.B.; Silva, S. Burnout Assessment Tool (BAT): Validity Evidence from Brazil and Portugal. Int. J. Environ. Res. Public Health 2022, 19, 1344. https://doi.org/10.3390/ijerph19031344

AMA Style

Sinval J, Vazquez ACS, Hutz CS, Schaufeli WB, Silva S. Burnout Assessment Tool (BAT): Validity Evidence from Brazil and Portugal. International Journal of Environmental Research and Public Health. 2022; 19(3):1344. https://doi.org/10.3390/ijerph19031344

Chicago/Turabian Style

Sinval, Jorge, Ana Claudia S. Vazquez, Claudio Simon Hutz, Wilmar B. Schaufeli, and Sílvia Silva. 2022. "Burnout Assessment Tool (BAT): Validity Evidence from Brazil and Portugal" International Journal of Environmental Research and Public Health 19, no. 3: 1344. https://doi.org/10.3390/ijerph19031344

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop