Next Article in Journal
Factors Associated with the Quality and Transparency of National Guidelines: A Mixed-Methods Study
Next Article in Special Issue
The North Italian Longitudinal Study Assessing the Mental Health Effects of SARS-CoV-2 Pandemic Health Care Workers—Part II: Structural Validity of Scales Assessing Mental Health
Previous Article in Journal
The Impact Relationships between Scientific and Technological Innovation, Industrial Structure Advancement and Carbon Footprints in China Based on the PVAR Model
Previous Article in Special Issue
Mental Illness and Work-Related Limitations in Healthcare Workers: A Preliminary Retrospective Study
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

The North Italian Longitudinal Study Assessing the Mental Health Effects of SARS-CoV-2 Pandemic on Health Care Workers—Part I: Study Design and Psychometric Structural Validity of the HSE Indicator Tool and Work Satisfaction Scale

Giovanni Veronesi
Emanuele Maria Giusti
Alessia D’Amato
Francesco Gianfagna
Rossana Borchini
Gianluca Castelnuovo
Licia Iacoviello
1,8 and
Marco Mario Ferrario
EPIMED Research Center, Department of Medicine and Surgery, University of Insubria, 21100 Varese, Italy
Psychology Research Laboratory, Istituto Auxologico Italiano IRCCS, 20149 Milan, Italy
Department of Psychology, Catholic University of the Sacred Heart, 20123 Milan, Italy
ASST Sette Laghi, 21100 Varese, Italy
Mediterranea Cardiocentro, 80122 Napoli, Italy
UOS Medicina Preventiva e Legale, ASST Lariana, 22100 Como, Italy
Psychology Research Laboratory, Istituto Auxologico Italiano IRCCS, 28824 Verbania, Italy
Department of Epidemiology and Prevention, IRCCS Neuromed, 86077 Pozzilli, Italy
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2022, 19(15), 9514;
Submission received: 21 June 2022 / Revised: 25 July 2022 / Accepted: 29 July 2022 / Published: 3 August 2022
(This article belongs to the Special Issue Mental Health at the Workplace)


Literature on the impact of the SARS-CoV-2 pandemic on the mental health of Health Care Workers (HCWs) is mostly based on cross-sectional surveys. We designed a longitudinal study to assess work-related stress and mental health before and after the pandemic onset in a university-hospital in Lombardia region, Italy. We report on sample representativeness and structural validity of questionnaires assessing work stress (HSE Indicator Tool, HSE-IT) and work satisfaction (WS), which were not validated in the HCWs population. n = 1287 HCWs from 67 hospital wards/offices were invited to an online survey in summer 2019 (pre-COVID-19 wave) and again during winter 2020 (COVID-19 wave). Selected hospital wards/offices did not differ from the remaining wards for turn-over and down-sizing rates, overload, sick leaves, and night shifts (Wilcoxon rank tests p-values > 0.05). Participation rates were 70% (n = 805) and 60% (n = 431) in the pre-COVID-19 and COVID-19 waves, respectively. Socio-demographic and work-related characteristics did not impact data completeness nor participation to the COVID-19 wave. While confirming a 7-component structure for HSE-IT, we identified a new factor related to participation in work organization. A one-factor model for WS had satisfactory fit. Our longitudinal study based on a representative sample and adopting validated questionnaires is well-suited to elucidate the role of work conditions on the development of mental health disorders in HCWs.

1. Introduction

From the inception of the COVID-19 pandemic, health care workers (HCWs) have been on the frontline against the disease, being one of the most exposed populations [1,2]. Along with the risk of contagion, the shortage of efficient protective equipment, and the moral distress due to the lack of effective treatments to fight the virus [3], HCWs were under overwhelming pressure due to institutional constraints. These include increased workload due to long working hours and irregular shifts, as well as frequent and sudden reorganizations of wards and departments occurring in a short time period to comply with the exponential growth of the pandemic. Coping with all these concurrent challenges is having severe consequences on the HCWs population in terms of wellbeing and mental health [4].
Since the onset of the pandemic, a large number of studies investigated its “impact” on burnout, post-traumatic stress disorders (PTSD), anxiety and depressive symptoms among HCWs [5,6,7,8,9,10,11,12]. Most of them were conducted using a cross-sectional design and with sampling strategies that are unable to assess and mitigate the risk of selection bias. All the 16 cross-sectional studies in the meta-analysis by Kisely et al. [5] were based on convenient samples or snowball sampling; nine did not report participation rates, and two had participation below 40%. In one study in Italy, the use of a convenient sample and the lack of psychometric proprieties in the adopted scales resulted in un-expected and counterintuitive associations between stress and burnout [12,13]. Moreover, the effect of exposures to pre-existing organizational constraints and work stress [14], as well as the pre-pandemic levels of the investigated mental health conditions on the same subjects are methodological issues of major importance to disentangle the true effect of the pandemic on mental health from that of other different components. Furthermore, the potential lack of measurement invariance of the adopted questionnaires after the pandemic outbreak needs to be assessed, as life stressors, natural disasters and major contextual changes might alter the psychometric characteristics of questionnaires [15].
The impact of pre-existing stressful working conditions on the mental health of HCWs have been widely documented [14,16,17]. Besides age and female gender, shift work scheduling, overload, poor sleep quality and circadian rhythm alteration have been associated with burnout onset [14]. Several longitudinal studies established a link between increasing work demand, decreasing control and managers’ and peers’ support with burnout incidence [16,17]. The valid assessment of these conditions in HCWs is therefore important. In Italy, the Health and Safety Executive Management Standards Indicator Tool (HSE) is the standard instrument to assess perceived work stress as recommended by the regulatory labor safety agency [18,19]. Nonetheless, studies assessing its structural validity employed heterogeneous methods and obtained different results [20,21,22]. In addition, the evaluation of structural validity of the Italian version of the HSE in HCWs is lacking.
We designed a longitudinal study with the aim to assess the mental health effects of SARS-CoV-2 pandemic on HCWs, considering the pre-pandemic levels of work strain, work satisfaction and burnout. In this paper, we report on sample representativeness and structural validity of the HSE and of a work satisfaction scale. A parallel paper will address the structural validity and the longitudinal invariance of the selected scales to appraise mental health disorders [23].

2. Materials and Methods

2.1. Study Population, Design and Recruitment

This is a longitudinal study carried out in a large University hospital in the city of Varese, Lombardia, the region with the highest number of SARS-CoV-2 cases and deaths in Italy [24]. The study flow chart is showed in Figure 1. N = 1286 healthcare workers were invited to participate from August to September 2019 to a periodic screening of work stress conditions required by the Italian safety legislation (from now on: pre-COVID-19 wave). These HCWs comprised all physicians, nurses, nurse assistants and front-office clerks either working in hospital wards/offices at expected high levels of work stress conditions according to representatives of the unions and of the hospital administration, or in a random sample of the remaining wards. Overall, the selected wards/offices were 67 from the following aggregated areas: emergency department and ICU, medical wards, surgery wards, front-office administration. HCWs received invitation in a sealed envelope distributed by their hospital ward referent. n = 129 invited workers who have been transferred to another ward or were on a long-term sick leave during the study period, were considered as not eligible. In December 2020, the n = 717 HCWs who responded to the pre-COVID-19 wave were still working were further invited to participate to a survey on the pandemic impact (from now on: COVID-19 wave). E-mail invitations were sent to their institutional account as required by the COVID-19 containment measures on place at that time. Questionnaires at both waves were hosted in a dedicated web site, external to the hospital organization. Participants received anonymized access credentials and were forced to change the password at first access. In addition, they were required to provide an informed consent at the beginning of the questionnaire. The study received ethical approval by the relevant ethical committee (approval ID 69/2020).

2.2. Assessment of Demographic and Work-Related Characteristics in the Pre-COVID-19 Wave

For each participant we collected information on age, sex, educational attainment, job title, work seniority, type of contract (permanent or fixed-term), type of employment (full time or part time), work schedule (day-time work, shift work without night shift, shift work with night shift), and area of hospital ward/office.

2.3. Assessment of Work Stress and Work Satisfaction in the Pre-COVID-19 Wave

We administered the Health and Safety Executive Management Standards Indicator Tool [18], revised and validated to the Italian context (HSE-IT; [19]). The HSE-IT was developed to measure presence of workplace stressors as perceived by workers, and hence the measurement model that underpins the instrument is formative, i.e., based on the assumption that the items cause the construct being measured. HSE-IT consists in 35 items on a 1 to 5 Likert scale assessing working conditions known to potentially cause work-related stress. The original questionnaire structure comprises seven constructs: demands, control, managers’ support, peer support, relationships, role and change [18].
Furthermore, we administered a 4-item work satisfaction scale investigating satisfaction level (1–4 Likert scale). The scale was purposefully-developed for the study, adapting two items from the Copenhagen Psychosocial Questionnaire [25] referring to work prospects and to how personal abilities are used, and adding two items on work results and salary. The underlying model for this scale is reflective, i.e., based on the assumption that a latent construct “work-satisfaction” causes the responses to the items.

2.4. Assessment of Burnout and Mental Health

In both the pre-COVID-19 and the COVID-19 waves, we administered the Maslach Burnout Inventory, to assess burnout [26]. In the COVID-19 wave, we employed the following questionnaires: i. General Health Questionnaire–12, a measure of mental well-being [27]; ii. PTSD Checklist for DSM-5-Short Form, to assess symptoms of post-traumatic stress disorder [28]; iii. Connor-Davidson Resilience scale, assessing resilience [29]; iv. Post-Traumatic Growth Inventory–Short Form, a self-rated questionnaire assessing positive outcomes reported by people who have experienced traumatic events [30]. We adopted the validated Italian version of all these questionnaires. The psychometric properties of these scales in our study sample are assessed and discussed in a companion paper [23].

2.5. Statistical Analysis

We assessed representativeness of the study sample at both waves. First, we evaluated possible differences in pre-COVID-19 work-related variables between selected vs. un-selected wards/offices by comparing the following metrics: turn-over rate, up-down sizing rate, overload, short-term sick leave spell, average number of night shifts, and prevalence of regular night shifts (i.e., followed by a daily rest). These metrics were obtained starting from the hospital administrative datasets according to an established method [31] and were referred to the latest available period prior to the pre-COVID-19 wave enrollment (January 2018 to April 2019). Due to their skewed distributions, for each metric we reported the median value with 25°–75° percentiles among selected and non-selected wards/offices, and tested the null hypothesis of homogeneity of metrics using non-parametric Wilcoxon tests. A detailed analysis by job titles is reported in the Supplementary Material. Second, we investigated the amount of missing data and the presence of missing data patterns in the HSE-IT and the work satisfaction scales administered at the pre-COVID-19 wave using absolute and relative frequencies and graphical methods, respectively. The Little’s MCAR test was performed once to assess if the missing data mechanism in the HSE and in the work satisfaction scale was Missing Completely At Random [32]. Then, we explored which socio-demographic and work-related characteristics (n = 9 variables) were associated with missingness in each item of the HSE-IT (n = 35 item) and work satisfaction scale (n = 4 item) by performing chi-square analyses or Mann-Whitney tests, as appropriate, between missingness in each item and each explanatory variable. We applied the Bonferroni correction to retain statistical significance in order to control for multiple testing (corrected alpha = 0.05/((35 + 4) × 9) = 0.0007). Finally, we evaluated sample representativeness at the COVID-19 wave by contrasting socio-demographic and work-related characteristics between respondents (n = 431) and not respondents (n = 286) at the COVID-19 wave.
We then focused on the structural validity of the questionnaires assessing pre-COVID-19 work stress and satisfaction. Since the HSE-IT is formative, the assumptions for Exploratory (EFA) or Confirmatory (CFA) Factor Analyses, as well as for Cronbach’s alpha, to assess its structure and internal consistency, respectively, are not met [33,34]. Therefore, we performed a Principal Component Analysis (PCA) to identify the components explaining the maximum variance of the HSE-IT items. Firstly, the components solutions obtained by extracting the components with eigenvalues > 1 and by examining the scree plot were compared. Since none of these methods produced a plausible components solution, we checked the components solution resulting from fixing the number of components to extract to 7, in accordance with the number of dimensions originally proposed by Cousins et al. [18]. All these solutions were interpreted after performing an Oblimin rotation to the components weights matrix. Then, as recommended for formative scales, we checked the presence of multicollinearity between the items. For each component, we performed a multiple regression in which the component was included as a dependent variable and the items that compose it as predictors, and we checked the tolerance of each item. Items whose tolerance was ≤0.35 were considered as being multicollinear.
Conversely, the factorial structure of the work satisfaction scale was assessed by performing a CFA. The fit of a unidimensional model was checked. Since the items of this scale are ordinal, a Diagonally Weighted Least Squares estimator was chosen. The fit indices that were used to evaluate the fit of the model were the Standardized Root Mean Square Residual (SRMR) and the Adjusted Goodness of Fit Index (AGFI), using the following cut-offs for model retention: SRMS < 0.08, AGFI > 0.90 [35]. Then, we assessed the internal consistency of the scale using Cronbach’s alpha, with a cutoff for the identification of adequate internal consistency at 0.70. The analyses were performed using SAS OnDemand for Academics software (release 9.04) (SAS Institute Inc., Cary, NC, USA).

3. Results

The timing of the study waves in relation to the pandemic curve is depicted in Figure 2. In the investigated hospital population as well as in the entire Province of Varese, the major spread of the SARS-CoV-2 pandemic started in October 2020. The COVID-19 wave took place during a downward trend in the weekly number of cases in the Varese Province, due to local mobility restriction periods. With the start of vaccination in January 2021 the epidemic reduces promptly among HCWs who were the first high-risk group involved.

3.1. Sample Representativeness at the Pre-COVID-19 Wave

The n = 67 selected hospital wards and offices invited to the pre-COVID-19 wave were not different from other wards/offices with respect to the considered work-related metrics (Table 1). Further breakdown by job title (Supplementary Table S1) revealed some differences, with selected wards showing higher median values of turn-over rates among nurse assistants and administrative clerks; higher overload working hours among nurses and nurse assistants; and higher spell of short-term sick leave, but also higher regularity in night shifts scheduling, among physicians.
Participation rate at the pre-COVID-19 wave was 69.6% of eligible (805/1157; Figure 1), and more in details 54.7%, 72.3%, 67.7% and 82.9% among physicians (104/190 eligible), nurses (429/593), nurse assistants (170/251) and clerks (102/123), respectively.
Among the 805 filled-in questionnaires, missing data was present in only 5% of the responses to the HSE-IT, and in 1.4% of the responses to the work satisfaction scale. No missing data pattern was detected by visual inspection of plots. The Little’s MCAR test was significant (χ2 = 1750.7, df = 1601, p = 0.005). However, no variable was associated with missingness in each of the HSE-IT and Work Satisfaction Scale items at the Bonferroni-corrected alpha (data not shown). Due to the low amount of missing data and the absence of variables associated with missingness in each of the items, observations with missing values were deleted from structure validity assessment.

3.2. Sample Representativeness at the COVID-19 Wave

Participation rate at the COVID-19 wave was 60.1% of eligible (431/717; Figure 1), and more in details 48.8%, 65.4%, 59.9% and 48.9% among physicians (42/86 eligible), nurses (261/399), nurse assistants (85/142) and clerks (43/88), respectively. Table 2 reports the socio-demographic and work-related characteristics at the pre-COVID-19 wave, among respondents and not respondents to the COVID-19 wave. We found no differences in mean age, gender distribution, work seniority, type of employment and work schedule (all p-values > 0.05). Conversely, participants were less likely to work in administration offices (8.6% vs. 15.7%; chi-square test p-value = 0.003), probably reflecting the slightly lower participation rate observed among office clerks.

3.3. Structural Validity of the HSE-IT Questionnaire

Eight components had eigenvalue >1 and were firstly extracted, but the corresponding component solution (explained variance: 61%) was not plausible since a component had no weights >0.35 and another component was associated only with two items, whose weights were one positive and one negative. The analysis of the scree plot suggested that six components could better represent the data. However, the corresponding component structure (explained variance: 55%) was not plausible, since the content of the items in two components were heterogeneous and another component was the combination of two items with positive loadings and two items with negative loadings. We then extracted 7 components and the corresponding component solution (explained variance: 58%) was deemed as plausible. Table 3 reports the standardized loadings of the final component solution.
Four of the identified components, namely demands, peer support, role and relationships, were identical to the original HSE questionnaire (Supplementary Table S2). We identified a new component comprising three items of the original change factor (“I have sufficient opportunities to question managers about change at work”, “Staff are always consulted about change at work” “When changes are made at work, I am clear how they will work out in practice”), one item from control (“working time can be flexible”) and one item from the managers’ support (“I am supported through emotionally demanding work”) factors. The items in this new component are bonded by the possibility to an active participation to shaping the changes in the work organization and to the organization of the own work. Therefore, this component was named “participation in work organization”. No item multicollinearity was apparent since the tolerance values of all items were >0.35 (data not shown).

3.4. Structural Validity of the Work Satisfaction Scale

The CFA on the work satisfaction scale showed that the unidimensional model had good fit (SRMR = 0.017, AGFI = 0.999). The four standardized loading ranged between 0.48 and 0.78. The Cronbach’s alpha value of 0.73 suggests adequate internal consistency.

4. Discussion

Most of research on the impact of COVID-19 on the mental health of health care workers relies on cross-sectional surveys conducted during the pandemic [5,6,7,8,9,10,11,12]. As such, they cannot consider pre-pandemic levels of the investigated mental health conditions, nor their determinants such as stressful working conditions [14,16,17]. In addition, there is scanty attention to methodological issues including sample selection [5] and measurement error due to poor psychometric proprieties of the adopted scales [13], resulting in counterintuitive associations [12]. Therefore, despite their aim, any causal claim coming from existing literature should not be taken as granted.
We designed a longitudinal study with repeated measures on the same health care workers taken prior to and during the COVID-19 pandemic. To our knowledge, only two studies have a similar design. However, in one of these studies a very small sample of HCWs responded to both pre- and post-pandemic waves (n = 15; [36]), while the second on n = 153 Intensive Care Unit professionals did not measure work-related stress in the pre-COVID-19 wave [37]. With respect to these studies, we sensibly expanded the sample size and generalized the sample to a number of heterogeneous hospital wards and settings. In this paper, we focused on potential selection bias from a number of different perspectives, including the characteristics of the hospital wards from which the sample was originated, participation rates at both study waves, and data completeness. Taken together, our findings consistently point towards a satisfactory representativeness of the study sample with respect to the operating health care workforce population.
From a methodological viewpoint, the adoption of the HSE and its Italian version to investigate perceived work stress may suffer from two major drawbacks: the use of inappropriate analyses to validate its psychometric validity, and the lack of a formal validation in the specific HCWs population. The study which developed the HSE [18] and the studies that assessed its structural validity employed EFAs or CFAs and Cronbach’s alphas [19,20,21,22], which are inappropriate if there is no latent factor causing the responses to the questionnaire, as in formative scales. The improper use of EFAs and CFAs may have important consequences, such as inadequate selection of items during the development of the scale, misrepresentation of constructs, biased estimates of the relationship between the item and the latent variable or of the relationship between the latent variable and other variables [33,34]. Regarding the Italian version, Magnavita [20] on a sample of 748 workers from 17 companies employed a Principal Component Analysis, as appropriate for the assessment of formative constructs, but interpreting results as if it were an EFA. Hence, he suggests the use of the identified structure only for the Italian version, not noting that the scale was previously analyzed as if it was reflective. Therefore, we suggest that other studies should address the structural validity of the HSE using the appropriate statistical methods.
The PCA allowed us to find a 7-component solution explaining 58% of the variance whose components had a plausible content. Similarly to the PCA performed by Magnavita [20], in both cases 7 components were extracted and the content of five of them overlap. In contrast, Magnavita did not interpret the 7th component since no significant loadings were present; and identified an “Elasticity” component which corresponds to the sum of the “Managers’ support” and the “Participation to the work organization” components found in our study. Employee participation is a key concept for the understanding of the relationship between the worker and the work context, besides being known as a determinant of organization performance [38]. Therefore, our newly identified component has a sound theoretical background. Similarly to the study by Magnavita, item 33 (“I am supported through emotionally demanding work”) cross-loaded on two components, namely “Participation to Work Organization” and “Peer support”. Rather than to the influence of the specific organizational setting, this might be due to the content of the item, which can be interpreted by the respondent as receiving either formal or informal support from multiple, unspecified sources. However, since ours is the first attempt to validate HSE-IT among HCWs, the removal of the item requires future confirmation in other studies. Until then, we recommend using our component structure to assess work stressors in HCWs samples.
We acknowledge the following limitations. First, our longitudinal design has only one assessment during the COVID-19 pandemic, and therefore it is unable to investigate whether the effects on mental health are short- or long-term ones. At the time we are writing this paper, the pandemic has not ended yet. Second, we did not administer HSE-IT at the COVID-19 wave. Therefore, we are not able to assess whether the 7 components’ structure observed in our study is time-invariant with respect to the pandemic outbreak. In addition, we will not be able to evaluate changes in perceived work stress and work satisfaction due to the COVID-19 pandemic. Finally, our study involved only one hospital in Northern Italy, and hence future research is desirable to confirm and generalize the structural validity of HSE-IT and work satisfaction in different healthcare settings. Among the study strengths, we mention: the longitudinal study design with a representative and large sample at both pre-COVID-19 and COVID-19 waves; the adoption of several mental health outcome scales; and the assessment of protective factors which help individuals to maintain the psychological wellbeing, such as resilience [29] and post-traumatic growth [30]. A parallel paper addresses the structural validity of the outcome scales, including measurement invariance over time, as well as of their potential mediators and effect modifiers [23]. Taken together, the two companion papers can provide useful suggestions on methodological issues related to the causal link between pandemic and mental health.
In conclusion, our study is based on a representative sample of HCWs from several hospital wards. The identified 7-component structure for HSE-IT, including a new “participation in work organization” component, explained a large amount of variance of perceived work stress, and it can be adopted in the health care workers population. Due to its longitudinal design, our study is well-suited to elucidate the role of work conditions, registered before the pandemic, on the development of mental health disorders among health care workers.

Supplementary Materials

The following supporting information can be downloaded at:, Table S1: Assessing representativeness of the surveyed sample, by job title: medians and interquartile ranges (25th–75th percentiles) of turn-over, up/down sizing, overload, spell of short-term sick leaves, number of night shifts and night shifts regularity in recruited and non-recruited wards; Table S2: Comparison between the results of the Principal Component Analysis and the results of the analyses performed by previous studies on the structural validity of the HSE.

Author Contributions

Conceptualization, G.V., F.G., L.I. and M.M.F.; Data curation, G.V.; Formal analysis, E.M.G.; Funding acquisition, L.I. and M.M.F.; Investigation, A.D.; Methodology, G.V., E.M.G. and M.M.F.; Project administration, F.G.; Validation, G.V.; Writing—original draft, G.V., E.M.G. and M.M.F.; Writing—review & editing, A.D., F.G., R.B., G.C. and L.I. All authors have read and agreed to the published version of the manuscript.


This study was sponsored by a grant provided by Regione Lombardia (DG-Welfare n. 7082/2020); a grant by Fondazione Veronesi (COVID-19 Insieme per la Ricerca 2020); and by a grant provided by Istituto Nazionale per l’Assicurazione Contro gli Infortuni sul Lavoro (Grant ID: BRIC-2019 ID 29A).

Institutional Review Board Statement

The study received ethical approval by the ASST Sette Laghi ethical committee (approval ID 69/2020).

Informed Consent Statement

Informed consent was obtained from all the subjects involved in the study.

Data Availability Statement

Anonymized study data are available upon motivated and reasonable request to the corresponding author.


The authors wish to thank Paolo Lasalvia, for creation and maintenance of the study dataset; Simona Boneschi, who contributed to questionnaire preparation and data collection; and Elena Brevi and Francesco Macchi, for their help in the bibliographic research.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study, in the collection, analyses, or interpretation of data, in the writing of the manuscript, or in the decision to publish the results.


  1. Sim, M.R. The COVID-19 pandemic: Major risks to healthcare and other workers on the front line. Occup. Environ. Med. 2020, 77, 281–282. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. Nafilyan, V.; Pawelek, P.; Ayoubkhani, D.; Rhodes, S.; Pembrey, L.; Matz, M.; Coleman, M.; Allemani, C.; Windsor-Shellard, B.; van Tongeren, M.; et al. Occupation and COVID-19 mortality in England: A national linked data study of 14.3 million adults. Occup. Environ. Med. 2021, 79, 433–441. [Google Scholar] [CrossRef]
  3. BMA. COVID-19: The Impact of Pandemic on the Medical Profession. May 2022. Available online: (accessed on 6 June 2022).
  4. Abbasi, J. Pushed to Their Limits, 1 in 5 Physicians Intends to Leave Practice. JAMA 2022, 327, 1435. [Google Scholar] [CrossRef]
  5. Kisely, S.; Warren, N.; McMahon, L.; Dalais, C.; Henry, I.; Siskind, D. Occurrence, prevention, and management of the psychological effects of emerging virus outbreaks on healthcare workers: Rapid review and meta-analysis. BMJ 2020, 369, m1642. [Google Scholar] [CrossRef] [PubMed]
  6. Galanis, P.; Vraka, I.; Fragkou, D.; Bilali, A.; Kaitelidou, D. Nurses’ burnout and associated risk factors during the COVID-19 pandemic: A systematic review and meta-analysis. J. Adv. Nurs. 2021, 77, 3286–3302. [Google Scholar] [CrossRef] [PubMed]
  7. Chigwedere, O.C.; Sadath, A.; Kabir, Z.; Arensman, E. The Impact of Epidemics and Pandemics on the Mental Health of Healthcare Workers: A Systematic Review. Int. J. Environ. Res. Public Health 2021, 18, 6695. [Google Scholar] [CrossRef] [PubMed]
  8. Holmes, M.R.; Rentrope, C.R.; Korsch-Williams, A.; King, J.A. Impact of COVID-19 Pandemic on Posttraumatic Stress, Grief, Burnout, and Secondary Trauma of Social Workers in the United States. Clin. Soc. Work J. 2021, 49, 495–504. [Google Scholar] [CrossRef]
  9. Caldas, M.P.; Ostermeier, K.; Cooper, D. When helping hurts: COVID-19 critical incident involvement and resource depletion in health care workers. J. Appl. Psychol. 2021, 106, 29–47. [Google Scholar] [CrossRef]
  10. Ghahramani, S.; Lankarani, K.B.; Yousefi, M.; Heydari, K.; Shahabi, S.; Azmand, S. A Systematic Review and Meta-Analysis of Burnout Among Healthcare Workers During COVID-19. Front. Psychiatry 2021, 12, 758849. [Google Scholar] [CrossRef]
  11. Magnavita, N.; Soave, P.M.; Antonelli, M. A One-Year Prospective Study of Work-Related Mental Health in the Intensivists of a COVID-19 Hub Hospital. Int. J. Environ. Res. Public Health 2021, 18, 9888. [Google Scholar] [CrossRef]
  12. Brera, A.S.; Arrigoni, C.; Dellafiore, F.; Odone, A.; Magon, A.; Nania, T.; Pittella, F.; Palamenghi, L.; Barello, S.; Caruso, R. Burnout syndrome and its determinants among healthcare workers during the first wave of the COVID-19 outbreak in Italy: A cross-sectional study to identify sex-related differences. Med. Lav. 2021, 112, 306–319. [Google Scholar] [CrossRef]
  13. Campanini, P. Methodological issues in assessing job stress and burnout in psychosocial research. Med. Lav. 2021, 112, 264–267. [Google Scholar] [CrossRef] [PubMed]
  14. Hacimusalar, Y.; Misir, E.; Kahve, A.C.; Hacimusalar, G.D.; Guclu, M.A.; Karaaslan, O. The effects of working and living conditions of physicians on burnout level and sleep quality. Med. Lav. 2021, 112, 346–359. [Google Scholar] [CrossRef]
  15. Olino, T.M.; Case, J.A.C.; Hawes, M.T.; Szenczy, A.; Nelson, B.; Klein, D.N. Testing Invariance of Measures of Internalizing Symptoms Before and After a Major Life Stressor: The Impact of COVID-19 in an Adolescent and Young Adult Sample. Assessment, 2021; online ahead of print. [Google Scholar] [CrossRef]
  16. Pisanti, R.; van der Doef, M.; Maes, S.; Meier, L.L.; Lazzari, D.; Violani, C. How Changes in Psychosocial Job Characteristics Impact Burnout in Nurses: A Longitudinal Analysis. Front. Psychol. 2016, 7, 1082. [Google Scholar] [CrossRef] [Green Version]
  17. Langballe, E.M.; Innstrand, S.T.; Aasland, O.G.; Falkum, E. The predictive value of individual factors, work-related factors, and work-home interaction on burnout in female and male physicians: A longitudinal study. Stress Health 2010, 27, 73–87. [Google Scholar] [CrossRef]
  18. Cousins, R.; Mackay, C.J.; Clarke, S.D.; Kelly, C.; Kelly, P.J.; McCaig, R.H. Management Standards and work-related stress in the UK: Practical development. Work Stress 2004, 18, 113–136. [Google Scholar] [CrossRef]
  19. Iavicoli, S.; Natali, E.; Rondinone, B.M.; Castaldi, T.; Persechino, B. Implementation and validation in the Italian context of the HSE management standards: A contribution to provide a practical model for the assessment of work-related stress. G. Ital. Med. Lav. Ergon. 2011, 32, 130–133. [Google Scholar]
  20. Magnavita, N. Validation of the Italian version of the HSE Indicator Tool. Occup. Med. Lond. 2012, 62, 288–294. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  21. Edwards, J.A.; Webster, S.; Van Laar, D.; Easton, S. Psychometric analysis of the UK Health and Safety Executive’s Management Standards work-related stress Indicator Tool. Work Stress 2008, 22, 96–107. [Google Scholar] [CrossRef]
  22. Brookes, K.; Limbert, C.; Deacy, C.; O’Reilly, A.; Scott, S.; Thirlaway, K. Systematic review: Work-related stress and the HSE Management Standards. Occup. Med. 2013, 63, 463–472. [Google Scholar] [CrossRef] [Green Version]
  23. Giusti, E.M.; Veronesi, G.; Callegari, C.; Calstelnuovo, G.; Iacoviello, L.; Ferrario, M.M. The North Italian Longitudinal Study assessing the Mental Health effects of SARS-CoV-2 pandemic Health Care Workers—Part II: Structural validity of scales assessing mental health paper. Int. J. Environ. Res. Public Health, 2022; accepted. [Google Scholar]
  24. Dong, E.; Du, H.; Gardner, L. An interactive web-based dashboard to track COVID-19 in real time. Lancet Infect. Dis. 2020, 20, 533–534. [Google Scholar] [CrossRef]
  25. Kristensen, T.S.; Hannerz, H.; Høgh, A.; Borg, V. The Copenhagen Psychosocial Questionnaire—A tool for the assessment and improvement of the psychosocial work environment. Scand. J. Work Environ. Health 2005, 31, 438–449. [Google Scholar] [CrossRef] [Green Version]
  26. Maslach, C.; Jackson, S.E. The measurement of experienced burnout. J. Organ. Behav. 1981, 2, 99–113. [Google Scholar] [CrossRef]
  27. Goldberg, D.P.; Hillier, V.F. A scaled version of the General Health Questionnaire. Psychol. Med. 1979, 9, 139–145. [Google Scholar] [CrossRef]
  28. Blevins, C.A.; Weathers, F.W.; Davis, M.T.; Witte, T.K.; Domino, J.L. The Posttraumatic Stress Disorder Checklist for DSM-5 (PCL-5): Development and Initial Psychometric Evaluation. J. Trauma. Stress 2015, 28, 489–498. [Google Scholar] [CrossRef] [PubMed]
  29. Connor, K.M.; Davidson, J.R.T. Development of a new resilience scale: The Connor-Davidson Resilience Scale (CD-RISC). Depress. Anxiety 2003, 18, 76–82. [Google Scholar] [CrossRef] [PubMed]
  30. Tedeschi, R.G.; Calhoun, L.G. The Posttraumatic Growth Inventory: Measuring the positive legacy of trauma. J. Trauma. Stress. 1996, 9, 455–471. [Google Scholar] [CrossRef] [PubMed]
  31. Ferrario, M.M.; Bertù, L.; Cimmino, L.; Conti, M.; Borsani, A.; Veronesi, G. A multi-dimensional evaluation of job stress integrating organizational and subjective risk indicators: One application to health care workers in Northern Italy. G. Ital. Med. Lav. Ergon. 2013, 33, 315–318. [Google Scholar]
  32. Little, R.J.A. A Test of Missing Completely at Random for Multivariate Data with Missing Values. J. Am. Stat. Assoc. 1988, 83, 1198. [Google Scholar] [CrossRef]
  33. Bollen, K.A.; Diamantopoulos, A. In defense of causal-formative indicators: A minority report. Psychol. Methods 2017, 22, 581–596. [Google Scholar] [CrossRef] [PubMed]
  34. Diamantopoulos, A.; Siguaw, J.A. Formative Versus Reflective Indicators in Organizational Measure Development: A Comparison and Empirical Illustration. Br. J. Manag. 2006, 17, 263–282. [Google Scholar] [CrossRef]
  35. Hu, L.; Bentler, P.M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct. Equ. Model. Multidiscip. J. 1999, 6, 1–55. [Google Scholar] [CrossRef]
  36. Gomez, S.; Anderson, B.J.; Yu, H.; Gutsche, J.; Jablonski, J.; Martin, N.; Kerlin, M.P.; Mikkelsen, M.E. Benchmarking Critical Care Well-Being: Before and After the Coronavirus Disease 2019 Pandemic. Crit. Care Explor. 2020, 2, e0233. [Google Scholar] [CrossRef]
  37. Kok, N.; van Gurp, J.; Teerenstra, S.; van der Hoeven, H.; Fuchs, M.; Hoedemaekers, C.; Zegers, M. Coronavirus Disease 2019 Immediately Increases Burnout Symptoms in ICU Professionals: A Longitudinal Cohort Study. Crit. Care Med. 2021, 49, 419–427. [Google Scholar] [CrossRef] [PubMed]
  38. Jiang, Y.; He, X.; Zhu, Y.; Wu, G.; Gao, X. The Economic Effects of Direct and Indirect Employee Involvement: Evidence From Corporate Social Responsibility Reports of Chinese Listed Companies. Front. Psychol. 2021, 12, 762608. [Google Scholar] [CrossRef]
Figure 1. Study flow chart: invited, eligible and respondents to pre-COVID-19 (blue boxes) and to COVID-19 (yellow boxes) waves.
Figure 1. Study flow chart: invited, eligible and respondents to pre-COVID-19 (blue boxes) and to COVID-19 (yellow boxes) waves.
Ijerph 19 09514 g001
Figure 2. Timing of the study waves, and time trend in the weekly number of COVID-19 cases among health care workers in the investigated Varese city hospital (solid black line) and in the population living in the Varese Province (dashed grey line; cases per 100).
Figure 2. Timing of the study waves, and time trend in the weekly number of COVID-19 cases among health care workers in the investigated Varese city hospital (solid black line) and in the population living in the Varese Province (dashed grey line; cases per 100).
Ijerph 19 09514 g002
Table 1. Assessing representativeness of the surveyed sample: medians and interquartile ranges (25th–75th percentiles) of turn-over, up/down sizing, overload, spell of short-term sick leaves, number of night shifts and night shifts regularity in recruited and non-recruited wards.
Table 1. Assessing representativeness of the surveyed sample: medians and interquartile ranges (25th–75th percentiles) of turn-over, up/down sizing, overload, spell of short-term sick leaves, number of night shifts and night shifts regularity in recruited and non-recruited wards.
MetricUnit of MeasurementRecruited Hospital Wards/Offices (n = 67)Other Hospital Wards/Offices (n = 297)p-Value ^
Turn-overnumber per 100 HCWs38.9 (15.8; 68.3)28.9 (0.0; 63.6)0.11
Up/down sizingnumber per 100 HCWs0.0 (−8.9; 4.8)0.0 (−9.3; 0.0)0.44
Overload hours/week per HCW1.5 (0.6; 2.4)1.1 (0.4; 2.2)0.11
Spell of short-term * sick leavenumber per 100 HCWs124.5 (62.6; 175.2)119.7 (42.0; 205.1)0.75
Night shiftsnumber per HCW58.5 (20.5; 75.9)47.9 (14.4; 79.2)0.56
Regular night shiftspercent85.3 (41.4; 96.6)65.5 (32.9; 97.5)0.54
In the table: median (25th–75th percentile). Abbreviations: HCWs = health care workers Metrics are referred to the period: 01 January 2018–30 April 2019 (prior to the pre-COVID-19 wave). *: <3 days. ^: Wilcoxon rank test.
Table 2. Socio-demographic and work-related characteristics assessed at the pre-COVID-19 wave, for respondents and not respondents at the COVID-19 wave.
Table 2. Socio-demographic and work-related characteristics assessed at the pre-COVID-19 wave, for respondents and not respondents at the COVID-19 wave.
(n = 431)
Not Respondents
(n = 286)
p-Value ^
Age, years46.5 ± 9.546.5 ± 10.00.94
Men, n (%)83 (19.3%)70 (24.1%)0.12
Educational attainment, n (%)
   Less than high school50 (11.6%)42 (14.7%)0.16
   High school185 (43.0%)104 (36.4%)
   University degree195 (45.4%)140 (49.0%)
Work seniority, years16.2 ± 10.915.1 ± 11.40.21
Type of employment, n (%)
   Full-time364 (85.7%)250 (87.7%)0.25
   Part-time66 (15.3%)35 (12.3%)
Work schedule, n (%)
   Day-time work69 (16.1%)60 (21.1%)0.18
   Shift work w/o night shift83 (19.3%)46 (16.1%)
   Shift work with night shift278 (64.6%)179 (62.8%)
Hospital ward/office, n (%)
   Emergency Department182 (42.2%)120 (42.0%)0.003
   Medical wards80 (18.6%)60 (21.0%)
   Surgical wards132 (30.6%)61 (21.3%)
   Administration offices37 (8.6%)45 (15.7%)
In the table: mean ± standard deviation for continuous variables, and n (%) for categorical variables. ^: p-value comparing respondents to not respondents, from t-test for continuous variables and chi-square test for categorical variables.
Table 3. Loadings of the HSE items on the components identified through Principal Component Analysis.
Table 3. Loadings of the HSE items on the components identified through Principal Component Analysis.
Item DemandsControlPeer
Manager’s SupportRoleParticipation in Work
Item 30.64−−0.06
Item 60.69−−0.110.05
Item 90.620.14−0.04−0.04−
Item 120.69−0.070−
Item 160.420.21−0.120.16−
Item 180.34−0.03−0.030.28−
Item 200.570.20−0.03−0.01−
Item 220.730.−0.040.01
Item 20.240.6000.04−0.14−0.03−0.06
Item 100.060.800.04−−0.02
Item 15−0.080.80−−0.020.03
Item 19−0.130.770.−0.05
Item 25−0.090.590.
Item 27−−
Item 3100.020.80−0.06−
Item 2300.0400.870.03−0.010.03
Item 29−−
Item 35−−0.03
Item 4−0.030.12−0.11−0.070.60−0.100.12
Item 1100.08−−0.110.18
Item 130.07−−0.07
Item 170.16−0.0600.090.560.37−0.13
Item 26−
Item 28−−0.020.640.10
Item 300.050.15−0.100−0.150.590.01
Item 320.05−0.030.05−−0.03
Item 330.060.130.350.150.010.39−0.09
Item 50.180.07−−0.030.67
Item 14−0.04−0.10.46−
Item 210.140.06−
Item 340.0700.2300.11−0.020.48
Note. Adjacent loadings belonging to the same component are marked in bold.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Veronesi, G.; Giusti, E.M.; D’Amato, A.; Gianfagna, F.; Borchini, R.; Castelnuovo, G.; Iacoviello, L.; Ferrario, M.M. The North Italian Longitudinal Study Assessing the Mental Health Effects of SARS-CoV-2 Pandemic on Health Care Workers—Part I: Study Design and Psychometric Structural Validity of the HSE Indicator Tool and Work Satisfaction Scale. Int. J. Environ. Res. Public Health 2022, 19, 9514.

AMA Style

Veronesi G, Giusti EM, D’Amato A, Gianfagna F, Borchini R, Castelnuovo G, Iacoviello L, Ferrario MM. The North Italian Longitudinal Study Assessing the Mental Health Effects of SARS-CoV-2 Pandemic on Health Care Workers—Part I: Study Design and Psychometric Structural Validity of the HSE Indicator Tool and Work Satisfaction Scale. International Journal of Environmental Research and Public Health. 2022; 19(15):9514.

Chicago/Turabian Style

Veronesi, Giovanni, Emanuele Maria Giusti, Alessia D’Amato, Francesco Gianfagna, Rossana Borchini, Gianluca Castelnuovo, Licia Iacoviello, and Marco Mario Ferrario. 2022. "The North Italian Longitudinal Study Assessing the Mental Health Effects of SARS-CoV-2 Pandemic on Health Care Workers—Part I: Study Design and Psychometric Structural Validity of the HSE Indicator Tool and Work Satisfaction Scale" International Journal of Environmental Research and Public Health 19, no. 15: 9514.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop