Next Article in Journal
Iodine Excretion and Intake in Women of Reproductive Age in South Australia Eating Plant-Based and Omnivore Diets: A Pilot Study
Next Article in Special Issue
The Relational Fit in Organizational Interventions—What Can Organizational Research Learn from Research in Psychotherapy?
Previous Article in Journal
Vertical Archaeology: Safety in the Use of Ropes for Scientific Research of Pre-Columbian Andean Cultures
Previous Article in Special Issue
A Hybrid Effectiveness-Implementation Study of a Multi-Component Lighting Intervention for Hospital Shift Workers
 
 
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Participatory Intervention to Improve the Psychosocial Work Environment and Mental Health in Human Service Organisations. A Mixed Methods Evaluation Study

1
Institute for Environmental Medicine, Karolinska Institutet, Unit of Occupational Medicine, 171 77 Stockholm, Sweden
2
Department of Public Health and Caring Sciences, Uppsala University, 751 22 Uppsala, Sweden
3
Center of Occupational and Environmental Medicine, Stockholm Region, 113 65 Stockholm, Sweden
4
Medical Management Centre, Procome Research Group, Department of Learning, Informatics, Management and Ethics, Karolinska Institutet, 171 77 Stockholm, Sweden
5
Department of Global Public Health, Karolinska Institutet, 171 77 Stockholm, Sweden
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2021, 18(7), 3546; https://doi.org/10.3390/ijerph18073546
Received: 26 February 2021 / Revised: 22 March 2021 / Accepted: 25 March 2021 / Published: 29 March 2021
(This article belongs to the Special Issue Workplace Interventions)

Abstract

:
Work-related stress is a global problem causing suffering and economic costs. In Sweden, employees in human service occupations are overrepresented among persons on sick leave due to mental health problems such as stress-related disorders. The psychosocial work environment is one contributing factor for this problem, making it urgent to identify effective methods to decrease stress at the workplace. The aim of the study is to evaluate a participatory intervention to improve the psychosocial work environment and mental health using an embedded mixed methods design. The study is a controlled trial with a parallel process evaluation exploring fidelity and participants’ reactions to the intervention activities, experiences of learning and changes in behaviours and work routines. We collected data through documentation, interviews and three waves of questionnaires. Our results show small changes in behaviours and work routines and no positive effects of the intervention on the psychosocial work environment nor health outcomes. One explanation is end-users’ perceived lack of involvement over the process causing the intervention to be seen as a burden. Another explanation is that the intervention activities were perceived targeting the wrong organisational level. A representative participation over both content and process can be an effective strategy to change psychosocial working conditions and mental health.

1. Introduction

Common mental disorders are the predominant cause of sick leave in Sweden today. During the last three decades, the share of psychiatric diagnoses in sick leave spells over two months increased from 13 to 45 percent among women and from 16 to 33 percent among men [1]. Stress-related disorders account for the largest increase among the psychiatric diagnoses, particularly exhaustion syndrome, a diagnosis with a specific ICD-10 code in Sweden, with sick leaves lasting approximately half a year [1]. Further, impaired sleep is a common symptom in common mental disorders, such as depression and exhaustion [2]. In this study, we measure burnout and impaired sleep as symptoms of common mental disorders. Burnout is an undesirable psychological state characterized by exhaustion, cynicism and feelings of reduced professional efficacy [3].
Human service occupations within the health and social care industry typically stand out as risk occupations for sick leave [4]; in particular, sick leave due to stress-related mental disorders [1]. One occupation within these industries with high risk of experiencing high levels of stress contributing to burnout is teachers [5,6].
The reasons for the increase in mental illness are multifaceted, but psychosocial work factors have been found to contribute [7,8]. High job demands, low job control, low social support from co-workers and the supervisor as well as role stress are examples of workplace factors that increase the risk of stress-related disorders. [9]. Cerdas et al. [10] show that there has been a negative development of job demands and decision authority in human service industries such as education, health, and social care in Sweden over time. Within these industries, teachers and elderly care workers are two large occupational groups. High workload and low decision authority have characterized the psychosocial work environment of teachers [11] and home care workers [12] in Sweden. Teachers also report lack of support from management [11] and home care workers experience a sense of being controlled rather than trusted by the management [12]. Interventions to improve the psychosocial working conditions in human service organisations appear warranted.

1.1. Primary Interventions to Reduce Work-Related Mental Illness

Organisational interventions targeting stressors in the work environment rather than the individual stress response have been proposed [13,14]. However, even though the amount of research on preventive organisational interventions has increased during the last three decades, it is still relatively scarce according to one systematic meta-review [15] and two systematic reviews [16,17] and findings show inconsistent results [17,18]. Nevertheless, existing research [15,19] on primary interventions suggests that interventions that are comprehensive, also called multi-components programs, tackling many levels at the same time are successful in bringing about positive health effects. Further, preventive interventions targeting enhancement of employee control have shown to have positive effects on mental health [15]. It is argued that the content of complex health interventions should be theory-driven [20,21] to create an understanding of the causal assumptions underpinning the intervention. This is referred to as theory of health, that is, the association between the exposure and the health outcomes [22]. The theory of health is not to be mixed up with the so-called program theory, which describes the association between the intended intervention and changes in exposure/behaviour, in other words, what activities (intervention) are assumed to put the theory of health into practice at the workplace. Further, theories of implementation also need to be considered when designing and evaluating occupational health interventions [19]. These theories could be seen as describing how the activities can be employed in the best way.
A participatory approach has been emphasized in the literature [23,24,25] and means that different stakeholders are involved in, for example, the planning and design of the intervention and/or have a say in the decision of the content of the intervention. However, it is important to clarify in what way participation is applied in intervention studies. Without such clarification, the ability to replicate and draw conclusions about what kind of participatory approach is successful will be limited. Abildgaard et al. [23] have presented a conceptual model of multiple dimensions of a participatory approach, which can be applied when designing or assessing interventions. They suggest a four-dimensional model specifying the aspects of (a) content, (b) process (design, planning, implementation process), (c) directedness, which refers to the use of full participation or representatives and (d) goal. The last dimension reflects whether the participatory approach is a goal in itself or a means to reach a goal, for example, a better work environment.
Organisational interventions are often considered challenging to evaluate. Therefore, it is suggested that elaborative evaluation frameworks are needed in order to uncover the so-called black box [26]. One proposed framework applied in this study is the framework for evaluating organizational-level interventions by Nielsen and Randall [27], which highlights three themes: the intervention and implementation design, the intervention context, and participants’ mental models. The first theme examines to what extent the intervention reached the target group and the latter two represent the factors that may moderate or mediate the link between any intervention exposure and its outcomes. Another process evaluation framework is the Medical Research Council guidance [28], which adopts the same three main dimensions as the Nielsen and Randal framework [27]; however, they also highlight the need of evaluating fidelity, whether or not the intervention was implemented as intended. Further, studies [29,30] have pinpointed the need of not only evaluating changes in working conditions and health outcomes but also the immediate effects such as behaviours and working procedures known as mediators of change. Additionally, whether these immediate achieved changes in turn lead to changes in working conditions and ultimately to changes in health. In Figure 1, the expected order of changes caused by the intervention is presented. First, it is assumed that the participants’ reactions to the intervention activities to some degree need to be positive. The participants also need to perceive that the intervention contributes to increased learning or knowledge. In this case it could be learning about how to systematically work with improvements in the psychosocial work environment. Finally, the participants need to report changes in, for example, behaviours and/or work routines due to the intervention in order to expect changes in distal outcomes such as the psychosocial work environment or health.

1.2. Aim and Research Questions

The overall aim of the current study is to evaluate a participatory intervention designed to improve the psychosocial working conditions and decrease symptoms of burnout among teachers and elderly care personnel. Three research questions are included.
  • Can improvements over time be observed in the psychosocial work environment and health outcomes for the intervention group compared to the control group?
  • Was the intervention implemented as intended (fidelity)?
  • What were the participants’ (a) reactions to the intervention activities, (b) experiences of learning, and (c) changes in behaviours and work routines, related to the intervention (mediators)?

2. Materials and Methods

2.1. Study Design

We conducted a mixed methods study using an embedded design [31], whereby interviews were carried out alongside three waves of questionnaires. We utilized a controlled trial to evaluate the effectiveness of a primary intervention targeting the psychosocial work environment in two different populations: (a) teachers and (b) elderly care personnel. The design was chosen to handle confounding variables [32]. Further, we applied a parallel process evaluation exploring fidelity and the following mediating factors: (a) reactions to the intervention activities (b) experiences of learning and changes in (c) behaviours and work routines. The Ethical Review Board in Stockholm (registration number 2018/303-31/5) has granted ethical approval.

2.2. Study Setting and Participants

The intervention took place in a middle-sized municipality in Sweden from 2016 to 2018. The administrations of early childhood and childhood education and the social services were enrolled. Additionally, the municipality management team and the political boards of each administration were involved. The control groups were matched with workplaces with similar type of work. Within the early childhood and childhood education administration, the elementary school department was chosen to generate both intervention and control groups. The two enrolled schools both represent pre-school to sixth grade. All occupations within the schools were included (intervention: n = 60; control n = 44) (Figure 2). Within social services, the elderly care department was chosen to generate both intervention and control groups. The home care unit was enrolled as the intervention group (n = 82 at baseline) and two nursing home units were invited and accepted to be controls (n = 121 at baseline). The intervention and the process evaluation (interviews) include all management levels within the two administrations. The selection for the effectiveness evaluation, however, includes only frontline workers and first-line managers since the aim of the intervention was to improve psychosocial work environment and quality of sleep, and decrease symptoms of burnout in these groups. The activity within the management groups and amongst politicians are seen as means to reach the goal of an improved work environment for frontline workers and first-line managers.
The selection of participants for the interviews was purposefully stratified [33]. The project manager invited the participants. Both those actively engaged in and those more hesitant to the intervention were included. A total of 49 interviews took place. For information about who were interviewed, when they were interviewed and which domains in the Nielsen and Randall framework [27] they were questioned about, see Supplementary Material Table S1. Both principals within the school were asked to participate, but one declined. The two principals were asked to invite two frontline workers from each team and one frontline worker from the administrative team. However, recruitment of teachers was difficult due to teachers’ lack of time, and only one participated. Within the home care department, we invited the unit manager and two (team 1 and 2) out of four team leaders. Two frontline workers from the teams 1 and 2 were also invited for the interviews. Everyone in the home care unit accepted and gave oral consent.

2.3. The Intervention

The intervention named “Me, myself, the team and the mission” was designed to promote mental health by enabling a dialogue on workplace challenges associated with stress as identified and prioritized by frontline employees. The job demands–resources theory (JD–R theory) [34] and the effort–reward imbalance model (ERI-model) [35] were used to guide frontline workers and managers in the formation of action plans, encouraging a simultaneous focus on reducing demands, increasing resources and identification of any imbalances regarding efforts and rewards. The intervention was coordinated by Human Resources representatives and one external consultant. The program theory of the evaluated intervention was built on a participatory approach and knowledge development operationalized into five main intervention components. Table 1 shows target groups and aim and content for all intervention components.
The implementation of the intervention focused, in line with research, on engagement from senior [36] and line managers [26,29]. It was initiated and supported by the politicians. The intervention activities among frontline workers were mandatory and held during working hours. When needed, substitutes were summoned.

2.4. Data Collection

To evaluate changes in outcomes over time, data were collected using questionnaires at baseline, after 18 and 24 months. Pencil-and-paper surveys were administered in September 2016 (baseline), in February 2017 (follow-up 1) and again in September 2018 (post-intervention). The surveys were distributed by the project manager in the municipality during work meetings. All employees were expected to attend the session, but participation was voluntary. Individuals who were absent on the given time for data collection were given approximately two weeks to fill out the questionnaire. Individual participants were not traceable; however, they were linked to their closest manager, meaning that all analyses are performed using aggregated data.
To evaluate whether the intervention components were delivered according to plan (fidelity) we studied number of intervention activities conducted as planned and proportions attending these activities (dose received). Fidelity was examined using documentation from the project management who kept careful notes of planned and completed activities with completed attendance lists.
Sequential semi-structured individual interviews collected data on reactions, learning and changes in behaviours and in work routines, see Figure 2. Each participant was interviewed up to three times over the duration of the study whereby the first round of the interviews (February 2017) was conducted face to face and phases two (October/November 2017) and three (May 2018) over the telephone.
The open-ended interview guide was informed by the Nielsen and Randall framework [27] for evaluating organisational-level interventions. The three main domains of the model: contextual factors, intervention and implementation design and mental models were all covered in the interview guide. Changes in routines or behaviours related to the intervention were added since the applied evaluation frameworks lacked this aspect. One chartered psychologist with experience of interviewing conducted all the interviews. Each interview lasted between 10 and 47 min and were recorded by way of an MP3 recorder. The interviews were transcribed verbatim by a transcription service.

2.5. Outcome Measures

2.5.1. Primary Outcome

The primary outcome measure is symptoms of burnout, which was determined with the Shirom Melamed Burnout Measurement (SMBM) [37], a 14-item scale from the 21-item long Shirom Melamed Burnout Questionnaire. The measure defines burnout as feelings of physical fatigue (6 items, e.g., “I am physically exhausted”), cognitive weariness (5 items, e.g., “My thinking process is slow”), and emotional exhaustion (3 items, e.g., “I feel like my emotional batteries are dead”). All items were answered on a 7-point rating scale with anchors 1: Almost never and 7: Almost always. Cronbach’s alpha for this scale = 0.95.

2.5.2. Secondary Outcomes and Covariates

Information on psychosocial work environment was collected through 9 scales from the General Nordic Questionnaire (QPSNordic) [38]. For information on how the scales were constructed, which items were included and Cronbach alpha for all scales, see Supplementary Material S2. Information on Quality of sleep was collected from one sub-scale of the Karolinska Sleep Questionnaire [39]. Information on educational level, occupation, job tenure, sex, age, part-time/full-time was also included. Additionally, in the second and third waves, we included a question on whether they had answered the previous questionnaire.

2.6. Analysis

Differences in background variables, work environment and mental health at baseline between control and intervention groups were tested using independent t-tests for continuous variables and Pearson’s chi-square test for categorical variables.
Since the individuals were not traceable, we applied weights and adjusted the analysis for background variables and baseline measures, respectively (cross-sectional analysis). Differences in work environment and mental health at 18 and 24 months between control and intervention groups were tested using independent t-tests for continuous variables. However, all outcome data is ordinal, which suggests that the median and inter quartile range should be reported and non-parametric tests be applied. However, normative data [38,40] and previous studies in the field [41,42] report means and standard deviations for the same variables as we apply. Thus, we will do the same to enable comparability between studies. We performed non-parametric tests (Mann–Whitney U-test) correspondingly with the t-tests to ensure the trustworthiness of our results. In order to test if homogeneity of variances existed between the intervention and control groups we performed Levene’s test.
Changes of outcome variables over time was calculated using t-test for partially overlapping samples in order to handle partially different study population at each time point caused by employee turnover and drop-out [43,44,45]. We also tested for mass significance [46]. See Supplementary Material S3 for a detailed description of the weighting procedure, comparison between intervention and control groups at different time points, comparison within intervention and control groups at different time points and the way of testing for mass significance. We used SPSS V26 (IBM, Armonk, NY, USA) to analyse quantitative data.
Before starting the analysis, we categorised the imported files in NVivo in order to organise informants by administration to enable identification of different perspectives from these two groups. The two administrations differ in several aspects, for example regarding educational level, which could mean that the intervention was perceived differently within the two administrations. The same was done to identify if the informant was a manager or a frontline worker. Dividing managers and frontline workers occurred because of their different roles in the intervention project. As the frontline workers were the main end-users of the intervention, the aim of the intervention was to improve their work environment. Hence, we compared and contrasted findings from two administrations and two stakeholder groups across time.
The analysis of interview data was organized in two stages. Through both stages, we were guided by Braun and Clarke’s thematic analysis [47], which is a method for identifying, analysing and understanding patterns (themes) within the data. The first stage was guided by the Nilsen and Randall framework [27], using their three broad pre-determined dimensions. The data was initially coded independently by two of the co-authors (EC and SSB) after which they met to discuss using NVivo version 11. To further understand and make sense of the first independent coding, the researchers discussed the coding and revisited the literature in an iterative process. During this process, Kirkpatrick’s learning evaluation model [48] was identified and in a second stage the data was reanalysed with this model in mind. The model includes four levels: (1) reactions to the intervention (2) learning/knowledge (3) behavioural changes and changes in work routines (4) organisational results. However, the fourth level reflects work and health effects, which in this study was assessed by the means of a questionnaire. Hence, the coding in stage two was guided by level one to three of the Kirkpatrick model. The model was chosen because it highlights the linkages between frontline workers and managers reactions to and learning from the intervention activities and the impact of those activities on the outcomes.
After categorising data into the three dimensions of the Kirkpatrick model [48], we searched for themes within these dimensions and reviewed them in relation to the entire data set. After that, the themes were named, defined and related to each other to create an overall story of the data. At last, the research group met to discuss this overall map and final adjustments were made.

3. Results

3.1. Baseline Participant Characteristics and Outcome Measures

No differences were found in any of the administrations between the intervention and control groups regarding baseline characteristics such as age, gender and job tenure (Table 2). However, the intervention and control groups within both administrations differed significantly concerning several psychosocial working conditions at baseline. Among the schools, the intervention group reported significantly higher social support from superior and control of work pacing at baseline than the control group. For the elderly care, on the other hand, the intervention group reported significantly poorer empowering leadership, social support from superior, role clarity, role conflict, control of work pacing and control of decision (Table 3).

3.2. Did the Intervention Improve the Psychosocial Working Conditions, Quality of Sleep and Decrease Symptoms of Burnout?

Regardless of which time points being compared, for both administrations no significant within-group differences over time after adjusting for background variables and the risk of mass significance were found. No data shown.
Regardless of administration, our findings show no significant improvements in the intervention groups compared with the control groups for any of the outcomes, see Table 4. Instead, the results show that all outcome variables within both intervention groups developed in a negative direction compared to the control groups. Further, within the elderly care four secondary outcomes (social support from manager, empowering leadership, control of work pacing and role clarity) differ significantly to the worse for the intervention group at the last follow-up. The primary outcome symptoms of burnout differ significantly to the worse for the intervention group at the last follow-up after adjusting for the baseline measure and the risk of mass significance but not when adjusted for background variables. Further, among the schools, no significant differences at the last follow-up were found. We also performed Mann–Whitney U-tests due to our ordinal data. The results from these tests did not differ from the results from the t-tests (Data not shown).

3.3. Fidelity

Most of the planned activities targeting frontline workers and first-line managers were delivered according to plan with a high participation rate. However, there is a discrepancy between the management levels, where the senior management and the political level have a lower fidelity compared to the middle and line managers. Due to time constraints and high turnover rates among the senior management, they cancelled most of the planned activities. For a detailed description see Table 5.

3.4. Reactions, Learning, Changes in Behaviours and Work Routines

The thematic analysis of the individual interviews led to the development of nine themes with two sub-themes, see Table 6. In general, our findings show both positive and negative reactions to the intervention but hardly any changes in learning, work routines, and or behaviours among frontline workers. However, managers reported changes such as emotional relief and skill development as a result of the coaching component.
Within the home care services managers agreed that the management development had contributed to clarification of goals and roles for the team. Further, frontline workers described increased learning about work routines and where and how decisions are made from participating in the horizontal dialogues. Another encouraging result for this group is the positive reporting of the impact of work groups. This intervention component contributed to both positive emotions and actual changes in behaviours and work routines. Within this activity, the participants perceived a meaningful participation in contrast to the other intervention components.
One theme not depicted in Table 6 represents perceptions about the intervention program as a whole. This theme is named “Not managing expectations” and represents the home care unit only. Both frontline workers and managers explain that they had wished for other types of measures than what was included in the intervention, that is, measures on a structural level such as additional personnel or shorter working hours. Still, the frontline workers report having put quite some hope into the intervention and in its capacity of bringing about better working conditions and a less stressful work environment. Nevertheless, as time passes and no or almost no changes occur, a lot of frustration, mostly among the frontline workers, is expressed.

4. Discussion

The results showed, contrary to our expectations, no improvements in psychosocial working conditions or mental health. Instead, the intervention group within elderly care reported a development in an unfavourable direction regarding the psychosocial work environment when compared to the control group. Our results showed that the intervention activities basically were implemented as intended among frontline workers, first-line and middle managers. Low fidelity among these groups can most likely not explain the lack of improvements in outcomes. However, among senior management teams and the political boards, fidelity was low. Furthermore, the intervention led to few changes in learning, behaviours and work routines, which was a prerequisite for improvements in work environment and mental health.
It is often assumed that organisational interventions, although they have no effect, at least do not cause harm [49]. However, other studies [30,50,51] which have evaluated similar organisational interventions as the one evaluated here, also report adverse developments of working conditions or health outcomes in the intervention groups compared to controls. It is important to pay attention to reasons for why some organisational preventive interventions lead to deteriorating working conditions. Below, we will discuss possible reasons for why the present intervention failed to bring about positive change and instead, to some extent, was associated with a negative development.
The studied intervention fulfilled several of the factors identified as important in order to improve psychosocial working conditions and mental health. It had a primary organisational focus, a theoretical starting point regarding theory of health, and a participatory approach. Despite this, it was unsuccessful in bringing about the intended effects. We propose three types of explanations: (1) The design and method of the evaluation did not capture the desired changes (2) Program theory failure (3) Insufficient implementation strategies. These are discussed in more detail below.

4.1. Was There a Methodological Failure?

As randomization was not possible, both confounding and selection bias may have affected the reported results. At baseline, the intervention group within elderly care reported less favourable working conditions in five of the nine items, compared to the control groups. It is possible that the intervention group were on a trajectory towards more negative working conditions when the intervention started, a trajectory that the intervention activities were not able to stop. Poorer baseline levels in the intervention group could, on the other hand, be associated with room for improvements, making a positive development more likely [50]. In the schools, the intervention group started out with more favourable scores than their controls on a few outcomes. A possible explanation of the negative development in the schools could, therefore, be attributed to a regression-toward-the-mean effect [52]. Another methodological issue concerns the outcome measures. These were defined and operationalized before the trial. It is possible that, as the intervention project developed and was implemented, the alignment between intervention activities and outcome measures to some extent was lost.

4.2. Was There a Program Theory Failure?

The core feature of the program theory was the participatory approach. The stakeholders were involved mainly in developing the content of the intervention. The interviews revealed that the participants perceived the aim of the intervention as meaningful; however, they had less confidence in the form in which it was delivered. The intervention design allowed participation in the development of the content of the intervention, but not over the process. The design, planning and implementation (process) of the intervention was primarily decided by the consultants and HR-representatives resulting in, it seems, a bad organisational fit [21]. Thus, obliging everyone to participate in a fixed process does not seem an effective way to create healthy psychosocial working conditions. It could possibly even be damaging, perhaps explained by the fact that participation increases expectations on the outcome.
Further, it seems that the choice of including all frontline workers instead of using representatives (directedness) was unfortunate. The process evaluation revealed that when representatives were used (vertical dialogues and work groups) the participants reported both positive reactions and actual changes in behaviours and work routines. The analysis also shows that the component (horizontal dialogues) where full participation was applied did not lead to any reported changes in behaviours or work routines, nor was it appreciated (reactions). To the contrary, participants felt that the intervention was very time consuming in relation to what came out of it, which led to frustration and disappointment. Gupta et al. [51] found similar results where the intervention was hypothesized to be seen as a burden instead of an enabler for decreased stress due to the fact that yet another task (the intervention) was introduced.

4.3. Was There an Implementation Failure?

The basic question here is whether the intervention activities were carried out as intended (fidelity). Fidelity was high among frontline workers and first-line managers. However, the intervention was not carried out as intended among the senior management teams and the political boards, suggesting that the anticipated support and engagement from them lacked. Another implementation strategy had the aim of enabling everyone to participate by placing the meetings on work hours with summoned substitutes. As most of the intervention components were delivered according to plan, this strategy fulfilled its function. However, it was also associated with high costs for, it seems, little effect.

4.4. Deteriorating Trend

Similar to Aust and colleagues [50], the interviews revealed few changes and learning as a result of the intervention activities and a negative trend for all outcome variables within the elderly care. Participants experienced that their expectations were not met by the intervention. They had hoped for more changes in work routines related to the psychosocial work environment and other types of changes, more related to the structural level such as more time to spend with the care recipient (client) and more personnel. It is important to highlight that the working conditions related to structural prerequisites within the home care are challenging to work in [12]. The studied municipality faced problems with high rates of sickness absence and staff turnover when the intervention was initiated [53]. Possibly, it could be damaging to introduce participatory occupational health interventions to workplaces which have undergone great reductions in employee and budgets, thereby lacking the basic prerequisites for a healthy work environment. Mellor et al. [54] refers to this as a lack of organisational capability. One should remember that the intention of the studied intervention was to identify all sorts of psychosocial risks, prioritized by the frontline workers, allowing them to participate and be more involved. However, the obstacles that were related to the structural level (human and technical resources) were never solved (during the two years of the intervention) and it is possible that the decision authority for these obstacles was not even on a municipality level but rather on a national level. Therefore, the hope for improvement was lit, however shaded, which perhaps is psychologically worse than carrying on as usual [50].

4.5. Strengths and Limitations of the Study

A strength in the study is the mixed methods design, allowing triangulation of data from the different methods. Without the process evaluation data in this study we would not have been able to understand the results of the controlled trial. Additionally, the longitudinal design with multiple measurements in the controlled trial and the qualitative design (3 phases of interviews) including a large number of informants (interviews) strengthens the study. The controlled trial evaluates the intervention program in total while the qualitative component allows for evaluation of the different intervention components. This gives an opportunity to disentangle the mechanisms of change within the intervention program. One limitation of this study is the lack of identification numbers of the participants, which did not allow to link individuals over time. This means that we cannot investigate within subject changes over time and we cannot control for the possible attrition bias [55]. However, we have tried to compensate for this by considering our overlapping samples, applying the formula by Derrick [44,45] and a weighting procedure, which made it possible to control for confounding variables.
One further limitation is the selection process of the two interventions groups, allowing possible confounding factors due to differences between intervention and control groups resulting from selection bias [32]. Participants’ perceived changes in routines and behaviours were evaluated through interviews, which implies a risk that the participants adapted their responses to ensure social desirability [56]. To alleviate this risk, the interviewer was not involved in the intervention.

4.6. Implications

The participants hoped for structural changes such as more time to spend with the care recipients and more personnel. Thus, they perceived that the intervention was targeting the wrong organisational level. This implies that researchers and practitioners, before implementing an intervention, need to thoroughly explore the context [21] and allow for tailoring of the intervention into the context. Examples of this are the use of formative evaluations with the aim of describing the perceived need for change among the end-users and the use of co-creation as a way to tailor the intervention, take contextual factors into consideration and enhance participation [57,58].
To consider the risk of unintended negative effects of an occupational health intervention, continuously monitoring of these effects are suggested. This is in line with previous research suggesting a dynamic evaluation model for occupational health interventions where results are examined and fed back to the organisation during the trial [59]. If a negative trend is observed, interrupting the intervention should be considered.

5. Conclusions

The studied participatory intervention with the aim of improving the psychosocial work environment and mental health failed to bring about the expected changes. We also found some negative trends regarding the psychosocial work environment for the home care unit. We have two suggestions to why. First, even though the program theory was based on a high level of participation the participants influence on the process was perceived as low and frustration over how the intervention was delivered was expressed. This failure contributed to that the intervention was seen as a burden and rendered a bad organisational fit. These consequences contributed to the lack of changes in psychosocial working conditions and mental health. Secondly, our results indicate a lack of fundamental organisational capability in the sense that basic prerequisites such as human and technical resources were insufficient in some respects. Hence, the intervention activities were perceived targeting the wrong organisational level.
Finally, we recommend that the design of future interventions utilize representative participation over both content and process, which may be an effective program theory to change psychosocial working conditions and mental health. This should also be a focus for future studies.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/ijerph18073546/s1, Table S1: description of participants in the interviews and which domains in the Nielsen and Randal (2013) framework they were questioned about; S2: information on items and Cronbach alpha for all outcome scales; S3: description of the weighting procedure, comparison between intervention and control groups at different time points, comparison within intervention and control groups at different time points and the way of testing for mass significance.

Author Contributions

Conceptualization, G.J. and E.C.; methodology, G.J. and E.C.; formal analysis, E.C., M.A., S.S.-B.; investigation, E.C.; data curation, E.C.; writing—original draft preparation, E.C.; writing—review and editing, E.C., A.N., S.S.-B., H.A., T.B., H.M.A. and G.J.; visualization, E.C.; supervision, G.J., A.N. and T.B.; project administration, E.C.; funding acquisition, G.J. All authors have read and agreed to the published version of the manuscript.

Funding

The municipality funded the data collection. No further external funding was obtained.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Ethical Review Board in Stockholm (protocol code 303 31 May 2018).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to legal restrictions.

Acknowledgments

We are grateful to all participants in the study for kindly sharing their views and experiences. We are also grateful to the project manager Linda Öhman and the external consultant Sara Göransson who generously shared all project documentation with us.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Försäkringskassan (Swedish Social Insurance Agency). Sjukfrånvaro i Psykiska Diagnoser: En Registerstudie av Sveriges Arbetande Befolkning i Åldern 20–69 år. Socialförsäkringsrapport 2020:8 (Sick Leave Due to Mental Disorders: A Register Based Study of the Working Population 20–69 Years in Sweden. Social Insurance Report 2020:8). Available online: https://www.forsakringskassan.se/wps/wcm/connect/e12b777c-e98a-488d-998f-501e621f4714/sjukfranvaro-i-psykiatriska-diagnoser-socialforsakringsrapport-2020-8.pdf (accessed on 28 March 2021).
  2. Akerstedt, T. Psychosocial stress and impaired sleep. Scand J. Work Environ. Health 2006, 32, 493–501. [Google Scholar] [CrossRef][Green Version]
  3. Maslach, C.; Schaufeli, W.B.; Leiter, M.P. Job burnout. Annu Rev. Psychol. 2001, 52, 397–422. [Google Scholar] [CrossRef][Green Version]
  4. Aronsson, V.; Toivanen, S.; Leineweber, C.; Nyberg, A. Can a poor psychosocial work environment and insufficient organizational resources explain the higher risk of ill-health and sickness absence in human service occupations? Evidence from a Swedish national cohort. Scand. J. Public Health 2019, 47, 310–317. [Google Scholar] [CrossRef]
  5. Gray, C.; Wilcox, G.; Nordstokke, D. Teacher Mental Health, School Climate, Inclusive Education and Student Learning: A Review. Can. Psychol. 2017, 58, 203–210. [Google Scholar] [CrossRef]
  6. Arvidsson, I.; Håkansson, C.; Karlson, B.; Björk, J.; Persson, R. Burnout among Swedish school teachers–a cross-sectional analysis. BMC Public Health 2016, 16, 823. [Google Scholar] [CrossRef][Green Version]
  7. Nieuwenhuijsen, K.; Bruinvels, D.; Frings-Dresen, M. Psychosocial work environment and stress-related disorders, a systematic review. Occup. Med. 2010, 60, 277–286. [Google Scholar] [CrossRef][Green Version]
  8. Seidler, A.; Thinschmidt, M.; Deckert, S.; Then, F.; Hegewald, J.; Nieuwenhuijsen, K.; Riedel-Heller, S.G. The role of psychosocial working conditions on burnout and its core component emotional exhaustion—A systematic review. J. Occup. Med. Toxicol. 2014, 9, 10. [Google Scholar] [CrossRef][Green Version]
  9. Harvey, S.B.; Modini, M.; Joyce, S.; Milligan-Saville, J.S.; Tan, L.; Mykletun, A.; Bryant, R.A.; Christensen, H.; Mitchell, P.B. Can work make you mentally ill? A systematic meta-review of work-related risk factors for common mental health problems. Occup. Environ. Med. 2017, 74, 301–310. [Google Scholar] [CrossRef]
  10. Cerdas, S.; Härenstam, A.; Johansson, G.; Nyberg, A. Development of job demands, decision authority and social support in industries with different gender composition–Sweden, 1991–2013. BMC Public Health 2019, 19, 758. [Google Scholar] [CrossRef] [PubMed][Green Version]
  11. Kjellström, J.; Almquist, B.; Modin, B. Lärares arbetsvillkor och hälsa efter 1990-talets skolreformer [Teachers’ working conditions and health after the school reforms of the 1990s]. Arbetsmarknad Arbetsliv 2016, 22, 52–69. [Google Scholar]
  12. Szebehely, M.; Stranz, A.; Strandell, R. Vem ska arbeta i framtidens äldreomsorg? Arbetsrapport 2017:1 (Who will work in the future elderly care? Working Paper 2017:1). Available online: https://www.socarb.su.se/polopoly_fs/1.320035.1486983623!/menu/standard/file/Slutversion%20rapport%20feb13.pdf (accessed on 28 March 2021).
  13. Irastorza, X.; Milczarek, M.; Cockburn, W. Second European Survey of Enterprises on New and Emerging Risks (ESENER-2): Overview Report: Managing Safety and Health at Work; Publications Office of the European Union: Luxembourg, 2016. [Google Scholar]
  14. Nielsen, K.; Miraglia, M. What works for whom in which circumstances? On the need to move beyond the ‘what works?’ question in organizational intervention research. Hum. Relat. 2016, 70, 40–62. [Google Scholar] [CrossRef][Green Version]
  15. Joyce, S.; Modini, M.; Christensen, H.; Mykletun, A.; Bryant, R.; Mitchell, P.B.; Harvey, S.B. Workplace interventions for common mental disorders: A systematic meta-review. Psychol. Med. 2016, 46, 683–697. [Google Scholar] [CrossRef]
  16. Richardson, K.M.; Rothstein, H.R. Effects of occupational stress management intervention programs: A meta-analysis. J. Occup. Health Psychol. 2008, 13, 69. [Google Scholar] [CrossRef]
  17. Ruotsalainen, J.H.; Verbeek, J.H.; Mariné, A.; Serra, C. Preventing occupational stress in healthcare workers. Cochrane Database Syst. Rev. 2014, 8, 12. [Google Scholar]
  18. Naghieh, A.; Montgomery, P.; Bonell, C.P.; Thompson, M.; Aber, J.L. Organisational interventions for improving wellbeing and reducing work-related stress in teachers. Cochrane Database Syst. Rev. 2015. [Google Scholar] [CrossRef][Green Version]
  19. Montano, D.; Hoven, H.; Siegrist, J. Effects of organisational-level interventions at work on employees’ health: A systematic review. BMC Public Health 2014, 14, 135. [Google Scholar] [CrossRef][Green Version]
  20. Craig, P.; Dieppe, P.; Macintyre, S.; Michie, S.; Nazareth, I.; Petticrew, M. Developing and evaluating complex interventions: The new Medical Research Council guidance. BMJ 2008, 337. [Google Scholar] [CrossRef][Green Version]
  21. Moore, G.F.; Evans, R.E. What theory, for whom and in which context? Reflections on the application of theory in the development and evaluation of complex population health interventions. SSM Popul. Health 2017, 3, 132–135. [Google Scholar] [CrossRef]
  22. Kristensen, T.S. Intervention studies in occupational epidemiology. J. Occup. Environ. Med. 2005, 62, 205–210. [Google Scholar] [CrossRef][Green Version]
  23. Abildgaard, J.S.; Hasson, H.; von Thiele Schwarz, U.; Løvseth, L.T.; Ala-Laurinaho, A.; Nielsen, K. Forms of participation: The development and application of a conceptual model of participation in work environment interventions. Econ. Ind. Democr. 2018. [Google Scholar] [CrossRef][Green Version]
  24. LaMontagne, A.D.; Keegel, T.; Louie, A.M.; Ostry, A.; Landsbergis, P.A. A systematic review of the job-stress intervention evaluation literature, 1990–2005. Int J. Occup. Med. Environ. Health 2007, 13, 268–280. [Google Scholar] [CrossRef]
  25. Nielsen, K.; Antino, M.; Rodríguez-Muñoz, A.; Sanz-Vergel, A. Is it me or us? The impact of individual and collective participation on work engagement and burnout in a cluster-randomized organisational intervention. Work Stress 2021, 1–24. [Google Scholar] [CrossRef]
  26. Nielsen, K.; Abildgaard, J.S. Organizational interventions: A research-based framework for the evaluation of both process and effects. Work Stress 2013, 27, 278–297. [Google Scholar] [CrossRef]
  27. Nielsen, K.; Randall, R. Opening the black box: Presenting a model for evaluating organizational-level interventions. Eur. J. Work Organ. Psychol. 2013, 22, 601–617. [Google Scholar] [CrossRef]
  28. Moore, G.F.; Audrey, S.; Barker, M.; Bond, L.; Bonell, C.; Hardeman, W.; Moore, L.; O’Cathain, A.; Tinati, T.; Wight, D.; et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ 2015, 350, h1258. [Google Scholar] [CrossRef][Green Version]
  29. Lundmark, R.; Hasson, H.; von Thiele Schwarz, U.; Hasson, D.; Tafvelin, S. Leading for change: Line managers’ influence on the outcomes of an occupational health intervention. Work Stress 2017, 31, 276–296. [Google Scholar] [CrossRef]
  30. Schelvis, R.M.C.; Wiezer, N.M.; van der Beek, A.J.; Twisk, J.W.R.; Bohlmeijer, E.T.; Oude Hengel, K.M. The effect of an organizational level participatory intervention in secondary vocational education on work-related health outcomes: Results of a controlled trial. BMC Public Health 2017, 17. [Google Scholar] [CrossRef][Green Version]
  31. Doyle, L.; Brady, A.-M.; Byrne, G. An overview of mixed methods research. J. Res. Nurs. 2009, 14, 175–185. [Google Scholar] [CrossRef]
  32. Schelvis, R.M.; Oude Hengel, K.M.; Burdorf, A.; Blatter, B.M.; Strijk, J.E.; van der Beek, A.J. Evaluation of occupational health interventions using a randomized controlled trial: Challenges and alternative research designs. Scand J. Work Environ. Health 2015, 41, 491–503. [Google Scholar] [CrossRef][Green Version]
  33. Onwuegbuzie, A.J.; Collins, K.M. A Typology of Mixed Methods Sampling Designs in Social Science Research. Qual. Rep. 2007, 12, 281–316. [Google Scholar]
  34. Demerouti, E.; Bakker, A.B.; Nachreiner, F.; Schaufeli, W.B. The job demands-resources model of burnout. J. Appl Psychol. 2001, 86, 499. [Google Scholar] [CrossRef] [PubMed]
  35. Siegrist, J. Adverse health effects of high-effort/low-reward conditions. J. Occup. Health Psychol. 1996, 1, 27–41. [Google Scholar] [CrossRef]
  36. Mellor, N.; Smith, P.; Mackay, C.; Palferman, D. The “Management Standards” for stress in large organizations. Int. J. Workplace Health Manag. 2013, 6, 4–17. [Google Scholar] [CrossRef]
  37. Shirom, A.; Melamed, S. A Comparison of the Construct Validity of Two Burnout Measures in Two Groups of Professionals. Int. J. Stress Manag. 2006, 13, 176–200. [Google Scholar] [CrossRef]
  38. Dallner, M.; Elo, A.-L.; Gamberale, F.; Gamberale, F.; Hottinen, V.; Knardahl, S.; Lindström, K.; Skogstad, A.; Lindström-Espeling, K. Validation of the General Nordic Questionnaire (QPS Nordic) for Psychological and Social Factors at Work; Nordic Council of Ministers [Nordiska ministerrådet]: Copenhagen, Denmark, 2000.
  39. Nordin, M.; Åkerstedt, T.; Nordin, S. Psychometric evaluation and normative data for the Karolinska Sleep Questionnaire. Sleep Biol. Rhythm. 2013, 11, 216–226. [Google Scholar] [CrossRef]
  40. Lundgren-Nilsson, Å.; Jonsdottir, I.H.; Pallant, J.; Ahlborg, G., Jr. Internal construct validity of the Shirom-Melamed Burnout Questionnaire (SMBQ). BMC Public Health 2012, 12, 1. [Google Scholar] [CrossRef] [PubMed][Green Version]
  41. Arapovic-Johansson, B.; Wåhlin, C.; Hagberg, J.; Kwak, L.; Björklund, C.; Jensen, I. Participatory work place intervention for stress prevention in primary health care. A randomized controlled trial. Eur. J. Work. Organ. Psychol. 2018, 27, 219–234. [Google Scholar] [CrossRef]
  42. Björklund, C.; Grahn, A.; Jensen, I.; Bergström, G. Does survey feedback enhance the psychosocial work environment and decrease sick leave? Eur. J. Work Organ. Psychol. 2007, 16, 76–93. [Google Scholar] [CrossRef]
  43. Costantini, G.; Saraulli, D.; Perugini, M. Uncovering the Motivational Core of Traits: The Case of Conscientiousness. Eur. J. Pers. 2020, 34, 1073–1094. [Google Scholar] [CrossRef]
  44. Derrick, B.; Toher, D.; White, P. How to compare the means of two samples that include paired observations and independent observations: A companion to Derrick, Russ, Toher and White (2017). Quant. Methods Psychol. 2017, 13. [Google Scholar] [CrossRef][Green Version]
  45. Derrick, B.; Russ, B.; Toher, D.; White, P. Test statistics for the comparison of means for two samples that include both paired and independent observations. J. Mod. Appl. Stat. Methods 2017, 16, 9. [Google Scholar] [CrossRef][Green Version]
  46. Shaffer, J.P. Multiple hypothesis testing. Annu. Rev. Psychol. 1995, 46, 561–584. [Google Scholar] [CrossRef]
  47. Braun, V.; Clarke, V. Using thematic analysis in psychology. Qual. Res. Psychol. 2006, 3, 77–101. [Google Scholar] [CrossRef][Green Version]
  48. Kirkpatrick, D. Great ideas revisited. Train. Dev. 1996, 50, 54–60. [Google Scholar]
  49. Semmer, N.K. Job stress interventions and organization of work. Scand. J. Work Environ. Health 2006, 32, 515–527. [Google Scholar] [CrossRef] [PubMed][Green Version]
  50. Aust, B.; Rugulies, R.; Finken, A.; Jensen, C. When workplace interventions lead to negative effects: Learning from failures. Scand. J. Public Health 2010, 38, 106–119. [Google Scholar] [CrossRef] [PubMed]
  51. Gupta, N.; Wahlin-Jacobsen, C.D.; Abildgaard, J.S.; Henriksen, L.N.; Nielsen, K.; Holtermann, A. Effectiveness of a participatory physical and psychosocial intervention to balance the demands and resources of industrial workers: A cluster-randomized controlled trial. Scand. J. Work Environ. Health 2017, 44, 58–68. [Google Scholar] [CrossRef][Green Version]
  52. Barnett, A.G.; Van Der Pols, J.C.; Dobson, A. Regression to the mean: What it is and how to deal with it. Int. J. Epidemiol 2005, 34, 215–220. [Google Scholar] [CrossRef][Green Version]
  53. Centrum för arbets- och miljömedicin [Center of Occupational and Environmental Medicine]. Arbetshälsorapport Stockholm Region [Occupational Health Report Stockholm Region]; EO Grafiska: Stockholm, Sweden, 2016. [Google Scholar]
  54. Mellor, N.; Mackay, C.; Packham, C.; Jones, R.; Palferman, D.; Webster, S.; Kelly, P. ‘Management standards’ and work-related stress in Great Britain: Progress on their implementation. Saf. Sci. 2011, 49, 1040–1046. [Google Scholar] [CrossRef]
  55. Hewitt, C.E.; Kumaravel, B.; Dumville, J.C.; Torgerson, D.J. Assessing the impact of attrition in randomized controlled trials. J. Clin. Epidemiol. 2010, 63, 1264–1270. [Google Scholar] [CrossRef]
  56. Crowne, D.P.; Marlowe, D. The Approval Motive: Studies in Evaluative Dependence; Wiley: New York, NY, USA, 1964. [Google Scholar]
  57. Cedstrand, E.; Nyberg, A.; Bodin, T.; Augustsson, H.; Johansson, G. Study protocol of a co-created primary organizational-level intervention with the aim to improve organizational and social working conditions and decrease stress within the construction industry–a controlled trial. BMC Public Health 2020, 20, 1–8. [Google Scholar] [CrossRef]
  58. Peters, S.E.; Nielsen, K.M.; Nagler, E.M.; Revette, A.C.; Madden, J.; Sorensen, G. Ensuring organization-intervention fit for a participatory organizational intervention to improve food service workers’ health and wellbeing: Workplace organizational health study. J. Occup. Environ. Med. 2020, 62, e33–e45. [Google Scholar] [CrossRef]
  59. Von Thiele Schwarz, U.; Lundmark, R.; Hasson, H. The dynamic integrated evaluation model (DIEM): Achieving sustainability in organizational intervention through a participatory evaluation approach. Stress Health 2016, 32, 285–293. [Google Scholar] [CrossRef] [PubMed][Green Version]
Figure 1. Program logic model of expected order of the changes.
Figure 1. Program logic model of expected order of the changes.
Ijerph 18 03546 g001
Figure 2. Flow diagram of the workers involved in the effectiveness and process evaluation.
Figure 2. Flow diagram of the workers involved in the effectiveness and process evaluation.
Ijerph 18 03546 g002
Table 1. Target groups and, aim and content for all intervention components.
Table 1. Target groups and, aim and content for all intervention components.
Intervention ComponentTarget GroupsAim/Content
Kick-offAll employees and politiciansInform about the project, anchoring.
Joint educationManagers, management teams and politiciansEducation on organizational and social work environment.
Feedback and discussionMunicipality management teamAnchoring of the project to enhance support from management.
Feedback and discussionPolitical board and the committees representing the two administrations met within each group together with the HR-representative and the external consultant.Anchoring of the project to enhance support from the politicians.
Discussions on the progress of the project and occupational health related issues.
Leadership coachingAll team managers (n = 5), unit managers (n = 3),
department managers (n = 2) and the two administrative managers.
The overall theme was “leading for health”. However, everyone enrolled in the coaching program formulated his or her own goal for the coaching.
Horizontal dialogue meetingsAll frontline employees and first-line managersMake team-specific risk assessments and action plans.
Management developmentManagement teams at 3 levels within the two administrations and the top management team within the municipality.Clarify goals and create a clear mission for each team.
All teams were offered support by the external consultant regarding the topic “leading for health”. A prioritized general focus for these meetings was the clarification of goals and roles, in order to create a clear mission for each team.
Vertical dialoguesChairman of the political committee, managers from all management levels and employee representatives.Resolve prioritized obstacles within the work environment. Questions discussed for example: parking permissions, subsidized clothes.
Work groups (This component was developed during the project. It was not in the original plan)Representatives from the home care unit.The work groups involved a couple of frontline workers from the different teams and were led by one team leader together with the external consultant or an HR representative.
5 work groups were organised on the topics: (1) staffing, planning, travel time for the unit (2) our meetings (3) employeeship (4) staffing, planning, travel time for a certain team (5) This is us.
Individual stress management (This component was developed during the project. It was not in the original plan)All employees at the school.Individual stress management
The bold names represent the five main intervention components.
Table 2. Characteristics of study participants.
Table 2. Characteristics of study participants.
School
Baseline18 Months24 Months
Background variablesIntervention
N = 52
Control
N = 32
Intervention
N = 44
Control
N = 32
Intervention
N = 48
Control
N = 34
AgeN (%)N (%)N (%)N (%)N (%)N (%)
<3510 (21)7 (22)11 (25)9 (28)9 (19)10 (29)
36–4513 (28)10 (31)13 (29.5)12 (38)17 (35)12 (35)
>4624 (51)15 (47)20 (45.5)11 (34)22 (46)12 (36)
Education
Elementary school (9 years)8 (16)4 (13.3)1 (2)5 (17)3 (6)4 (12)
Upper elementary school (>9 years)5 (10)2 (6.7)8 (19)3 (10)8 (17)1 (3)
University/college38 (74)24 (80)34 (79)22 (73)37 (77)29 (85)
Females36 (70.6)28 (87.5)24 (54.5)26 (87)32 (67)28 (82)
Occupation
Teacher26 (59)18 (56.3)24 (60)21 (68)28 (65)17 (68)
Early childhood educator4 (9)4 (12.5)5 (12.5)1 (3)3 (7)1 (4)
Recreation leader5 (11)5 (15.6)2 (5)3 (10)1 (2)3 (12)
Others9 (20)5 (15.6)9 (28.5)6 (19)11 (26)4 (16)
Work time
Full time37 (74)29 (90.6)39 (91)28 (88)42 (89)29 (85)
Part time (chosen)12 (24)3 (9.4)3 (7)3 (9)4 (9)5 (15)
Part time (not chosen)1 (2)01 (2)1 (3)1 (2)0
Job tenure
<116 (31.4)6 (18.7)7 (16)6 (19)10 (20.8)4 (12)
1–211 (21.6)4 (12.5)16 (36)7 (22)7 (14.6)8 (23)
3–56 (11.7)5 (15.6)3 (7)3 (9)10 (20.8)7 (21)
>518 (35.3)17 (53.2)18 (41)16 (50)21 (43.8)15 (44)
Answered before (at 24 months) N (%)N (%)
Yes, both times24 (50)16 (47.1)
Answered one of them8 (17)11 (32.4)
No, none of them11 (23)6 (17.6)
Do not remember4 (8)1 (2.9)
Missing1 (2)0
Elderly Care
Baseline18 Months24 Months
Background variablesIntervention
N = 74
Control
N = 81
Intervention
N = 50
Control
N = 92
Intervention
N = 57
Control
N = 74
AgeN (%)N (%)N (%)N (%)N (%)N (%)
<3516 (25)12 (21)11 (22)11 (12)15 (26)7 (10)
36–4513 (20)5 (8)11 (22)19 (21)13 (23)15 (20)
>4636 (55)41 (71)27 (56)70 (67)29 (51)51 (70)
Education
Elementary school9 (12.7)10 (14)6 (12)18 (20)6 (10.5)14 (19)
Upper elementary school51 (71.8)53 (75)33 (69)61 (68)39 (68.5)49 (66)
University/college11 (15.5)8 (11)9 (19)11 (12)12 (21)11 (15)
Females66 (90)73 (94.8)42 (89)80(89)49 (86)62 (84)
Occupation
Care assistant23 (34.8)16 (22)11 (33)12 (16.5)19 (41)11 (19)
Assistant nurse43 (65.2)56 (78)21 (64)60 (82)27 (59)48 (81)
Nurse001 (3)000
Cleaner0001 (1.5)00
Work time
Full time42 (58)39 (53.5)22 (46)49 (56.5)28 (51)37 (53)
Part time (chosen)28 (39)31 (42.5)26 (54)35 (40)27 (49)32 (45.5)
Part time (not chosen)2 (3)3 (4)03 (3.5)01 (1.5)
Job tenure
<110 (13.7)10 (13)4 (8)7 (8)4 (7)5 (7)
1–29 (12.3)13 (17)3 (6)14 (15)3 (5)12 (16)
3–515 (20.5)8 (10)8 (16.5)12 (13)16 (28)11 (15)
>539 (53.5)46 (60)34 (69.5)58 (64)34 (60)46 (62)
Answered before (at 24 months) N (%)N (%)
Yes, both34 (60)31 (42)
Answered one of them4 (7)16 (22)
No, none of them4 (7)7 (10)
Do not remember10 (17.5)16 (22)
Missing5 (8.5)4 (4)
Table 3. Mean and Standard deviation for all outcome variables at all three time points.
Table 3. Mean and Standard deviation for all outcome variables at all three time points.
School
Baseline18 Months24 Months
Intervention
N = 52
Control
N = 32
Intervention
N = 44
Control
N = 32
Intervention
N = 48
Control
N = 34
M (Sd)M (Sd)M (Sd)M (Sd)M (Sd)M (Sd)
Primary outcome
Burnout (1–7)2.5 (1.2)2.1 (1.1)2.5 (1.2)2.3 (1.1)2.7 (1.2)2.1 (1.0)
Secondary outcomes
Role conflict (1–5)2.5 (0.8)2.5 (0.7)2.7 (0.8)2.3 (0.8)3.0 (0.8)2.6 (0.6)
Role clarity (1–5)4.4 (0.6)4.5 (0.5)4.2 (0.9)4.7 (0.5)4.1 (0.8)4.6 (0.5)
Social support manager (1–5)4.4 * (0.6)4.0 (0.7)3.7 (1.0)4.2 (0.9)3.4 (1.0)4.1 (0.6)
Empowering leadership (1–5)3.6 (1.1)3.3 (0.7)3.0 (1.0)3.5 (0.9)2.9 (1.1)3.4 (0.7)
Social support colleagues (1–5)4.5 (0.6)4.3 (0.7)4.2 (0.6)4.3 (0.7)4.0 (0.7)4.5 (0.6)
Control of decision (1–5)2.9 (0.6)2.9 (0.6)2.9 (0.5)2.8 (0.6)2.7 (0.6)2.8 (0.6)
Control of work pacing (1–5)2.3 * (0.8)1.8 (0.5)2.4 (0.9)1.8 (0.6)2.2 (0.9)2.0 (0.7)
Quantitative job demands (1–5)2.9 (1.0)3.0 (0.9)3.0 (0.8)3.0 (0.8)3.3 (1.0)3.1 (0.7)
Decision demands (1–5)3.9 (0.7)4.0 (0.6)3.9 (0.6)4.0 (0.6)4.0 (0.6)4.0 (0.5)
Quality of sleep (1–6) 12.8 (0.8)2.6 (1.1)2.8 (1.0)2.8 (0.8)2.9 (0.9)2.7 (0.8)
Elderly Care
Baseline18 Months24 Months
Intervention
N = 74
Control
N = 81
Intervention
N = 50
Control
N = 92
Intervention
N = 57
Control
N = 74
Primary outcome
Burnout (1–7)2.4 (1.1)2.2 (1.2)2.4 (1.0)1.9 (1.1)2.9 (1.2)1.9 (0.9)
Secondary outcomes
Role conflict (1–5)2.9 * (0.7)2.6 (0.9)3.0 (0.6)2.4 (0.9)3.0 (0.8)2.3 (0.9)
Role clarity (1–5)3.9 * (0.8)4.5 (0.5)4.0 (0.8)4.6 (0.5)4.1 (0.6)4.6 (0.6)
Social support manager (1–5)3.8 * (0.8)4.2 (0.9)3.5 (0.9)4.3 (0.7)3.4 (1.0)4.3 (0.8)
Empowering leadership (1–5)3.0 * (1.1)3.6 (1.0)2.5 (0.9)4.0 (1.0)2.6 (1.1)3.9 (1.0)
Social support colleagues (1–5)4.2 (0.8)4.3 (0.7)4.2 (0.8)4.3 (0.7)4.2 (0.7)4.2 (0.9)
Control of decision (1–5)2.3 * (0.5)2.7 (0.8)2.3 (0.6)2.9 (0.8)2.3 (0.7)2.9 (0.8)
Control of work pacing (1–5)2.1 * (0.6)2.5 (0.8)2.0 (0.8)3.0 (0.8)2.0 (0.8)2.9 (0.8)
Quantitative job demands (1–5)2.9 (0.7)2.7 (0.8)3.0 (0.7)2.5 (0.9)3.0 (0.7)2.5 (0.8)
Decision demands (1–5)3.6 (0.7)3.7 (0.8)3.5 (0.7)3.3 (0.8)3.6 (0.6)3.3 (0.7)
Quality of sleep (1–6) 12.7 (0.7)2.7 (1.1)2.7 (0.8)2.6 (1.0)3.1 (0.8)2.5 (1.0)
1 Higher score indicate lower quality of sleep. * p < 0.05.
Table 4. Between group differences at second and third follow-up.
Table 4. Between group differences at second and third follow-up.
School
Experiment vs. Control 18 MonthsExperiment vs. Control 24 Months
Weighted for
Background Variables
Weighted for
Baseline Measure
Weighted for
Background Variables
Weighted for
Baseline Measure
Primary outcomeMean diff.p-value 1Mean diff.p-value 1Mean diff.p-value 1Mean diff.p-value 1
Burnout0.391.00.1210.950.130.311.0
Secondary outcomes
Role conflict−0.530.13 *−0.131.0−0.930.001−0.241.0
Role clarity−0.171.0−0.241.0−0.620.001−0.270.66
Social support manager−0.910.001 *−0.251.0−1.180.001−0.281.0
Empowering leadership−0.850.001−0.351.0−0.920.001−0.061.0
Social support colleagues−0.241.0−0.121.0−0.590.001−0.251.0
Control of decision−0.310.660.061.0−0.331.0 *−0.011.0
Control of work pacing0.610.260.221.0−0.181.0 *0.021.0
Quantitative job demands0.291.00.061.00.620.530.121.0
Decision demands0.600.001 *0.041.00.141.0 *0.021.0
Quality of sleep0.161.0 *−0.041.00.511.0−0.061.0
Elderly Care
Experiment vs. Control 18 MonthsExperiment vs. Control 24 Months
Weighted for
Background Variables
Weighted for
Baseline Measure
Weighted for
Background Variables
Weighted for
Baseline Measure
Primary outcomeMean diff.p-value 1Mean diff.p-value 1Mean diff.p-value 1Mean diff.p-value 1
Burnout0.401.00.371.00.750.130.620.001 *
Secondary outcomes
Role conflict−0.670.001−0.241.0 *−0.670.001−0.350.13
Role clarity−0.410.13 *−0.380.001 *−0.450.001−0.370.001
Social support manager−1.030.001−0.660.001−0.780.001−0.650.001
Empowering leadership−1.290.001 *−1.030.001−1.080.001−0.800.001
Social support colleagues0.261.0−0.021.00.291.0 *−0.151.0 *
Control of decision−0.300.40−0.260.66−0.350.13−0.300.13
Control of work pacing−0.670.001−0.510.001−0.790.001−0.500.001
Quantitative job demands0.540.0010.420.0010.321.0 *0.360.001
Decision demands0.650.0010.121.00.121.00.221.0
Quality of sleep−0.021.00.041.00.291.00.311.0
1 All p-values are Bonferroni-adjusted. * Indicates different variances between the intervention and control groups as a result of the Levene’s test p < 0.05. Outcomes significant after adjusting for background variables and the baseline measure at 24 months are in bold.
Table 5. Dose delivered (numbers planned/delivered) and dose received (N/%) for all intervention components.
Table 5. Dose delivered (numbers planned/delivered) and dose received (N/%) for all intervention components.
Type of Intervention ComponentNo. Planned/DeliveredDose Received
N/%
Kick-off (all employees and politicians)2 times × 2 h/2 times × 2 h149/not available
Joint education (managers, management teams and politicians)6 times, 2 parts. Each part was supposed to be delivered 3 times each/Part one was delivered twice and part two was delivered once.Part one = 11/not available
Feedback and discussion (municipality management team)7 times × 1 h/2 times × 75 minNot available
Feedback and discussion (political board)7 times × 1 h/2 times × 2 hNot available
Leadership coachingSchool
6 h per person/4.3 h per person
Social services
6 h per person/5.4 h per person
Not available
Horizontal dialogue meetings (workshops in the team or at the unit)School
7 workshops per team/7 workshops per team
Social services
7 workshops per team/6–8 among the four teams
Mandatory
Management development (management teams at 3 levels)School
Senior 12 h/2 times × 2 h
Middle 12 h/3 times × 2 h
First-line 12 h/15 times × 2 h
Social services
Senior 12 h/4 times × 2 h
Middle 12 h/5 times × 3 h
First-line 12 h/26 times × 2 h
Not available
Vertical dialogues
(with politicians, managers and employee representatives)
School
Not planned/7 times × 1 h
Social services
Not planned/5 times × 1 h
Work groupsNot planned
5 groups, 28 times ×2 h (4–8 meetings per group).
Individual stress managementNot planned/8 times × 90 min
The bold names represent the five main intervention components.
Table 6. Main findings from the thematic analysis.
Table 6. Main findings from the thematic analysis.
Type of Intervention ComponentReactionsLearningChanges in Behaviours and Routines
Leadership coachingThemes: Rewarding (emotional relief)
Appreciated, meaningful. They report getting support in resolving complicated situations with employees, co-workers or their closest manager.
Themes: Rewarding (skill development)
Contributed to both personal and professional development. Improved skills in conflict management, how to organize meetings and how to coach employees are reported.
Horizontal dialogue meetings (workshops in the team or at the unit)Themes: Lack of change and disagreement on the format.
Disappointment due to the lack of changes coming out from the project.
Positive about the possibility to listen to each other and discuss work-related challenges (all). Frustration about how the meetings are organised.
Themes: Better understanding of work routines and colleagues.
How decisions formally are taken and who is responsible for what (home care).
Management development (management teams at 3 levels) Themes: Mutual mental models (home care)
Helped them clarify the goals and roles of the team, giving its members a mutual understanding of each other and of what they wish to achieve
Themes: Mutual mental models (home care)
The four work groups operate more equal, applying the same work routines more often than before
Vertical dialogues
(with politicians, managers and employee representatives)
Themes: A sense of being listened to
Appreciated the opportunity to communicate the needs of the organisation directly to the politicians.
Themes: Potentially moving towards change
One example of an actual change in work routines due to the vertical dialogues is reported. It is within the home care where the suggestion of purchasing winter jackets for the workers has been approved.
Work groupsThemes: Meaningful participation
Experiences of being listened to and taken seriously (involved).
Themes: Meaningful participation
Introduction of dedicated time to write and read reports from a day’s work.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Cedstrand, E.; Nyberg, A.; Sanchez-Bengtsson, S.; Alderling, M.; Augustsson, H.; Bodin, T.; Mölsted Alvesson, H.; Johansson, G. A Participatory Intervention to Improve the Psychosocial Work Environment and Mental Health in Human Service Organisations. A Mixed Methods Evaluation Study. Int. J. Environ. Res. Public Health 2021, 18, 3546. https://doi.org/10.3390/ijerph18073546

AMA Style

Cedstrand E, Nyberg A, Sanchez-Bengtsson S, Alderling M, Augustsson H, Bodin T, Mölsted Alvesson H, Johansson G. A Participatory Intervention to Improve the Psychosocial Work Environment and Mental Health in Human Service Organisations. A Mixed Methods Evaluation Study. International Journal of Environmental Research and Public Health. 2021; 18(7):3546. https://doi.org/10.3390/ijerph18073546

Chicago/Turabian Style

Cedstrand, Emma, Anna Nyberg, Sara Sanchez-Bengtsson, Magnus Alderling, Hanna Augustsson, Theo Bodin, Helle Mölsted Alvesson, and Gun Johansson. 2021. "A Participatory Intervention to Improve the Psychosocial Work Environment and Mental Health in Human Service Organisations. A Mixed Methods Evaluation Study" International Journal of Environmental Research and Public Health 18, no. 7: 3546. https://doi.org/10.3390/ijerph18073546

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop