Next Article in Journal
Resilience and Regulation of Emotions in Adolescents: Serial Mediation Analysis through Self-Esteem and the Perceived Social Support
Previous Article in Journal
High-Resolution Urban Air Quality Mapping for Multiple Pollutants Based on Dense Monitoring Data and Machine Learning
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Managing Minds at Work: Development of a Digital Line Manager Training Program

School of Health Sciences, University of Nottingham, Nottingham NG7 2HA, UK
NIHR Nottingham Biomedical Research Centre, Nottingham NG7 2UH, UK
Institute of Mental Health, Nottinghamshire Healthcare NHS Trust, Nottingham NG3 6AA, UK
Birkbeck, University of London, London WC1H 0PD, UK
School of Sport, Exercise & Health Sciences, Loughborough University, Loughborough LE11 3TU, UK
Institute for Mental Health, University of Birmingham, Birmingham B15 2TT, UK
Office of the Provost, The British University in Egypt, El Sherouk City 11837, Cairo, Egypt
Faculty of Health and Life Sciences, Coventry University, Coventry CV1 5FB, UK
Thrive at Work, West Midlands Combined Authority, Birmingham B19 3SD, UK
Executive Office, Warwick University, Coventry CV4 7AL, UK
School of Medicine, University of Nottingham Jubilee Campus, Nottingham NG8 1BB, UK
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2022, 19(13), 8006;
Submission received: 21 April 2022 / Revised: 13 June 2022 / Accepted: 22 June 2022 / Published: 29 June 2022
(This article belongs to the Section Mental Health)


Mental ill-health is the leading cause of sickness absence, creating a high economic burden. Workplace interventions aimed at supporting employers in the prevention of mental ill-health in the workforce are urgently required. Managing Minds at Work is a digital intervention aimed at supporting line managers in promoting better mental health at work through a preventative approach. This intervention was developed as part of the Mental Health and Productivity Pilot, a wider initiative aimed at supporting employers across the Midlands region of the United Kingdom to improve the future of workplace mental health and wellbeing. The aim of the study is to describe the design and development of the Managing Minds at Work digital training program, prior to feasibility testing. We adopted a collaborative participatory design involving co-design (users as partners) and principles of user-centred design (pilot and usability testing). An agile methodology was used to co-create intervention content with a stakeholder virtual community of practice. Development processes were mapped to core elements of the Medical Research Council (MRC) framework for developing and evaluating complex interventions. The program covers five broad areas: (i) promoting self-care techniques among line managers; (ii) designing work to prevent work-related stress; (iii) management competencies to prevent and reduce stress; (iv) having conversations with employees about mental health; (v) building a psychologically safe work environment. It was considered by stakeholders to be appropriate for any type of organization, irrespective of their size or resources. Pilot and usability testing (n = 37 surveys) aligned with the Technology Acceptance Model (TAM) demonstrated that the program was perceived to be useful, relevant, and easy to use by managers across sectors, organization types, and sizes. We identified positive impacts on manager attitudes and behavioral intentions related to preventing mental ill-health and promoting good mental wellbeing at work. The next step is to explore the feasibility and acceptability of Managing Minds at Work with line managers in diverse employment settings.

1. Introduction

Mental health problems affect one in six workers each year and are the leading cause of sickness absence, with stress, anxiety, and depression being responsible for approximately half of the working days lost [1,2]. Mental ill-health can lead to unemployment [3], which has broader implications for individuals, employers, and society. The economic burden of mental ill-health is high and likely to increase due to the rising prevalence of mental ill-health due to the COVID-19 pandemic [4], coupled with the impacts of the pandemic on the labor market [5]. A systematic review [6] focusing only on work-related stress (WRS) and including studies from Australia, Canada, Denmark, France, Sweden, Switzerland, the United Kingdom, and the EU-15 suggested that there is a sizable financial burden imposed by WRS on society, ranging from US$221.13 million to $187 billion. Most of the cost of WRS comes from productivity losses—between 70–90%—with the remaining 10–30% coming from health and medical costs [6].
Preventing and managing mental ill-health at work is therefore a priority, not only from a public health perspective, but for organizations and their ‘bottom line’ (net profits). Yet, many employers are unaware of their important role in supporting workers’ mental health [2]. Interventions to mitigate the impact of mental ill-health in the workplace have been typically categorized in ways that distinguish between the stages of prevention and their associated targets of change: primary-level interventions (taking action to eliminate the sources of stress or poor mental health in the workplace), secondary-level interventions (detecting and managing experienced stress or mental health concerns by increasing employees’ awareness, knowledge, skills, and coping resources), or tertiary-level interventions (minimizing the effects of poor mental health at work once they have occurred through the treatment of symptoms and provision of remedial support). A holistic approach incorporating all three types of interventions is needed and advocated as best practice in European guidelines, including the European Foundation for the Improvement of Living and Working Conditions and the European Agency for Safety and Health at Work [7]. However, primary prevention is advocated as being particularly important to ‘flatten the curve’ and avoid a surge in the incidence of mental disorders stemming from the COVID-19 pandemic [8].
Line managers play an important role in the primary prevention of mental ill-health [9]. They can act proactively through the design and management of work tasks, communicating with and managing their employees with respect and clarity, developing a supportive and psychologically safe work environment for their teams, and encouraging people to have conversations about their mental wellbeing at work [10,11,12,13,14,15]. To effectively fulfil this role, managers need to be equipped with the right skills and knowledge, but there is a dearth of rigorously developed guidance to assist with this [16]. Training for line managers has been identified as an important part of the process of supporting employee mental health [12,17]. A systematic review and meta-analysis indicates that training managers in workplace mental health can improve their knowledge, attitudes, and self-reported behavior in supporting employees experiencing mental health problems [18]. Yet, a recent survey showed that line managers are not being given enough support and training to protect the mental health and wellbeing of staff at work, and more than half of organizations (57%) indicated that their organization offers no mental health and wellbeing training and/or support for managerial staff [19]. Where training is offered and shows promise, the development of content and the evaluation of training is often poorly described and/or lacks rigor, hindering its replicability (e.g., [20])
Several studies have examined the effectiveness of line manager training to improve the mental health of employees using trial methodology [17,21,22,23] These have broadly shown that training can lead to improvements in line managers’ knowledge of mental health [22], communication about mental health and related resources [17], confidence in and self-efficacy for creating a mentally healthy workplace [21,22], and lowered work-related sickness absences of employees [23]. However, the precise content of the training interventions in these trials has varied. Much of this training has focused on increasing line managers’ awareness of mental health and improving their skills in how to support employees who are already experiencing poor mental health i.e., a secondary level intervention. There are few studies taking a primary prevention approach. Stansfeld and colleagues [24] focused on the line managers’ competencies that are required to prevent work-related stress [25]; however, their intervention did not improve employee outcomes, and line manager outcomes were not explored. The authors identified a need for greater focus on reflective and experiential learning to encourage behavior changes [24]. We have therefore endeavored to address this in our study through the rigorous development of a primary prevention intervention that includes reflection and experiential learning, delivered online.
Online approaches to workplace training are advocated to allow for greater flexibility in learning and increase workplace training capability [26]. E-learning platforms are a webspace or portal for educational content and resources that bring together all the resources a user needs into a single place. The use of e-learning has increased in recent years, and there is a plethora of platforms available (e.g., Coursera, Skillshare, Udemy, Codecademy, Edx, Pluralsight, Future Learn, Xerte, Moodle, and many more). The use of e-learning is escalating, following the rapid shift to online learning during the coronavirus (COVID-19) pandemic, potentially providing employers with a contemporary solution to increasing flexibility in workplace training and learning [27]. The increasingly competitive environment of modern times has led organizations to focus on how they can establish a sustainable competitive advantage [28]. Training delivered using e-learning offers a distinct, competitive advantage to employers. Since facilitators are not required, it is cheap to deliver and is seen to be cost effective [29]. Since travel is not required, this reduces personal risks for employees (e.g., lone travel, fear of failure in front of others) and the costs associated with course attendance (e.g., financial, and person-time), and reduces the environmental impact (e.g., from travel, printing, etc.). Online training is highly versatile and can be accessed at any time or place, therefore fitting with employees’ flexible work patterns and lifestyles. This approach allows ongoing access to resources, and therefore ongoing opportunities for learning and development. This flexibility of use allows users to have greater control over their learning, which can increase satisfaction and motivation for learners [28,30]. Training delivered through e-learning ensures a certain level of consistency and standardization in the materials being delivered. This cannot be guaranteed with face-to-face modes of delivery, which can be heterogenous in terms of content and delivery, as demonstrated elsewhere [20]. Digital training is also highly scalable, giving broad accessibility to employees in diverse geographical regions. For users, e-learning provides opportunities to link learning activities directly to the day-to-day experiences people have at work, and this opportunity to provide relevant learning activities (e.g., inclusion of reflection, experiential learning, or competency-based learning [30]) may motivate better engagement and retention of learners than other modes of training delivery [29].
Our overall aim, therefore, is to develop and test a digital intervention aimed at supporting employers in the prevention of mental ill-health at work. This primary prevention research was conducted as part of the Mental Health and Productivity Pilot (MHPP) [31], a wider initiative aimed at supporting employers across the Midlands region of the United Kingdom (UK) to improve the future of workplace mental health and wellbeing. The MHPP aims to (a) reduce the impact of poor mental health in the workplace and to remove barriers to employability and productivity, (b) help to reduce stigma around the workplace, (c) deliver evidence-based interventions. The Managing Minds at Work project is one of several interventions developed within the MHPP program, in which a digital interactive training program to support line managers will be developed and tested. The research will be conducted over several phases, including intervention development, feasibility testing, evaluation, and exploration of implementation processes. In this paper, we report the first stage of the research, which is the intervention development. The intervention is a digital training program called Managing Minds at Work (MMW). The aim of the intervention is to support line managers by increasing their knowledge, skills, and confidence for promoting and protecting the mental wellbeing of the people they manage.

2. Methods

2.1. Design

This study reports on a rigorous intervention development process adopting a collaborative participatory design [32], involving co-design (with users as partners for content development) and incorporating principles of user-centred design (conducting usability testing while considering human factors and ergonomics). Therefore, we adopted a person-centred approach as described in guidelines for the development of digital health interventions [33,34,35]. The study involved an ‘agile design’ process, which is a dynamic and flexible approach to co-design, adopting ‘kanban’ methods (where team members are allocated specific tasks without a pre-determined timeframe) as used in previous studies [36,37,38]. The approach draws on agile software development and generates directionally useful feedback that enables rapid learning and iterative improvements to both the content and technical design during the development of training materials. The intervention was then tested in a usability pilot study that was aligned to the Technology Acceptance Model (TAM [39]). This intervention development phase is an early stage of a registered trial ( Identifier: NCT05154019). The intervention development process follows the recommended principles and actions to allow links to be made in the future between intervention development processes and the subsequent success of the intervention [40]. Ethical approval for the research study was obtained from the institutional ethics review board (FMHS Ref: 299-0621).

2.2. Procedure

Intervention development took place between February and July 2021. The study adhered to the development phase of the Medical Research Council (MRC) framework for developing and evaluating complex interventions [41] and was mapped to the six core elements including: (i) context, (ii) developing and refining program theory, (iii) engaging stakeholders, (iv) identifying key uncertainties, (v) refining the intervention, and (vi) economic considerations (Table 1). Study reporting draws on relevant items from the ‘Checklist of questions to consider when planning, reporting, or analysing co-design activities’ [42] (Supplementary File S1).
The intervention development was informed by the four critical elements of co-design outlined in Chisholm’s co-design model ([43] Figure 1). A virtual panel was established in May 2021 consisting of 10 expert stakeholders who contributed to the co-design of training content. As members of the virtual panel, stakeholders liaised with the project team through a range of methods including individual email correspondence, telephone discussion, and/or group discussion. Stakeholders included individuals from academic institutions, local authorities, and the mental health charity Mind and Public Health England. The stakeholder group had expertise in education and training, involving specialists in human resources, employment, and/or mental health, and people with lived experiences of mental ill-health and line management, including the supervision or management of employees with mental health concerns. The research team had expertise in occupational and health psychology, mental health, workforce and employment issues, digital health education, and equality, diversity, and inclusion. Collaborative working between the research team and stakeholders allowed for evidence-based (top-down) and experience-based (bottom-up) inputs to design. At each stage of the study, to merge inputs, we adopted existing bidirectional strategies to enable full co-design; such strategies have been used previously for digital intervention development, for example, in the creation of eHealth initiatives [44]. These include selecting (satisfying one need but not the other), combining (keeping multiple options in the design), integrating (designing a new and coherent functionality that serves both needs), and reframing (redefining perspectives in a way that dissolves the conflict).
After preliminary discussion with stakeholders, the initial outline of module headings was drafted by the research team as subject matter experts. Module headings were created using a ‘Wall Storm’ approach, in which key topic areas were proposed by team members and processed as a group through interactive virtual meetings. A storyboard was then co-created together with stakeholders. A storyboard is a document used to describe the text, visuals, audio, interactive elements, and navigation that will be used in a digital training program. The Managing Minds at Work storyboard was co-ordinated by the project researchers and co-created through multiple methods including a design charette (larger group meeting to sketch storyboard ideas), design jams (smaller group meetings to develop multiple iterations of user experiences e.g., as used in [43]), an online form for the collection of feedback data, and email communication. In this way, end-users and stakeholders worked together in playing an active role in design decision making which is ongoing [45,46] and highly flexible to maximise stakeholder engagement. The stakeholder group was a ‘virtual community of practice’ (vCoP; [47]); defined for the purpose of this study as ‘a group of people with a common interest who interact and share knowledge, using online platforms’. Stakeholders were invited to consider six broad areas:
  • Order of modules and materials.
  • Frequency and timing of module delivery.
  • Content revisions (additions/removals).
  • Targeting of information (e.g., specific/generic).
  • What are the barriers that could prevent line managers from accessing/engaging in the training?
  • What are the facilitators that could support line managers in completing the training?
This process was agile, with multiple co-creation and review activities being undertaken concurrently. Reviews and revisions were made iteratively, until a final storyboard was agreed that aligned with the program’s theory of change (see Section 3.1). The process included the consideration of future implementation of the training, and the relationship between interactive activities (e.g., reflections, quiz), mechanisms of change (e.g., improved understanding, acquiring manager competencies, learning communication skills), potential outcomes (e.g., improvement in communication or willingness to promote mental health), and context, including barriers and facilitators to use (e.g., availability of devices, ease of use).
Technical development was then undertaken by the project team, by creation of a prototype using Xerte, a multimedia authoring tool. In line with prototype development processes described elsewhere [48], design specifications were refined according to the level of importance assigned to them by stakeholders, and their views towards the likely acceptability, ease of use, technical difficulty, time required, and development costs. Following stakeholder review, revisions were made in five broad categories (Table 2).
An explanation of the value of the training was included in the introductory text to line managers to encourage institutional ‘buy-in’. Further revisions by module are shown in Supplementary File S2. Stakeholders agreed to the final module topics (Figure 2; May 2021) and the final training program content (June 2021). Managing Minds at Work was then hosted on a Web platform in September 2021 and is ready for feasibility testing, a future phase of the research in which the training program will be delivered to managers in real-world settings, with an evaluation undertaken on ‘what’ was delivered, ‘how’ it was delivered, and how recipients respond.
Since the study was undertaken during the global coronavirus pandemic (COVID-19), we were guided by published practical actions to promote and maintain meaningful exchange with stakeholders in times of social distancing and lockdowns [49]. As described, we were able to sustain stakeholder exchange by offering alternative approaches to involvement and deducting which formats of engagement were suitable for different groups (Action 1: ‘know your stakeholders’). These options included brief, one-to-one conversations (Action 2: ‘strengthen stakeholder relationships’), emails, telephone contact (Action 3: do not go 100% digital’), and small-group meetings (Action 3: ‘re-think your offline methods’). To accommodate stakeholders, we allowed a significant period for development and included extended email correspondence to maintain the conversation (Action 4: ‘stay flexible and keep it simple’). Given that stakeholder input was gathered remotely, there is a potential for bias in that those involved with virtual stakeholder input may have been more likely to be familiar with technology and digital approaches to training (Action 5: ‘learn lessons for post-pandemic engagement’). Finally, given the pandemic-related increase in mental ill-health [50] and the impact of the pandemic on the labor market [51], we aim to consider the impact of the circumstances of the pandemic on intervention development and future feasibility testing (Action 6: ‘account for the COVID-19 circumstances in your research results’).

2.3. Usability and Pilot Testing

The intervention was pilot tested by the project team between October 2021 and March 2022, with the aim of testing the acceptability and usability of MMW to line managers in ‘real-world’ settings. The design was a single group intervention study with post-intervention data collection, aligned with the Technology Acceptance Model (TAM [39]) (Figure 3).
TAM is an information systems theory that models how users come to accept and use a technology. Participants were organizations (site participants) and line managers (individual participants). Since the entire study was conducted during a pandemic, which was a challenging time for organizations, we adopted opportunity sampling. There is no established consensus for the sample size required for usability testing, with a range of suggested options (e.g., 3, 4, 5, 7 ± 2, 16 ± 4) depending on the approach taken. We therefore opted to recruit the largest suggested sample size (16 ± 4) given the diversity of workplace training needs across organizations and sectors, and the changing needs in the context of the COVID-19 pandemic and its aftermath—it has been proposed that group size should typically be increased along with the study’s complexity and the criticality of its context [52].

3. Results

3.1. Managing Minds at Work Intervention

Intervention reporting aligns with the Template for Intervention Description and Replication (TIDieR) checklist and guide [53] (Supplementary File S3). The resulting intervention is a workplace line manager training program called ‘Managing Minds at Work’. The intended audience is line managers who have direct managerial or supervisory responsibilities. The purpose of Managing Minds at Work is to develop line managers’ knowledge and confidence in preventing work-related stress and promoting mental health at work. The training is based on a theory of change, which is commonly used in the development of digital interventions [54]. The theory of change is a purposeful model of how the MMW intervention will contribute through a chain of early and intermediate outcomes to the intended result:
“Managing Minds at Work will develop line manager’s knowledge and confidence in preventing work-related stress and promoting mental health at work. This will be achieved through learning activities to increasing line managers’’ awareness of mental health (including legal requirements and employer responsibilities around work-related stress), encouraging the creation of psychologically safe working environments and work designs that promote mental wellbeing, and increasing managers’ competencies in preventing work-related stress and having open conversations about mental health in the workplace. Ultimately, the longer-term outcomes of this will be to reduce the prevalence of mental ill-health in working-age adults, and related economic burden of presenteeism and sickness absence to individuals, employers, and society”.
The program was developed using Xerte, a multimedia authoring tool focused on accessibility and simplicity. It consists of an introduction followed by five modules of online learning (Figure 2), accessed by URL using an individual user login and password. It is accessible via a secure website using a personal computer (PC), laptop, tablet, or mobile phone, although larger screen devices (e.g., PC or standard laptop) are recommended to maximize readability. It is intended to be accessed as a standalone training program, requiring no additional materials. The training materials are underpinned by an interactive and multi-media format. Each module includes evidence-based guidance, interactive tasks with opportunities for reflection, short videos, embedded links, and downloadable checklists. Each module ends with a ‘summary and next steps’ section, that provides a broad recap on key messages and key resources. A ‘next steps’ checklist at the end of each module invites the user to reflect on practical steps: (a) things you can do as a manager, (b) things to ask your organization to do, (c) things to ask your team to do.
The content is designed to be relevant to any line manager, in any organization and any sector. Therefore, it intentionally provides generic advice rather than sector-specific or tailored advice, which will allow for future intervention scalability. The modules are independent and stand-alone, although it is recommended that they are completed in succession (Introduction, then Module 1 through to Module 5), at a rate of 1 module per week, undertaken over a period of 6 weeks. Each module takes approximately 30 min to complete. The training is designed in this way so that learning can be undertaken flexibly during working hours. The training is self-led (requiring no prior knowledge or training to access and engage with it), so each participant can progress individually through each module at their own pace and at a time that suits them. Module content can be revisited at any time.
The final Managing Minds at Work content is shown in Figure 4. The management competencies that prevent work-related stress [55,56,57,58] are included in Module 3 (Figure 5). Approaches to having conversations with people about mental health at work are included in Module 5 (strategies shown in Figure 6). An example interactive activity is provided (Figure 7).

3.2. Usability and Pilot Testing Results

In October 2021, we approached three organizations that were taking part in mental health initiatives organised by MHPP (a regional consortium [31]) and therefore had ‘gatekeepers’ that were known to the project team. All organizations agreed to be pilot test sites and to approach managers employed within their organizations to invite them to take part in MMW piloting and usability testing. One of the organizations was from the third sector (a charity), one was from the private sector (a cooperative financial institution), and the other was from the public sector (an educational institution). One organization was small (0–49 employees), one was medium-sized (50–249 employees) and one was large (>250 employees). These gatekeepers provided information to all their employees about the usability study and invited them to directly contact the project team if they were interested in being involved. We aimed to recruit a sample size of 16 ± 4, and this was achieved. Twenty-seven managers expressed an interest in participating in the pilot study. Of these, 18 gave informed consent (67%), and were then invited to engage with the prototype training package and provide user feedback aligned with the Technology Acceptance Model (TAM; [39]) which is one of the most influential models of technology acceptance. Quantitative data were collected using a brief survey designed to assess the two primary factors influencing an individual’s intention to use new technology: perceived ease of use and perceived usefulness. This contained 11 question items, including 2 categorical items (yes/no), 5 Likert scale items (strongly agree, agree, disagree, strongly disagree), including 2 items that were reverse coded (i.e., phrased negatively), and 4 open-ended question items which allowed for free text responses to expand on attitudes towards the training, mental health promotion, and behavioral intentions (i.e., individual intention to use a particular technology that directly affects actual usage). Managers could choose whether they completed one or more modules as part of the pilot test.
Of the 18 managers, 12 (67%) completed at least 1 module, and a total of 37 survey responses were gathered across 5 modules (62% of a possible 60). This provided feedback from between 5–11 managers per module, which was considered adequate to establish their usability. Due to the pandemic context and impact on businesses during this time, we did not advocate that all the managers complete all five modules unless they wished to. Feedback related to module 1 (n = 11), module 2 (n = 10), module 3 (n = 5), module 4 (n = 6), and module 5 (n = 5). Age of managers ranged from 24–59 years (61% female). Participants had worked in a managerial role between 6 months and 30 years (two managers had supervised other people for 1 year or less, 11 managers had supervised other people for ≤10 years). Managers currently supervised up to 25 employees; half of the managers supervised ≤10 people (n = 9). Quantitative survey data were analyzed using descriptive statistics, in IBM SPSS Statistics for Windows, version 26.0 [59]. Findings are shown in Table 3.
Quantitative data and qualitative free text responses were then coded into the five TAM categories and narratively reported.
External variable: The MMW training was delivered as planned. However, while almost two-thirds of the managers could easily complete the materials within the working day, finding time to complete the modules was a barrier for over one-third of managers (37.8%).
Perceived usefulness: All the managers (100%) perceived that the module content was relevant to their managerial role. Most of the managers found the case examples relevant to their role, and only two offered reasons for non-relevance. One manager indicated that the case examples were not in their direct field of work, although they did acknowledge the similarities. Another raised an issue with a case example “putting the onus on the manager to control workload” when in some roles (e.g., in large organizations or expert teams), this is not possible at an individual level, and it may be more relevant to make workload decisions collaboratively as a team as to what can or cannot be delivered by the team. In such instances, acknowledging the views of employees ensures they have some level of input and control. It was proposed that future versions could contain more examples as resources for users to draw on. Almost two thirds of respondents indicated that they had learned something new from the training. Those that had not learned new information still found the materials relevant as they highlighted that the program served as a useful prompt or reminder to apply their knowledge in the workplace.
Ease of use: All the managers (100%) perceived that the modules were an appropriate length, and all but one of them found the content easy to understand and the online package easy to navigate (albeit one reported a technical issue). One manager proposed that audio recordings of the content would be a useful addition in the future.
Attitude: Managers referred to becoming more aware of the heterogenous impact of stressors on individuals after engaging with the training:
“It’s given me a better understanding of the different sources of stress for colleagues and subordinates. I’ll be sure to be more considerate of the different factors in my approach going forward”.
Some indicated that they had learned the importance of prioritizing their own mental wellbeing and role modelling positive behavior in the workplace, particularly by enacting self-care behavior, such as taking work breaks:
“I have underestimated how important it is that I look after myself. The information in the module makes sense in that you cannot support others if you aren’t in a good place yourself”.
“a good reminder of the need to take care of yourself”.
Behavioral Intention: Most of the managers indicated that they would be making practical changes following their use of the training program. Proposed actions were diverse but included monitoring stress levels within their organization (e.g., through use of a workforce stress survey). Several managers suggested that they would further review the sources of stress in their organization and seek to remove stressors and/or identify ways in which stress could be managed. Managers reported that they would use their learning to better recognize the signs of stress in colleagues, and their new understanding of their own responsibilities as a manager would help them to decide on actions to take. One manager commented that they would be more likely to act in a timelier way, and others indicated they intended to use the tools advocated within the materials (e.g., HSE Management Standards Indicator Tool [58]). Managers reported that they would re-consider job design and job demands in the future—in particular, providing staff with more clarity around their job roles, and focusing on whether job demands are both manageable and challenging.
“It was useful to learn about breaking down the job demands into areas I can easily understand. This will help me to help my team”.
There was a recognition that the COVID-19 pandemic had exerted a negative impacted on organizations and commonly resulted in higher workloads for staff. Managers committed to exploring the impact of workloads on staff stress levels and ways in which this could be mitigated, for example, the employment of additional staff, or reviewing workplace policies and behavior and actively encouraging behavior that supports wellbeing (e.g., advocating work breaks and self-care activities). One manager indicated they would be involving their employees more in problem solving rather than attempting to generate ‘top-down’ solutions themselves:
“I think, as the manager I need to have all the answers but it’s important to share problems with the team”.
Many of the proposed actions related to improving communication approaches between managers and other employees. For example, managers indicated that they would more regularly and consistently ‘check-in’ with staff, initiating more one-to-one conversations about mental health with colleagues, and consider the language they used to do this:
“I will make an effort to ensure that both my team and I are more conscious about the things we say and how we say them”.
“Made me rethink how to approach people - and the boundaries of things I might ask about”
Managers recognized the value of establishing the right workplace culture and environment to facilitate this. Several managers pledged actions related to improving the workplace culture, such as: “be kinder to colleagues”, “note down positive occurrences in the working day”, and “establish greater psychological safety within the workplace”:
“I had heard of psychological safety but didn’t really know what it was. I can now make more effort to ensure we have psychological safety in the team”.
“the more conversations I have, the easier it is to talk about mental health”.
Although the managers primarily referred to behavioral intentions in terms of the positive actions they would take in the future, one manager highlighted the potential challenges of implementing change, in terms of balancing the actions recommended within the training program with the expectations and demands of the business.
Overall, managers perceived the package to be easy to use, with high content readability (designed for a reading age of 12 years), high usability, and high fidelity (relevance to employment context). Some minor technical issues were raised when accessing the package from a device with a small screen (e.g., phone or tablet), resulting in a recommendation for the training to be accessed on a PC or laptop with a larger screen size. Strategies to maintain or enhance fidelity will be explored in a future feasibility trial.

4. Discussion

In this paper, we report on the theoretically informed development processes and pilot testing of a digital training program for line managers called Managing Minds at Work (MMW). The aim of MMW is to support employers in preventing mental ill-health and promoting good mental wellbeing at work. The key areas covered within the training are self-care, designing and managing work to promote mental wellbeing, management competencies to prevent work-related stress, developing a psychologically safe work environment, and having conversations about mental health at work. To our knowledge MMW is the first digital training to be co-created with line managers, using case examples and reflexive learning, which aims to equip line managers with the skills, knowledge, and competencies needed to design and operationalize a healthy workplace that promotes mental health.
Our findings indicate that managers perceive MMW to be appropriate and relevant for their needs, with high fidelity, ease of use, and relevance for managers from the public, private, and third sector. This is important given that managers play a key role in primary prevention, but many organizations offer no training for their staff [19], and there are few existing studies of interventions with a primary prevention focus [24]. Nevertheless, national and international guidance advocates the importance of a preventative approach and the essential role of line managers in mental health at work [60,61,62,63,64].
We included an element of reflexive and experiential learning within MMW training content as recommended in previous research [24] and opted for online training delivery given the high value placed on online, flexible learning approaches [26,27,28,29,30]—both elements were valued by stakeholders and managers for the ease of access to training and their assistance in supporting learning.
Co-creation of content using collaborative participatory design (e.g., [32,42]) has resulted in a training program that is directly relevant to the learning and support needs of line managers. The adoption of a user-centered active learning pedagogy with professional reflection on scenarios means that line managers are actively engaged in the learning process, gathering information, thinking, and problem solving, which is known to have better outcomes than more passive learning approaches [65]. Use of agile methodology has involved a rigorous development process [32,36] that allowed flexibility in virtual engagement with stakeholders and timings for the review processes. This was essential given that the study took place during the COVID-19 pandemic, a time of significant employment impact [66], when stakeholders were harder to reach, had less time, had priorities elsewhere, and/or government social distancing measures designed to stop the spread of the virus restricted face-to-face participatory activities [49].
The participatory design and rigorous co-creation processes have been described in detail, including the practical actions taken for managing the effects of the COVID-19 pandemic on participatory research. The importance of documenting intervention development processes, as a key research activity, has been outlined by O’Cathain and colleagues [40]. This is deemed to be an essential research activity which allows for judgements to be made about the quality of the process and outcome of future interventions and supports future replicability. This study is the first step in a research process which will test the feasibility and acceptability of MMW to managers in diverse employment settings, and then assess the effectiveness of MMW in improving outcomes for managers, employees, and organizations.

Study Strengths and Limitations

A strength of the study was the rigorous process for content and technical development, involving participatory design with multi-method approaches to co-creation, adapted in line with guidance for conducting participatory research during the COVID-19 pandemic. Our intervention was based on a ‘theory of change’ and user-centered active learning pedagogy, and the research was theoretically informed, using Chisholm’s co-design model [43], and TAM [39], an information systems theory. The resource has been pilot tested across sectors (public, private, and third) and organization types and sizes, which has allowed us to explore the usability and fidelity of the training program prior to full feasibility testing. We did not collect demographic data for the individuals who participated in the expert panel to maintain confidentiality. In our empirical pilot study, the sample size was appropriate for usability testing, although not all managers were able to complete all the modules, due to the pressures and additional work demands related to the COVID-19 pandemic. Nevertheless, between 5–11 managers provided feedback on each module, and this was deemed sufficient to meet the aims of the pilot testing and usability study. Managers involved in our expert panel and usability study were primarily from office-based roles, and the training program is yet to be feasibility tested in other types of employment setting.

5. Conclusions

Managing Minds at Work is a new digital training program for line managers which aims to support employers in preventing mental ill-health and promoting good mental wellbeing at work. The intervention was deemed to have high usability and fidelity by managers from the public, private, and third sectors. The next step is to fully explore the feasibility and acceptability of the Managing Minds at Work intervention with managers in diverse employment settings, explore the mechanisms for its most effective delivery, and examine whether and how the intervention can be tailored for different types of organizations and integrated alongside existing polices, practices, and interventions.

Supplementary Materials

The following supporting information can be downloaded at:, File S1: Checklist of questions to consider when planning, reporting, or analysing co-design activities; File S2: Module revisions following stakeholder review; File S3: TiDieR Checklist.

Author Contributions

Conceptualization, H.B., L.T., J.H., S.R., G.D., C.M. and S.M.; data curation, B.V. and C.B.; formal analysis, B.V. and C.B.; funding acquisition, L.T., H.B., J.H., F.M., S.R., G.D., C.M. and S.M.; investigation, B.V., C.B., J.Y., L.T., H.B., J.H. and F.M.; methodology, H.B., L.T. and J.H.; project administration, B.V. and C.B.; writing—original draft, H.B.; writing—review and editing, all authors. All authors have read and agreed to the published version of the manuscript.


The study is part of the Mental Health and Productivity Pilot, which is funded by Employers, Health and Inclusive Employment (EHIE). The EHIE had no role in the design of the Managing Minds at Work study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Research Ethics Committee of the University of Nottingham Faculty of Medicine and Health Sciences (FMHS 299-0621; July 2021). This intervention development phase is an early stage of a registered trial ( Identifier: NCT05154019).

Informed Consent Statement

Informed verbal consent was obtained from all participants involved in the study. Stakeholders involved in the study were contributing to patient and public involvement and engagement in research (PPIE). As such, research ethical approval and written consent is not required, but all contributors were provided with full information about what they were being asked to do and had the opportunity to ask questions and the option to withdraw from the work. Their personal contact details remain confidential.

Data Availability Statement

This was an intervention development study that involved co-creation and usability testing with participants rather than the collection of research data. However, further details on the development activities, stakeholder feedback, and usability testing are available from the authors on reasonable request.


The authors would like to thank all the stakeholders who contributed to the development processes, the Midlands Engine Mental Health and Productivity Pilot consortium partners, including Mind (in particular, Errin Riley and Nemmy Johnston), the University of Warwick (in particular, Charlotte Kershaw), Coventry University (in particular, Caroline Geraghty, Chitro Ghose), the West Midlands Combined Authority, Loughborough University, University of Birmingham, University of Lincoln, and the University of Derby.

Conflicts of Interest

The authors declare no conflict of interest.


  1. Health and Safety Executive (HSE). Work Related Stress, Anxiety and Depression Statistics in Great Britain. 2019. Available online: (accessed on 23 May 2022).
  2. Deloitte. Mental Health and Employers: The Case for Investment-Pandemic and Beyond. September–October 2021. Available online: (accessed on 5 April 2022).
  3. Butterworth, P.; Leach, L.S.; Pirkis, J.; Kelaher, M. Poor mental health influences risk and duration of unemployment: A prospective study. Soc. Psychiatry Psychiatr. Epidemiol. 2011, 47, 1013–1021. [Google Scholar] [CrossRef] [PubMed]
  4. Torales, J.; O’Higgins, M.; Castaldelli-Maia, J.M.; Ventriglio, A. The outbreak of COVID-19 coronavirus and its impact on global mental health. Int. J. Soc. Psychiatry 2020, 66, 317–320. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Hassard, J.; Teoh, K.; Thomson, L.; Blake, H. Understanding the cost of mental ill-health at work: An integrative framework. In The SAGE Handbook of Organizational Wellbeing; Wall, T., Cooper, C.L., Brough, P., Eds.; SAGE Publications, Ltd.: London, UK, 2021. [Google Scholar]
  6. Hassard, J.; Teoh, K.R.H.; Visockaite, G.; Dewe, P.; Cox, T. The cost of work-related stress to society: A systematic review. J. Occup. Health Psychol. 2018, 23, 1–17. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  7. EOSHA. Psychosocial Risks in Europe: Prevalence and Strategies for Prevention; Publications Office of the European Union: Luxembourg, 2014. [Google Scholar]
  8. Carbone, S.R. Flattening the curve of mental ill-health: The importance of primary prevention in managing the mental health impacts of COVID-19. Ment. Health Prev. 2020, 19, 200185. [Google Scholar] [CrossRef] [PubMed]
  9. Joyce, J. Facing the challenge of mental ill health in the workplace. J. Public Ment. Health 2013, 12, 93–97. [Google Scholar] [CrossRef]
  10. Newman, S.A.; Ford, R.C. Five Steps to Leading Your Team in the Virtual COVID-19 Workplace. Organ. Dyn. 2021, 50, 100802. [Google Scholar] [CrossRef]
  11. Gray, P.; Senabe, S.; Naicker, N.; Kgalamono, S.; Yassi, A.; Spiegel, J.M. Workplace-Based Organizational Interventions Promoting Mental Health and Happiness among Healthcare Workers: A Realist Review. Int. J. Environ. Res. Public Health 2019, 16, 4396. [Google Scholar] [CrossRef] [Green Version]
  12. Martin, A.; Woods, M.; Dawkins, S. How managers experience situations involving employee mental ill-health. Int. J. Work. Health Manag. 2018, 11, 442–463. [Google Scholar] [CrossRef]
  13. Davenport, L.J.; Allisey, A.F.; Page, K.M.; LaMontagne, A.D.; Reavley, N.J. How can organizations help employees thrive? The de-velopment of guidelines for promoting positive mental health at work. Int. J. Work. Health Manag. 2016, 9, 411–427. [Google Scholar] [CrossRef]
  14. Dextras-Gauthier, J.; Marchand, A.; Haines, V., III. Organizational culture, work organization conditions, and mental health: A proposed integration. Int. J. Stress Manag. 2012, 19, 81–104. [Google Scholar] [CrossRef]
  15. Edmondson, A.D. The Fearless Organization: Creating Psychological Safety in the Workplace for Learning, Innovation, and Growth; John Wiley & Sons Inc.: Hoboken, NJ, USA, 2019. [Google Scholar]
  16. Nexø, M.A.; Kristensen, J.V.; Grønvad, M.T.; Kristiansen, J.; Poulsen, O.M. Content and quality of workplace guidelines developed to prevent mental health problems: Results from a systematic review. Scand. J. Work. Environ. Health 2018, 44, 443–457. [Google Scholar] [CrossRef] [PubMed]
  17. Dimoff, J.K.; Kelloway, E.K. With a little help from my boss: The impact of workplace mental health training on leader behaviors and employee resource utilization. J. Occup. Health Psychol. 2019, 24, 4–19. [Google Scholar] [CrossRef] [PubMed]
  18. Gayed, A.; Milligan-Saville, J.S.; Nicholas, J.; Bryan, B.T.; La Montagne, A.D.; Milner, A.; Madan, I.; Calvo, R.A.; Christensen, H.; Mykletun, A.; et al. Effectiveness of training workplace managers to understand and support the mental health needs of employees: A systematic review and meta-analysis. Occup. Environ. Med. 2018, 75, 462–470. [Google Scholar] [CrossRef] [PubMed]
  19. Institute for Occupational Safety and Health. White Paper—Workplace Wellbeing: The Role of Line Managers in Promoting Positive Nental Health. Management Today. 2019. Available online: (accessed on 20 April 2022).
  20. Weston, D.; Hudson, C.; Carroll, D.; Coomber, S.; Amlôt, R. Evaluating a pilot mental health awareness for managers’ training course. Occup. Med. 2019, 69, 251–257. [Google Scholar] [CrossRef] [PubMed]
  21. Gayed, A.; Bryan, B.T.; LaMontagne, A.D.; Milner, A.; Deady, M.; Calvo, R.A.; MacKinnon, A.; Christensen, H.; Mykletun, A.; Glozier, N.; et al. A cluster randomized controlled trial to evaluate HeadCoach: An online mental health training programme for workplace managers. Occup. Environ. Med. 2019, 61, 545–551. [Google Scholar] [CrossRef] [Green Version]
  22. Dimoff, J.K.; Kelloway, E.K.; Burnstein, M.D. Mental health awareness training (MHAT): The development and evaluation of an intervention for workplace leaders. Int. J. Stress Manag. 2016, 23, 167–189. [Google Scholar] [CrossRef]
  23. Milligan-Saville, J.S.; Tan, L.; Gayed, A.; Barnes, C.; Madan, I.; Dobson, M.; A Bryant, R.A.; Christensen, H.; Mykletun, A.; Harvey, S.B. Workplace mental health training for managers and its effect on sick leave in employees: A cluster randomised controlled trial. Lancet Psychiatry 2017, 4, 850–858. [Google Scholar] [CrossRef]
  24. Stansfeld, S.A.; Kerry, S.; Chandola, T.; Russell, J.; Berney, L.; Hounsome, N.; Lanz, D.; Costelloe, C.; Smuk, M.; Bhui, K. Pilot study of a cluster randomised trial of a guided e-learning health promotion intervention for managers based on management standards for the improvement of employee well-being and reduction of sickness absence: GEM Study. BMJ Open 2015, 5, e007981. [Google Scholar] [CrossRef] [Green Version]
  25. Health and Safety Executive (HSE). Tackling Work-Related Stress Using the Management Standards Approach. 2019. Available online: (accessed on 4 April 2022).
  26. Jones, M.A. A Study of Satisfaction with Online Learning in Workplace Training. Thesis in Public Policy and Administration. Walden University, 2016. Available online: (accessed on 20 April 2022).
  27. Mitsakis, F.; Karageorgakis, T. E-learning: A Temporary ‘By-Product’ of Covid-19 Pandemic or a Contemporary Solution to Workplace Training and Learning? In The Future of HRD; Loon, M., Stewart, J., Nachmias, S., Eds.; Palgrave Macmillan: Cham, Switzerland, 2020; Volume 1. [Google Scholar] [CrossRef]
  28. Caudill, J.G. Employee Motivations for Workplace Learning and the Role of Elearning in the Workplace. Internet Learn. 2015, 4. Available online: (accessed on 23 May 2022). [CrossRef]
  29. David, O.; Salleh, M.; Iahad, N. The impact of elearning in workplace: Focus on organizations and healthcare environments. IAJIT 2012, 2, 203–209. [Google Scholar]
  30. Cheng, B.; Wang, M.; Yang, S.J.; Kinshuk; Peng, J. Acceptance of competency-based workplace e-learning systems: Effects of individual and peer learning support. Comput. Educ. 2011, 57, 1317–1333. [Google Scholar] [CrossRef] [Green Version]
  31. Mental Health and Productivity Pilot (MHPP). Supporting Workplace Mental Health Across the East Midlands Engine Region. Available online: (accessed on 20 April 2022).
  32. Blake, H.; Somerset, S.; Greaves, S. The Pain at Work Toolkit for Employees with Chronic or Persistent Pain: A Collaborative-Participatory Study. Healthcare 2022, 10, 56. [Google Scholar] [CrossRef]
  33. Yardley, L.; Morrison, L.; Bradbury, K.; Muller, I. The Person-Based Approach to Intervention Development: Application to Digital Health-Related Behavior Change Interventions. J. Med. Internet Res. 2015, 17, e30. [Google Scholar] [CrossRef]
  34. Mohr, D.C.; Schueller, S.M.; Montague, E.; Burns, M.N.; Rashidi, P. The Behavioral Intervention Technology Model: An Integrated Conceptual and Technological Framework for eHealth and mHealth Interventions. J. Med. Internet Res. 2014, 16, e146. [Google Scholar] [CrossRef]
  35. Oinas-Kukkonen, H.; Harjumaa, M. Persuasive systems design: Key issues, process model, and system features. CAIS 2009, 24, 28. [Google Scholar] [CrossRef]
  36. Blake, H.; Somerset, S.; Evans, C. Development and Fidelity Testing of the Test@Work Digital Toolkit for Employers on Workplace Health Checks and Opt-In HIV Testing. Int. J. Environ. Res. Public Health 2020, 17, 379. [Google Scholar] [CrossRef] [Green Version]
  37. Blake, H.; Bermingham, F.; Johnson, G.; Tabner, A. Mitigating the Psychological Impact of COVID-19 on Healthcare Workers: A Digital Learning Package. Int. J. Environ. Res. Public Health 2020, 17, 2997. [Google Scholar] [CrossRef]
  38. Thabrew, H.; Fleming, T.; Hetrick, S.; Merry, S. Co-design of eHealth Interventions with Children and Young People. Front. Psychiatry 2018, 9, 481. [Google Scholar] [CrossRef] [Green Version]
  39. Davis, F.D. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Q. 1989, 13, 319–340. [Google Scholar] [CrossRef] [Green Version]
  40. O’Cathain, A.; Croot, L.; Duncan, E.; Rousseau, N.; Sworn, K.; Turner, K.M.; Yardley, L.; Hoddinott, P. Guidance on how to develop complex interventions to improve health and healthcare. BMJ Open 2019, 9, e029954. [Google Scholar] [CrossRef] [Green Version]
  41. Skivington, K.; Matthews, L.; Simpson, S.; Craig, P.; Baird, J.; Blazeby, J.; Boyd, K.; Craig, N.; French, D.; McIntosh, E.; et al. A new framework for developing and evaluating complex interventions: Update of Medical Research Council guidance. BMJ 2021, 374, n2061. [Google Scholar] [CrossRef]
  42. Bevan Jones, R.; Stallard, P.; Agha, S.S.; Rice, S.; Werner-Seidler, A.; Stasiak, K.; Kahn, J.; Simpson, S.A.; Alvarez-Jimenez, M.; Rice, F.; et al. Practitioner review: Co-design of digital mental health technologies with children and young people. J. Child Psychol. Psychiatry 2020, 61, 928–940. [Google Scholar] [CrossRef]
  43. Hodson, E.; Dadashi, N.; Delgado, R.; Chisholm, C.; Sgrignoli, R.; Swaine, R. Co-design in mental health; Mellow: A self-help holistic crisis planning mobile application by youth, for youth. Des. J. 2019, 22, 1529–1542. [Google Scholar] [CrossRef] [Green Version]
  44. Austin, J.; Van Dijk, J.; Drossaert, C. When Theory Meets Users in Co-Design: Four Strategies towards Synergy between Bottom-Up and Top-Down Input. In Proceedings of the Synergy-DRS International Conference, held online, 11–14 August 2020. [Google Scholar]
  45. Sanders, E.B.-N.; Stappers, P.J. Co-creation and the new landscapes of design. CoDesign 2008, 4, 5–18. [Google Scholar] [CrossRef] [Green Version]
  46. Saad-Sulonen, J.; Eriksson, E.; Halskov, K.; Karasti, H.; Vines, J. Unfolding participation over time: Temporal lenses in participatory design. CoDesign 2018, 14, 4–16. [Google Scholar] [CrossRef] [Green Version]
  47. Toledo-Chávarri, A.; Ramos-García, V.; Koatz, D.; Torres-Castaño, A.; Perestelo-Pérez, L.; Ramírez-Puerta, A.B.; Tello-Bernabé, M.-E.; García-García, J.-M.; García-García, J.; Pacheco-Huergo, V.; et al. Co-Design Process of a Virtual Community of Practice for the Empowerment of People with Ischemic Heart Disease. Int. J. Integr. Care 2020, 20, 9. [Google Scholar] [CrossRef]
  48. Bevan Jones, R.; Thapar, A.; Rice, F.; Beeching, H.; Cichosz, R.; Mars, B.; Smith, D.J.; Merry, S.; Stallard, P.; Jones, I.; et al. A Web-Based Psychoeducational Intervention for Adolescent Depression: Design and Development of MoodHwb. JMIR Ment. Health 2018, 5, e13. [Google Scholar] [CrossRef] [Green Version]
  49. Köpsel, V.; de Moura Kiipper, G.; Peck, M.A. Stakeholder engagement vs. social distancing—how does the COVID-19 pandemic affect participatory research in EU marine science projects? Marit. Stud. 2021, 20, 189–205. [Google Scholar] [CrossRef]
  50. Cullen, W.; Gulati, G.; Kelly, B.D. Mental health in the COVID-19 pandemic. QJM 2020, 113, 311–312. [Google Scholar] [CrossRef]
  51. Powell, A.; Francis-Devine, B.; Clark, H. Coronavirus: Impact on the Labour Market. Research Briefing; House of Commons Library, UK Parliament: London, UK, 2022; Available online: (accessed on 20 April 2022).
  52. Al Roobaea, R.; Mayhew, P.J. How Many Participants are Really Enough for Usability Studies? In Proceedings of the Science and Information Conference, London, UK, 27–29 August 2014. [Google Scholar]
  53. Hoffmann, T.; Glasziou, P.; Boutron, I.; Milne, R.; Perera, R.; Moher, D.; Altman, D.; Barbour, V.; Macdonald, H.; Johnston, M.; et al. Better reporting of interventions: Template for intervention de-scription and replication (TIDieR) checklist and guide. BMJ 2014, 348, g1687. [Google Scholar] [CrossRef] [Green Version]
  54. Rad, D.; Rad, G. Theory of Change in Digital Behavior Change Interventions (Dbcis) And Community-Based Change Initiatives —A General Framework. Tech. Soc. Sci. J. 2021, 21, 554–569. [Google Scholar] [CrossRef]
  55. Yarker, J.; Lewis, R.; Donaldson-Feilder, E.; Flaxman, P. Management Competencies for Preventing and Reducing Stress at Work: Identifying and Developing the Management Behaviors Necessary to Implement the HSE Management Standards; Rr553; HSE Books: London, UK, 2007. [Google Scholar]
  56. Yarker, J.; Lewis, R.; Donaldson-Feilder, E. Management Competencies for Preventing and Reducing Stress at Work: Identifying the Management Behaviors Necessary to Implement the Management Standards: Phase Two; RR633; HSE Books: London, UK, 2008. [Google Scholar]
  57. Donaldson-Feilder, E.; Lewis, R.; Yarker, J. Preventing Stress: Promoting Positive Manager Behavior; CPID: London, UK, 2009; Available online: (accessed on 6 April 2022).
  58. Health and Safety Executive (HSE). Stress Management Competency Indicator Tool. 2009. Available online: (accessed on 20 April 2022).
  59. IBM SPSS Statistics for Windows; Version 26.0; IBM Corp.: Armonk, NY, USA, 2019.
  60. National Institute for Health and Care Excellence (NICE). Mental Wellbeing at Work. NICE Guideline [NG212]. 2022. Available online: (accessed on 4 April 2022).
  61. ISO 45003; Psychological Health in the Workplace. British Standards Institution (BSI): Loughborough, UK, 2022. Available online: (accessed on 4 April 2022).
  62. Mental Health Commission of Canada (MHCC). National Standard of Canada for Psychological Health and Safety in the Workplace. Available online: (accessed on 4 April 2022).
  63. Safe Work Australia. Work-Related Psychological Health and Safety: A Systematic Approach to Meeting Your Duties. National Guidance Material. 2019. Available online: (accessed on 4 April 2022).
  64. World Health Organization. Prevention and Promotion in Mental Health. Mental Health: Evidence and Research; Department of Mental Health and Substance Dependence, World Health Organization: Geneva, Switzerland, 2002. [Google Scholar]
  65. Michael, J. Where’s the evidence that active learning works? Adv. Physiol. Educ. 2006, 30, 159–167. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  66. Fana, M.; Torrejón Pérez, S.; Fernández-Macías, E. Employment impact of Covid-19 crisis: From short term effects to long terms prospects. J. Ind. Bus. Econ. 2020, 47, 391–410. [Google Scholar] [CrossRef]
Figure 1. Application of Chisholm’s co-design model to Managing Minds at Work development.
Figure 1. Application of Chisholm’s co-design model to Managing Minds at Work development.
Ijerph 19 08006 g001
Figure 2. Overview of Managing Minds at Work modules.
Figure 2. Overview of Managing Minds at Work modules.
Ijerph 19 08006 g002
Figure 3. Technology Acceptance Model (from [39]).
Figure 3. Technology Acceptance Model (from [39]).
Ijerph 19 08006 g003
Figure 4. Managing Minds at Work Training Materials.
Figure 4. Managing Minds at Work Training Materials.
Ijerph 19 08006 g004
Figure 5. Management competencies that prevent work-related stress (module 3).
Figure 5. Management competencies that prevent work-related stress (module 3).
Ijerph 19 08006 g005
Figure 6. Strategies for having conversations with people about mental health at work (module 5).
Figure 6. Strategies for having conversations with people about mental health at work (module 5).
Ijerph 19 08006 g006
Figure 7. Example interactive activities (module 2) . Two slides from the same module. First example requires users to identify which sources of stress are the cause of each of the examples by dragging four sources of stress shown along the bottom (job control; job demands; relationships at work; work and organizational change) to their appropriate grey boxes. Second example requires users to click on the image to gain practical examples.
Figure 7. Example interactive activities (module 2) . Two slides from the same module. First example requires users to identify which sources of stress are the cause of each of the examples by dragging four sources of stress shown along the bottom (job control; job demands; relationships at work; work and organizational change) to their appropriate grey boxes. Second example requires users to click on the image to gain practical examples.
Ijerph 19 08006 g007
Table 1. Mapping of core elements of the MRC framework to ‘Managing Minds at Work’ development.
Table 1. Mapping of core elements of the MRC framework to ‘Managing Minds at Work’ development.
Core ElementsIntervention Development Considerations
(i) ContextContext of the employment settings and job role of the line manager may influence the way in which the intervention is accessed and used. Content must be relevant across employment settings (sector, size, and type of organization) for line managers at all levels of the hierarchy.
(ii) Developing and refining program theoryProgram theory was established prior to intervention development with the involvement of diverse stakeholders and based on evidence and theory from relevant fields. This focused on identifying the key areas of line managers’ influence in preventing poor mental health, specific actions associated with these, and the likely outcomes. Can be refined during successive phases to inform transferability of the intervention across settings.
(iii) Engaging stakeholdersCollaborative participatory approach involved stakeholders at every stage: development of program theory, co-creation of intervention content, iterative peer review, and revisions. Multiple methods for stakeholder engagement were triangulated and included individual/group discussion, online data collection, and email feedback to share and develop ideas. Participation must be both emergent and ongoing (long term).
(iv) Identifying key uncertaintiesUncertainties related to design and delivery: timescale for development of the intervention, appropriateness of the level of language complexity, most appropriate format for delivery. Potential challenges with engaging stakeholders in research intervention development during a global pandemic. Consideration of the global public health and economic impact of a global pandemic on intervention content and future implementation.
(v) Refining the interventionUsing an agile approach, stakeholder consultation and review is an iterative process, allowing for continuous delivery and a resource-efficient approach to toolkit development.
(vi) Economic considerationsStakeholder input supported by charitable bodies and professional input via the wider MHPP program. Specific costs for intervention web-hosting and individual user logins.
Medical Research Council (MRC) framework for developing and testing complex interventions.
Table 2. Revisions to Managing Minds at Work content and design during co-creation and review.
Table 2. Revisions to Managing Minds at Work content and design during co-creation and review.
CategorySummary of Stakeholder Revisions
Presentation of materialscover page, colors, logos, image diversity, balance of text, activities and images, use of bullet points, signposting, typographical errors
Functionalityfont and illustration size, scrolling, web links, transcript availability, use of hyperlinks, video quality
Clarity of informationdefinition of terms, not assuming prior knowledge or skills, rephrasing, additional explanation, removal of repetition
Additional resourcesadding confidential helplines, downloadable resources page
Incentives for completionmodule-by-module completion for flexibility, provision of feedback or explanation for incorrect answers on tasks, encouragement to revisit tasks, observable progression points, confirmation of completion, reminders and encouragement for completion, downloadable certificate
Consideration of current contextrelevant to virtual and remote working due to the global coronavirus pandemic
Table 3. Usability testing and associated TAM constructs.
Table 3. Usability testing and associated TAM constructs.
Question Item
(N = 37 Responses +)
TAM Constructn (%)
Knowledge attainment % Yes
Did you learn anything that you did not know before?U24 (64.9)
Content Relevance % Yes
Did you think the module content was relevant to your managerial role?U37 (100)
Case example relevance n (%) strongly disagree, or disagree
The examples provided throughout the module were not relevant to my role as a managerU30 (81.1)
Ease of understanding n (%) strongly disagree,
or disagree
I found some of the information presented in the module difficult to understandE36 (97.3)
Usability n (%) strongly agree,
or agree
The online module was an appropriate length E37 (100)
The online module was easy to navigateE36 (97.3)
Barriers to use n (%) strongly agree,
or agree
It was easy to find the time to complete this moduleEV23 (62.2)
TAM Technology Acceptance Model; U, perceived usefulness; E, perceived ease of use; EV, external variable. + 37 survey completions from 12 managers.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Blake, H.; Vaughan, B.; Bartle, C.; Yarker, J.; Munir, F.; Marwaha, S.; Daly, G.; Russell, S.; Meyer, C.; Hassard, J.; et al. Managing Minds at Work: Development of a Digital Line Manager Training Program. Int. J. Environ. Res. Public Health 2022, 19, 8006.

AMA Style

Blake H, Vaughan B, Bartle C, Yarker J, Munir F, Marwaha S, Daly G, Russell S, Meyer C, Hassard J, et al. Managing Minds at Work: Development of a Digital Line Manager Training Program. International Journal of Environmental Research and Public Health. 2022; 19(13):8006.

Chicago/Turabian Style

Blake, Holly, Benjamin Vaughan, Craig Bartle, Jo Yarker, Fehmidah Munir, Steven Marwaha, Guy Daly, Sean Russell, Caroline Meyer, Juliet Hassard, and et al. 2022. "Managing Minds at Work: Development of a Digital Line Manager Training Program" International Journal of Environmental Research and Public Health 19, no. 13: 8006.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop