Next Article in Journal
Patients with Heart Failure and Preserved Ejection Fraction Are at Risk of Gastrointestinal Bleeding
Next Article in Special Issue
ZIEL: Internet-Based Self-Help for Adjustment Problems: Results of a Randomized Controlled Trial
Previous Article in Journal
Association between Secondhand Smoke Exposure and Hypertension in 106,268 Korean Self-Reported Never-Smokers Verified by Cotinine
Previous Article in Special Issue
Current State and Future Directions of Technology-Based Ecological Momentary Assessment and Intervention for Major Depressive Disorder: A Systematic Review
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
JCM
Volume 8
Issue 8
10.3390/jcm8081239
jcm-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

From Research to Practice: Ten Lessons in Delivering Digital Mental Health Services

by
Nickolai Titov
1,*,
Heather D. Hadjistavropoulos
2,
Olav Nielssen
1,
David C. Mohr
3,
Gerhard Andersson
4,5 and
Blake F. Dear
1
1
MindSpot Clinic and Department of Psychology, Macquarie University, Sydney, NSW 2109, Australia
2
Online Therapy Unit, Department of Psychology, University of Regina, Regina, Saskatchewan, SK S4S 0A2, Canada
3
Center for Behavioral Intervention Technologies, Department of Preventive Medicine, Northwestern University, Chicago, IL 60611, USA
4
Department of Behavioural Sciences and Learning, Linköping University, Linköping SE-581 83, Sweden
5
Department of Clinical Neuroscience, Karolinska Institute, Stockholm 171 77, Sweden
*
Author to whom correspondence should be addressed.
J. Clin. Med. 2019, 8(8), 1239; https://doi.org/10.3390/jcm8081239
Submission received: 16 July 2019 / Revised: 10 August 2019 / Accepted: 15 August 2019 / Published: 17 August 2019
(This article belongs to the Special Issue Innovative Technology Based Interventions for Psychological Treatment of Common Mental Disorders)
Review Reports Versions Notes

Abstract

:
There is a large body of research showing that psychological treatment can be effectively delivered via the internet, and Digital Mental Health Services (DMHS) are now delivering those interventions in routine care. However, not all attempts to translate these research outcomes into routine care have been successful. This paper draws on the experience of successful DMHS in Australia and Canada to describe ten lessons learned while establishing and delivering internet-delivered cognitive behavioural therapy (ICBT) and other mental health services as part of routine care. These lessons include learnings at four levels of analysis, including lessons learned working with (1) consumers, (2) therapists, (3) when operating DMHS, and (4) working within healthcare systems. Key themes include recognising that DMHS should provide not only treatment but also information and assessment services, that DMHS require robust systems for training and supervising therapists, that specialist skills are required to operate DMHS, and that the outcome data from DMHS can inform future mental health policy. We also confirm that operating such clinics is particularly challenging in the evolving funding, policy, and regulatory context, as well as increasing expectations from consumers about DMHS. Notwithstanding the difficulties of delivering DMHS, we conclude that the benefits of such services for the broader community significantly outweigh the challenges.

1. Introduction

Globally, mood and anxiety disorders affect more than 700 million people each year and are associated with considerable burden and disability [1,2]. However, in a 12-month period, fewer than half of those affected seek or receive evidence-based treatments [3,4,5] for reasons that include cost, limited availability of services in many areas, limited awareness of both illness and the potential benefit of treatment, stigma and preference to self-manage [6].
Delivering psychological services via the internet is one way of increasing access to care. A large number of randomised controlled trials have demonstrated that internet-delivered cognitive behavioural therapy (ICBT) is effective at treating anxiety and depression [7,8,9,10]. However, attempts to extend ICBT to routine care have produced mixed results. Several attempts at implementation have either been unsuccessful [11] or were not found to have added value to existing face to face services [12], which has raised doubt as to whether internet-delivered psychological services can be implemented successfully in typical health care settings [13]. Notwithstanding the challenges, the successful use of ICBT as part of routine clinical care has been reported in Sweden [14,15,16,17], the Netherlands [18,19], Norway, Denmark [20,21], Canada [22,23,24,25], and Australia [26,27,28,29,30]. In addition to reports of outcomes from individual clinics, and reflecting the maturing state of the field, there is now an increasing number of studies describing barriers [20,23,31,32], guidelines for implementation [33,34], and comparisons of clinics across different countries [35]. The successful clinics typically deliver ICBT interventions via so-called virtual or digital mental health services (DMHS). Common features of the successful clinics include high standards of both clinical and organisational governance, and robust systems for staff training and supervision [35].
This paper describes key lessons learned during our own efforts to develop and deliver DMHS. The MindSpot Clinic, Australia, and the Online Therapy Unit (OTU) in the province of Saskatchewan, Canada accept referrals directly from consumers as well as via general practitioners. Together, these DMHS have provided assessments to more than 100,000 people and treatment to more than 30,000 people. The authors have worked closely together for several years and have served on advisory bodies to each other’s services. The lessons we describe draw on our shared experiences in service development, delivery and collaboration.
We narrowed down our experience to ten key lessons that were not fully described in other papers. These lessons were not immediately apparent to us when we set about translating our research findings to routine care but have been of fundamental importance in how we developed and now operate our DMHS. Hence, we anticipate that these lessons may help those launching similar clinics.
We intentionally avoided specific frameworks of reporting [36,37], because an aim of this paper was to describe the experiences of operating mature services, rather than just the implementation phase. We also acknowledge that some of these lessons overlap, may not apply in other jurisdictions or even to other DMHS within our own countries.

2. Lessons

We chose to organise our lessons according to the model shown in Table 1, which represents the lessons learned from working with (1) consumers, (2) therapists, (3) operating DMHS, and (4) when operating in the broader health systems, including when engaging with funders and policy makers.
Before further describing the lessons learned, key aspects of the MindSpot Clinic and OTU are summarised below.

2.1. The MindSpot Clinic, Sydney, Australia

MindSpot was launched in 2013 and operates from Macquarie University, Sydney. MindSpot is funded by the Australian Government Department of Health, with funding initially provided for a 3-year period as a result of the competitive tender process. MindSpot aims to improve access to evidence-based education, triage, assessment, referral, and treatment services throughout Australia to adults with symptoms of depression and anxiety [28,29]. Clinic services are provided free of charge.
Patients can either self-refer after learning about MindSpot via the website (mindspot.org.au), online advertising, links from other mental health websites, recommendations by previous users or can take up referrals from health professionals. Patients first register online or via telephone and complete a detailed assessment questionnaire followed by telephone or secure email contact with a therapist to discuss symptoms and treatment options. Patients then choose between information to assist with self-management, referral to another service or ICBT. The clinic offers seven ICBT programs that have been validated in clinical trials, including transdiagnostic treatments designed to treat symptoms of anxiety and depression in several age groups [24,38,39,40,41,42,43,44,45,46,47] and disorder-specific treatments for obsessive compulsive disorder [48,49], post-traumatic stress disorder [50], and chronic pain [51]. All the treatment programs comprise of five lessons which provide the core information, delivered over eight-weeks. Additional resources targeting specific symptoms or difficulties are made available during treatment to assist patients tailor treatment to their own needs. Outcomes are measured using validated symptom scales that are administered weekly during treatment, on completion, and at a three-month follow-up. The therapists are all registered or provisionally registered mental health professionals who contact and monitor participants weekly during treatment via a secure email system or by telephone. The treatment patients are enrolled in cohorts every two weeks, with therapists each responsible for 50 or so patients. To date, more than 100,000 people have registered to use the clinic, and 25,000 have opted to receive ICBT.

2.2. Online Therapy Unit, Canada

The OTU has operated from the University of Regina in Saskatchewan since October 2010. Initial funding was provided by a federal research grant, but since 2015, the OTU has received stable funding from the Saskatchewan Ministry of Health. The OTU aims to provide therapist-guided ICBT for depression and anxiety and to educate providers of mental health care and conduct research on ICBT in routine practice [25]. Clinic services are also provided to patients free of charge.
The OTU promotes services to patients via word of mouth primarily from health care providers, media reports, and both digital and print communication. Patients are encouraged to visit the clinic online (onlinetherapyuser.ca) and can either self-refer or are referred by a health professional. Patients first complete an online screening followed by telephone assessment.
The OTU delivers several ICBT programs including an adaptation of the Wellbeing course developed at Macquarie University and used at the MindSpot Clinic [44,47,52]. Clinically validated patient reported outcome measures (PROMS) of anxiety and depression are administered regularly during treatment, at post-treatment, and at three-month follow-up. Therapists are registered mental health professionals or graduate students under supervision employed by the clinic or by publicly funded community clinics located in other parts of Saskatchewan. During treatment, patients receive weekly therapist contact primarily via secure email or by telephone to assist in applying the skills taught during treatment. Since October 2010, the clinic had assessed more than 5400 patients, 4200 of whom have received ICBT [23,24,35].

3. The Ten Lessons

3.1. Level 1: Lessons Working with Consumers

The first three lessons refer to the way DMHS can improve access to care and serve a broad section of the community, and how they deliver services other than treatment.

3.1.1. Lesson 1: DMHS Can Improve Access to Care for Those Who Really Need Care

One of the most profound lessons we learned is that DMHS improve access to mental health services for many people who would not otherwise seek care. For example, at least a third of the users of both MindSpot and OTU report that they have not previously spoken to a health professional about symptoms and 80% of MindSpot and 55% of OTU users are not using other mental health services at the time of assessment [28,53].
The users of these services often have chronic and disabling symptoms. A third of MindSpot users have been troubled by their symptoms for between one and five years with a further third reporting symptoms for more than six years. Among users of the OTU, 45% report symptoms for two or more years and those in paid employment report an average of 7 days off work in the last 30 days due to symptoms. Moreover, the mean symptom scores reported by users of MindSpot are in the moderate–severe range, a quarter report suicidal thoughts and 2.4% disclose suicidal plans.
We were also surprised at how people use our DMHS. While most users access content online, many others download information and refer to it when they do not have internet access [54]. In addition, for those who prefer or require printed materials, we send all the treatment course content in printed form [28]. For these reasons, we sometimes think of our services as delivering virtual rather than digital care, however, we are reluctant to introduce new terms to describe this already definitionally challenged field.

3.1.2. Lesson 2: DMHS Deliver More than Treatment

Contrary to our expectations, not all consumers using our DMHS seek treatment, especially in Australia, where most users report they are primarily seeking a confidential assessment and recommendations about treatment options. This should not have come as a complete surprise, given the high proportion who had not previously sought treatment. However, we learned that for many people, the assessment itself serves as a brief but helpful clinical intervention, particularly when the consumer and therapist create a shared clinical formulation, discuss treatment options that include self-help strategies, and explore barriers and facilitators to recovery. This assessment process is highly regarded by the patients of both clinics.
Consumers also frequently report enormous difficulty understanding and navigating the existing mental health service eco-system and some report using our services to seek advice about other health services, a theme which we return to below. Consequently, our second lesson is that DMHS need to offer a range of services in addition to online treatment, including information, assessment, triage and referral to other services. It should be noted that there are differences between our two clinics, with more people using the OTU reporting they are seeking treatment compared with users of MindSpot [55]. Thus, the specific services offered by DMHS may vary between jurisdictions, possibly reflecting differences in how the clinics are perceived and promoted, and differences in the needs of the local community.

3.1.3. Lesson 3: DMHS are used by a Broad Cross Section of the Population

Our third lesson is that a diverse cross section of our communities access our services, including indigenous Australians and Canadians [56], people on low incomes, people living in rural and remote regions, and other groups who often under-utilise traditional health services. We stress that our DMHS are not a panacea in this regard. Instead, we note that the widespread use of internet-enabled devices to access a range of services, including education, banking, entertainment and other domains, has extended to healthcare. In a similar way, DMHS have considerable potential to reduce, to some degree, the inequality of access to mental health.
A striking example is the ability of DMHS to reach people living in rural areas. Almost 40% of MindSpot users report living outside major metropolitan areas, with many living in rural or remote parts of Australia, including islands off the mainland. Similarly, 28% of OTU users are from rural locations and 32% from small cities. Collectively, these people live in locations where access to health services is limited or sometimes non-existent [55]. A further example relates to engagement by older adults. Across both clinics, approximately 6% of users are over the age of 60, which is a group that often experience difficulty accessing mental health services, including for reasons related to physical impairments. The experience at both clinics is that older adults strongly engage in treatments and often obtain large improvements in symptoms [42].
A final example relates to socio-economic disadvantage. A recent analysis of MindSpot data found that users come from all socio-economic backgrounds, including 33% from the lower four deciles of socio-economic status, a group who are more prone to experience a disadvantage and difficulties accessing mental health services.

3.2. Level 2: Lessons Working with Therapists

Both clinics have now trained large numbers of therapists, previously experienced in delivering face-to-face care, to deliver DMHS. Key lessons have been that therapists working in DMHS require specialised skills and therefore, require specialised training and supervision to acquire and maintain those skills.

3.2.1. Lesson 4: DMHS Require Specialised Therapist Skills

An important lesson is that the skills and knowledge required for effective delivery of DMHS are sufficiently different to those associated with traditional models of care to warrant specialised training and supervision [57]. Obvious examples include the need for DMHS therapists to become competent in the use of clinical software platforms and in engaging with patients via telephone or text-based communication, including responding appropriately to very long messages or technical questions, skills that are not taught in most clinical training programs [23,40,58].
However, there are less obvious reasons for providing specialised training and supervision. First, therapists new to DMHS often intellectually understand that DMHS treatments can result in similar outcomes to face-to-face treatment [10,59] but may initially expect DMHS treatments to produce poorer outcomes, a sentiment they may communicate to consumers [23]. New therapists also often find that their assumptions about mechanisms or facilitators of recovery are challenged, particularly when they learn that DMHS patients can develop very strong therapeutic alliances [60] and may obtain large clinical improvements, even after choosing not to have regular therapist contact [55]. Related to this, therapists are often surprised to learn that the structured educational aspects and resources associated with DMHS offer significant advantages over traditional clinical care where such resources may not be used.
A second reason for providing specialised training and supervision is to support DMHS therapists to successfully process the transference and other dynamics that occur when delivering DMHS to large groups of patients at once. An example of this is the temporary increase in symptoms experienced by many consumers at mid-treatment when they begin to apply skills learned in treatment in their everyday lives. These lapses usually resolve and most continue to recover. This process is familiar to experienced therapists, who can help patients understand this trajectory and can manage their own reactions. However, the effect can be magnified by the large numbers of patients in each treatment cohort, leading to feelings of intense elation or sometimes disillusionment, particularly when some patients choose not to engage with the therapist but have not made this clear at the outset. A strong framework of training and supervision can assist therapists to understand and adapt to these patterns and maintain confidence in both their own performance and the effectiveness of the treatment programs.

3.2.2. Lesson 5: DMHS Require Specialised Clinical Processes

This lesson reflects important differences in the clinical procedures and processes used in DMHS compared to traditional mental health clinics. An obvious example is the use of structured ICBT interventions, questionnaires and outcome measures used in DMHS, the delivery of which are governed by procedures which regulate therapists’ actions more than they would in typical in face-to-face services.
This level of structure reflects how DMHS attempt to manage both quality assurance and treatment to large numbers of consumers. Therapists experienced in working at DMHS can efficiently deliver individualised care within these structured frameworks, but this is more difficult for less experienced therapists. Another lesson has been the importance of robust systems for not only training and supervision [58,61], but also the recruitment and retention of clinicians who are comfortable with relatively high levels of structure and process [23].
Another example of how DMHS differ from traditional face-to-face mental health services, at least in our jurisdictions, relates to the use of PROMS and patient-reported experience measures (PREMS) [62,63,64]. Despite the documented utility of PROMS and PREMS in clinical care, they are infrequently used in traditional services, and rarely as a therapeutic tool for guiding discussions or decisions about treatment or as a method for improving the quality of care. Since our clinics routinely administer PROMS and PREMS during and after treatment, we provide specific training for new therapists to increase their comfort and competence in using measures of outcome and experience [61].

3.3. Level 3: Lessons Operating Services

3.3.1. Lesson 6: The Operation of DMHS Require Specialised Systems and Skills

This lesson is obvious, but we include it here because we underestimated the complexity of developing and delivering safe and effective DMHS. We expected that our DMHS would be similar to traditional face-to-face clinical mental health services or an extension of the operations used in our large-scale clinical research trials. However, we quickly learned that safely and effectively operating DMHS required attention in at least four areas.
First, DMHS require robust procedures to define and effectively manage safety risks for people presenting with more severe and complex needs than seen in our clinical trials [30] and who often live in remote locations. This requires developing expertise in evaluation of risk via telephone or online communication, the ability to contact emergency services that are available near where the patient is located, how to refer to such services, and how to stay abreast of changes in their referral and contact details.
Second, although operating DMHS involve similar skills as those required for operating traditional mental health services, including management, human resources, marketing and IT [25], we found that DMHS were sufficiently different to warrant employing people with additional expertise, including skills relevant to telehealth, social media and online marketing.
Third, and in addition to the urgent requirement for establishing robust systems of organizational and clinical governance we recognised that in order to effectively lead our services, we personally needed to develop commercial and management skills, domains in which we, as clinical researchers, had little or no experience. We also needed to address challenges relating to regulation; for example at MindSpot, we needed to determine which of the myriad of possible regulatory frameworks applied to our activities [65] given that most [66] had been developed for traditional face-to-face services. Within the OTU, a similar issue requiring attention was how to meet the different regulatory requirements for services provided by therapists from psychology and social work.
Establishing, maintaining, and subsequently improving our operational systems has required a considerable work effort. Whilst we have recruited specialist staff to assist with such efforts, due to the novelty of DMHS, we often also sometimes found it necessary to train and develop our own staff in these operational and managerial roles.

3.3.2. Lesson 7: Digital Mental Health Clinics Evolve

This lesson represents another difference between DMHS and traditional clinical services. Traditional services may change frequently, but the change is usually limited to organisational structure or branding, with less frequent changes to the service or delivery models. By contrast, our DMHS regularly undergo significant changes in procedures, systems, and even service delivery models due to developments in research, technology, the changing expectations of consumers, and changes in the policy priorities of the funding bodies. However, the most frequent changes stem from reviewing our outcomes and procedures.
For example, within the OTU, changes within recent years include (1) replacing disorder-specific ICBT programs with a transdiagnostic treatment program in light of the extensive comorbidity found among patients seeking services and the efficiency of delivery compared to disorder-specific programs [24], (2) ongoing trials to determine the best level of support and specialisation by therapists [40,67], and (3) the expansion of services to address other needs in the community, such as ICBT for pain [22].
Regular change has significant implications for the operation of our clinics. For example, management needs systems for reliably collecting and analysing data, the ability to develop and test alternative models, and the skills and authority to make decisions. The individuals given the task of implementing change need project leadership and change management skills and procedures to plan and deploy changes. Furthermore, therapists and other staff need to be prepared for and willing to implement changes. This means that all staff members need to be adaptable and change needs to become part of the culture of the DMHS.

3.4. Level 4: Lessons Working with Health Systems, Funders, and Policy Makers

This final group of lessons summarises key learnings derived from working with and influencing health systems, in particular, the future role and value of DMHS within health systems.

3.4.1. Lesson 8: Integrating DMHS within Health Systems Is Challenging

Our DMHS reside within enormously complex health systems which might be more accurately described as interconnected nodes of care rather than true systems. For example, mental health services, while all purporting to share the aim of improving mental wellbeing, often target different groups and may be accountable to different policy and regulatory frameworks. Health services also differ with respect to funding, and in both Canada and Australia, funding for different types of mental health services can be provided by the federal government, state/provincial government, state mental health commissions, non-governmental organisations and individual consumers. As a result, mental health services are often fragmented, poorly connected, and difficult for consumers to navigate.
The complexity of health systems and their resistance to change created a number of threats to the sustainability and stability of our DMHS, particularly in our early years of service delivery. We were able to overcome such challenges by building strong relationships with key stakeholders and in particular, by publishing outcome data that documented the value of the services. This data has also assisted in defining the role of DMHS in the broader mental healthcare system, including as services which improve access to patients who would otherwise not access mental health services.
These so-called external-facing activities have required considerable time and effort. Participation in such activities has required us to learn the sometimes-subtle rules of engaging with other organisations, to commit to regular participation in networking activities, and to make frequent efforts to build and maintain collaborations. Such activities can be enormously time consuming but we have found that they are an essential component of successfully delivering our DMHS.

3.4.2. Lesson 9: DMHS May Change the Mental Health System

By providing services to large numbers of consumers and because of our routine collection of outcome data, our clinics are having a growing influence on the health system and are increasingly seen as agents or examples of change. In Canada, the experiences and activities of the OTU have influenced the development of e-mental health in several ways [68]. In Australia, data from MindSpot has helped inform long-term government planning and funding strategies for mental health services [69], not only DMHS. The data and outcome driven reports prepared by our services are often in marked contrast to submissions by other groups, which may be based more on opinion rather than on evidence. Hence, an important role of DMHS is to inform policy makers and funders to help improve the broader mental health system by presenting data drawn from a broad cross section of the community and also to nudge traditional services to adopt systems of measurement and reporting of outcomes.

3.4.3. Lesson 10: DMHS Are Not a Panacea

We are struck by how often mental health funders and policy makers, when presented with the evidence from our DMHS, become enthusiastic about their potential without an appreciation of their limits. These limits include the so-called digital divide, that is, the group in society who does not use digital devices, those with very low levels of literacy, and those in crisis, who can benefit from contact with DMHS, but may be better off with a mental health service that includes direct human interaction, as well as those who prefer to see someone face-to-face.
In all our communication, we emphasise that DMHS should complement and not replace existing services. We also emphasise that attention must be paid to systematically evaluating delivery methods that combine the best elements of DMHS with traditional services, including blended care [70,71,72,73,74]. We also stress that consumer knowledge of DMHS is still limited and that even brief education about DMHS can improve consumer perceptions and uptake of services [75]. These observations lead to our final lesson that whilst acknowledging that DMHS can significantly improve access to safe, clinical and cost-effective care, our DMHS are not a panacea.

4. General Discussion

This paper aimed to assist other emerging DMHS by sharing ten lessons we learned from successfully delivering DMHS to very large numbers of consumers. Some of these lessons might seem obvious, but their importance was not always apparent when we started our services. Several key themes are discussed below, followed by recommendations.
One theme is that we expect that demand for this service model will grow. The number of patients treated using ICBT in the OTU has more than doubled in the past four years. The threshold for accessing this model of care is significantly lower than traditional face-to-face services and consumers are becoming increasingly comfortable with using technology to access a broad range of services, including health services. Along with a growth in numbers, we expect that existing DMHS will become more tailored to different populations, for example, people in certain occupations, different cultures, or who have been referred from different pathways, although our experience is that the extent to which the treatment course materials need to be customised is considerably less than expected [56,76,77,78].
A second theme is that the workforce requires specialised training, clinical supervision and support. This raises broader issues about workforce planning and training programs. We note that many professional bodies have recognised the importance of education and training of mental health professionals and standards in this model of care [79,80,81,82] but that few training programs in any mental health discipline offer courses or training opportunities specifically for digital mental health. The absence of such training opportunities poses significant risks for the future sustainability and quality of the field.
A third theme is that the delivery of DMHS requires specialist skills in both clinical and operational domains. We also note that although the costs to entry of developing a DMHS, especially a low volume service, might be relatively low, the costs of maintaining quality services can be high. Inadequate funding and inadequate organisational governance can affect the reputation, credibility, and therefore, the potential of the emerging field of DMHS [83]. Hence, we strongly encourage anyone seeking to launch a DMHS to carefully consider the governance frameworks that will ensure the safe and sustainable delivery of services, or to consider licensing their interventions to groups who have proven success in implementing similar services.
Another theme is that the field of DMHS is rapidly evolving. We encourage those seeking to start a DMHS to consider trialing different models of care to those currently used by existing DMHS, including testing different levels of therapist support [24,40,55] and testing care which combines both face-to-face and online delivery. We note the important work conducted by our European colleagues on blended care [70,71,72,73,74] and by others on mobile services [13] and encourage collaboration in order to collectively develop the most effective models of care.
Our final theme relates to recognising the true value proposition of DMHS. We maintain that they are not a panacea but instead serve several valuable functions, including as a useful complement to existing services, as a way of improving equity of access to mental health care for common psychological disorders, and as a stepping stone to other services. Over-promising may increase the likelihood of short-term funding, but poorly designed and delivered services might harm consumers, disappoint stakeholders and risk the future of DMHS.

4.1. Recommendations

These observations lead to several recommendations which we encourage those contemplating developing DMHS to consider. First, we recommend that new DMHS recruit not only appropriately skilled therapists, but also people with commercial and professional skills, ideally with experience in digital service delivery. Second, given the unique challenges of DMHS, we recommend the development of both thorough initial training of therapists, as well as of systems for ongoing training and supervision. Third, given the likelihood that demand for DMHS will grow, we strongly encourage that organisations involved in training and certification of mental health professionals add content and training opportunities relevant to the competencies required in DMHS.
Fourth, we recommend that emerging DMHS measure and publish their outcomes, including disappointing and negative effects outcomes [84]. We also encourage DMHS to engage with policy makers and funders to develop mental health policy grounded in evidence rather than in opinion. Finally, we strongly encourage DMHS to engage with their consumers in appropriate co-design and evaluation activities to ensure services are not only effective but acceptable to consumers.

4.2. Strengths and Limitations

We believe that the main strength of this paper stems from the authors’ shared experience in launching and steadily improving successful high volume DMHS. However, we acknowledge several weaknesses, including that the list of lessons is non-exhaustive and did not include some of the significant challenges associated with managing funding insecurity or bureaucratic and professional challenges within the field of mental health, topics we will return to in a subsequent publication. We also acknowledge that our experiences may not reflect those of other DMHS.

4.3. Conclusions

This paper described ten key lessons learned by the authors when developing, delivering, and evaluating DMHS. Despite the challenges, provided they are delivered safely, effectively and with strong clinical, operational and organisational governance, we remain highly optimistic about the potential of DMHS to reduce the global burden of the high prevalence of mental disorders.

Author Contributions

Conceptualisation, N.T. and B.F.D.; Writing—Original Draft Preparation, N.T., H.D.H., O.N., and B.F.D.; Writing—Review & Editing, N.T., H.D.H., O.N., D.C.M., G.A. and B.F.D.

Acknowledgments

The Online Therapy Unit is funded by Saskatchewan Ministry of Health. Research conducted by the Unit is currently funded by the Canadian Institutes of Health Research (152917), the Saskatchewan Health Research Foundation, and the Saskatchewan Centre for Patient-Oriented Research. The MindSpot Clinic is funded by the Australian Department of Health and supported by MQ Health, Macquarie University. The authors gratefully acknowledge their funders, supporting institutions, the patients for allowing the use of their data, and their clinical, management, and technical teams for their efforts in launching and operating the Clinics. We also acknowledge the achievements and leadership shown by early and current researchers in the field of psychological internet interventions. Their work has strongly influenced the models and outcomes reported here and continue to inspire our attempts at reducing barriers to evidence-based psychological services.

Conflicts of Interest

H.D.H. is funded by Saskatchewan Ministry of Health to operate the Online Therapy Unit. Research conducted by the Unit is currently funded by the Canadian Institutes of Health Research (152917), the Saskatchewan Health Research Foundation, and the Saskatchewan Centre for Patient-Oriented Research. Funders had no involvement in the design of the paper, collection, analysis, or interpretation of the data. N.T. and B.F.D. are funded by the Australian Government to operate the national MindSpot Clinic. O.N. is a member of a Lundbeck advisory board for an antipsychotic medication. The other authors report no financial relationships with commercial interests. This paper was investigator initiated. It was funded by Departmental funds from the authors’ university, which had no role in the design, execution, interpretation, or writing of the study.

References

  1. Kessler, R.C.; Aguilar-Gaxiola, S.; Alonso, J.; Chatterji, S.; Lee, S.; Ormel, J.; Üstün, T.B.; Wang, P.S. The global burden of mental disorders: An update from the WHO World Mental Health (WMH) surveys. Epidemiol. E Psichiatr. Soc. 2009, 18, 23–33. [Google Scholar] [CrossRef]
  2. Whiteford, H.A.; Degenhardt, L.; Rehm, J.; Baxter, A.J.; Ferrari, A.J.; Erskine, H.E.; Charlson, F.J.; Norman, R.E.; Flaxman, A.D.; Johns, N.; et al. Global burden of disease attributable to mental and substance use disorders: Findings from the Global Burden of Disease Study 2010. Lancet 2013, 382, 1575–1586. [Google Scholar] [CrossRef]
  3. Urbanoski, K.; Inglis, D.; Veldhuizen, S. Service Use and Unmet Needs for Substance Use and Mental Disorders in Canada. Can J. Psychiat. 2017, 62, 551–559. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Wang, P.S.; Matthias, A.; Borges, G.; Bruffaerts, R.; Chiu, T.; De Girolamo, W.; Fayyad, G.; John, G.; Ronald, C.; Kovess, V.; et al. Steven Delay and failure in treatment seeking after first onset of mental disorders in the World Health Organization’s World Mental Health Survey Initiative. World Psychiatry. 2007, 6, 177–185. [Google Scholar] [PubMed]
  5. Whiteford, H.A.; Buckingham, W.J.; Harris, M.G.; Burgess, P.M.; Pirkis, J.E.; Barendregt, J.J.; Hall, W.D. Estimating treatment rates for mental disorders in Australia. Aust. Health Rev. 2014, 38, 80–85. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Mojtabai, R.; Olfson, M.; Sampson, N.A.; Jin, R.; Druss, B.; Wang, P.S.; Wells, K.B.; Pincus, H.A.; Kessler, R.C. Barriers to mental health treatment: Results from the National Comorbidity Survey Replication. Psychol. Med. 2011, 41, 1751–1761. [Google Scholar] [CrossRef] [PubMed]
  7. Andrews, G.; Cuijpers, P.; Craske, M.G.; McEvoy, P.; Titov, N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: A meta-analysis. PLoS ONE 2010, 5, e13196. [Google Scholar] [CrossRef]
  8. Andersson, G.; Titov, N. Advantages and limitations of Internet-based interventions for common mental disorders. World Psychiatry Off. J. World Psychiatr. Assoc. 2014, 13, 4–11. [Google Scholar] [CrossRef]
  9. Andersson, G.; Carlbring, P.; Titov, N.; Lindefors, N. Internet Interventions for Adults with Anxiety and Mood Disorders: A Narrative Umbrella Review of Recent Meta-Analyses. Can. J. Psychiatry 2019, 64, 465–470. [Google Scholar] [CrossRef]
  10. Carlbring, P.; Andersson, G.; Cuijpers, P.; Riper, H.; Hedman-Lagerlöf, E. Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: An updated systematic review and meta-analysis. Cogn. Behav. Ther. 2018, 47, 1–18. [Google Scholar] [CrossRef]
  11. Gilbody, S.; Littlewood, E.; Hewitt, C.; Brierley, G.; Tharmanathan, P.; Araya, R.; Barkham, M.; Bower, P.; Cooper, C.; Gask, L.; et al. Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): Large scale pragmatic randomised controlled trial. BMJ 2015. [Google Scholar] [CrossRef]
  12. Kenter, R.M.F.; Van De Ven, P.M.; Cuijpers, P.; Koole, G.; Niamat, S.; Gerrits, R.S.; Willems, M.; Van Straten, A. Costs and effects of Internet cognitive behavioral treatment blended with face-to-face treatment: Results from a naturalistic study. Internet Interv. 2015, 2, 77–83. [Google Scholar] [CrossRef] [Green Version]
  13. Mohr, D.C.; Weingardt, K.R.; Reddy, M.; Schueller, S.M. Three Problems with Current Digital Mental Health Research and Three Things We Can Do About Them. Psychiatr. Serv. 2017, 68, 427–429. [Google Scholar] [CrossRef]
  14. El Alaoui, S.; Hedman, E.; Kaldo, V.; Hesser, H.; Kraepelien, M.; Andersson, E.; Rück, C.; Andersson, G.; Ljótsson, B.; Lindefors, N. Effectiveness of Internet-based cognitive–behavior therapy for social anxiety disorder in clinical psychiatry. J. Consult. Clin. Psychol. 2015, 83, 902. [Google Scholar] [CrossRef]
  15. Hedman, E.; Ljótsson, B.; Kaldo, V.; Hesser, H.; El Alaoui, S.; Kraepelien, M.; Andersson, E.; Rück, C.; Svanborg, C.; Andersson, G.; et al. Effectiveness of Internet-based cognitive behaviour therapy for depression in routine psychiatric care. J. Affect. Disord. 2014, 155, 49–58. [Google Scholar] [CrossRef]
  16. Hedman, E.; Ljótsson, B.; Rück, C.; Bergström, J.; Andersson, G.; Kaldo, V.; Jansson, L.; Andersson, E.; Andersson, E.; Blom, K.; et al. Effectiveness of Internet-based cognitive behaviour therapy for panic disorder in routine psychiatric care. Acta Psychiatri. Scand. 2013, 128, 457–467. [Google Scholar] [CrossRef]
  17. Hedman, E.; Ljótsson, B.; Andersson, E.; Rück, C.; Andersson, G.; Lindefors, N. Effectiveness and cost offset analysis of group CBT for hypochondriasis delivered in a psychiatric setting: An open trial. Cogn. Behav. Ther. 2010, 39, 239–250. [Google Scholar] [CrossRef]
  18. Ruwaard, J.; Lange, A.; Schrieken, B.; Dolan, C.V.; Emmelkamp, P. The Effectiveness of Online Cognitive Behavioral Treatment in Routine Clinical Practice. Plos One 2012, 7, e40089. [Google Scholar] [CrossRef]
  19. Ruwaard, J.; Lange, A.; Schrieken, B.; Emmelkamp, P.M. Efficacy and effectiveness of online cognitive behavioral treatment: A decade of interapy research. Stud. Health Technol. Inform. 2011, 167, 9–14. [Google Scholar]
  20. Folker, A.P.; Mathiasen, K.; Lauridsen, S.M.; Stenderup, E.; Dozeman, E.; Folker, M.P. Implementing internet-delivered cognitive behavior therapy for common mental health disorders: A comparative case study of implementation challenges perceived by therapists and managers in five European internet services. Internet Interv. 2018, 11, 60–70. [Google Scholar] [CrossRef]
  21. Mathiasen, K.; Riper, H.; Ehlers, L.H.; Valentin, J.B.; Rosenberg, N.K. Internet-based CBT for social phobia and panic disorder in a specialised anxiety clinic in routine care: Results of a pilot randomised controlled trial. Internet Interv. 2016, 4, 92–98. [Google Scholar] [CrossRef] [Green Version]
  22. Hadjistavropoulos, H.D.; Schneider, L.H.; Hadjistavropoulos, T.; Titov, N.; Dear, B.F. Effectiveness, acceptability and feasibility of an Internet-delivered cognitive behavioral pain management program in a routine online therapy clinic in Canada. Can. J. Pain 2018, 2, 62–73. [Google Scholar] [CrossRef] [Green Version]
  23. Hadjistavropoulos, H.; Nugent, M.; Dirkse, D.; Pugh, N. Implementation of internet-delivered cognitive behavior therapy within community mental health clinics: A process evaluation using the consolidated framework for implementation research. BMC Psychiatry 2017, 17, 331. [Google Scholar] [CrossRef]
  24. Hadjistavropoulos, H.; Nugent, M.; Alberts, N.; Staples, L.; Dear, B.; Titov, N. Transdiagnostic Internet-delivered cognitive behaviour therapy in Canada: An open trial comparing results of a specialized online clinic and nonspecialized community clinics. J. Anxiety Disord. 2016, 42, 19–29. [Google Scholar] [CrossRef] [Green Version]
  25. Hadjistavropoulos, H.D.; Thompson, M.; Ivanov, M.; Drost, C.; Butz, C.J.; Klein, B.; Austin, D.W. Considerations in the Development of a Therapist-Assisted Internet Cognitive Behavior Therapy Service. Prof. Psycol. Res. Pr. 2011, 42, 463–471. [Google Scholar] [CrossRef]
  26. Newby, J.M.; Mackenzie, A.; Williams, A.D.; McIntyre, K.; Watts, S.; Wong, N.; Andrews, G. Internet cognitive behavioural therapy for mixed anxiety and depression: A randomized controlled trial and evidence of effectiveness in primary care. Psychol. Med. 2013, 43, 2635–2648. [Google Scholar] [CrossRef]
  27. Williams, A.D.; Andrews, G. The Effectiveness of Internet Cognitive Behavioural Therapy (iCBT) for Depression in Primary Care: A Quality Assurance Study. PloS One 2013, 8, e57447. [Google Scholar] [CrossRef]
  28. Titov, N.; Dear, B.F.; Staples, L.G.; Bennett-Levy, J.; Klein, B.; Rapee, R.M.; Andersson, G.; Purtell, C.; Bezuidenhout, G.; Nielssen, O.B. The first 30 months of the MindSpot Clinic: Evaluation of a national e-mental health service against project objectives. Aust. New Zealand J. Psychiatry 2017, 51, 1227–1239. [Google Scholar] [CrossRef]
  29. Titov, N.; Dear, B.F.; Staples, L.G.; Bennett-Levy, J.; Klein, B.; Rapee, R.M.; Shann, C.; Richards, D.; Andersson, G.; Ritterband, L.; et al. MindSpot Clinic: An Accessible, Efficient, and Effective Online Treatment Service for Anxiety and Depression. Psychiatr. Serv. 2015, 66, 1043–1050. [Google Scholar] [CrossRef]
  30. Nielssen, O.; Dear, B.F.; Staples, L.G.; Dear, R.; Ryan, K.; Purtell, C.; Titov, N. Procedures for risk management and a review of crisis referrals from the MindSpot Clinic, a national service for the remote assessment and treatment of anxiety and depression. BMC Psychiatry 2015, 15, 304. [Google Scholar] [CrossRef]
  31. Mathiasen, K.; Riper, H.; Andersen, T.E.; Roessler, K.K. Evaluation of Clinical Effect in Routine Secondary Care of Guided Internet Based CBT for Adult Depression and Anxiety: An Observational Study. JMIR Preprints. 2018. [Google Scholar] [CrossRef]
  32. Vis, C.; Mol, M.; Kleiboer, A.; Bührmann, L.; Finch, T.; Smit, J.; Riper, H. Improving Implementation of eMental Health for Mood Disorders in Routine Practice: Systematic Review of Barriers and Facilitating Factors. JMIR Ment. Health 2018, 5, e20. [Google Scholar] [CrossRef]
  33. Vis, C.; Ruwaard, J.; Finch, T.; Rapley, T.; de Beurs, D.; van Stel, H.; van Lettow, B.; Mol, M.; Kleiboer, A.; Riper, H.; et al. Toward an Objective Assessment of Implementation Processes for Innovations in Health Care: Psychometric Evaluation of the Normalization Measure Development (NoMAD) Questionnaire Among Mental Health Care Professionals. J. Med. Int. Res. 2019, 21, e12376. [Google Scholar] [CrossRef]
  34. ImpleMentAll: Getting eHealth implementation right. Available online: https://www.implementall.eu/ (accessed on 25 June 2019).
  35. Titov, N.; Dear, B.; Nielssen, O.; Staples, L.; Hadjistavropoulos, H.; Nugent, M.; Adlam, K.; Nordgreen, T.; Bruvik, K.H.; Hovland, A.; et al. ICBT in routine care: A descriptive analysis of successful clinics in five countries. Internet Interv. 2018, 13, 108–115. [Google Scholar] [CrossRef]
  36. Damschroder, L.J.; Aron, D.C.; Keith, R.E.; Kirsh, S.R.; Alexander, J.A.; Lowery, J.C. Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implement. Sci. 2009, 4, 50. [Google Scholar] [CrossRef]
  37. Gaglio, B.; Shoup, J.A.; Glasgow, R.E. The RE-AIM framework: A systematic review of use over time. Am. J. Public Health 2013, 103, e38–e46. [Google Scholar] [CrossRef]
  38. Staples, L.G.; Dear, B.F.; Johnson, B.; Fogliati, V.; Gandy, M.; Fogliati, R.; Nielssen, O.; Titov, N. Internet-delivered treatment for young adults with anxiety and depression: Evaluation in routine clinical care and comparison with research trial outcomes. J. Affect. Disord. 2019, 256, 103–109. [Google Scholar] [CrossRef]
  39. Dear, B.F.; Fogliati, V.J.; Fogliati, R.; Johnson, B.; Boyle, O.; Karin, E.; Gandy, M.; Kayrouz, R.; Staples, L.G.; Titov, N. Treating anxiety and depression in young adults: A randomised controlled trial comparing clinician-guided versus self-guided Internet-delivered cognitive behavioural therapy. Aust. New Zealand J. Psychiatry 2018, 52, 668–679. [Google Scholar] [CrossRef]
  40. Hadjistavropoulos, H.D.; Schneider, L.H.; Edmonds, M.; Karin, E.; Nugent, M.N.; Dirkse, D.; Dear, B.F.; Titov, N. Randomized controlled trial of internet-delivered cognitive behaviour therapy comparing standard weekly versus optional weekly therapist support. J. Anxiety Disord. 2017, 52, 15–24. [Google Scholar] [CrossRef]
  41. Fogliati, V.J.; Dear, B.F.; Staples, L.G.; Terides, M.D.; Sheehan, J.; Johnston, L.; Kayrouz, R.; Dear, R.; McEvoy, P.M.; Titov, N. Disorder-specific versus transdiagnostic and clinician-guided versus self-guided internet-delivered treatment for panic disorder and comorbid disorders: A randomized controlled trial. J. Anxiety Disord. 2016, 39, 88–102. [Google Scholar] [CrossRef] [Green Version]
  42. Staples, L.G.; Fogliati, V.J.; Dear, B.F.; Nielssen, O.; Titov, N. Internet-delivered treatment for older adults with anxiety and depression: Implementation of the Wellbeing Plus Course in routine clinical care and comparison with research trial outcomes. Brit. J. Psychiat. Open 2016, 2, 307–313. [Google Scholar] [CrossRef]
  43. Dear, B.; Staples, L.; Terides, M.; Fogliati, V.; Sheehan, J.; Johnston, L.; Kayrouz, R.; Dear, R.; McEvoy, P.; Titov, N. Transdiagnostic versus disorder-specific and clinician-guided versus self-guided internet-delivered treatment for social anxiety disorder and comorbid disorders: A randomized controlled trial. J. Anxiety Disord. 2016, 42, 30–44. [Google Scholar] [CrossRef]
  44. Titov, N.; Dear, B.F.; Staples, L.G.; Terides, M.D.; Karin, E.; Sheehan, J.; Johnston, L.; Gandy, M.; Fogliati, V.J.; Wootton, B.M.; et al. Disorder-specific versus transdiagnostic and clinician-guided versus self-guided treatment for major depressive disorder and comorbid anxiety disorders: A randomized controlled trial. J. Anxiety Disord. 2015, 35, 88–102. [Google Scholar] [CrossRef] [Green Version]
  45. Titov, N.; Fogliati, V.J.; Staples, L.G.; Gandy, M.; Johnston, L.; Wootton, B.; Nielssen, O.; Dear, B.F. Treating anxiety and depression in older adults: randomised controlled trial comparing guided v. self-guided internet-delivered cognitive–behavioural therapy. Brit. J. Psychiat. Open 2016, 2, 50–58. [Google Scholar] [CrossRef]
  46. Titov, N.; Dear, B.F.; Johnston, L.; McEvoy, P.M.; Wootton, B.; Terides, M.D.; Gandy, M.; Fogliati, V.; Kayrouz, R.; Rapee, R.M. Improving adherence and clinical outcomes in self-guided internet treatment for anxiety and depression: A 12-month follow-up of a randomised controlled trial. PLoS ONE 2014, 9, e89591. [Google Scholar] [CrossRef]
  47. Titov, N.; Dear, B.F.; Johnston, L.; Lorian, C.; Zou, J.; Wootton, B.; Spence, J.; McEvoy, P.M.; Rapee, R.M. Improving adherence and clinical outcomes in self-guided internet treatment for anxiety and depression: Randomised controlled trial. PloS One 2013, 8, e62873. [Google Scholar] [CrossRef]
  48. Wootton, B.M.; Dear, B.F.; Johnston, L.; Terides, M.D.; Titov, N. Self-guided internet-delivered cognitive behavior therapy (iCBT) for obsessive–compulsive disorder: 12 month follow-up. Internet Interv. 2015, 2, 243–247. [Google Scholar] [CrossRef]
  49. Wootton, B.M.; Dear, B.F.; Johnston, L.; Terides, M.D.; Titov, N. Remote treatment of obsessive-compulsive disorder: A randomized controlled trial. J. Obs. Compuls. Relat. Disord. 2013, 2, 375–384. [Google Scholar] [CrossRef]
  50. Spence, J.; Titov, N.; Johnston, L.; Jones, M.P.; Dear, B.F.; Solley, K. Internet-based trauma-focused cognitive behavioural therapy for PTSD with and without exposure components: A randomised controlled trial. J. Affect. Disord. 2014, 162, 73–80. [Google Scholar] [CrossRef]
  51. Dear, B.F.; Titov, N.; Perry, K.N.; Johnston, L.; Wootton, B.M.; Terides, M.D.; Rapee, R.M.; Hudson, J.L. The Pain Course: A randomised controlled trial of a clinician-guided Internet-delivered cognitive behaviour therapy program for managing chronic pain and emotional well-being. Pain 2013, 154, 942–950. [Google Scholar] [CrossRef]
  52. Dear, B.; Staples, L.; Terides, M.; Karin, E.; Zou, J.; Johnston, L.; Gandy, M.; Fogliati, V.; Wootton, B.; McEvoy, P. Transdiagnostic versus disorder-specific and clinician-guided versus self-guided internet-delivered treatment for generalized anxiety disorder and comorbid disorders: A randomized controlled trial. J. Anxiety Disord. 2015, 36, 63–77. [Google Scholar] [CrossRef] [Green Version]
  53. Hadjistavropoulos, H.D. Online Therapy Unit Outcome Report 2013-2017; University of Regina: Regina, SK, Canada, 2018. [Google Scholar]
  54. Hadjistavropoulos, H.D.; Faller, Y.N.; Klatt, A.; Nugent, M.N.; Dear, B.F.; Titov, N. Patient Perspectives on Strengths and Challenges of Therapist-Assisted Internet-Delivered Cognitive Behaviour Therapy: Using the Patient Voice to Improve Care. Community Ment. Health J. 2018, 54, 944–950. [Google Scholar] [CrossRef] [Green Version]
  55. Hadjistavropoulos, H.D.; Schneider, L.H.; Mehta, S.; Karin, E.; Dear, B.F.; Titov, N. Preference trial of internet-delivered cognitive behaviour therapy comparing standard weekly versus optional weekly therapist support. J. Anxiety Disord. 2019, 63, 51–60. [Google Scholar] [CrossRef]
  56. Titov, N.; Schofield, C.; Staples, L.; Dear, B.F.; Nielssen, O. A comparison of Indigenous and non-Indigenous users of MindSpot: An Australian digital mental health service. Australas. Psychiatry 2018, 27, 352–357. [Google Scholar] [CrossRef]
  57. Hadjistavropoulos, H.D.; Thompson, M.J.; Klein, B.; Austin, D.W. Dissemination of Therapist-Assisted Internet Cognitive Behaviour Therapy: Development and Open Pilot Study of a Workshop. Cog. Behav. Ther. 2012, 41, 230–240. [Google Scholar] [CrossRef]
  58. Hadjistavropoulos, H.D.; Gullickson, K.M.; Schneider, L.H.; Dear, B.F.; Titov, N. Development of the Internet-Delivered Cognitive Behaviour Therapy Undesirable Therapist Behaviours Scale (ICBT-UTBS). Internet Interv. 2019. [Google Scholar] [CrossRef]
  59. Andersson, G.; Cuijpers, P.; Carlbring, P.; Riper, H.; Hedman, E. Guided Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: A systematic review and meta-analysis. World Psychiatry 2014, 13, 288–295. [Google Scholar] [CrossRef]
  60. Hadjistavropoulos, H.D.; Pugh, N.E.; Hesser, H.; Andersson, G. Therapeutic Alliance in Internet-Delivered Cognitive Behaviour Therapy for Depression or Generalized Anxiety. Clin. Psychol. Psychother. 2016, 24, 451–461. [Google Scholar] [CrossRef]
  61. Hadjistavropoulos, H.D.; Schneider, L.H.; Klassen, K.; Dear, B.F.; Titov, N. Development and evaluation of a scale assessing therapist fidelity to guidelines for delivering therapist-assisted Internet-delivered cognitive behaviour therapy. Cogn. Behav. Ther. 2018, 47, 447–461. [Google Scholar] [CrossRef]
  62. Hatfield, D.R.; Ogles, B.M. The Use of Outcome Measures by Psychologists in Clinical Practice. Prof. Psychol. Res. Pract. 2004, 35, 485–491. [Google Scholar] [CrossRef]
  63. Boswell, J.F.; Kraus, D.R.; Miller, S.D.; Lambert, M.J. Implementing routine outcome monitoring in clinical practice: Benefits, challenges, and solutions. Psychother. Res. 2015, 25, 6–19. [Google Scholar] [CrossRef]
  64. Jensen-Doss, A.; Haimes, E.M.B.; Smith, A.M.; Lyon, A.R.; Lewis, C.C.; Stanick, C.F.; Hawley, K.M. Monitoring Treatment Progress and Providing Feedback is Viewed Favorably but Rarely Used in Practice. Adm. Policy Ment. Health Ment. Health Serv. Res. 2018, 45, 48–61. [Google Scholar] [CrossRef]
  65. Australian Government. National Standards for Mental Health Services; Department of Health, Ed.; Australian Government: Canberra, Australian, 2010.
  66. Commonwealth of Australia. Quality Framework for Telephone Counselling and Internet-based Support Services; Department of Health and Ageing, Australian Government: Canberra, Australian, 2009.
  67. Hadjistavropoulos, H.D.; Pugh, N.E.; Hesser, H.; Andersson, G. Predicting Response to Therapist-Assisted Internet-Delivered Cognitive Behavior Therapy for Depression or Anxiety Within an Open Dissemination Trial. Behav. Ther. 2016, 47, 155–165. [Google Scholar] [CrossRef]
  68. Mental Health Commission of Canada. E-Mental Health in Canada: Transforming the Mental Health System Using Technology; Mental Health Commission of Canada: Ottawa, ON, Canada, 2014. [Google Scholar]
  69. National Mental Health Service Planning Framework. Available online: https://nmhspf.org.au/ (accessed on 27 June 2019).
  70. Urech, A.; Krieger, T.; Möseneder, L.; Biaggi, A.; Vincent, A.; Poppe, C.; Meyer, B.; Riper, H.; Berger, T. A patient post hoc perspective on advantages and disadvantages of blended cognitive behaviour therapy for depression: A qualitative content analysis. Psychother. Res. 2018, 31, 1–13. [Google Scholar] [CrossRef]
  71. Mol, M.; Dozeman, E.; Provoost, S.; van Schaik, A.; Riper, H.; Smit, J.H. Behind the Scenes of Online Therapeutic Feedback in Blended Therapy for Depression: Mixed-Methods Observational Study. J. Med Internet Res. 2018, 20, e174. [Google Scholar] [CrossRef] [Green Version]
  72. Erbe, D.; Eichert, H.C.; Riper, H.; Ebert, D.D. Blending face-to-face and internet-based interventions for the treatment of mental disorders in adults: Systematic review. J. Med Internet Res. 2017, 19, e306. [Google Scholar] [CrossRef]
  73. Mathiasen, K.; Andersen, T.E.; Riper, H.; Kleiboer, A.A.M.; Roessler, K.K. Blended CBT versus face-to-face CBT: A randomised non-inferiority trial. BMC Psychiatry 2016, 16, 432. [Google Scholar] [CrossRef]
  74. Kleiboer, A.; Smit, J.; Bosmans, J.; Ruwaard, J.; Andersson, G.; Topooco, N.; Berger, T.; Krieger, T.; Botella, C.; Baños, R.; et al. European COMPARative Effectiveness research on blended Depression treatment versus treatment-as-usual (E-COMPARED): Study protocol for a randomized controlled, non-inferiority trial in eight European countries. Trials 2016, 17, 387. [Google Scholar] [CrossRef]
  75. Soucy, J.N.; Owens, V.A.M.; Hadjistavropoulos, H.D.; Dirkse, D.A.; Dear, B.F. Educating patients about Internet-delivered cognitive behaviour therapy: Perceptions among treatment seekers and non-treatment seekers before and after viewing an educational video. Internet Interv. 2016, 6, 57–63. [Google Scholar] [CrossRef] [Green Version]
  76. Choi, I.; Zou, J.; Titov, N.; Dear, B.F.; Li, S.; Johnston, L.; Andrews, G.; Hunt, C. Culturally attuned Internet treatment for depression amongst Chinese Australians: A randomised controlled trial. J. Affect. Disord. 2012, 136, 459–68. [Google Scholar] [CrossRef]
  77. Kayrouz, R.; Dear, B.F.; Kayrouz, B.; Karin, E.; Gandy, M.; Titov, N. Meta-analysis of the efficacy and acceptability of cognitive-behavioural therapy for Arab adult populations experiencing anxiety, depression or post-traumatic stress disorder. Cogn. Behav. Ther. 2018, 47, 412–430. [Google Scholar] [CrossRef]
  78. Kayrouz, R.; Dear, B.F.; Karin, E.; Fogliati, V.J.; Titov, N. A pilot study of a clinician-guided internet-delivered cognitive behavioural therapy for anxiety and depression among Arabs in Australia, presented in both English and Arabic languages. Internet Interv. 2016, 5, 5–11. [Google Scholar] [CrossRef] [Green Version]
  79. American Psychological Association, Guidelines for the Practice of Telepsychology. Am. Psychol. 2013, 68, 791–800. [CrossRef]
  80. Association of Canadian Psychology Regulatory Organizations. Model Standards for Telepsychology Service Delivery; ACPRO: Saskatchewan, Canadian, 2011; pp. 1–3. [Google Scholar]
  81. Royal Australian and New Zealand College of Psychiatrists. Benefits of E-Mental Health Treatments and Interventions; RANZCP: Melbourne, Australian, 2019. [Google Scholar]
  82. Available online: http://www.emhprac.org.au/e-Mental Health in Practice (accessed on 26 June 2019).
  83. Ruwaard, J.; Kok, R. Wild West eHealth: Time to hold our horses? Eur. Health Psychol. 2015, 17, 45–49. [Google Scholar]
  84. Rozental, A.; Andersson, G.; Boettcher, J.; Ebert, D.D.; Cuijpers, P.; Knaevelsrud, C.; Ljótsson, B.; Kaldo, V.; Titov, N.; Carlbring, P. Consensus statement on defining and measuring negative effects of Internet interventions. Internet Interv. 2014, 1, 12–19. [Google Scholar] [CrossRef] [Green Version]
Table
Table 1. Lessons learned at four levels of analysis of digital mental health services (DMHS).
Table 1. Lessons learned at four levels of analysis of digital mental health services (DMHS).
Level Lesson
1.
Consumers
  • DMHS can improve access to care for those who really need care
  • DMHS deliver more than treatment services
  • DMHS are used by a broad cross-section of the population
2.
Therapists
  • DMHS require specialised therapist skills
  • DMHS require specialised clinical processes
3.
Operating DMHS
  • The operation of DMHS require specialised systems and skills
  • DMHS evolve
4.
Health Systems, Funders, and Policy Makers
  • Integrating DMHS within health systems is challenging
  • DMHS may change the mental health system
  • DMHS are not a panacea

Share and Cite

MDPI and ACS Style

Titov, N.; Hadjistavropoulos, H.D.; Nielssen, O.; Mohr, D.C.; Andersson, G.; Dear, B.F. From Research to Practice: Ten Lessons in Delivering Digital Mental Health Services. J. Clin. Med. 2019, 8, 1239. https://doi.org/10.3390/jcm8081239

AMA Style

Titov N, Hadjistavropoulos HD, Nielssen O, Mohr DC, Andersson G, Dear BF. From Research to Practice: Ten Lessons in Delivering Digital Mental Health Services. Journal of Clinical Medicine. 2019; 8(8):1239. https://doi.org/10.3390/jcm8081239

Chicago/Turabian Style

Titov, Nickolai, Heather D. Hadjistavropoulos, Olav Nielssen, David C. Mohr, Gerhard Andersson, and Blake F. Dear. 2019. "From Research to Practice: Ten Lessons in Delivering Digital Mental Health Services" Journal of Clinical Medicine 8, no. 8: 1239. https://doi.org/10.3390/jcm8081239

APA Style

Titov, N., Hadjistavropoulos, H. D., Nielssen, O., Mohr, D. C., Andersson, G., & Dear, B. F. (2019). From Research to Practice: Ten Lessons in Delivering Digital Mental Health Services. Journal of Clinical Medicine, 8(8), 1239. https://doi.org/10.3390/jcm8081239

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop