Long Term Outcomes of Blended CBT Compared to Face-to-Face CBT and Treatment as Usual for Adolescents with Depressive Disorders: Analyses at 12 Months Post-Treatment

: Depression is a major problem in youth mental health and identified as the leading cause of disability worldwide. There is ample research on the acute effects of treatment, with estimated small-to-moderate effect sizes. However, there is a lack of research on long-term outcomes. A total of 129 adolescents with clinical depression (82.2% female), aged 13–22 ( M = 16.60, SD = 2.03), received blended CBT, face-to-face CBT or treatment as usual. Data were collected at 12 months after the intervention and compared between treatment conditions. Clinical diagnosis, depressive symptoms, suicide risk, internalizing symptoms and externalizing symptoms decreased significantly over time, from baseline to the 12-month follow-up, and also from post-treatment to the 12-month follow-up in all three conditions. Changes were not significantly different between conditions. At the long-term, improvements following the treatment continued. Due to the large amount of missing data and use of history control condition, our findings need to be interpreted with caution. However, we consider these findings as a clinical imperative. More evidence might contribute to convincing adolescents to start with therapy, knowing it has lasting effects. Further, especially for adolescents for whom it is not possible to receive face-to-face treatment, blended treatment might be a valuable alternative. Our findings might contribute to the implementation of blended CBT.


Introduction
Depressive disorders in adolescents are highly prevalent (Kessler et al. 2012) and are identified as the leading cause of disability worldwide (World Health Organization 2017). Adolescent depressive disorders have severe negative impacts on social and family functioning (Jaycox et al. 2009;Verboom et al. 2014), as well as on academic and occupational performance (Verboom et al. 2014;Wickrama et al. 2008). They are also related to poor physical and mental health in adulthood (Ellis et al. 2017;Seeley et al. 2009) and pose a major risk for suicidal behavior and completed suicides (Gould et al. 2003;Portzky and Van Heeringen 2009).
Cognitive Behavioral Therapy (CBT) is often the first-choice treatment, and there is ample research showing the short-term effects in reducing depressive symptoms or depressive disorders in adolescents (e.g., Cuijpers et al. 2020;Klein et al. 2007;Weisz et al. 2006;Weisz et al. 2013), with estimated small-to-moderate effect sizes after the intervention. These meta-analyses included only face-to-face CBT treatment and did not include the more recent introduced blended CBT. Blended treatment is defined as treatment containing face-to-face sessions with a mental health professional combined with computerized therapy which patients follow independently, combined into an integrated treatment protocol (Van der Vaart et al. 2014). There are only a few review studies focusing on blended treatment, and despite that, they presented positive short-term effects after the intervention; the authors were not able to statistically analyze the synthesized results because of the small number of studies on blended treatment (Erbe et al. 2017;Rasing et al. 2019a, Rasing 2020. Importantly, none of these reviews or meta-analyses on either faceto-face or blended treatment presented follow-up treatment effects (i.e., 12 months or longer after finishing treatment), because only a few of the original empirical studies included measures after the immediate phase of treatment.
Nonetheless, the few original empirical studies presenting outcomes of psychotherapy (Birmaher et al. 2000;Clarke et al. 1999;Clarke et al. 2002;Lewinsohn et al. 1990;Topooco et al. 2019; Treatment for Adolescents with Depression Study (TADS) Team 2009) showed rather consistently that improvement in depressive symptoms or rate of remission continues after face-to-face treatment. Birmaher et al. (2000) also presented that, during the follow-up period, up to 12 months after treatment, equal improvement in symptoms and rate of remission from depressive disorders was found in patients across various treatment conditions. To the best of our knowledge, only one study presented findings of the 12-month follow-up of blended treatment, albeit without control condition, and showed that participants did not show additional improvement in depressive symptoms from post-treatment to the 12-month follow-up . Taken together, effects of face-to-face psychotherapy on depressive symptoms seem to continue up to 12 months after treatment, but the same continuation was not found after blended treatment. It is also important to notice that this is a small number of studies and that there is a paucity of information regarding the long-term outcomes of adolescents treated for depressive disorders, especially for blended CBT. The purpose of our study is to contribute to previous research by studying the maintenance of effects of blended CBT when compared to face-to-face CBT and treatment as usual and increase the evidence base for blended CBT as treatment option for adolescents with a depressive disorder. The main aim was to exploratively evaluate the maintenance of effects of blended CBT, face-to-face CBT and TAU and to compare the remission rate of depressive disorders between blended CBT and face-to-face CBT and between blended CBT and TAU. The second aim was to explore the maintenance of reduction in depressive symptoms, suicide risk, internalizing symptoms and externalizing symptoms and the differences between blended CBT and face-toface CBT and between blended CBT and TAU at the 12-month follow-up.
Previous publications from the current trials described the rationale and design (Rasing et al. 2019b;) and the outcomes immediately after treatment and at the 6-month follow-up Stikkelbroek et al. 2020). To briefly summarize, we evaluated the outcomes of blended CBT and compared them to the outcomes of face-to-face CBT and treatment as usual (TAU). We found no difference in remission rate between blended CBT and face-to-face CBT or TAU post-treatment and at 6-month follow-up. Depressive symptoms decreased significantly over time in adolescents in the three treatment conditions but changes were not significantly different between blended CBT and face-to-face CBT or TAU. More specifically, participants were evenly likely to show a clinically relevant decline in depressive symptoms directly after the treatment and at the 6-month follow-up between blended CBT and face-to-face CBT or TAU. Further, changes in other outcomes (i.e., suicide risk, internalizing and externalizing symptoms, severity of depression and global functioning) were also not significantly different between blended CBT and face-to-face CBT or TAU Stikkelbroek et al. 2020).
In the current study, outcomes at the 12-month follow-up were examined. Data of the open trial with blended CBT as treatment condition (Rasing et al. 2019b) were compared to the data of the randomized controlled trial with face-to-face CBT and TAU as treatment conditions . The main aim was to exploratively evaluate the maintenance of effects of blended CBT, face-to-face CBT and TAU and to compare the remission rate of depressive disorders between blended CBT and face-to-face CBT and between blended CBT and TAU in adolescents with a clinical depression 12 months after treatment. The second aim was to explore the maintenance of reduction in the secondary outcomes depressive symptoms, suicide risk, and internalizing and externalizing symptoms, in blended CBT, face-to-face CBT and TAU and the differences in the secondary outcomes between blended CBT and face-to-face CBT and between blended CBT and TAU at the 12-month follow-up.

Ethics
Both the study protocol for blended CBT and the study protocol for face-to-face CBT and TAU were approved by the Medical Research Ethics Committee METC Utrecht, The Netherlands (NL61804.041.17 and NL34064.041.10) and were registered in the Dutch Trial Register (NTR) as NTR6759 and NTR2676. Informed and written consent was provided by all adolescents, and if the adolescents was under the age of 16 years also their parents. Findings were reported in accordance with the CONSORT 2010 statement (Moher et al. 2012;Schulz et al. 2010) and the SPIRIT guidelines (Chan et al. 2013).

Design and Procedure
The study followed a pragmatic quasi-experimental controlled design. Specifically, data from the open trial with blended CBT as treatment condition (Rasing et al. 2019b) were compared to data from the randomized controlled trial with face-to-face CBT and TAU as treatment conditions (Stikkelbroek et al. 2020). Data from participants receiving blended CBT (Rasing et al. 2019b) were collected between November 2017 and June 2020, as well as data from participants receiving face-to-face CBT or TAU between December 2011 and December 2015 (Stikkelbroek et al. 2020;). The trials followed the same recruitment procedure, eligibility criteria for participants and duration of treatment, and they were executed by the same research team. Adolescents with a depressive disorder who were referred for treatment in psychiatric care were informed about the study, together with their parents, and they were asked to participate.
In both trials, treatment was provided for 15 weeks and could be prolonged to 20 weeks when intermitted by holidays or illness. Assessments were conducted at baseline (T0) during the intervention after 5 (T1) and 10 (T2) weeks, at 1-4 weeks post-treatment (T3), and at the 6-month (T4) and 12-month follow-up (T5). In this article, we reported the outcomes at 12-month follow-up (T5). The participant flow is presented in Figure 1.

Sample Size
Based on previous research, the within-effect sizes of blended CBT and face-to-face CBT were estimated to be moderate (d = 0.76 and d = 0.53, respectively) (Andrews et al. 2010;Klein et al. 2007). To detect a difference in remission between conditions (assuming alpha = 0.05, power (1 − β) = 0.80 and dropout of 20%), 70 adolescents per condition were required.

Interventions
Face-to-face CBT consisted of the Dutch protocolized CBT program Doepressie (Stikkelbroek et al. 2005), which is based on the evidence-based treatment program Coping with Depression course for Adolescents (CWD-A) ). The program consists of 15 weekly sessions at 45 min each. The blended CBT condition contained Doepressie Blended (Stikkelbroek and Van Dijk 2013), adapted from the face-to-face protocol. The online content of the program is combined with face-to-face sessions with a therapist of each 45 min, with a minimum of 5 and maximum of 15 sessions. The control intervention is defined as treatment as usual (TAU), consisting of a range of different treatments with 15 weekly sessions. In this study, mental health institutions offered treatments among which are Interpersonal Therapy (IPT), family therapy, parent counseling, antidepressant medication and acceptance commitment therapy (ACT). For the purpose of this study, CBT was not allowed within the control condition. A more detailed description of the treatment conditions and the therapists can be found in .

Primary Outcome
The presence of the diagnosis of depression was assessed with the Kiddie-Schedule for Affective Disorders and Schizophrenia, present and lifetime version (K-SADS-PL) (Kaufman et al. 1997;Reichart et al. 2000). This semi-structured diagnostic interview assesses present and life-time DSM diagnoses, taking adolescents' and parents' view into account. Previous research revealed excellent test-retest reliability and high interrater agreement (93-100%) (Kaufman et al. 1997).

Secondary Outcomes
Depressive symptoms were measured by using the self-report measure Child Depression Inventory-2 (CDI-2) (Bodden et al. 2016;Kovacs 2011). The questionnaire contains 28 items, each consisting of three statements rated from 0 to 2. Item scores were summed, with higher scores representing more depressive symptoms. The CDI-2 showed good psychometric properties (Bodden et al. 2016).
Suicide risk was assessed with the self-report questionnaire Suicide Risk Taxation (SRT), based on the Suicide Ideation Questionnaire-Jr (Reynolds 1988) and the Suicide Severity Rating Scale (Posner et al. 2011). The questionnaire consists of six items rated on a 3-point scale and assesses frequency of suicidal thoughts, wishes, plans and actions over the past two weeks. Item scores were summed, with higher scores representing higher risk.

Missing Data
The proportions of missing data are presented in Figure 1. Missing data from the clinical interview K-SADS (i.e., dichotomous data) were not imputed. To handle the missing data in data measured with questionnaires (i.e., continuous data), multiple imputation was used (Rubin 1987). The pattern of missing data can be found in Supplementary Materials Table S1. Using the R (R Core Team 2020) package Mice (Van Buuren and Groothuis-Oudshoorn 2011) (25 iterations), we constructed 10 datasets by predictive mean matching. Traditional choices for the number of imputed datasets are m = 3, m = 5 and m = 10. The larger the number of imputed datasets, the smaller the effect of simulation error on the total variance. Based on Van Buuren (2018), we used m = 5 during the building of the imputation model. During the final round of imputation, we increased the number of imputed datasets to 10 (Van Buuren 2018). Intent-to-treat analyses were performed on each imputed dataset and subsequently pooled, using Rubin's rules (Rubin 1987). Imputation performance was assessed by comparing the results from the analyses on imputed datasets to results from analyses on non-imputed data. To test the robustness of the findings, we conducted sensitivity analyses.

Statistical Analyses
Differences in remission of depressive disorder as primary outcome between blended CBT and face-to-face CBT and between blended CBT and TAU were analyzed with binomial logistic regression models controlling for age and gender.
To examine differences in depressive symptoms as a secondary outcome, linear mixed models were constructed using the R package lme4 (Bates et al. 2015) with a random intercept for each participant, while controlling for age and gender. Interaction terms between condition and dummy timepoints were included to test for differences between conditions over time (Twisk et al. 2018). Subsequently, the participants' Reliable Change Index (RCI) was calculated by dividing the baseline to follow-up difference in depressive symptoms by the standard error of this difference. RCIs smaller than −1.96 SDs were qualified as significant improvement in symptoms and RCI's larger than −1.96 SDs as no improvement (Jacobson and Truax 1991). To test for differences in improvement rates between conditions, binomial logistic regression models controlling for age and gender were fitted with improved RCI as dependent variable. Finally, the same linear mixed models as for depressive symptoms were fitted with suicide risk, internalizing symptoms and externalizing symptoms as dependent variable.

Sample Characteristics
No differences between participants' characteristics were found between the three conditions: age (F(2,126) = 0.73, p = 0.48), gender (χ2 (2, N = 129) = 1.26, p = 0.53), educational level (χ2 (4, N = 129) = 2.79, p = 0.59) and ethnicity (χ2 (2, N = 129) = 0.48, p = 0.79). Clinically, there were no differences between conditions in the presence of clinical diagnoses at baseline, except from the diagnosis social phobia, which was more present in the conditions face-to-face CBT and TAU. An overview of the clinical diagnoses at baseline can be found in Supplementary Materials Table S2. This table was published before in . Participants did not differ in level of depressive symptoms, suicide risk, internalizing symptoms and externalizing symptoms between conditions. Further, no differences were found in drop-out rate or number of adverse events between conditions. A detailed description of drop-out, adverse events and treatment dosage can be found in .

Depressive Symptoms
Significant time effects for depressive symptoms were found from baseline to 12 months after the intervention (B = −7.04, SE = 1.54, p < 0.001), and from post-treatment to 12 months after the intervention (B = −6.01, SE = 1.47, p < 0.001). The decline in depressive symptoms was not significantly different in the blended CBT condition compared to the decline in the face-to-face CBT condition nor compared to the decline TAU from baseline to the 12-month follow-up, neither from post-treatment to the 12-month follow-up. Means and standard deviations of this secondary outcome are presented in Table 1; results of the linear mixed models are presented in Table 2. The between-group effects sizes were non-

Reliable Change in Depressive Symptoms
We found that, between baseline and the 12-month follow-up, 57.3% of the participants receiving blended CBT showed a clinically relevant decrease (i.e., reliable change) in depressive symptoms, compared to 68.6% of the participants receiving face-to-face CBT and 56.6% of the participants receiving TAU. This means that participants who received blended CBT were evenly likely to show a clinically relevant decrease in depressive symptoms compared to participants who received face-to-face CBT (OR = 0.58, 95% CI [0.18, 1.84]) and compared to participants who received TAU (OR = 1.09, 95% CI [0.36, 3.23]).

Other Outcomes
Our findings showed a significant effect of time on suicide risk (B = −1.27, SE = 0.45, p = 0.005), internalizing symptoms (B = −8.38, SE = 1.74, p < 0.001) and externalizing symptoms (B = −3.45, SE = 1.10, p = 0.002) from baseline to the 12-month follow-up. Furthermore, we found a significant effect of time from post-treatment to the 12-month follow-up on suicide risk (B = −1.30, SE = 0.42, p = 0.002), internalizing symptoms (B = −7.53, SE = 1.65, p < 0.001) and externalizing symptoms (B = −3.75, SE = 1.07, p < 0.001). No significant differences in the decrease of suicide risk, internalizing symptoms or externalizing symptoms were found between the blended CBT condition and the face-to-face CBT condition nor between the blended CBT condition and the TAU condition from baseline to 12-months follow-up, neither from post-treatment or the 12-month follow-up. Means and standard deviations of the secondary outcomes are presented in Table 1; results of the linear mixed models are presented in Table 2.

Sensitivity Analyses
Completer-only analyses for differences in decline in depressive symptoms, suicide risk, internalizing symptoms and externalizing symptoms also showed no differences between treatment conditions. Thus, results were comparable to the intent-to-treat analyses.

Discussion
In the current study, outcomes at 12-month follow-up of an open trial with blended CBT as treatment condition (Rasing et al. 2019b) were compared to the outcomes at 12month follow-up of an RCT with face-to-face CBT and treatment as usual as treatment conditions . The main aim was to exploratively evaluate the maintenance of effects of blended CBT, face-to-face CBT and TAU and to compare the remission rate of depressive disorders between blended CBT and face-to-face CBT and between blended CBT and TAU. The second aim was to explore the maintenance of reduction in depressive symptoms, suicide risk, internalizing symptoms and externalizing symptoms and the differences between blended CBT and face-to-face CBT and between blended CBT and TAU at the 12-month follow-up.
Our findings show that, 12 months after treatment, 87.5% of the adolescents were in remission from a depressive disorder. Findings also show no differences between blended CBT and face-to-face CBT and between blended CBT and treatment as usual in likelihood to be in remission. Further, all three conditions resulted in a significant decrease in depressive symptoms, suicide risk, internalizing symptoms and externalizing symptoms from baseline to the 12-month follow-up and from post-treatment to 12-month follow-up, with no significant differences between blended CBT and face-to-face CBT, nor between blended CBT and treatment as usual.
Previous research already showed that the immediate effects of face-to-face CBT persisted during follow-up (Birmaher et al. 2000;Clarke et al. 2002; Treatment for Adolescents with Depression Study (TADS) Team 2009). However, this was not certain for blended CBT treatment ); our findings show they did. A possible explanation for finding positive long-term outcomes of face-to-face CBT as well as blended CBT, is the similarity of the content of the treatment protocols; the Doepressie Blended online protocol is the digitalized version of the depression face-to-face treatment protocol. The therapy techniques used during the treatments are therefore the same. Furthermore, in the faceto-face sessions of the blended CBT, therapists were able to guide adolescents through the online protocol and used the personal contact to give feedback, which closely resemble the guidance provided in face-to-face CBT. In the TAU condition, also evidence-based techniques were used, such as IPT, which explains the significant decrease in disorders and symptoms in the TAU condition. Important to notice, as mentioned before in , we found no differences in therapist experience between treatment conditions. Previous research by Vernmark et al. (2019) showed that they found no evidence to assume that therapist-related alliance is different between blended treatment and face-toface treatment modalities. Despite these findings, we cannot rule out the influence of these nonspecific therapeutic factors, and they need to be taken into consideration.
Despite depression being a highly recurrent disorder, the reduction of depressive symptoms, suicide risk, internalizing and externalizing symptoms continues; the decrease from post-treatment to 12 months after treatment is also significant. The remission rate increases from 54.5% in blended CBT, 68.0% in face-to-face CBT and 60.0% in TAU at posttreatment (see  to 87.5% in blended CBT, 92.9% in face-to-face CBT and 97.7% in TAU at 12-months follow-up. This is where our findings regarding blended CBT differed from previous research showing that symptom level between post-treatment and the 12-month follow-up remained the same ). This might be explained by the duration of the actual treatment, which is eight weeks in the study by  compared to 15 weeks in our study, possibly leading to a better endurance of learned skills.
Furthermore, while the average symptom level of the adolescents in the conditions decreased, symptoms were still present in a third to slightly less than half of the adolescents across the three conditions 12 months after treatment. That means that none of the treatments is effective for everyone. An important step in future research and clinical practice would be to move towards precision medicine, gaining more understanding as to which treatment works for whom. Personalizing treatment based on the use of prognostic and prescriptive characteristics could improve the effects for individuals.

Strengths and Limitations
The interpretation of the findings has some limitations. First, the main limitation is the extent of missing data. The attrition rate at the 12-month follow-up is 73.6% on the primary outcome (i.e., clinical interview) and 48.8% on the secondary outcomes (i.e., questionnaires). The attrition was partly caused by participants no longer willing to participate and partly by participants dropping out of treatment. The rate of adolescents dropping out of treatment was 45.0%, albeit comparable to other studies on depression treatment (i.e., 50%) (De Haan et al. 2013). Second, as previously mentioned in , the sample size of the study was rather small, as we were not able to recruit the required number of participants (i.e., 129 of the required 210). Both, starting with a small sample size and high attrition rate, resulted in a low number of participants in the 12-month follow-up assessment. This results in lower power and reduces the likelihood that statistically significant results reflect a true effect. This means that we need to interpret our results with caution. However, 12-month follow-up results of treatment outcome for depressive disorders in adolescents are rare, and it is important to contribute to an evidence base. Third, there is a lack of detail regarding utilization of healthcare in the follow-up phase. The treatments subjected to examination were finished after duration of 15 to 20 weeks and whether participants received other forms of treatment or relapse prevention afterwards is unclear. However, this situation is most likely to happen in routine care of depressed adolescents. Lastly, as blended CBT was compared to history control conditions, adolescents were not randomized between treatment conditions. We cannot completely rule out any differences between participants in the different conditions, such as participants preferring blended treatment and different expectations regarding blended CBT compared to treatment conditions used in the RCT. In order to minimize the differences, in the open trial with the blended treatment condition, we explicitly followed the same recruitment procedure, eligibility criteria for participants and duration of treatment of the RCT, and both trials were executed by the same research team. Despite the fact that we found no baseline differences between participants in the conditions, differences between trials could not be ruled out.
Despite these limitations, we consider these findings as a clinical imperative. Importantly, we compared the outcomes between active treatment conditions, all three studied in adolescents with a clinical depression referred for treatment, suggesting a high generalizability to clinical care (Weisz et al. 2015). Moreover, as we mentioned before, there is a paucity of information regarding the long-term outcomes of treatment for adolescents with clinical depression, especially for blended CBT. It goes without saying that therapists want clarity about the effects before introducing blended treatment to their patients.

Clinical Implications
Depressive disorders are the most important cause of disability worldwide. A majority of depressive disorders start during adolescence (Kim-Cohen et al. 2003), and effective treatment with long-term effects are a necessity. However, evidence on long-term effects is scarce. Our findings contribute to knowledge of outcomes a year after treatment. We found that adolescents are difficult to motivate for therapy, and they are aware that depressive episodes are recurrent. More evidence of the long-term effects of treatment might contribute to convincing adolescents to start with therapy, knowing it has lasting effects.
Additionally, more knowledge about specifically blended CBT could improve the therapists' confidence in blended treatment and might convince them to use it as treatment for adolescents with depressive disorders. Especially for adolescents for whom it is not possible to receive face-to-face treatment, due to reasons such as travel costs or distance to a mental health facility or not being able to visit during office hours, blended treatment might be a valuable alternative. Our findings might contribute to the implementation of blended CBT.

Conclusions
Despite the limitations, the outcomes of blended CBT were promising, and no differences in outcomes at the 12-month follow-up could be established between the blended and face-to-face treatment or TAU. All three treatment conditions resulted in a decline in depressive disorders, depressive symptoms, suicide risk, and internalizing and externalizing symptoms.

Informed Consent Statement:
Written informed consent to participate in the study was obtained from adolescents and from parents.

Data Availability Statement:
The data for the current study is not publicly available due to them containing information that could compromise research participant privacy, but they are available from the corresponding author upon reasonable request.