Long-Term Effects of Positive Psychotherapy Compared to Cognitive Behavior Therapy in Clinical Depression: An 18-Month Follow-Up Randomized Controlled Trial

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

After a careful reading of the manuscript, several positive aspects can be highlighted. One important strength of the paper lies in the introduction, where the authors clearly acknowledge the limitations of previous research. In particular, they note that existing studies have rarely examined the long-term follow-up outcomes of Positive Psychotherapy (PPT) in clinical depression. By explicitly stating this gap, the authors provide a clear and well-justified rationale for the current study, which aims to extend knowledge regarding the long-term clinical effectiveness of PPT compared to CBT in reducing depressive and general psychological symptoms, as well as in enhancing positive emotional states such as happiness, over an 18-month follow-up period.

Another notable strength of the study is the detailed description of the therapeutic content delivered within the positive psychotherapy group. This level of transparency allows readers to better understand the intervention process and increases the methodological clarity and replicability of the study.

Additionally, the inclusion of a clear and well-structured flowchart represents a valuable asset for the reader, as it explicitly outlines the inclusion and exclusion criteria, thereby facilitating a better understanding of the participant selection process. An additional strength of the manuscript is the transparent and well-balanced discussion of study limitations. The authors demonstrate a high level of methodological rigor by openly addressing issues related to sample size reduction at the 18-month follow-up and by clearly reporting attrition rates. Another positive aspect is the authors’ ethical sensitivity, as they clearly explain the impossibility of restricting participants from seeking additional psychotherapy during the follow-up period. By explicitly acknowledging this limitation and its potential impact on long-term treatment effects, the authors provide a realistic and clinically grounded interpretation of their findings. While the authors appropriately highlight the practical relevance of PPT, the clinical implications remain relatively general. The manuscript would benefit from a clearer translation of the findings into concrete therapeutic guidance, specifying how clinicians might adjust their interventions or therapeutic focus based on these results.

Author Response

Comments 1: After a careful reading of the manuscript, several positive aspects can be highlighted. One important strength of the paper lies in the introduction, where the authors clearly acknowledge the limitations of previous research. In particular, they note that existing studies have rarely examined the long-term follow-up outcomes of Positive Psychotherapy (PPT) in clinical depression. By explicitly stating this gap, the authors provide a clear and well-justified rationale for the current study, which aims to extend knowledge regarding the long-term clinical effectiveness of PPT compared to CBT in reducing depressive and general psychological symptoms, as well as in enhancing positive emotional states such as happiness, over an 18-month follow-up period.

Another notable strength of the study is the detailed description of the therapeutic content delivered within the positive psychotherapy group. This level of transparency allows readers to better understand the intervention process and increases the methodological clarity and replicability of the study.

Additionally, the inclusion of a clear and well-structured flowchart represents a valuable asset for the reader, as it explicitly outlines the inclusion and exclusion criteria, thereby facilitating a better understanding of the participant selection process. An additional strength of the manuscript is the transparent and well-balanced discussion of study limitations. The authors demonstrate a high level of methodological rigor by openly addressing issues related to sample size reduction at the 18-month follow-up and by clearly reporting attrition rates. Another positive aspect is the authors’ ethical sensitivity, as they clearly explain the impossibility of restricting participants from seeking additional psychotherapy during the follow-up period. By explicitly acknowledging this limitation and its potential impact on long-term treatment effects, the authors provide a realistic and clinically grounded interpretation of their findings. While the authors appropriately highlight the practical relevance of PPT, the clinical implications remain relatively general. The manuscript would benefit from a clearer translation of the findings into concrete therapeutic guidance, specifying how clinicians might adjust their interventions or therapeutic focus based on these results.

 

Response 1: Thanks for this helpful feedback. We added now at the end of our paper clinical implications:

From a practical perspective, our findings indicate that PPT may be a valuable alternative or complement to traditional CBT, particularly for patients who aim not only to alleviate symptoms but also to improve overall well-being and life satisfaction. Clinicians may benefit from moving beyond a model focused solely on symptom reduction and incorporating structured positive psychological interventions to foster long-term recovery, resilience, and life satisfaction in patients with depressive disorders. Greater emphasis could be placed on interventions that cultivate positive emotions, meaning, engagement, and interpersonal connectedness. Such approaches may help expand patients’ emotional and cognitive repertoire, especially in the later stages of treatment and in relapse prevention.

Reviewer 2 Report

Comments and Suggestions for Authors

I have serious concerns about the validity of this manuscript. The authors are presenting the same research results from 2014 for the third time, each time with a slightly different statistical framework. Three analyses of the same data do indeed yield the same results (eureka!). However, this practice seems to weaken the science, although perhaps the authors believe that publishing the same data three times strengthens their findings. Unfortunately, there is no mention anywhere in the manuscript that the data are being used for the third time. Overall, the study sample is very poorly described, as is the procedure (for example, it is unclear where and when the studies were conducted, and the dates are missing, making replication impossible). The flowchart is identical to the previous ones in this study. Previous studies also used the same variables and questionnaires. Worse still, three of them lack German validation, so the validity of the obtained results is unknown. The translation method deviates from the standard, and it is unclear whether it was a back-translation or how the final result was obtained (was one of the two translations chosen, or was something else used?). The Positive Psychotherapy Inventory (PPTI) consists of five scales, but their reliability is unknown in both previous and this study. Positive therapy focuses on positive outcomes, such as flourishing and well-being, so it's hardly surprising that these dimensions were significantly higher in positive therapy. It's a pity that this study didn't also measure variables specific to CBT, such as cognitive distortions (cognitive restructuring), behavioral patterns (behavioral activation, exposure to anxiety stimuli), or psychological flexibility. Perhaps this would demonstrate that positive therapy, in contrast, changes little in these areas of functioning. It seems the authors use tautology to prove their point, without offering alternative possibilities, thereby creating confusion. There are also many weaknesses in the reporting of results, such as the use of a t-test instead of ANOVA in Table 4 and the lack of reporting of effect sizes in Table 3. The limitations section also fails to address numerous methodological issues, including those related to self-assessment in the questionnaires and measurement errors (the desire to meet researchers' expectations after free-of-charge therapy). The article is also poorly written, with numerous incorrect references to literature (e.g., SWLS - reference [30] appears as number 29 in the reference list), and "X" instead of the appropriate reference or text (e.g., lines 220, 224, 226, 478, 501, 503, 507, 513, 514). Considering all the problems described above and the fact that the same data were collected three times, it should be concluded that this article adds nothing new to what the authors previously presented in the following publications:

• Furchtlehner, L. M., Schuster, R., & Laireiter, A. R. (2020). A comparative study of the efficacy of group positive psychotherapy and group cognitive behavioral therapy in the treatment of depressive disorders: A randomized controlled trial. The Journal of Positive Psychology, 15(6), 832-845. https://doi.org/10.1080/17439760.2019.1663250
• Furchtlehner, L. M., Fischer, E., Schuster, R., & Laireiter, A. R. (2024). A comparative study on the effectiveness of group positive 566 psychotherapy and group cognitive behavioral therapy on flourishing, happiness and satisfaction with life: A randomized 567 controlled trial. Journal of Happiness Studies, 25(7), 104. https://doi.org/10.1007/s10902-024-00806-y

Therefore, I recommend rejecting this manuscript because it does not add anything new to the existing findings and has too many flaws to qualify as a study.

Author Response

Comments 1: I have serious concerns about the validity of this manuscript. The authors are presenting the same research results from 2014 for the third time, each time with a slightly different statistical framework. Three analyses of the same data do indeed yield the same results (eureka!). However, this practice seems to weaken the science, although perhaps the authors believe that publishing the same data three times strengthens their findings. Unfortunately, there is no mention anywhere in the manuscript that the data are being used for the third time.

Response 1: We would like to clarify that we are not publishing the same data for a third time in an overlapping or redundant manner. Rather, the datasets stem from one large study, from which we developed three separate manuscripts in order to analyze and present the findings in a differentiated and focused way.

In the first paper, we examined negative outcomes in relation to short-term effects following therapy. In the second paper, we also investigated positive outcomes related to the short-term effects following treatment. The current manuscript focuses on data from an 18-month long-term study. These data clearly differ from those from the first two studies in that completely new data, which have never been published before, have been evaluated here. So, this long-term follow-up data has not been presented or published elsewhere.

In addition, we mentioned and referenced the two related publications in the manuscript:

1. „This study is a long-term follow-up of two previously published pre-post and six-month follow-up studies comparing PPT and CBT [17, 21]“
2. „Other limitations concerning the present study have been discussed in detail in the previous papers [17, 21].“

To avoid any misunderstanding, we have now added a further explicit statement in the manuscript clarifying that other components of the overall study have been reported in separate publications, each addressing distinct research questions and outcome periods. („Findings from this large-scale study have been disseminated across multiple publications to allow for differentiated analyses of specific research questions and time points. While prior papers reported immediate post-treatment outcomes, the present manuscript reports exclusively on 18-month long-term follow-up data that have not been presented elsewhere.“)

We hope this clarification explains our rationale for disseminating the findings across multiple papers and demonstrates that the present manuscript offers novel and previously unpublished results. We thank you for raising this important point and for the opportunity to make this clarification.

 

Comments 2: Overall, the study sample is very poorly described, as is the procedure (for example, it is unclear where and when the studies were conducted, and the dates are missing, making replication impossible).

Response 2: Due to the blinding requirements of the peer-review process, we anonymized the study locations as “XX” in this section and have left this placeholder in place for the time being. The relevant information on where the study was conducted can be found on page 7.: „Participants were recruited nationally in two different treatment centers in Austria (two-center-study) with different kinds of acquisition. At the Outpatient Centre of the Department of Psychology of the University of XX, individuals were informed about the intended study through a university press release, a newspaper advertisement and through an email to all students and employees registered or working at the university. At XX, potential participants were recruited at the residential psychiatric hospital after their discharge… „

We have now added the study dates and the duration of the study: „The study’s interventions and data collection containing baseline assessment (t1), post-intervention assessment (t2), a 6-month follow-up as well as an 18-month follow-up measurement, took place over a period of more than 2 years, beginning in May 2014 and finishing in October 2016. To evaluate long-term outcomes, self-report and observer rated measurements are available. In the present study, only the results of the 18-month follow-up measure will be presented.”

 

Comments 3: The flowchart is identical to the previous ones in this study.

Response 3: For the sake of completeness and transparency and given that this manuscript is based on the same large-scale study, the upper sections of the flowchart present information identical to that reported in the previous publications. However, the lower section depicting the 18-month follow-up has not been included in earlier papers and reflects the central focus of the present paper.

Should the reviewer prefer, we would be happy to provide a shortened version of the flowchart, for example by omitting the information on post-treatment assessments.

Comments 4: Previous studies also used the same variables and questionnaires. Worse still, three of them lack German validation, so the validity of the obtained results is unknown. The translation method deviates from the standard, and it is unclear whether it was a back-translation or how the final result was obtained (was one of the two translations chosen, or was something else used?). The Positive Psychotherapy Inventory (PPTI) consists of five scales, but their reliability is unknown in both previous and this study.

Response 4: Thank you for this important comment. We would like to clarify the translation and validation procedure in more detail.

The German version of the Positive Psychotherapy Inventory (PPTI) was developed using a standardized translation and back-translation procedure, consistent with established cross-cultural adaptation guidelines. Specifically, the process involved the following steps:

1. Forward translation: One bilingual translators (fluent in English and German and familiar with psychological terminology) translated the original PPTI into German.
2. Back-translation: This preliminary German version was then translated back into English by one independent translator.
3. Revision: The back-translated versions was systematically compared with the original English questionnaire within the research team. Deviations in meaning, nuance, or psychological construct representation were identified and discussed. Where necessary, items were revised through a consensus process to ensure semantic and conceptual fidelity.

Thus, the final German version was not based on selecting one translation over another, but rather on a collaborative process integrating expert translations and discussions, which reflects recommended best practice.

If you do not agree with us, please specify further why this should not be standard proceedings.

Regarding reliability, we agree that reporting psychometric properties is essential. Therefore, we have now added the internal consistency estimates (Cronbach’s α) for all five PPTI scales in the revised manuscript (see Methods section). The reliability coefficients in our sample were within an acceptable to good range, supporting the use of the German PPTI version in the present study.

We have revised the manuscript to describe the translation procedure and psychometric results in greater detail to ensure transparency and reproducibility.

 

Comments 5: Positive therapy focuses on positive outcomes, such as flourishing and well-being, so it's hardly surprising that these dimensions were significantly higher in positive therapy. It's a pity that this study didn't also measure variables specific to CBT, such as cognitive distortions (cognitive restructuring), behavioral patterns (behavioral activation, exposure to anxiety stimuli), or psychological flexibility. Perhaps this would demonstrate that positive therapy, in contrast, changes little in these areas of functioning. It seems the authors use tautology to prove their point, without offering alternative possibilities, thereby creating confusion.

Response 5: Thank you for this comment. You are right that it would have been highly informative to examine CBT-specific variables in addition to the outcomes assessed. We have now incorporated this consideration into the Limitations section of the manuscript.

Specifically, we added the following point: “Another limitation is that we did not investigate variables specific to CBT, such as cognitive distortions (cognitive restructuring), behavioral patterns (behavioral activation, exposure to anxiety stimuli), or psychological flexibility. Future research should examine these outcomes in the context of PPT compared to CBT in order to better understand potential mechanisms of change and differential treatment effects.“

At the same time, our findings provide several noteworthy insights. Interestingly, we observed no significant long-term differences between PPT and CBT in flourishing, PPTI total score and subscales, or observer-rated depressive symptoms (MADRS) (see Table 5). Satisfaction with life was the only positive outcome, showing a significant advantage for PPT compared to CBT. In contrast, improvements were evident across negative outcomes assessed. Significant Time × Group interactions emerged for the DHS, BDI-II, and BSI (Table 5), indicating that participants in the PPT condition reported lower levels of depressive and psychological distress symptoms over time than those in the CBT condition.

Taken together, these findings suggest that PPT contributes to a substantial reduction in negative symptoms. This pattern points toward a broader, more general therapeutic impact rather than a narrowly specific effect limited to positive outcomes. Consequently, we maintain that the present results support the comparatively strong long-term efficacy of PPT.

 

Comments 6: There are also many weaknesses in the reporting of results, such as the use of a t-test instead of ANOVA in Table 4 and the lack of reporting of effect sizes in Table 3.

Response 6: We have now calculated the effect sizes and included them in Table 3. In Table 4, we replaced the t-tests with ANOVAs and reported the corresponding results.

 

Comments 7: The limitations section also fails to address numerous methodological issues, including those related to self-assessment in the questionnaires and measurement errors (the desire to meet researchers' expectations after free-of-charge therapy).

Response 7: We have now added the use of self-assessment and one observer-rated measure as a limitation: „Also, the present paper relied primarily on self-report and one observer-rated measures. Future studies should incorporate additional assessment methods, such as behavioral tasks or physiological indicators (e.g., heart rate variability, cortisol) to obtain a more comprehensive and multi-method evaluation. “

Regarding your second point: In German-speaking countries, psychotherapy is covered by health insurance. Thus, when clinically indicated, it is usually free of charge or “worse case” covered in part by health insurance. Our study context reflects the standard care available to patients in this setting.

Furthermore, as noted in the manuscript, additional limitations related to the large-scale study are discussed in detail in our two companion papers.

Comments 8: The article is also poorly written, with numerous incorrect references to literature (e.g., SWLS - reference [30] appears as number 29 in the reference list), and "X" instead of the appropriate reference or text (e.g., lines 220, 224, 226, 478, 501, 503, 507, 513, 514).

Response 8: Thank you for pointing out the incorrect reference. We have now corrected this and carefully reviewed all references again. As mentioned above, we used “X” to ensure blinding. We apologize for any confusion this may have caused and for not stating this more clearly. While the manuscript remains under review, we will retain the blinding. For the final publication, we will of course remove it.

Comments 9: Considering all the problems described above and the fact that the same data were collected three times, it should be concluded that this article adds nothing new to what the authors previously presented in the following publications:

• Furchtlehner, L. M., Schuster, R., & Laireiter, A. R. (2020). A comparative study of the efficacy of group positive psychotherapy and group cognitive behavioral therapy in the treatment of depressive disorders: A randomized controlled trial. The Journal of Positive Psychology, 15(6), 832-845. https://doi.org/10.1080/17439760.2019.1663250
• Furchtlehner, L. M., Fischer, E., Schuster, R., & Laireiter, A. R. (2024). A comparative study on the effectiveness of group positive 566 psychotherapy and group cognitive behavioral therapy on flourishing, happiness and satisfaction with life: A randomized 567 controlled trial. Journal of Happiness Studies, 25(7), 104. https://doi.org/10.1007/s10902-024-00806-y

Therefore, I recommend rejecting this manuscript because it does not add anything new to the existing findings and has too many flaws to qualify as a study.

Response 9: We very much hope that our explanations have clarified our perspective and made it more understandable why we consider this work worthy of publication. In particular, long-term outcome data in the context of Positive Psychotherapy remain scarce, and we therefore see our paper as an important contribution to the ongoing development of the field. We report core findings in tables and the figure for providing context, but focus lies on 18 months follow-up, which constitutes first-published results.

We also hope that the now more transparent clarification in the manuscript — namely, that two additional publications exist within the framework of the larger-scale study — will help readers appropriately contextualize and interpret our findings.

We thank the reviewers for their careful evaluation and constructive input, which has helped us to strengthen the manuscript.

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

Although the authors' explanations regarding self-plagiarism have somewhat alleviated my concerns, the issue of splitting the same material into slides and then re-publishing it several times remains, a practice that is not generally recommended in the current scientific world. Furthermore, several unresolved issues remain that must be addressed before the manuscript can be considered for publication.

Replication of this study is still impossible. The authors wrote, "Those who met basic criteria for diagnosis and demographic variables were then invited to an individual assessment session of approximately 3 hours." However, it remains unclear where the studies were conducted and under what conditions (e.g., in a teaching hospital, clinic, university, or the patient's home). What was the room like where the study took place? The explanation that data marked with an "X" cannot be used for review but will later be used for publication is unclear. Please explain why the reviewer cannot know this data when evaluating the manuscript, but the reader will (the "X"s are supposed to be filled in with specific data after the review). If the session lasted 3 hours, how reliable are the results of such a study? Were individuals "having a mild to moderate major depressive disorder or dysthymia" able to answer questions in the medical interview and questionnaires for three hours? This issue should be addressed in the limitations section, as the results of such studies may be unreliable.

Nothing is known about the effect of medication on the study results. How many participants were taking medication, and what types of antidepressants and antianxiety medications? How long had depression been present since the initial diagnosis? All this information is essential for assessing both groups, as pharmacological treatment and psychiatric treatment status can significantly impact the 18-month follow-up measurement. Please include this information in the manuscript or address this issue in the discussion and limitations section of the study.

References to the German validation of the SWLS and MADRS are still missing. If these questionnaires were translated for this publication, please describe the translation process. If no published validation is available for these questionnaires, as with the translated versions of the other two questionnaires, the DHS and the PPTI, this issue should be addressed in the limitations section. It is unclear whether questionnaires in other languages ​​and cultures will have the same factor structure.

Please provide the reliability of the first and last measurements (after 18 months) for all questionnaires in this study.

It is also unknown what the percentage of missing data was and how the researchers dealt with this issue. What method did they use to replace missing data? The limitations section should address this issue by explaining how it could have distorted the data and affected the results of the statistical analyses.

The manuscript does not describe anywhere how the results were distributed in terms of normality and homogeneity of variance between groups. Was the use of the ANOVA test even justified? Were the assumptions for multiple regression analysis met, including: independence of residuals (autocorrelation in residuals), homoscedasticity, normality of residuals, normality of random effects, no multicollinearity, model selection criteria, for example, the Akaike Information Criteria?

Author Response

Comment 1: Although the authors' explanations regarding self-plagiarism have somewhat alleviated my concerns, the issue of splitting the same material into slides and then re-publishing it several times remains, a practice that is not generally recommended in the current scientific world. Furthermore, several unresolved issues remain that must be addressed before the manuscript can be considered for publication.

Response 1: Thank you very much for your prompt and renewed feedback on our revised manuscript.

With regard to your concern about self-plagiarism, we would like to clarify that plagiarism refers to the unauthorized use of others’ intellectual work, such as texts, ideas, or data, without appropriate citation. This does not apply in our case. The pre-intervention data are used solely as reference values for the 18-month follow-up outcomes, which is methodologically necessary in a longitudinal study. Beyond this, no prior material has been reused inappropriately.

We have carefully re-examined the entire manuscript to identify any potential overlap with our previous publications and have critically considered where content could be shortened or removed. After thorough review and in light of the journal’s CONSORT checklist requirements, which we are obliged to follow, we have concluded that further reductions would compromise the transparency, clarity, and replicability of the study.

However, if you believe that specific sections can be removed without affecting the scientific integrity or comprehensibility of the manuscript, we would greatly appreciate your guidance on the exact passages you consider non-essential.

 

Comment 2: Replication of this study is still impossible. The authors wrote, "Those who met basic criteria for diagnosis and demographic variables were then invited to an individual assessment session of approximately 3 hours." However, it remains unclear where the studies were conducted and under what conditions (e.g., in a teaching hospital, clinic, university, or the patient's home). What was the room like where the study took place? The explanation that data marked with an "X" cannot be used for review but will later be used for publication is unclear. Please explain why the reviewer cannot know this data when evaluating the manuscript, but the reader will (the "X"s are supposed to be filled in with specific data after the review). If the session lasted 3 hours, how reliable are the results of such a study? Were individuals "having a mild to moderate major depressive disorder or dysthymia" able to answer questions in the medical interview and questionnaires for three hours? This issue should be addressed in the limitations section, as the results of such studies may be unreliable.

Response 2: We had originally masked certain words with “XX” because the information contained in them could have allowed conclusions to be drawn about our identities as authors, thereby compromising the required anonymity of the peer-review process.

Since you are now the final reviewer of the paper and are already familiar with our previous publications and therefore our names we have decided to remove this anonymization. We are aware that this entails a certain risk, for which we take responsibility, but we believe it is appropriate at this stage.

The words that were previously masked with “XX” have now been de-anonymized, so that the information which was previously unavailable should now be fully clear and comprehensible to you.

We have now added information to the manuscript specifying where the therapies and interviews were conducted. In Salzburg, they took place in the therapy rooms of the Outpatient Center of the Department of Psychology, and in Linz, they were conducted in the therapy facilities of Kepler Universitätsklinikum Linz. Both settings are typical facilities for outpatient therapy sessions.

For your information, we also included site as a covariate in all statistical analyses.

The initial three-hour interviews did not overwhelm the patients. We did not recruit individuals with severe major depressive disorder or psychotic disorders; patients with severe depression or psychosis were excluded beforehand. In addition, breaks were possible at any time during the interviews. Patients did not report that the three-hour sessions were excessive or overly demanding. Likewise, the therapists who conducted the interviews did not gain the impression that the duration was overwhelming for the participants. Furthermore, in German-speaking countries it is common and considered standard clinical practice to conduct structured diagnostic interviews such as the Structured Clinical Interview for DSM-IV at the beginning of treatment.

Finally, these interviews constituted only the baseline diagnostic assessment. We are therefore surprised by the suggestion that this aspect could render the entire study unreliable. The therapy sessions themselves lasted two hours each, which is a typical duration for group therapy sessions in German-speaking countries.

 

Comment 3: Nothing is known about the effect of medication on the study results. How many participants were taking medication, and what types of antidepressants and antianxiety medications? How long had depression been present since the initial diagnosis? All this information is essential for assessing both groups, as pharmacological treatment and psychiatric treatment status can significantly impact the 18-month follow-up measurement. Please include this information in the manuscript or address this issue in the discussion and limitations section of the study.

Response 3: We collected information on whether participants were taking psychopharmacological medication. We subsequently included this variable as a covariate in our mixed-effects model to examine whether medication had a meaningful impact on the results. No significant effects were found.

Nevertheless, in response to your concern, we have acknowledged the use of antidepressants as a potential limitation of the study, even though it did not show a statistically significant effect in our analyses.

Comment 4: References to the German validation of the SWLS and MADRS are still missing. If these questionnaires were translated for this publication, please describe the translation process. If no published validation is available for these questionnaires, as with the translated versions of the other two questionnaires, the DHS and the PPTI, this issue should be addressed in the limitations section. It is unclear whether questionnaires in other languages ​​and cultures will have the same factor structure.

Response 4: We used the versions of both scales validated for German-speaking populations and have now included the corresponding German versions in the citations, referencing them in the description of the scales.

Glaesmer, H., Grande, G., Brähler, E., & Roth, M. (2011). The German version of the Satisfaction with Life Scale (SWLS): Psychometric properties, validity, and population-based norms. European Journal of Psychological Assessment, 27(2), 127–132.

Schmidtke, A., Fleckenstein, P., Moises, W., & Beckmann, H. (1988). Studies of the reliability and validity of the German version of the Montgomery-Asberg Depression Rating Scale (MADRS). Schweizer Archiv für Neurologie und Psychiatrie, 139(2), 51-65.

 

Comment 5: Please provide the reliability of the first and last measurements (after 18 months) for all questionnaires in this study.

Response 5: We have now calculated the reliabilities for all scales for the 18-month follow-up and added them to the manuscript. All newly calculated reliabilities fall within the high range.

Comment 6: It is also unknown what the percentage of missing data was and how the researchers dealt with this issue. What method did they use to replace missing data? The limitations section should address this issue by explaining how it could have distorted the data and affected the results of the statistical analyses.

Response 6: We have already reported the percentage of missing data and discussed it in detail in the limitations section, as excerpted below:

First, although we started with a sufficiently large sample of 92 subjects, the 18-month sample must now be considered not particularly large. Despite a prolonged inclusion period and efforts to enhance the enrolment rate, we could not include more than 49 patients (53%) who completed assessments at 18 months follow-up. The most common reason for incomplete questionnaires was that participants did not respond anymore. To investigate possible biases due to missing data, we conducted independent t-tests. Across all outcome measures, demographics, and main diagnoses, we found no significant differences between the characteristics at baseline measurement of those who were lost to follow-up compared to those who completed the last questionnaires. Moreover, attrition was unrelated to treatment condition. Therefore, it can be assumed that the absence or non-completion is random, and the principal study findings should hold true. However, the statistical power is reduced due to the missing data, which should be considered when interpreting the results.

We deliberately chose not to replace the missing data, as doing so can introduce additional limitations and bias. As noted in the text, we did not find any systematic biases associated with the missing data, which appear to be random.

Comment 7: The manuscript does not describe anywhere how the results were distributed in terms of normality and homogeneity of variance between groups. Was the use of the ANOVA test even justified? Were the assumptions for multiple regression analysis met, including: independence of residuals (autocorrelation in residuals), homoscedasticity, normality of residuals, normality of random effects, no multicollinearity, model selection criteria, for example, the Akaike Information Criteria?

Response 7: Here, too, we already have the necessary information included in the paper.

To assess normality and homoscedasticity, the residual plots were visually examined, utilizing the sjPlot package in R.

And further down:

The residual errors and random effects for all variables exhibit a normal distribution, indicating that the statistical assumptions for mixed-effects models are satisfied.

 

Regarding your second point: We did not perform ANOVAs for our primary and secondary hypotheses. Rather, in line with good scientific practice, we used linear mixed models. To our knowledge, we checked and could confirm all necessary assumptions for these models. For model comparison, we of course used model selection criteria such as AIC. Detailed information is available for interested readers in the published R-Studio code, which is available alongside the data on OSF (https://osf.io/etgbh/overview?view_only=9d406b4f793c46fd870f728c1aed664f).

If you believe that we have not sufficiently assessed the assumptions, we kindly ask for clarification and a detailed description of the additional analyses you recommend, along with references supporting why further assumption checks would be necessary.

As we understand your comment, you are suggesting an additional test of the independence of residuals using the Durbin–Watson test. To our knowledge, this is relevant for linear regression but not for mixed-effects models. For completeness, we also ran our analyses using simple linear regression and checked this parameter, as well as multicollinearity using VIF and tolerance values. We found no violations of assumptions in any of these analyses (we have attached these outputs for your reference). We did not include these additional checks in the paper because, to our understanding, the assumptions we reported and the provided R code are sufficient and appropriate for mixed-effects models.

If you disagree, please let us know. We are willing to perform additional analyses and include them in the manuscript, provided we understand and can interpret them appropriately.

Author Response File: Author Response.pdf