The Use of Methylphenidate to Improve Executive Functioning in Pediatric Traumatic Brain Injury: A Systematic Review and Meta-Analysis
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThank you for the opportunity to review this systematic review and meta-analysis of the use of methylphenidate to improve executive functioning in pediatric traumatic brain injury.
The manuscript is well written, well-reasoned, and on the whole reflects a disciplined approach to the methodologies of systematic reviews and meta-analyses. I enjoyed reading this, and whilst the caveats of study quality and bias beset this systematic review/meta-analysis like many others, it will contribute to the literature.
I am supportive of the publication of this manuscript but would like to suggest some minor revisions. These minor revisions are suggested in the spirit of ensuring this already adequate manuscript maximizes its readability for the diverse disciplines reached by trauma-focused journals and research. I am confident that the Editor and Authors can determine which minor revisions are necessary to address, and which reflect differences in written expression.
INTRODUCTION
Page 1, line 35: Contemporary trauma literature often avoids the terminology of "road traffic accidents" in favor of "road traffic crashes". This relates to concerns that the word "accident" may downplay the preventability over these trauma events; for example: https://www.roadpeace.org/working-for-change/crash-not-accident/. Please consider revising to "road traffic crashes".
Page 1, line 37: Please consider revising "Severity of the injury is determined using" to read, "Severity of the injury may be determined using"
Page 2, lines 50-52: Please consider revising, "Attention deficit is therefore a significant consequence of pediatric TBI. The syndrome can also be described as new-onset (or secondary) attention-deficit hyperactivity disorder (ADHD) [7]." to read "Attention deficit is therefore a significant consequence of pediatric TBI, and in this setting may also be described as new-onset (or secondary) attention-deficit hyperactivity disorder (ADHD) [7]."
Page 2, lines 55-56: Please consider revising, "One study [9] concluded that secondary ADHD might occur up to ten years" to read, "One study [9] concluded that secondary ADHD may occur up to ten years".
Page 2, lines 57-58. Please consider revising, "This study excluded mild TBI data from their analysis, meaning these effects are unclear." to read, "However, this study excluded mild TBI data from its analysis [9], and the relevance of these results to the setting of mild TBI remain unclear".
Page 2, line 61: Please consider revising, "Pharmacological treatment for ADHD includes" to read, "Pharmacological treatment options for ADHD include". There are yet further medication options for the treatment of ADHD not detailed here and I consider the original wording less clear on this.
Page 2 line 67 and line 71: Where the authors refer to "review", if this is indeed a "systematic review" I would recommend that this be stated explicitly.
Page 2, line 70: Please consider revising, "However, this focused on an exclusively adult" to read, "However, this meta-analysis was limited to adult".
MATERIALS AND METHODS
Page 4: lines 146-147: I am assuming that these Table numbers have been kept intentionally blank, pending finalization of the manuscript text. If not, please correct.
RESULTS
Page 10, line 238: Please consider revising, "while two of the studies [25-26]" to read, "while the remaining two studies". I feel this correctly emphasizes the point that no analyzed studies were without bias concerns.
Page 10, line 250: Please consider revising, "Across the studies, multiple measurement scales were used." to read, "The attention measurement scales used varied between included studies." I feel the original wording could equally imply that studies had in common the multiple measurement scales that they used. If this was the intended meaning, my apologies, but my concerns regarding ambiguity stand.
Page 10, line 251 to end of that paragraph on the next page: I would recommend including citations after each of the listed measurement scales to indicate which of the included studies used which scale. I am aware this comes out in part in the following words/paragraphs/supplementary material, but I don’t think readers should be made to wait (or hunt) for this information.
Page 11, line 272: Please consider revising, "The heterogeneity for the studies included in this meta-analysis was calculated at 0%" to read, "The heterogeneity for the three studies included in this CPT meta-analysis was calculated at 0%". I feel this increases clarity for the readers.
Page 12, line 306: Please consider revising, "Additional secondary outcomes were recorded" to read, "Additional secondary outcomes are recorded".
CONCLUSIONS
Page 14, line 410: Please consider revising, "similar results" to read, "similar statistically non-significant results".
Page 14, line 412: Please consider revising, "Higher quality studies with larger sample sizes and a subgroup analysis are needed" to read, "Higher quality studies with larger sample sizes, consistent dosing and outcome measurement scales, and powered subgroup analyses are needed".
Thank you again for the opportunity to review this paper.
Comments on the Quality of English LanguageThe written English is acceptable.
Author Response
Comment 1: INTRODUCTION - Page 1, line 35: Contemporary trauma literature often avoids the terminology of "road traffic accidents" in favor of "road traffic crashes". This relates to concerns that the word "accident" may downplay the preventability over these trauma events; for example: https://www.roadpeace.org/working-for-change/crash-not-accident/. Please consider revising to "road traffic crashes".
Response 1: Amended to “road traffic crashes”
Comment 2: Page 1, line 37: Please consider revising "Severity of the injury is determined using" to read, "Severity of the injury may be determined using"
Response 2: Amended to “Severity of injury may be…”
Comment 3: Page 2, lines 50-52: Please consider revising, "Attention deficit is therefore a significant consequence of pediatric TBI. The syndrome can also be described as new-onset (or secondary) attention-deficit hyperactivity disorder (ADHD) [7]." to read "Attention deficit is therefore a significant consequence of pediatric TBI, and in this setting may also be described as new-onset (or secondary) attention-deficit hyperactivity disorder (ADHD) [7]."
Response 3: Amended as recommended. See lines 53-55.
Comment 4: Page 2, lines 55-56: Please consider revising, "One study [9] concluded that secondary ADHD might occur up to ten years" to read, "One study [9] concluded that secondary ADHD may occur up to ten years".
Response 4: Amended. See line 58-60.
Comment 5: Page 2, lines 57-58. Please consider revising, "This study excluded mild TBI data from their analysis, meaning these effects are unclear." to read, "However, this study excluded mild TBI data from its analysis [9], and the relevance of these results to the setting of mild TBI remain unclear".
Response 5: Amended. See lines 60-62.
Comment 6: Page 2, line 61: Please consider revising, "Pharmacological treatment for ADHD includes" to read, "Pharmacological treatment options for ADHD include". There are yet further medication options for the treatment of ADHD not detailed here and I consider the original wording less clear on this.
Response 6: Amended. See line 65.
Comment 7: Page 2 line 67 and line 71: Where the authors refer to "review", if this is indeed a "systematic review" I would recommend that this be stated explicitly.
Response 7: Amended. See lines 70 and 71.
Comment 8: Page 2, line 70: Please consider revising, "However, this focused on an exclusively adult" to read, "However, this meta-analysis was limited to adult".
Response 8: Amended. See Lines 74-75.
MATERIALS AND METHODS
Comment 9: Page 4: lines 146-147: I am assuming that these Table numbers have been kept intentionally blank, pending finalization of the manuscript text. If not, please correct.
Response 9: Apologies, this has been deleted and is not necessary here. See Lines 161-162.
RESULTS
Comment 10: Page 10, line 238: Please consider revising, "while two of the studies [25-26]" to read, "while the remaining two studies". I feel this correctly emphasizes the point that no analyzed studies were without bias concerns.
Response 10: Amended. See line 261.
Comment 11: Page 10, line 250: Please consider revising, "Across the studies, multiple measurement scales were used." to read, "The attention measurement scales used varied between included studies." I feel the original wording could equally imply that studies had in common the multiple measurement scales that they used. If this was the intended meaning, my apologies, but my concerns regarding ambiguity stand.
Response 11: Amended. See line 274.
Comment 12: Page 10, line 251 to end of that paragraph on the next page: I would recommend including citations after each of the listed measurement scales to indicate which of the included studies used which scale. I am aware this comes out in part in the following words/paragraphs/supplementary material, but I don’t think readers should be made to wait (or hunt) for this information.
Response: Citations now included. See lines 275-290.
Comment 13: Page 11, line 272: Please consider revising, "The heterogeneity for the studies included in this meta-analysis was calculated at 0%" to read, "The heterogeneity for the three studies included in this CPT meta-analysis was calculated at 0%". I feel this increases clarity for the readers.
Response 13: Amended. See line 305.
Comment 14: Page 12, line 306: Please consider revising, "Additional secondary outcomes were recorded" to read, "Additional secondary outcomes are recorded".
Response 14: Amended. See line 347.
CONCLUSIONS
Comment 15: Page 14, line 410: Please consider revising, "similar results" to read, "similar statistically non-significant results".
Response 15: Amended. See line 494-495.
Comment 16: Page 14, line 412: Please consider revising, "Higher quality studies with larger sample sizes and a subgroup analysis are needed" to read, "Higher quality studies with larger sample sizes, consistent dosing and outcome measurement scales, and powered subgroup analyses are needed".
Response 16: Amended. See Lines 496-497.
Reviewer 2 Report
Comments and Suggestions for AuthorsThe authors have undertaken a systematic review to consider the efficacy of methylphenidate (MPH) medication in the treatment of cognitive problems following pediatric TBI. There is very little published in this area to date broadly, and particularly within such a narrow range, and as concluded by researchers what is available has many shortcomings.
The methodology that the authors have followed is appropriate for a systematic review and meta-analysis. Additional methods of evaluating the pooled data were also considered.
Figure 1 and Table 3 are organised and easy to decipher.
In terms of Table 3 it would be recommended that the decimal place for GCS be removed as it is not reported in this way (it is a whole number). It would also be helpful to report a range.
The reporting in 3.3 risk of bias assessment has some errors - in that there is a difference between what is reported in the text and what is outlined in Figure 2 and 3. Specifically in line 239 it is stated : "For the RCT's, much of the risk of bias resulted from the randomisation process, deviations from the intended interventions, and missing outcome data". However, Figure 2 shows that the high risk was for missing data and bias in selection of the reported result. There were no issues deviations from intended interventions.
There is a similar issue in the reporting for the non-RCT's - in that bias in classification of interventions which was shown in Figure 3 was not reported in the text description.
Figure 4 is difficult to read and needs to be enlarged.
The statement in line 265 in relation to a negative mean difference, could be explained, further relation to what the lower score signifies, did the mean of the MPH group fall below mean indicating that they were experiencing more difficulty than the placebo group - i.e. those that were not on the medication?
Table 5 - Conners 3 rating scales is confusing as the Nickels study seems to be using scaled scores (mean of 10 and SD of 3) while Williams has T scores. There should be a footnote outlining what the means actual refer to i.e. what kind of scores that they are. This is also an issue in all supplementary tables : there is no indication of the type of score that they are: what is the actual mean and SD for the score - mean = 100, SD =15 etc.
In Table 6 - a footnote should denote what the self-rated overall score is - i.e. is it the GEC (Global Executive Score). Also what does a mean of 122.24 +/- 5.98 mean? The ceiling T scores for the self report indexes are 84, 87 and 90. The GEC ceiling T score is 88. Unless what is reported is not a T score?
Line 298 which describes the difference between the stimulant group and placebo group, goes on to call the placebo group the control group but it would be clearer to say that those not treated with stimulants had worse executive functioning.
The discussion needs to be re-written to increase clarity.
There is a statement that there were differences in results but that these were not significant (i.e in cpt and brief scores). If this is the case is it necessary to outline that there were differences, if none were significant? It could be reframed as there was a trend in the expected direction but this did not reach significance and methodological weaknesses were considered to impact this - if this is what was thought. Or were the differences in fact quite small with major overlap so really it is not worth commenting on the differences?
In line 355 it is stated " the hypothesized benefits of MPH after pediatric TBI are therefore not supported strongly by the evidence in this review" whereas in fact they are not supported given that statistical significance was not reached. Again there needs to be clarity around whether the differences that observed, although not significant could potentially be useful as they may represent a trend and for example, with larger samples etc significance might be reached. I note that there is a stronger statement in the abstract : that the results do not support MPH treatment to remediate cognitive effects of pediatric TBI.
In discussion regarding limitations of individual studies and the risk of bias analysis there could be explanation of what the domains represent from a research perspective rather than just re-stating the results. E.G what does confounding actual mean in the research.
Line 362 states that there were differences in the characteristics of TBI - it would be helpful to state what these were.
The statement in line 364 that some studies had a baseline requirement for executive functioning but it is not clear what this means.
There is a statement in line 374 that says that parent and teacher reports are not directly comparable, however, this is not accurate. It is possible to statistically investigate the difference across raters (i.e. between parent and teacher report), and determine whether there is (or is not) a reliable difference in the scores that are being reported.
It is stated that "inaccurate recall may also introduce bias" however, it is more that informant report can be subjective, and therefore formal report combined with direct measure can increase reliability of the findings.
A limitation of studies that were included (line 381) was that they did not also measure additional secondary outcomes, such as behavioural problems, mental fatigue and aggression. I note that the brief does look at behavioural and emotional regulation as well as cognition. Perhaps it could be suggested that future studies would benefit from considering both behaviour and cognition – although.
Line 387 states “but not all included trials” – this needs to be clarified – is it meant that not all studies included trials to account for placebo effects?
It would be helpful to outline key factors that need to be addressed for future studies to be successful in the final paragraph.
There is a similar issue with the conclusion needing to be more targeted – was there a trend, or in fact no support due to lack of significance.
Author Response
Comment 1: In terms of Table 3 it would be recommended that the decimal place for GCS be removed as it is not reported in this way (it is a whole number). It would also be helpful to report a range.
Response 1: Mean and SD are now reported and where available a range (only in one study) is now also reported. See changes to Table 3.
Comment 2: The reporting in 3.3 risk of bias assessment has some errors - in that there is a difference between what is reported in the text and what is outlined in Figure 2 and 3. Specifically in line 239 it is stated : "For the RCT's, much of the risk of bias resulted from the randomisation process, deviations from the intended interventions, and missing outcome data". However, Figure 2 shows that the high risk was for missing data and bias in selection of the reported result. There were no issues deviations from intended interventions.
Response 2: Our apologies for not getting this right. The text for the risk of bias analysis has now been corrected. See lines 256-266.
Comment 3: There is a similar issue in the reporting for the non-RCT's - in that bias in classification of interventions which was shown in Figure 3 was not reported in the text description.
Response 3: Apologies again. This has now been corrected. See Lines 256-266.
Comment 4: Figure 4 is difficult to read and needs to be enlarged.
Response 4: Now enlarged to make it more legible.
Comment 5: The statement in line 265 in relation to a negative mean difference, could be explained, further relation to what the lower score signifies, did the mean of the MPH group fall below mean indicating that they were experiencing more difficulty than the placebo group - i.e. those that were not on the medication?
Response 5: The sentence has been clarified by amending it to read: “All three studies reported a reduction in the mean CPT score for the MPH condition compared to placebo, meaning that MPH treatment led to an improvement in CPT performance.” See lines 298-300.
Comment 6: Table 5 - Conners 3 rating scales is confusing as the Nickels study seems to be using scaled scores (mean of 10 and SD of 3) while Williams has T scores. There should be a footnote outlining what the means actual refer to i.e. what kind of scores that they are. This is also an issue in all supplementary tables: there is no indication of the type of score that they are: what is the actual mean and SD for the score - mean = 100, SD =15 etc.
Response 6: A note has been added to the table to explain the scores and also added into the text in lines 314-316.
Comment 7: In Table 6 - a footnote should denote what the self-rated overall score is - i.e. is it the GEC (Global Executive Score). Also what does a mean of 122.24 +/- 5.98 mean? The ceiling T scores for the self report indexes are 84, 87 and 90. The GEC ceiling T score is 88. Unless what is reported is not a T score?
Response 7: Scores for values now specified in Table 6.
Comment 8: Line 298 which describes the difference between the stimulant group and placebo group, goes on to call the placebo group the control group but it would be clearer to say that those not treated with stimulants had worse executive functioning.
Response 8: Amended. See lines 339-341.
Comment 9: The discussion needs to be re-written to increase clarity.
Response 9: The discussion has been amended as per reviewer’s comments. See below.
Comment 10: There is a statement that there were differences in results but that these were not significant (i.e in cpt and brief scores). If this is the case is it necessary to outline that there were differences, if none were significant? It could be reframed as there was a trend in the expected direction but this did not reach significance and methodological weaknesses were considered to impact this - if this is what was thought. Or were the differences in fact quite small with major overlap so really it is not worth commenting on the differences?
Response 10: This has been amended. See lines 376-378.
Comment 11: In line 355 it is stated " the hypothesized benefits of MPH after pediatric TBI are therefore not supported strongly by the evidence in this review" whereas in fact they are not supported given that statistical significance was not reached. Again there needs to be clarity around whether the differences that observed, although not significant could potentially be useful as they may represent a trend and for example, with larger samples etc significance might be reached. I note that there is a stronger statement in the abstract : that the results do not support MPH treatment to remediate cognitive effects of pediatric TBI.
Response 11: Amended. See lines 381-382.
Comment 12: In discussion regarding limitations of individual studies and the risk of bias analysis there could be explanation of what the domains represent from a research perspective rather than just re-stating the results. E.G what does confounding actual mean in the research.
Response: 12: This has now been added. See lines 403-421.
Comment 13: Line 362 states that there were differences in the characteristics of TBI - it would be helpful to state what these were.
Response 13: Explanation added. See lines 427-429.
Comment 14: The statement in line 364 that some studies had a baseline requirement for executive functioning but it is not clear what this means.
Response 14: Statement clarified. See Lines 429-431.
Comment 15: There is a statement in line 374 that says that parent and teacher reports are not directly comparable, however, this is not accurate. It is possible to statistically investigate the difference across raters (i.e. between parent and teacher report), and determine whether there is (or is not) a reliable difference in the scores that are being reported.
Response 15: We agree that this is inaccurate and the statement has now been deleted. See Line 438.
Comment 16: It is stated that "inaccurate recall may also introduce bias" however, it is more that informant report can be subjective, and therefore formal report combined with direct measure can increase reliability of the findings.
Response 16: This statement is also not correct and now has been deleted. See line 438.
Comment 17: A limitation of studies that were included (line 381) was that they did not also measure additional secondary outcomes, such as behavioural problems, mental fatigue and aggression. I note that the brief does look at behavioural and emotional regulation as well as cognition. Perhaps it could be suggested that future studies would benefit from considering both behaviour and cognition – although.
Response 17: We agree and hence we have added the sentence: “It would be useful to also be able to compare these outcomes in future studies”. See line 445.
Comment 18: Line 387 states “but not all included trials” – this needs to be clarified – is it meant that not all studies included trials to account for placebo effects?
Response 18: These sentences have been clarified in Lines 494-495.
Comment 19: It would be helpful to outline key factors that need to be addressed for future studies to be successful in the final paragraph.
Response 19: This has been stated in lines 496-497.
Comment 20: There is a similar issue with the conclusion needing to be more targeted – was there a trend, or in fact no support due to lack of significance.
Response 20: The conclusions have now been re-worked to make them more targeted. See lines 471-501.
Round 2
Reviewer 1 Report
Comments and Suggestions for AuthorsI am grateful to the authors for their detailed and sincere responses to the reviews required. I feel the end result is a quality paper and contribution to the literature related to pediatric traumatic brain injury. I would be happy to support the publication of this paper in Trauma Care.
