Assessment of ‘Cool’ and ‘Hot’ Executive Skills in Children with ADHD: The Role of Performance Measures and Behavioral Ratings

Round 1

Reviewer 1 Report

DEAR AUTHORS.

Some minor observations are suggested below:

The introductory text greatly rescues the neuropsychological context, especially of cold executive functioning, especially in ADHD. This is valued and congratulated. The theoretical bases to explore hot executive functions are also sufficient, although it is worth asking what the study contributes to the understanding of the neuropsychological aspects of ADHD. In the last paragraph I suggest writing something oriented to it, What does hot executive functioning contribute to cold? And answering this in the discussion could be of clinical interest.

In material and methods:

The choice of tests in both hot and cold Executive Functions is well supported in the literature and discriminates performances in both groups of the selected population. Likewise, the other methodological aspects are pertinent.

The statistical analyses and indicators are well expressed in the results, discussion, and conclusions. I suggest communicating the comparative results through bar graphs to show the reader those that present statistical significance, with error bars included. The limitations expressed are significant and may affect the generalization of the findings, as the authors point out in the text. I suggest that, in the lines of discussion and conclusions, indicate that the results show these alterations in this population diagnosed with ADHD in the profile of hot and cold executive functioning, projecting the care that these statements refer to for diagnostic considerations. Finally, the question remains, how is the study of hot executive functions justified as a contribution to the development of cognition?

Best regards

Author Response

First, we would like to thank you so much for your valuable comments and recommendations. We outline our answers below.

1. The theoretical bases to explore hot executive functions are also sufficient, although it is worth asking what the study contributes to the understanding of the neuropsychological aspects of ADHD. In the last paragraph I suggest writing something oriented to it, What does hot executive functioning contribute to cold? And answering this in the discussion could be of clinical interest.

Thank you for the very relevant recommendations. As suggested, a few additions to the introduction were made to explicitly point out the contribution of ‘hot’ executive functions to pure cognitive processes (‘cold’ executive functioning) (lines 116-120; 123-126). These considerations were also included in the clinical implications subsection (lines 548-555).

2. I suggest communicating the comparative results through bar graphs to show the reader those that present statistical significance, with error bars included.

Thanks for this suggestion. We added five graphs comparing the results between groups (with error bars; see Fig. 1).

3. I suggest that, in the lines of discussion and conclusions, indicate that the results show these alterations in this population diagnosed with ADHD in the profile of hot and cold executive functioning, projecting the care that these statements refer to for diagnostic considerations.

The limitation regarding the reduced generalization of results and need for careful consideration before referring to them for diagnostic confirmation was included in the respective section (lines 488 – 490). Thanks for this comment.

4. The question remains, how is the study of hot executive functions justified as a contribution to the development of cognition?

We weight in on why it is important to consider ‘hot’ executive functions along with cognition for clinical purposes in the respective section of the discussion (lines 548-555).

Reviewer 2 Report

Before beginning to comment on the review, I would like to thank you the opportunity to review this manuscript and congratulate the authors for their work. The study is very interesting, it brings novelty to the field and adresses some questions regarding the discriminant value of hot and cool executive functions. However, I would like to make some comments on the manuscript that I hope will contribute to enrich it.

Materials and Methods

Participants

The authors reported that the children belonging to the clinical group had an already established diagnosis of ADHD. My question focuses on the way in which this diagnosis was established, how the assessment was conducted, since no information is provided regarding. In this line I would like to ask for more information on what were the strategies mobilized by pediatricians and child psychiatrists to support the diagnostic. Were multimodal strategies, based on multiple informants (e.g. teachers, parents,..)? If so, please describe it in more detail.

Also the informants on Conners Rating Scales were only the parents.  However, you were able to access the teachers' report through the BRIEF. I wonder, why you didn't collect also the teachers' perspective on ADHD symptoms that would be so relevant given the need for symptoms to be present in two different contexts.

Furthermore, I would like to know if you tried to assess the presence of ADHD symptoms in the control group? If so, please describe it.

If possible, I would also like to have some information regarding the provenance/origin of the sample.

Measures

- (Lines 169-173) the raw result of the Digit Span subtest was used as a measure of working memory. I wonder why you didn't just use the digit span backward raw result, given that this seems to be a more proxy measure of working memory, and considering that you didn't use standardized results.

- (Lines 198-207). I assume that the BRIEF version used in the study was just a translation of the original questionnaire, which are not validated to the Portuguese population. I think it  would be mentioned on the limitations of the study.

Discussion

- (Lines 373-386) the results related to cognitive flexibility do not corroborate previous studies, since the authors reported unimpaired cognitive flexibility competencies in the children within ADHD sample. It is said that this may be due to the small sample size, but also that it may result from the use of different tasks or different scores to characterize this domain. What I  would like to ask is whether you tested the same analyses, but using the difference  between Trail B - Trail A as a score, given that some authors consider this as a purer measure of cognitive flexibility (Andersson, 2008; Drane et al., 2002).

Author Response

First, we would like to thank you so much for your valuable comments and recommendations. We outline our answers below.

1. My question focuses on the way in which this diagnosis was established, how the assessment was conducted, since no information is provided regarding. In this line I would like to ask for more information on what were the strategies mobilized by pediatricians and child psychiatrists to support the diagnostic. Were multimodal strategies, based on multiple informants (e.g. teachers, parents,..)? If so, please describe it in more detail.

Thank you so much for your questions. Children were recruited either within a school or clinical context. The clinical judgment was made by a specialist psychiatrist or pediatrician with training and expertise in the diagnosis of ADHD. The process leading to the diagnosis included information gathered from multiple sources (children, parents, and teachers). Experts diagnosed ADHD based on the application of the DSM-5 criteria. This information was added to the paper (lines 153 – 154; 156 – 158).

2. I wonder, why you didn't collect also the teachers' perspective on ADHD symptoms that would be so relevant given the need for symptoms to be present in two different contexts.

It was not possible to collect teachers' perspective on ADHD symptoms due to institutional constraints (we were only allowed to choose one questionnaire). 

3. I would like to know if you tried to assess the presence of ADHD symptoms in the control group? If so, please describe it.

Unfortunately, we were not able to assess the presence of ADHD symptoms in the control group. This was added as a limitation to the study (lines 487 – 488).

4. I would also like to have some information regarding the provenance/origin of the sample.

Thank you for your question. The sample was obtained in the north of Portugal. This information was added to the manuscript (line 153).

5. ... the raw result of the Digit Span subtest was used as a measure of working memory. I wonder why you didn't just use the digit span backward raw result, given that this seems to be a more proxy measure of working memory, and considering that you didn't use standardized results.

Thank you for your comment. We considered the use of backward items to characterize working memory, and we think this is a valuable suggestion. However, several studies also used both forward and backward digits to characterize working memory (e.g., Rajan & Bell, 2015; Bellaj et al., 2015). Furthermore, research has pointed out that impairments in working memory could be undetected when using digit span backwards and that short-term memory (digit forward) reflects the memory component of working memory (e.g., Wells et al., 2018), thus having an important role in working memory performance. As such, we believed working memory could be better characterized considering the overall score, and, in fact, significant differences were found between groups resorting to this outcome. To better clarify this on the manuscript, we replaced the term ‘working memory’ with ‘short-term and working memory’.

6. I assume that the BRIEF version used in the study was just a translation of the original questionnaire, which are not validated to the Portuguese population. I think it would be mentioned on the limitations of the study.

We agree. As suggested, this limitation was added to the respective section (line 486 – 487).

7. The results related to cognitive flexibility do not corroborate previous studies, since the authors reported unimpaired cognitive flexibility competencies in the children within ADHD sample. It is said that this may be due to the small sample size, but also that it may result from the use of different tasks or different scores to characterize this domain. What I would like to ask is whether you tested the same analyses but using the difference between Trail B - Trail A as a score, given that some authors consider this as a purer measure of cognitive flexibility (Andersson, 2008; Drane et al., 2002).

Thank you for your question. We also consider Trails B – A to be a purer measure of cognitive flexibility, however, unfortunately, in our sample this variable did not comply with the testing assumptions and thus had to be removed from the analysis. This information was added to the discussion section (lines 411 – 416).

Reviewer 3 Report

This paper presents a study explored which measures of executive functions (EF) may lead to better diagnostic prediction and assessed whether participants were appropriately assigned to the ADHD group on the basis of identified predictors. Seventeen 6- to 10-year-old children with ADHD were matched with 17 typically developing (TD) peers for age, gender, and nonverbal intelligence. Performance-based measures and "cold" and "hot" EF behavior ratings were used.

Predictably, a significant group effect was observed in the linear combination of measures, indicating that the ADHD children showed significant difficulties with EF compared to the TD group. In fact, significant differences were found on measures of working memory, planning, delay aversion, and EF-related behaviors as reported by parents and teachers.

There are some related papers such as "Use of an ANN to Value MTF and Melatonin Effect on ADHD Affected Children" and "Analysis of Different Melatonin Secretion Patterns in Children With Sleep Disorders: Melatonin Secretion Patterns in Children" in which the role of melatonin in children with ADHD is observed. As well as in "Clinical Considerations Derived From the Administration of Melatonin to Children With Sleep Disorders", they have also published a paper on "Melatonin Secretion Patterns in Children with ADHD".


However, discriminant function analysis revealed only three significant predictors: the General Executive Composite of the Executive Function Behavior Rating Inventory (parent and teacher forms) and the Delayed Gratification Task, with 97.1% correct ratings. These findings highlight the importance and contribution of both behavioral ratings and "hot" measures of EF to the characterization of ADHD in children.

Author Response

First, we would like to thank you so much for your valuable comments and recommendations. We outline our answers below.

1. There are some related papers such as "Use of an ANN to Value MTF and Melatonin Effect on ADHD Affected Children" and "Analysis of Different Melatonin Secretion Patterns in Children With Sleep Disorders: Melatonin Secretion Patterns in Children" in which the role of melatonin in children with ADHD is observed. As well as in "Clinical Considerations Derived From the Administration of Melatonin to Children With Sleep Disorders", they have also published a paper on "Melatonin Secretion Patterns in Children with ADHD".

Thank you for your comment. The mentioned references were added when relevant (lines 39-42), specifically the one concerning ADHD.