Effect of Cognitive Distractors on Neonatal Endotracheal Intubation Performance: Insights from a Dual-Task Simulator

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Dear Authors, 

I was pleased to read your manuscript and learn about your interest in overcoming cognitive distraction, as this is known to impair performance in different activities, including neonatal medical care. Neonatal ETI is one of the most commonly performed procedures by a neonatologist, however it is accompanied by many possible complications and requires thoroughly developed skills, a task which is difficult to be achieved. Some of my observations are listed below.

Introduction and Section 2 present in a logical and systematic manner all important information about this procedure, emphasizing the need to be able to successfully perform it in a narrow time interval of 30 seconds. The short-comings of existing simulation programs are also described and the impact of real- life conditions upon trainees used to simulation ideal environments are presented, in order to sustain the need for better approaches to reduce cognitive distractions in the neonatal intensive care units. The method of dual-task training is clearly described and its advantages are presented, emphasizing that it allows individuals to perform multiple tasks simultaneously, leading to enhancements in cognitive performance and skill automaticity. It is argued that this is also a tool that can be integrated into new medical training programs.

Specific comments for Introduction: This section is well-structured and offers the necessary background for one to understand the limitation of simulator-based training in acquiring transferable skills. However, I would find interesting, if you could mention some proficiency levels required in clinical practice. Available data in the literature suggest a number of 26-75 intubations attempts with a necessary proficiency level of 90% in order to achieve competent skills. 

Evans P, Shults J, Weinberg DD, Napolitano N, Ades A, Johnston L, Levit O, Brei B, Krick J, Sawyer T, Glass K, Wile M, Hollenberg J, Rumpel J, Moussa A, Verreault A, Abou Mehrem A, Howlett A, McKanna J, Nishisaki A, Foglia EE. Intubation Competence During Neonatal Fellowship Training. Pediatrics. 2021 Jul;148(1):e2020036145. doi: 10.1542/peds.2020-036145. Epub 2021 Jun 25. 

Doglioni N, Cavallin F, Zanardo V, Trevisanuto D. Intubation training in neonatal patients: a review of one trainee's first 150 procedures. J Matern Fetal Neonatal Med. 2012 Aug;25(8):1302-4. doi: 10.3109/14767058.2011.632035. Epub 2011 Nov 17. 

Section 3 (Study design) and 4 (Methods) describe the components of the proposed simulator with all the facilities offered by this new system, data collection and statistical analysis of the recordings.

Section 5 (Results and Discussions):  a comparison between expert and novice scores is analyzed, using appropriate statistical methods and the results are presented in numerous tables and figures, confirming the value of this research methodology.

I consider that the accuracy p values would more easily understandable when represented as for example as p<0.001. 

Section 6 (Conclusions) captures most important aspects based on the analysis of the data and shows the important clinical impact of this new simulation approach. This multi-task simulator represents an important step forward in achieving true automaticity and improving specialist response under intense cognitive disturbances. This section seems but a little too long. I think the authors should list in 3-4 lines the most important findings of their study. 

Author Response

Thank you for the thoughtful review.

Comment 1: Specific comments for Introduction: This section is well-structured and offers the necessary background for one to understand the limitation of simulator-based training in acquiring transferable skills. However, I would find interesting, if you could mention some proficiency levels required in clinical practice. Available data in the literature suggest a number of 26-75 intubations attempts with a necessary proficiency level of 90% in order to achieve competent skills. 

Response 1: This is a good point, we included this background information in line 26-28 with relevant reference.

Comment 2: I consider that the accuracy p values would more easily understandable when represented as for example as p<0.001. 

Response 2: This is highlighted in the beginning of method section line 252, and we chose p<0.05 as the dataset is relatively small, which is widely adopted in similar studies. A stricter threshold such as p<0.001 would substantially increase the risk of failing to detect a true effect due to the reduced statistical power in small samples.

Comment 3: This (conclusion) section seems but a little too long. I think the authors should list in 3-4 lines the most important findings of their study. 

Respond 3: The conclusion section is revised as suggested and highlighted in the manuscript.

 

Reviewer 2 Report

Comments and Suggestions for Authors

Aiming to improve training realism and feedback, the research offers a Mixed Reality simulator for newborn intubation that evaluates performance under cognitive stress, hence revealing professionals outperforming beginners. The paper needs minor revisions.

Here are some comments:

• The motivation of the work should be spot-on.
• The novelty of the work and also the added value of it should be mentioned.
• In page 3, a long series of references [12-18] is written without any overview of these references.
• Abbreviations of the names of the journals should be replaced by their complete names according to the style of MDPI.
• Please check the punctuation throughout the paper.
• Proofreading is required.

Author Response

Thank you for the thoughtful review.

Comment 1: The motivation of the work should be spot-on.

Response 1: The research motivation is described in the abstract and introduction section line 58-62 with highlighted text

Comment 2: The novelty of the work and also the added value of it should be mentioned.

Response 2: We revised the manuscript to emphasize this. The novelties of the work are listed in line 69-76, added value is in line 77-79

Comment 3: In page 3, a long series of references [12-18] is written without any overview of these references.

Respond 3: These references describe influential surgical simulators that inspired aspects of our design approach. Although they are not directly related to the specific procedure we focus on, we include them here for interested readers. The revised paragraph briefly outlines the key features of these simulators in line 88-95

Comment 4: Abbreviations of the names of the journals should be replaced by their complete names according to the style of MDPI.

Respond 4: Complete names are provided.

Reviewer 3 Report

Comments and Suggestions for Authors

The objective of the paper is to develop and validate a mixed reality dual-task simulator for neonatal endotracheal intubation that evaluates the impact of cognitive distractions on performance and establishes effective, automated metrics for distinguishing expertise levels.
The study is interesting with potential for future research.
The title is relevant to the paper’s objectives.
The introduction provides a well-justified motivation for the research, articulates a relevant gap in current training practices, and clearly outlines the study's contribution. The discussion on cognitive load theory and automaticity is somewhat scattered. Consider briefly defining "automaticity" earlier to aid non-specialist readers. It would be useful to further discuss cognitive load theory and its implications. In addition, it would be important yo to elaborate more on how the proposed first dual-task MR simulator differs from other simulators or dual-task paradigms.
The following section adequately analyzes related studies.
The study design section is adequately described.  It would be useful to provide more scientific justification for design choices. While each component is described, there is limited justification for why specific tools or technologies were chosen. The term “hit event” is used in the context of the secondary task, but is not clearly defined.
In the data collection section. , it is mentioned that the initial trial type was “counterbalanced,” but no detail is provided.
In the “methods” section, it would be useful to better describe the repeated-measures structure. It would be helpful to clarify the sensor sampling rate, smoothing algorithms, or event detection methods used to compute these features. The hypotheses should be more clearly tied to specific statistical tests.
In the discussion section, the interpretations around novice recalibration and attention allocation are interesting, but it would be useful to tie these claims back to existing cognitive load or skill acquisition theories. In the final paragraph about limitations of single-task metrics can be further analyzed, focusing on how this dual-task paradigm can be integrated into training assessment and whether the paradigm is scalable or generalizable across clinical domains.
In the conclusions section, the phrase about future work is vague. Provide clearer objectives.

Author Response

Thank you for the thoughtful review.

Comment 1: The discussion on cognitive load theory and automaticity is somewhat scattered. Consider briefly defining "automaticity" earlier to aid non-specialist readers. It would be useful to further discuss cognitive load theory and its implications. In addition, it would be important yo to elaborate more on how the proposed first dual-task MR simulator differs from other simulators or dual-task paradigms.

Response 1: Cognitive load theory and its implication are included in line 46-51, where automaticity is also defined. The related work section discussed the existing simulators, and line 124-127 emphasize how first dual-task MR simulator differs from other simulators. Currently, no other surgical simulator incorporates dual-task functionality to simulate environmental factors.

Comment 2: It would be useful to provide more scientific justification for design choices. While each component is described, there is limited justification for why specific tools or technologies were chosen. The term “hit event” is used in the context of the secondary task, but is not clearly defined.

Response 2: The design justification is included in line 162-164. And the selection of specific tools and technologies are described in line 176-187, 207-210. The hit event is defined in line 210-212.

Comment 3: In the data collection section. , it is mentioned that the initial trial type was “counterbalanced,” but no detail is provided.

Respond 3: Counterbalancing involves varying the order of conditions across participants so that each condition appears in each position (first, second, etc.) an equal number of times.

Comment 4: In the “methods” section, it would be useful to better describe the repeated-measures structure. It would be helpful to clarify the sensor sampling rate, smoothing algorithms, or event detection methods used to compute these features. The hypotheses should be more clearly tied to specific statistical tests.

Respond 4: The sensor sampling rate is provided in study design line 177. To keep the rich information from the collected data, no data smooth is done with the kinematic data except the pitch and yaw peaks, where Fast Fourier Transformation is used to filter out high frequency data. Moreover, in practice, high force and penetration may damage the tissue just appearing for a very short moment. The clarification is provided in line 260-261.

Comment 5: In the discussion section, the interpretations around novice recalibration and attention allocation are interesting, but it would be useful to tie these claims back to existing cognitive load or skill acquisition theories. In the final paragraph about limitations of single-task metrics can be further analyzed, focusing on how this dual-task paradigm can be integrated into training assessment and whether the paradigm is scalable or generalizable across clinical domains.

Respond 5: Agreed. This is included and revised in line 397-404.

Comment 6: In the conclusions section, the phrase about future work is vague. Provide clearer objectives.

Respond 6: The future work is clarified in line 431-438.

Round 2

Reviewer 3 Report

Comments and Suggestions for Authors

Dear authors, 

I have received your response, and I have checked the revised version. 

I believe that you made significant efforts to improve the quality of your manuscript.

I have no additional comments.

Kind regards,