Skip to Content
MDPIMDPI

Advanced

by
  • Nino Shiukashvili 1,*,
  • Gvantsa Vardosanidze 1 and
  • Eka Ekaladze 1
  • et al.

Reviewer 1: Anonymous Reviewer 2: Anonymous Reviewer 3: Anonymous

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

This manuscript addresses an important and timely topic in medical education. The integration of actor-led simulations and multi-source feedback represents a pedagogically sound approach. Despite its potential contribution, the manuscript raises several substantial methodological and analytical concerns that must be addressed before it can be considered for publication.

  1. The introduction would benefit from a more comprehensive engagement with the literature on simulation-based education as a pedagogical approach. In particular, the authors may consider citing recent work such as https://www.frontiersin.org/journals/public-health/articles/10.3389/fpubh.2023.1202598/full, which highlights simulation as a powerful experiential learning method for developing communication, leadership, and decision-making skills.
  2. The manuscript does not clearly state explicit research hypotheses. The authors are encouraged to articulate clear, testable hypotheses aligned with their study design.
  3. The reported effect sizes (Cohen’s d ranging up to 12.5) are exceptionally large and far exceed what is typically observed in educational or behavioral research. Such values are highly unusual and raise important concerns regarding the validity and interpretation of the findings. Additionally, the presentation of Cohen’s d as negative values is unconventional and may confuse readers. The authors must: verify and clearly report how Cohen’s d was calculated, provide justification for these unusually large values, and consider re-analysis if necessary.
  4. The manuscript reports multiple statistical comparisons across numerous items without any correction for multiple testing, raising the possibility of inflated significance levels. Furthermore, the tables provide limited information on variability (e.g., lack of confidence intervals), making it difficult to fully assess the robustness of the findings.
  5. The authors do not sufficiently distinguish between statistical significance and clinical or educational significance, which limits the interpretability of the reported improvements.
  6. The study employs a pre-post design without a comparison or control group. This significantly limits causal inference. The authors should explicitly acknowledge this limitation and temper causal claims throughout the manuscript.
  7. The use of non-blinded evaluators (including faculty, peers, and actors involved in the intervention) introduces a substantial risk of observer bias. Given that the same individuals participated in both training and assessment, score inflation cannot be ruled out. The authors should report any steps taken to mitigate bias.
  8. The Empathetic Communication Assessment Form appears to be adapted from previous work, but there is insufficient information regarding its psychometric properties in the current study. Please clarify whether reliability (e.g., inter-rater reliability) was assessed, how the instrument was validated, and whether it has been previously used in comparable contexts.
  9. The study includes a relatively small sample (n = 32) from a single institution, which limits generalizability. While this is acceptable for an initial study, the authors should more explicitly acknowledge this limitation and avoid overgeneralizing the findings.
  10. The study evaluates outcomes immediately post-intervention, with no assessment of long-term retention or transfer to real clinical settings. This limitation should be discussed more explicitly, particularly given the focus on behavioral change.
  11. Some claims in the discussion and conclusion sections appear stronger than warranted given the study design (e.g., implications for clinical outcomes or broad curricular adoption). The authors should moderate their language to better align with the study’s methodological constraints.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Thank you for allowing me to reviw this article. I thank the author about trying to address this issue. Here is my comments: 

  1. The first part of the title is catchy and caught my eyes( Beyond Self-Report:) but the rest is a mouthfull and should be made more concise.
  2. The methods in the abstract is way too long and should be shorten 
  3. The abstract results need some demographic data
  4. The abstract conclusion is too strong and overgeneralizing. You only measured the short term effect and should make your conclusion around that until you do a follow up study about retention of skills on the long term. 
  5. The introduction has no local data about communication skills and empathy in Georgia 
  6. The introducation did not explicitly define the problem and how this intervention can help!
  7. Methodology is missy and needs multiple subheadings. 
  8. You need to explain how the students were choosen, how did you minimize selection bias
  9. While the students being observed, how did you try to minimize the Howthorne bias? 
  10. Table one is length and mention every student. It might belong to supplemental data. 
  11. Table 2 and figure 2 represent the same thing: please choose one and my vote goes to the table as the figure is not impressive with the bars close to each other in height. 
  12. Have you done sub-group analysis as female doctors tend to have better communication? 
  13. What does the negative value mean in the t-test in table 3?
  14. The discussion is undercited and does not fully discuss the results. It is hard to accept that introduction has 11 references while discussion has only 7!!!
  15. This is an example of under citation: "Compared to earlier studies, which often report modest or inconsistent outcomes, these results demonstrate substantially greater educational impact." This sentence deserves a citation. 
  16. How do you say abscense of control group in the limitation while you did a pre-post study!!! Pre is your control!!! please, edit your limitation. 
  17. I would tone down the conclusion due to small sample size and short duration. The written conclusion is too strong and overgeneralizing!
  18. The references are with exception of one, not recent. 

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

Thank you. It was well-written and was an interesting read. This reviewer has only a few comments.

  1. Claiming a single-group pre–post design without a concurrent control demonstrating “the program’s strong educational impact” and “substantially greater educational impact” than prior work might be a bit of overreach. The effect size was also quite big. The authors might consider reframing as a pilot/feasibility study with tempered claims, or adding a historical or wait-list control in a revised analysis. In the discussion, please expand the Limitation with explicit comparison to controlled trials.
  2. Some psychometrics details seemed to be missing. There seemed to be no report of the content or construct validity of the shortened version, nor the internal consistency (Cronbach’s α) or factor structure. 
  3. The authors might wish to report the inter-rater reliability (ICC or κ) among the raters. In addition, please report the rater training or calibration procedures.
  4. Was there any blinding of raters?
  5. The authors appeared to treat the 19 items as independent replicates for each student, which is unorthodox in statistics. 
  6. Please perform a Bonferroni/Holm correction.
  7. Did the authors check for normality? If yes, please report the details.
  8. Please provide a power analysis or justification for the sample size.
  9. For Limitation, the authors might wish to consider that, for a single institution, a homogeneous cohort, results may not translate to other curricula, cultures, or clinical clerkships. There was no longitudinal follow-up to assess skill retention or transfer to real patients.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The authors have addressed all the comments.

Author Response

We thank the reviewer for their valuable feedback and contribution to this work! 

Reviewer 2 Report

Comments and Suggestions for Authors

I thank the authors for their hard work and the paper is suitable for publication. The only minor edits I suggest is to move demographic data to the results section in the abstract rather than the methodology. Beyond this minor edit, I have no further edits. And most importantly, I agree with the authors that empathy can be taught. Keep up the good work and cannot wait to read the final product when it is published. 

Author Response

We thank the reviewer for their valuable feedback and contribution to this work! Minor edit regarding abstract has be addressed in the revised version. 

Reviewer 3 Report

Comments and Suggestions for Authors

Thank you. This reviewer has no further comments.

Author Response

We thank the reviewer for their valuable feedback and contribution to this work!