Review Reports

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Thank you for  submitting your work.  The topic of the research is of significant interest to educators worldwide, particularly since the end of the pandemic.  The impact of instructional delivery on learning outcomes has been discussed widely but with little empiricism to back assertions made by proponents and critics.  Your  study provides a useful contribution to the literature to guide instructors and organizations in assessing  teaching delivery formats.

The study design, data  analysis, and discussion are all well crafted and  align with the study objectives.  Conclusions made are proportionate to the data presented.  I find no significant concerns with respect to statistical analyses used.  Overall this is  a well executed study examining  an issue of timely importance.

I have one  (small) recommendation to consider:  it appears as though students self-select into  the cohort of their  choice in this study.  While this is acknowledged in the manuscript, this reality fundamentally clouds the findings of  the study and raises questions as to causation and correlation.  Is there something unique  about those who self select  into the F2F group that would mean they would be more likely to succeed on exams anyway?  Is the instructional  delivery format the actual variable  that predicts performance,  or  is it the psychology/temperament of the learner who self-selects into the F2F group that is the more meaningful influence?  Of course, this  study cannot fully answer this question, but further reflection on this point would be useful to include in the discussion, with reference to existing literature that may help readers make  sense of this.

Beyond that, I  have no other significant concerns.  Some additional editing of  the text to make the prose more concise and easier to read may be beneficial.   Congratulations on this interesting piece of research!

Comments on the Quality of English Language

The prose is somewhat verbose and repetitive at times - some additional editing may be of benefit.

Author Response

Comment 1: Thank you for submitting your work. The topic of the research is of significant interest to educators worldwide, particularly since the end of the pandemic.  The impact of instructional delivery on learning outcomes has been discussed widely but with little empiricism to back assertions made by proponents and critics.  Your study provides a useful contribution to the literature to guide instructors and organizations in assessing teaching delivery formats.

Response 1: We sincerely thank the reviewer for the positive assessment of our study and for recognizing its relevance to educators and institutions considering instructional delivery formats. We appreciate the reviewer’s view that this work contributes empirical evidence to the ongoing discussion of face-to-face, online, and HyFlex education.

Comment 2: The study design, data analysis, and discussion are all well crafted and align with the study objectives. Conclusions made are proportionate to the data presented. I find no significant concerns with respect to statistical analyses used. Overall this is a well executed study examining an issue of timely importance.

Response 2: We thank the reviewer for the encouraging evaluation of the study design, statistical analysis, discussion, and conclusions. We are pleased that the reviewer considered the analyses appropriate and the conclusions proportionate to the data presented.

Comment 3: I have one (small) recommendation to consider: it appears as though students self-select into the cohort of their choice in this study. While this is acknowledged in the manuscript, this reality fundamentally clouds the findings of the study and raises questions as to causation and correlation. Is there something unique about those who self select into the F2F group that would mean they would be more likely to succeed on exams anyway? Is the instructional delivery format the actual variable that predicts performance, or is it the psychology/temperament of the learner who self-selects into the F2F group that is the more meaningful influence? Of course, this study cannot fully answer this question, but further reflection on this point would be useful to include in the discussion, with reference to existing literature that may help readers make sense of this.

Response 3: We agree that voluntary selection of attendance format is an important limitation and that learner characteristics may have influenced both attendance choice and learning outcomes. We therefore revised the Discussion to distinguish the potential effects of instructional format from the characteristics of learners selecting each format and added Athens (2023) as reference [22] to contextualize relevant self-regulatory, motivational, and practical factors (L324-345). The Limitations section now states concisely that the retrospective observational design and self-selected attendance format preclude causal inference and that the observed F2F advantage may partly reflect residual confounding (L359-370). We also added a brief qualification to the Abstract (L22-27). In the Conclusions, we retained the principal finding and its practical implication as the main emphasis: F2F attendance was associated with greater course-period learning gain and may be beneficial for supporting learning during the structured course period, while HyFlex support should remain responsive to individual learning needs and conditions. The qualification regarding self-selection is presented briefly at the end, together with the need for prospective multicenter studies to confirm the association and clarify the independent contribution of attendance format (L372-382).

Comment 4: Beyond that, I have no other significant concerns. Some additional editing of the text to make the prose more concise and easier to read may be beneficial. Congratulations on this interesting piece of research!

Response 4: We thank the reviewer for this helpful suggestion and the encouraging closing comment. We carefully edited the manuscript throughout to improve concision, readability, and sentence flow, while preserving the intended meaning and the precision of the scientific statements.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Nagakura et al. investigated the association between attendance format—face-to-face (F2F), live online (LO), and mixed (MIX), and learning outcomes in a four-month HyFlex preparatory course for sixth-year pharmacy students. Using retrospective data over three academic years (FY2023–FY2025), the authors compared learning gains between pre- and post-course examinations and NEP pass rates across the three attendance groups. The F2F group demonstrated significantly greater course-period learning gains than the LO and MIX groups; however, differences in NEP pass rates did not reach statistical significance. This study addresses a timely and relevant issue, as HyFlex delivery is increasingly used in pharmacy education, while its relationship with course-level and licensure-examination outcomes remains insufficiently understood. The manuscript is well organized, clearly presented, and supported by appropriate statistical analyses, including a sensitivity analysis addressing the attendance-based inclusion criterion. Nevertheless, several methodological, statistical, and interpretative concerns should be addressed before the manuscript can be considered suitable for publication.

1. Because students self-selected their attendance format rather than being randomly assigned, selection bias cannot be excluded. Factors such as motivation, self-regulated learning ability, commuting burden, and learning environment may have influenced both attendance choice and academic performance. In particular, students who deliberately selected F2F attendance may already have been more motivated, disciplined, or academically engaged. This could have inflated the apparent advantage of F2F attendance and should be discussed more clearly as an important limitation of the study.
2. The novelty of this study relative to the authors’ previous report (reference 15; line 77), which analysed FY2023–FY2024 data, should be clarified in the Introduction. The authors should explicitly identify the knowledge gap remaining after the earlier study and explain how including a third academic year and NEP pass/fail outcomes provides a distinct contribution.
3. The substantial imbalance between the F2F and LO groups (n = 38 vs. n = 199) may reduce the precision and generalizability of the F2F-related findings. The authors should discuss this issue more thoroughly and report confidence intervals for the estimated group differences.
4. Authors has to double-check the numbers in the tables. For example, Table 2 lists "45.1 ± 0.6" for the LO group's pre-course score, which looks like a typo (the SD seems too small compared to other groups). Also, Table 4's row "Practical Pharmacy 95)" is missing an opening parenthesis.
5. The methods state that the laws/regulations/ethics component contained 30 questions (line 125), whereas Tables 2 and 4 list a maximum score of 40. Please correct this inconsistency.

Author Response

Comment 1: Nagakura et al. investigated the association between attendance format—face-to-face (F2F), live online (LO), and mixed (MIX), and learning outcomes in a four-month HyFlex preparatory course for sixth-year pharmacy students. Using retrospective data over three academic years (FY2023–FY2025), the authors compared learning gains between pre- and post-course examinations and NEP pass rates across the three attendance groups. The F2F group demonstrated significantly greater course-period learning gains than the LO and MIX groups; however, differences in NEP pass rates did not reach statistical significance. This study addresses a timely and relevant issue, as HyFlex delivery is increasingly used in pharmacy education, while its relationship with course-level and licensure-examination outcomes remains insufficiently understood. The manuscript is well organized, clearly presented, and supported by appropriate statistical analyses, including a sensitivity analysis addressing the attendance-based inclusion criterion. Nevertheless, several methodological, statistical, and interpretative concerns should be addressed before the manuscript can be considered suitable for publication.

Response 1: We sincerely thank the reviewer for the careful evaluation of our manuscript and for recognizing the relevance, organization, and analytical approach of the study. We have addressed each methodological, statistical, and interpretative concern below and revised the manuscript accordingly.

Comment 2: Because students self-selected their attendance format rather than being randomly assigned, selection bias cannot be excluded. Factors such as motivation, self-regulated learning ability, commuting burden, and learning environment may have influenced both attendance choice and academic performance. In particular, students who deliberately selected F2F attendance may already have been more motivated, disciplined, or academically engaged. This could have inflated the apparent advantage of F2F attendance and should be discussed more clearly as an important limitation of the study.

Response 2: We agree that self-selection is an important limitation. Because attendance format was chosen voluntarily, students selecting F2F attendance may have differed in unmeasured learner characteristics that influenced both attendance choice and academic performance. Although pre-course scores and sex distribution did not differ significantly among groups, these comparisons do not eliminate residual confounding. We therefore expanded the Discussion to explain how learner characteristics may have contributed to the observed association and added Athens (2023) as reference [22] (L324-345). To avoid duplication, the revised Limitations section states concisely that the retrospective observational design and self-selected attendance format preclude causal inference and that the observed F2F advantage may partly reflect residual confounding (L359-370). We also added a brief qualification to the Abstract (L22-27). In the revised Conclusions, however, we emphasize the main result and its practical relevance: F2F attendance was associated with greater course-period learning gain and may be beneficial during the structured course period, while HyFlex support should consider individual learning needs and conditions. The self-selection qualification is retained briefly in the final sentence, together with the need for prospective multicenter studies to confirm the association and better address selection effects (L372-382).

Comment 3: The novelty of this study relative to the authors’ previous report (reference 15; line 77), which analysed FY2023–FY2024 data, should be clarified in the Introduction. The authors should explicitly identify the knowledge gap remaining after the earlier study and explain how including a third academic year and NEP pass/fail outcomes provides a distinct contribution.

Response 3: We appreciate this important point. We clarified the novelty of the present study in the Introduction. Our previous report analyzed FY2023–FY2024 data and focused on course-period examination-based learning outcomes; it could not evaluate the relationship between attendance format and the final NEP outcome because the available sample was insufficient. The present study adds FY2025 data, increases the sample size, evaluates NEP pass/fail or non-attendance outcomes approximately one month after course completion, and includes a sensitivity analysis incorporating students below the 90% attendance threshold. We now explicitly identify the remaining knowledge gap and explain that the distinct contribution of this study is the joint examination of course-period learning gain and subsequent licensure-examination performance across three academic years (L74-93).

Comment 4: The substantial imbalance between the F2F and LO groups (n = 38 vs. n = 199) may reduce the precision and generalizability of the F2F-related findings. The authors should discuss this issue more thoroughly and report confidence intervals for the estimated group differences.

Response 4: We agree that the marked imbalance in group sizes, particularly the small F2F group, limits precision, statistical power, and the generalizability of estimates concerning F2F attendance. We expanded the Limitations section to emphasize this issue and tempered the interpretation of F2F-related findings (L362-364). We also revised the statistical reporting to include 95% confidence intervals for the estimated between-group differences in overall learning gain, in addition to effect sizes and adjusted P values (L224-241). These intervals make the uncertainty surrounding the group comparisons more transparent.

Comment 5: Authors has to double-check the numbers in the tables. For example, Table 2 lists "45.1 ± 0.6" for the LO group's pre-course score, which looks like a typo (the SD seems too small compared to other groups). Also, Table 4's row "Practical Pharmacy 95)" is missing an opening parenthesis.

Response 5: Thank you for identifying these errors. We carefully rechecked all numerical entries and labels in the tables against the source data. In Table 2, the LO-group pre-course overall score has been corrected from “45.1 ± 0.6” to the verified value “45.1 ± 10.6.” In Table 4, the label has been corrected to “Practical Pharmacy (95).” We also reviewed the remaining tables for similar typographical or transcription errors and corrected them where necessary.

Comment 6: The methods state that the laws/regulations/ethics component contained 30 questions (line 125), whereas Tables 2 and 4 list a maximum score of 40. Please correct this inconsistency.

Response 6: We thank the reviewer for noting this inconsistency. The laws/regulations/ethics component contained 30 questions, as stated in the Methods. We therefore corrected the maximum score shown in Tables 2 and 4 from 40 to 30 and confirmed that the reported percentages were calculated using the correct denominator.

Author Response File: Author Response.pdf