Assessment-Focused Pedagogical Methods for Improving Student Learning Process and Academic Outcomes in Accounting Disciplines

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The paper presents an interesting and timely study on how assessment-focused pedagogical methods, specifically practice testing and student-generated questions, enhance student learning outcomes in accounting within a blended learning environment, grounded in Self-Determination Theory (SDT). Conducted at a top Romanian economics university, the study follows 107 students over two semesters. In general, the article is written correctly and fits into the scope of the Education Sciences journal.

The strengths of the paper, in my opinion, are:

- Longitudinal Design - the study tracks students across two semesters,

- The study effectively applies Self-Determination Theory to frame its pedagogical interventions and interpret the results,

- The authors used a rigorous statistical analysis to validate findings.

- The study provides practical recommendations for educators, demonstrating the efficacy of assessment-based strategies in blended learning environments.

I see also some weaknesses:

-             Limited Sample Size – with only 107 students from a single institution, the findings may not be generalizable to other universities or disciplines. Of course, I realize that it is not easy to conduct such research on a larger group.

- Problem with control group, because the study relies on post-hoc clustering rather than a pre-determined experimental and control group, limiting causal interpretations.

- The assessment of learning effectiveness includes self-reported data (subjective student perceptions).

- Lack of qualitative insights (students’ experiences in depth), so missing potential interesting information from interviews or open-ended surveys.

I have some recommendation to improve for Authors:

1. If it is possible: incorporate qualitative data (use student interviews or focus groups to complement quantitative findings if  you have it).

2. Include studies more newly published studies from 2022–2024 to incorporate the latest findings in this area of education.

3. Provide more practical guidelines for educators based on the study’s findings.

To strengthen the literature review and discussion part, the following recently published studies (2022–2024) could be included, e.g.:

Accounting education and digitalization: A new perspective after the pandemic (2023), https://www.sciencedirect.com/science/article/pii/S147281172300085X

Retaining remote teaching and assessment methods in accounting education: Drivers and challenges in the post-pandemic era (2023), https://www.sciencedirect.com/science/article/abs/pii/S1472811723000484

PACE-IT: designing blended learning for accounting education in the challenging context of a global pandemic (2022) https://www.tandfonline.com/doi/abs/10.1080/09639284.2022.2090851

In my opinion this additions could improve the quality of the paper.

Author Response

Reviewer 1

Comments and Suggestions for Authors

The paper presents an interesting and timely study on how assessment-focused pedagogical methods, specifically practice testing and student-generated questions, enhance student learning outcomes in accounting within a blended learning environment, grounded in Self-Determination Theory (SDT). Conducted at a top Romanian economics university, the study follows 107 students over two semesters. In general, the article is written correctly and fits into the scope of the Education Sciences journal.

The strengths of the paper, in my opinion, are:

- Longitudinal Design - the study tracks students across two semesters,

- The study effectively applies Self-Determination Theory to frame its pedagogical interventions and interpret the results,

- The authors used a rigorous statistical analysis to validate findings.

- The study provides practical recommendations for educators, demonstrating the efficacy of assessment-based strategies in blended learning environments.

Thank you for the comment, this helps us better structure our paper! Also, the entire paper was carefully proofread with the help of Writefull premium. The most important additions were added in red, the other corrections in purple.

I see also some weaknesses:

-             Limited Sample Size – with only 107 students from a single institution, the findings may not be generalizable to other universities or disciplines. Of course, I realize that it is not easy to conduct such research on a larger group.

- Problem with control group, because the study relies on post-hoc clustering rather than a pre-determined experimental and control group, limiting causal interpretations.

- The assessment of learning effectiveness includes self-reported data (subjective student perceptions).

- Lack of qualitative insights (students’ experiences in depth), so missing potential interesting information from interviews or open-ended surveys.

All of these elements, which have been correctly highlighted by the reviewer, have been included as limitations. Regarding the sample size, Table 1 has been included to prove that there is a high variability in sample sizes in this domain and that 107 in a matched sample is a satisfactory sample (not for generalization purposes). The absence of the control group is due to administrative restrictions, because doing experimental design research in educational settings is implicitly prohibited in our university. The questionnaire is based on self-reported data, but the main assessment of learning effectiveness is based on performance scores collected for different tasks. Overall, the limitation of the sample size is due to the focus on maintaining the same research settings for the entire group (same tutor at the lecture for two semesters, same evaluation methods etc.). The homogeneity of pedagogical methods is a strength of this research.

I have some recommendation to improve for Authors:

1. If it is possible: incorporate qualitative data (use student interviews or focus groups to complement quantitative findings if  you have it).

Thank you for the comment! We did not collect qualitative data from the sample. We collected two types of data, which are already included in the study: 1) the data referring to the activity of the students (every week and learning outcomes); 2) students’ answers to an ad-hoc questionnaire distributed at the end of second semester included in this study. The questionnaire is self-reported, but the performance evaluation data is collected by the professor using homogenous assessment scales and criteria. Therefore, the study links the performance data with the opinions of students regarding their performance and motivation. 

2. Include more newly published studies from 2022–2024 to incorporate the latest findings in this area of education.

Thank you for the comment! We read and integrated additional references from the period 2022-2024. Also, most of the sources cited in the paper are related to accounting education, which is the focus of this paper. By limiting our literature analysis in Table 1 to accounting research, we want to emphasize that this set of disciplines have certain particularities, being technical, logical, and practice-oriented.

3. Provide more practical guidelines for educators based on the study’s findings.

We provided more practical guidelines for educators in the final section of the paper.

To strengthen the literature review and discussion part, the following recently published studies (2022–2024) could be included, e.g.:

Accounting education and digitalization: A new perspective after the pandemic (2023), https://www.sciencedirect.com/science/article/pii/S147281172300085X

Retaining remote teaching and assessment methods in accounting education: Drivers and challenges in the post-pandemic era (2023), https://www.sciencedirect.com/science/article/abs/pii/S1472811723000484

PACE-IT: designing blended learning for accounting education in the challenging context of a global pandemic (2022) https://www.tandfonline.com/doi/abs/10.1080/09639284.2022.2090851

In my opinion these additions could improve the quality of the paper.

Thank you for your generous support!

Reviewer 2 Report

Comments and Suggestions for Authors

This study aims to present and validate a pedagogical method based on practice testing and student-generated questions in a blended learning environment. While the research objectives and foundation are meaningful to some extent, several improvements are necessary to enhance the clarity and rigor of the manuscript.  

1. The research questions must be explicitly stated at the end of the introduction. Currently, the research hypotheses are presented sporadically in Section 3, causing the core of the study to appear fragmented. It is essential to restructure and clearly specify these elements to ensure coherence. Additionally, the results section should directly address the research questions, explicitly explaining how the questions are answered. The introduction should also provide a clearer rationale for the research questions, supported by relevant literature.

 2. Irrelevant content (Lines 152–153):

The following statement seems out of context and does not fit well within the surrounding discussion: “The objective of the present study is to present and validate a pedagogical method based on practice testing and student-generated questions, delivered in a blended learning environment.”

Please clarify its purpose or consider revising or removing it to maintain logical flow.

3. Section 2.2 introduces various pedagogical methods, but the content currently reads as a simple list of previous studies. Instead of merely describing these methods, restructure this section to focus on how such pedagogical approaches have been implemented in management accounting courses, including their effectiveness, challenges, and limitations. Moreover, the current title, "Pedagogical Methods," is overly broad and abstract. It should be revised to better reflect the specific focus of this study.

4. Sample size limitation (n = 107)

The sample size of 107 students is quite small for generalizing the results in a quantitative study. To strengthen the research, consider either: expanding the sample size through repeated studies to validate the findings, or providing a strong rationale for why the current sample is sufficient, supported by relevant methodological literature. Additionally, the limited sample size must be explicitly mentioned as a limitation of the study. Addressing this issue is crucial for the manuscript to meet the publication standards of an international journal.

5. Misclassification as a longitudinal study

The study is described as a longitudinal study, but this classification is not entirely accurate. Longitudinal studies typically involve data collection over several years to analyze changes over time. Since this study spans only two semesters and does not explicitly analyze temporal changes, it would be more appropriate to refer to it as a short-term longitudinal study or, preferably, a repeated measures study. Please revise this terminology accordingly to maintain methodological accuracy.

6. The discussion section should offer a more thoughtful interpretation of the results. Begin by clearly articulating the key findings and explaining how these findings confirm or challenge existing theories. Furthermore, please discuss whether the results are consistent with or differ from previous research in detail. If there are unexpected or significant findings, address them explicitly and provide possible explanations. Please expand on both the theoretical and practical implications of the study. Additionally, please acknowledge the study’s limitations more thoroughly, explaining how they may have influenced the results. 

Comments on the Quality of English Language

This study aims to present and validate a pedagogical method based on practice testing and student-generated questions in a blended learning environment. While the research objectives and foundation are meaningful to some extent, several improvements are necessary to enhance the clarity and rigor of the manuscript.  

1. The research questions must be explicitly stated at the end of the introduction. Currently, the research hypotheses are presented sporadically in Section 3, causing the core of the study to appear fragmented. It is essential to restructure and clearly specify these elements to ensure coherence. Additionally, the results section should directly address the research questions, explicitly explaining how the questions are answered. The introduction should also provide a clearer rationale for the research questions, supported by relevant literature.

 2. Irrelevant content (Lines 152–153):

The following statement seems out of context and does not fit well within the surrounding discussion: “The objective of the present study is to present and validate a pedagogical method based on practice testing and student-generated questions, delivered in a blended learning environment.”

Please clarify its purpose or consider revising or removing it to maintain logical flow.

3. Section 2.2 introduces various pedagogical methods, but the content currently reads as a simple list of previous studies. Instead of merely describing these methods, restructure this section to focus on how such pedagogical approaches have been implemented in management accounting courses, including their effectiveness, challenges, and limitations. Moreover, the current title, "Pedagogical Methods," is overly broad and abstract. It should be revised to better reflect the specific focus of this study.

4. Sample size limitation (n = 107)

The sample size of 107 students is quite small for generalizing the results in a quantitative study. To strengthen the research, consider either: expanding the sample size through repeated studies to validate the findings, or providing a strong rationale for why the current sample is sufficient, supported by relevant methodological literature. Additionally, the limited sample size must be explicitly mentioned as a limitation of the study. Addressing this issue is crucial for the manuscript to meet the publication standards of an international journal.

5. Misclassification as a longitudinal study

The study is described as a longitudinal study, but this classification is not entirely accurate. Longitudinal studies typically involve data collection over several years to analyze changes over time. Since this study spans only two semesters and does not explicitly analyze temporal changes, it would be more appropriate to refer to it as a short-term longitudinal study or, preferably, a repeated measures study. Please revise this terminology accordingly to maintain methodological accuracy.

6. The discussion section should offer a more thoughtful interpretation of the results. Begin by clearly articulating the key findings and explaining how these findings confirm or challenge existing theories. Furthermore, please discuss whether the results are consistent with or differ from previous research in detail. If there are unexpected or significant findings, address them explicitly and provide possible explanations. Please expand on both the theoretical and practical implications of the study. Additionally, please acknowledge the study’s limitations more thoroughly, explaining how they may have influenced the results. 

Author Response

Reviewer 2

Comments and Suggestions for Authors

This study aims to present and validate a pedagogical method based on practice testing and student-generated questions in a blended learning environment. While the research objectives and foundation are meaningful to some extent, several improvements are necessary to enhance the clarity and rigor of the manuscript.  

1. The research questions must be explicitly stated at the end of the introduction. Currently, the research hypotheses are presented sporadically in Section 3, causing the core of the study to appear fragmented. It is essential to restructure and clearly specify these elements to ensure coherence. Additionally, the results section should directly address the research questions, explicitly explaining how the questions are answered. The introduction should also provide a clearer rationale for the research questions, supported by relevant literature.

Thank you for the comment, this helps us better structure our paper! Also, the entire paper was proofread with the help of Writefull premium. The most important additions were added in red, the other corrections in purple.

In the introduction, the research question of our paper is mentioned: How are the student learning process and learning outcomes (in the field of management accounting) influenced by the application of a pedagogical method consisting of practice testing and student-generated questions in the blended learning environment?

In order to answer the research question, the following hypotheses were summarized at the beginning of Section 4 Results.:

H1: Learning outcomes (quiz average, bonus, tutorial, and exam scores) are positively correlated with consistent learning.

H2: Consistent learning at the MA discipline is positively correlated with consistent learning at the PMC discipline.

H3: Maintaining the learning strategy for two consecutive semesters in related disciplines significantly improves the students’ learning outcomes in the second discipline.

H4: Learning outcomes are positively correlated for the two disciplines.

H5: The voluntary adoption of the learning strategy leads to a significant improvement in the student learning outcomes in both disciplines.

H6: Consistent learning is correlated with the importance attached to quizzes and homework, and with the students’ choice to solve additional tasks.

H7: Consistent learning is a direct predictor of exam scores in the second discipline.

We restructured the introduction and supporting evidence on the effectiveness of the pedagogical method suggested. The results section is structured so that it follows the research hypotheses. Also, in the final part of the section there is a table summarising the validation or rejection of the hypotheses.

2. Irrelevant content (Lines 152–153): The following statement seems out of context and does not fit well within the surrounding discussion: “The objective of the present study is to present and validate a pedagogical method based on practice testing and student-generated questions, delivered in a blended learning environment.” Please clarify its purpose or consider revising or removing it to maintain logical flow.

Thank you for the comment! We deleted the marked statement.

3. Section 2.2 introduces various pedagogical methods, but the content currently reads as a simple list of previous studies. Instead of merely describing these methods, restructure this section to focus on how such pedagogical approaches have been implemented in management accounting courses, including their effectiveness, challenges, and limitations. Moreover, the current title, "Pedagogical Methods," is overly broad and abstract. It should be revised to better reflect the specific focus of this study.

Thank you for the comment! We changed the title of the section, which is now “Pedagogical methods for improving student learning process and academic outcomes.” The section starts with the most representative studies regarding the management accounting pedagogical methods. It continues with paragraphs for the pedagogical methods used for the present study (practice testing and question generation), the use of digital technologies for the pedagogical methods and developments in accounting education after the COVID 19 pandemic. Most of the literature cited is from the accounting domain.

4. Sample size limitation (n = 107). The sample size of 107 students is quite small for generalizing the results in a quantitative study. To strengthen the research, consider either: expanding the sample size through repeated studies to validate the findings, or providing a strong rationale for why the current sample is sufficient, supported by relevant methodological literature. Additionally, the limited sample size must be explicitly mentioned as a limitation of the study. Addressing this issue is crucial for the manuscript to meet the publication standards of an international journal.

We provided this rationale in Section 3. The data described in Table 3 was obtained for the entire cohort that attended the lectures, allowing the creation of a homogeneous database. As a limitation, the sample size of 107 students is small for generalizing the results beyond the studied population. As in any study, the sample size is a trade-off between reaching statistical power and maintaining the rigorous nature of data collection. This view is supported by Hu et al. (2023, p.662), which stated that “the use of balanced panel data reduces the noise introduced by participant heterogeneity.” In our case, the pedagogical method was applied to a single cohort of students with the same professor at the lecture, who maintained a rigorous application of the evaluation methods for both disciplines. The academic context did not allow extending the research to other cohorts, other disciplines, or other time periods. 

5. Misclassification as a longitudinal study. The study is described as a longitudinal study, but this classification is not entirely accurate. Longitudinal studies typically involve data collection over several years to analyze changes over time. Since this study spans only two semesters and does not explicitly analyze temporal changes, it would be more appropriate to refer to it as a short-term longitudinal study or, preferably, a repeated measures study. Please revise this terminology accordingly to maintain methodological accuracy.

Thank you for the comment! We revised the terminology accordingly. 

6. The discussion section should offer a more thoughtful interpretation of the results. Begin by clearly articulating the key findings and explaining how these findings confirm or challenge existing theories. Furthermore, please discuss whether the results are consistent with or differ from previous research in detail. If there are unexpected or significant findings, address them explicitly and provide possible explanations. Please expand on both the theoretical and practical implications of the study. Additionally, please acknowledge the study’s limitations more thoroughly, explaining how they may have influenced the results. 

We provided more practical guidelines for educators in the final section of the paper. The limitations section was significantly expanded to integrate the reviewers’ comments and explanations of these limitations. Thank you for your generous support!

 

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

The submitted manuscript is not academically outstanding, as it is constrained by fundamental limitations in research design, methodology, and sample selection. However, the reviewers' comments have been largely incorporated, resulting in a generally moderate-quality manuscript.

Comments on the Quality of English Language

The submitted manuscript is not academically outstanding, as it is constrained by fundamental limitations in research design, methodology, and sample selection. However, the reviewers' comments have been largely incorporated, resulting in a generally moderate-quality manuscript.