Review Reports

Round 1

Reviewer 1 Report

The overall manuscript is neat and written concisely—with relevant information for existing literature. One aspect that you can focus on is consistency in terminology and a more formal writing style. Some sections have an odd word choice and, therefore, might not be easily connected to existing literature. In most cases, the verb can be replaced by a more concise academic alternative.

Comments for author File: Comments.pdf

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

The article describes a study to explore the effectiveness of students' learning in science courses utilizing the model of constructive alignment. The article is based on a case study – the "Management Information System" course in China. The study is based on the comparison of results from a control group of students utilizing traditional teaching and learning techniques and approaches, and an experimental group utilizing the model of constructive alignment for the setup of the learning outcomes, activities and assessment tasks. The article describes the methodology with scientific rigours. The results are reported and discussed, underlining the positive impact of the application of the new model both in the grades of the students and in the qualitative assessment of the course. 

The literature review is well articulated and previous studies are reported and discussed. The article is overall well-referred. To improve clarity and easiness to the reader, I would recommend dividing the literature review into thematic sub-secitons.

In the conclusions, the limitations and the strengths of the study are reported. However, the article would benefit from a description of what are the implications of the study for education and discussing how the results of the study can be applied or useful for future research or extended to other contexts if this is possible. 

In the methodology, please emphasize and make very clear what are the strategy to approach your question about the effectiveness of the new model. Please, explain why you use choose to compare both the grades and a qualitative assessment of the course. In the discussion of the results, please emphasize if and how the results answer your research questions or objectives.

In the whole paper, clarity and style must be improved (e.g., lines 42, 45, 66, 110, 164 are just a few examples of unclear sentences due to the construction or wording). In general, I would suggest a moderate English check and changes, and elevating the style of the whole paper to a scientific standard by a proper selection of wording in some sentences (e.g., line 196, "But..."). Please check punctuation and spacing as there might be some typing mistakes (e.g., lines 80, 115, 155). A capital letter at line 65 to correct.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

The manuscript entitled "An Experimental Study on the Effectiveness of Students’ Learning in Scientific Course through Constructive Alignment—A Case from MIS Course" is an interesting work, trying to evaluate the learning effectiveness of two different approaches to the learning-teaching process.

1. The hypothesis "there is diversity among students in the mass stage of higher education, and based on the implementation of the constructive alignment initiative in science courses, every student can have a highly challenging and supportive learning experience, and students who originally had a weak academic foundation can benefit from it and meet the intended learning outcomes" is inappropriate to the study design and was not verified here. The authors did not examine "diversity among students in the mass stage of higher education", or "students who originally had a weak academic foundation", or who "can benefit from it and meet the intended learning outcomes". This study simply compared two different samples of students who attended the same MIS course, but using two different methods. The design of this study is cross-sectional, which does not allow causal conclusions. Therefore, the hypothesis should be rewritten, accessible and verifiable using statistical methods.
2. The Method section should be structuralized into subsections, such as "Procedure", "Participants characteristic", "Measurement", and "Statistical analysis". Each section should be exhaustively described to allow replication of the study. 
3. In is unclear what type of factor analysis was performed (EFA, PCA, or CFA), what extraction methods, etc. All information should be described in the Methods section, and all data presented in tables and described in the Results section (e.g., KMO and Bartlett test, factor weights, eigenvalues, reliability, etc.).
4. If one general factor was identified, why did the authors demonstrate analysis of particular questions in the Course Experience Questionnaire? The composite score should be considered for further statistical analysis.
5. It is unclear what is "The second effectiveness variable" and how it was accessed? How "academic performances" were achieved, and why it was converted to ordinal categories?
6. Analysis of parametric properties should be performed  (i.e., descriptive statistics, includeng M, SD, Mdn, Skewness, Kurtosis, normality test), presented in table, and commented, whether the normality assumption was met in the first and second sample, respectively.
7. Analysis of Student's t-test should include more statistics (apart of M and SD, t and p-value), like df and effect size stats.
8. If sample differences were examined 16 times (for each item separately, see Table 1), Bonferroni correction should be used for p-value.
9. It is unclear why Mann-Whitney U test was performed, and why scores were converted. Please see APA guideline to learn how this test should be reported.
10. The proportion of scores should be examined using Pearson's χ² test, with appropriate statistics (df, p, effect size).
11. Discussion should be changed and focus more on the results of this study.
12. Conclusions are not related to results. Practical conclusions should be stated. 

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 3 Report

Student's t-test was performed in the same sample of subjects 16 times and a Bonferoni correction is necessary. Even without it, the authors maintain that the groups differ statistically significantly, but they are not, e.g. in the first question p = 0.0531274 which means no statistical significance (although the authors interpret this result as significant). The effect size is 0.32 (authors claim 0.34). As such, the results are not credible, and interpretation is overused.

The df values in Table 1 do not make sense. These should be integers, not decimals. Apparently, the authors do not understand what the degrees of freedom statistic is.

The Pearson chi-square test should be performed separately for each grade and appropriate correction should be added for a small samples. 

Still nothing is known about the characteristics of the study participants (gender, age, field of study, year of study, place of residence, etc.). The replication of the results is therefore impossible.

In addition, if the author of this manuscript conducted an experiment and judged students on their achievements, the results may be inadequate due to a conflict of interest. 

Author Response

Please see the attachment.

Author Response File: Author Response.docx