Investigating Metacognitive Strategies and Exam Performance: A Cross-Sectional Survey Research Study

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

82%  response rate in answering survey from the class: very impressive!  121 of the 182 students taking the course completed survey.  Responses included a highly representative sampling of all demographics in the course and at the institution.  Congratulations.

Author Response

Greetings. Please see the attached Word document for our point-by-point response to your review. Thank you for participating in this masked review process. The revised manuscript is much better due to comments from all three reviewers.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

Title: “Impact of Student Behaviors Preparing for and During Exams on Test Scores”

The manuscript aims to investigate investigated impact of student behaviors preparing for and during examinations upon those scores in an undergraduate history course. While the authors’ topic of research contributes something worthwhile to the field of higher education teaching, learning, and assessment, it does have several conceptual and methodological issues. In my view, the authors need to address the following comments.

1. Beginning from the manuscript’s topic, the authors should use a more suitable word to highlight the ‘retrospective nature of the study’ and its specific focus on ‘test scores of a History course’. In addition, words such as contribution, relationship, or predictability’ is more appropriate than ‘impact’.

2. The introduction section was not enough as it states about undergraduate instructors’ academic support and actions and the study context. Re-write the introduction section with a focus on providing a general overview of the topic of interest and a clear understanding of the variables used in the study. Often, the introduction serves as the basis for the paper, directing your reader’s attention quickly to the issues that your research project addresses and providing the reader with a concise statement of the research study itself.

In the introduction section, it was not clear whether the study answers new questions or introduces novel interventions. In addition, it was not clear whether the problem statement provides evidence of an important barrier, gap, or inefficiency that is not currently addressed through other interventions. What is the evidence that suggests that the proposed intervention, ‘actions that a classroom history instructor took to assist students to earn higher grades on their first examination of the academic term' will improve target outcomes? This empirical evidence serves as a base for framing the study hypothesis.

3. Please, move the third paragraph on page 1 to the setting section, on page 3.

4. The authors did not seem to give enough attention to the research questions. The one on page 3, lines 123 to 124, was inadequate as it addresses only exam preparation behaviors by students and how these correlate with their first major exam grade. The authors should include the research questions in much fuller terms, including research questions that guide the descriptive statistics, group difference tests, and the hierarchical regression analysis. In addition, the different parts of the manuscript, including the methods, results, and conclusions, must align with the research questions addressed. Due to the absence of research questions related to these core issues, it was difficult to understand why the authors conducted such statistics.

On top of this, although the authors used group difference tests and regression analysis, I did not see the hypotheses that need to be derived from the literature (theory or empirically based). In the introduction section, there is limited justification as to why the authors chose to investigate group differences and relationships between variables of interest; as well as, what the previous literature says about group differences and relationships of the selected variables.

While addressing these, please make sure that your hypothesis includes a statement that expresses a possible relationship between variables or phenomena, based on existing knowledge, theory, or observation. Again, make sure that your research question is a question that seeks to explore or explain an aspect of your topic, without making any assumptions or claims.

5. The literature review’s three components did not seem appropriate because this section tells very little about the core issues presented in the title. The stated three components are relevant topics, but they are far removed from the central case of the manuscript. For me, the literature review section’s last paragraph, on pages 2 and 3, lines 90 to 101, is the only relevant piece related to the topic of interest. I strongly suggest the authors review the literature on test scores in undergraduate history courses, students’ preparation for exams, and students’ behaviors during exams. In addition, the ‘Constructivist Theory’ would have served as a theoretical foundation if the authors had explained its linkage to the substantive contents of the manuscript.

6. The other issue I have is with the Materials and Methods section, specifically with the study design section. I was not sure that the study design was a survey. In my view, the study design sounds like an ‘ex post facto research design,’ in which ‘students groups with some behaviors of preparing for and during exams’ that already exist are compared on the dependent variable ‘test scores of History exams’. I am not accepting the word impact in the title of the manuscript because an ex post facto design is considered quasi-experimental because the subjects are not randomly assigned as they are grouped based on a particular characteristic or trait. Hence, the authors should convince their readers that this manuscript reports on a study based on a survey design.

7. Again, in the abstract and data analysis section, the word inferential statistics is too broad as it includes a wide variety of analytical methods. I suggest the authors use the specific types of inferential statistics used (independent samples t-test and the appropriate regression analysis method). In addition, they have to specify the kinds of descriptive statistics used like frequency, percentage, or others. While the established data analysis was hierarchical regression analysis, this did not occur in the results section. Hence, this should be clarified.

8.     In the results section, the authors used SPSS outputs instead of standard output reporting for regression analysis results. In addition, the authors used an item-level analysis for quantifying frequencies and categorizing student groups into 'completed' and 'did not complete' categories. This item-level analysis is erroneous for holistically categorizing students into the assumed groups and answering the research question related to the relationship. At the end of the day, the authors should classify students based on their behaviors in preparing for and during exams. For this, the authors should have two independent variables or more depending on justifiable reasons from the literature. An item-level analysis did not work for that.

9.  Most importantly, the authors need to include a more accurate version of the limitations of the study based on the ex post facto nature of the study design, its underlying assumptions, and the variables measured. The aim of this suggestion is to make the reading more affordable so that readers would take the conclusions cautiously.

10. This manuscript benefits from language editing. There are some language issues in this manuscript. For example, in the abstract, words or phrases such as ‘Made an outline of,’ and 'spent' should be corrected as 'outlined' and 'spend,' respectively. In addition, phrases such as 'submitted to SPSS' (p. 5, line 109) and 'nearly statistically significant (p. 11, line 358),' should be replaced with phrases such as 'entered to SPSS' and 'marginally significant,' respectively. In addition, the phrase 'narrowly missed statistical significance' (p. 10, line 339) should be replaced by the phrase 'marginally significant'.   

I hope these suggestions are of use to the author(s). 

Comments on the Quality of English Language

Title: “Impact of Student Behaviors Preparing for and During Exams on Test Scores”

Thank you for the opportunity to review this manuscript!

The manuscript aims to investigate investigated impact of student behaviors preparing for and during examinations upon those scores in an undergraduate history course. While the authors’ topic of research contributes something worthwhile to the field of higher education teaching, learning, and assessment, it does have several conceptual and methodological issues. In my view, the authors need to address the following comments.

1. Beginning from the manuscript’s topic, the authors should use a more suitable word to highlight the ‘retrospective nature of the study’ and its specific focus on ‘test scores of a History course’. In addition, words such as contribution, relationship, or predictability’ is more appropriate than ‘impact’.

2. The introduction section was not enough as it states about undergraduate instructors’ academic support and actions and the study context. Re-write the introduction section with a focus on providing a general overview of the topic of interest and a clear understanding of variables used in the study. Often, the introduction serves as the basis for the paper, directing your reader’s attention quickly to the issues that your research project addresses and providing the reader with a concise statement of the research study itself.

In the introduction section, it was not clear whether the study answers new questions or introduces novel interventions. In addition, it was not clear whether the problem statement provides evidence of an important barrier, gap, or inefficiency that is not currently addressed through other interventions. What is the evidence that suggests that the proposed intervention, ‘actions that a classroom history instructor took to assist students to earn higher grades on their first examination of the academic term' will improve target outcomes? This empirical evidence serves as a base for framing the study hypothesis.

3. Please, move the third paragraph on page 1 to the setting section, on page 3.

4. The authors did not seem to give enough attention to the research questions. The one on page 3, lines 123 to 124, was inadequate as it addresses only exam preparation behaviors by students and how these correlate with their first major exam grade. The authors should include the research questions in much fuller terms, including research questions that guide the descriptive statistics, group difference tests, and the hierarchical regression analysis. In addition, the different parts of the manuscript, including the methods, results, and conclusions, must align with the research questions addressed. Due to the absence of research questions related to these core issues, it was difficult to understand why the authors conducted such statistics.

On top of this, although the authors used group difference tests and regression analysis, I did not see the hypotheses that need to be derived from the literature (theory or empirically based). In the introduction section, there is limited justification as to why the authors chose to investigate group differences and relationships between variables of interest; as well as, what the previous literature says about group differences and relationships of the selected variables.

While addressing these, please make sure that your hypothesis includes a statement that expresses a possible relationship between variables or phenomena, based on existing knowledge, theory, or observation. Again, make sure that your research question is a question that seeks to explore or explain an aspect of your topic, without making any assumptions or claims.

5. The literature review’s three components did not seem appropriate because this section tells very little about the core issues presented in the title. The stated three components are relevant topics, but they are far removed from the central case of the manuscript. For me, the literature review section’s last paragraph, on pages 2 and 3, lines 90 to 101, is the only relevant piece related to the topic of interest. I strongly suggest the authors review the literature on test scores in undergraduate history courses, students’ preparation for exams, and students’ behaviors during exams. In addition, the ‘Constructivist Theory’ would have served as a theoretical foundation if the authors had explained its linkage to the substantive contents of the manuscript.

6. The other issue I have is with the Materials and Methods section, specifically with the study design section. I was not sure that the study design was a survey. In my view, the study design sounds like an ‘ex post facto research design,’ in which ‘students groups with some behaviors of preparing for and during exams’ that already exist are compared on the dependent variable ‘test scores of History exams’. I am not accepting the word impact in the title of the manuscript because an ex post facto design is considered quasi-experimental because the subjects are not randomly assigned as they are grouped based on a particular characteristic or trait. Hence, the authors should convince their readers that this manuscript reports on a study based on a survey design.

7. Again, in the abstract and data analysis section, the word inferential statistics is too broad as it includes a wide variety of analytical methods. I suggest the authors use the specific types of inferential statistics used (independent samples t-test and the appropriate regression analysis method). In addition, they have to specify the kinds of descriptive statistics used like frequency, percentage, or others. While the established data analysis was hierarchical regression analysis, this did not occur in the results section. Hence, this should be clarified.

8.     In the results section, the authors used SPSS outputs instead of standard output reporting for regression analysis results. In addition, the authors used an item-level analysis for quantifying frequencies and categorizing student groups into 'completed' and 'did not complete' categories. This item-level analysis is erroneous for holistically categorizing students into the assumed groups and answering the research question related to the relationship. At the end of the day, the authors should classify students based on their behaviors in preparing for and during exams. For this, the authors should have two independent variables or more depending on justifiable reasons from the literature. An item-level analysis did not work for that.

9.  Most importantly, the authors need to include a more accurate version of the limitations of the study based on the ex post facto nature of the study design, its underlying assumptions, and the variables measured. The aim of this suggestion is to make the reading more affordable so that readers would take the conclusions cautiously.

10. This manuscript benefits from language editing. There are some language issues in this manuscript. For example, in the abstract, words or phrases such as ‘Made an outline of,’ and 'spent' should be corrected as 'outined' and 'spend,' respectively. In addition, phrases such as 'submitted to SPSS' (p. 5, line 109) and 'nearly statistically significant (p. 11, line 358),' should be replaced with phrases such as 'entered to SPSS' and 'marginally significant,' respectively. In addition, the phrase 'narrowly missed statistical significance' (p. 10, line 339) should be replaced by the phrase 'marginally significant'.   

I hope these suggestions are of use to the author(s). 

Many thanks for the opportunity to review this paper!

Author Response

Greetings. Please see the attached Word document for our point-by-point response to your review. Thank you for participating in this masked review process. The revised manuscript is much better due to comments from all three reviewers.

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

Please see attached document.

Comments for author File: Comments.pdf

Comments on the Quality of English Language

Some minor editorial reviewing needed (found 1-2 incorrect/misspelled words)

Author Response

Greetings. Please see the attached Word document for our point-by-point response to your review. Thank you for participating in this masked review process. The revised manuscript is much better due to comments from all three reviewers.

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

Dear Authors,

Please accept my sincere gratitude for making the effort to significantly improve your manuscript. I am happy with the improvement the authors made with this revised version. However, I have observed some areas where the authors need further revision.

1. I'm not sure what the authors were attempting to say with the new title. I believe that you can compare between groups (groups that are not the same) or within groups (the same group taken at two different time points), but not differences in one variable on another specific variable. Rephrase the title within the scope of the study.

2. I believe the revised title makes it obvious that the primary variable the authors are comparing is whether or not there are differences in the ‘major exam results in a high-enrollment undergraduate course’. I was wondering, where is the summary of results you reported on the exam scores for the students registered for the course?

3. Additionally, on p. 6, lines 263-266, those two statements do not agree with one another. While the first statement deals with examination preparation behaviors, the second statement talks about the comparison of strategies. Rephrase those statements considering the central point of comparison within the scope of your study.

4. The term ‘cognitive task strategy’ needs to be conceptually clarified within the scope of the study. Make sure this conceptualization did not confuse with other related terms found in the manuscript, for example, ‘metacognitive strategies’ (p. 1, line 26), cognitive strategies (p. 4, line 153), task strategies' (p. 4, line 156), and strategies (p. 6, line 265)

5. Specify whether you employed a cross-sectional or longitudinal survey design in both the abstract and the methods section.

6.     On p. 5, line 243, I did not understand the word ‘hirst’? Also, on the same page, line 245, change ‘the’ to ‘The’. Additionally, on p. 5, line 229, ‘ini’ rewrite this word correctly. Also, on p. 11, line 450, remove the word ‘influence’.

7. I did not understand what you wanted to communicate with the statement on p. 5, lines 245-247. Lines 248-256 on pages 5 and 6 had secondary research questions that were illogical.  Delete those questions.

8.  Be consistent in presenting information about the software package used for the analysis of the data (p. 8, line 349 and p. 11, line 443).

9. Page 11, lines 473–480, the independent sample t-test findings for items 7 and 4b were described but no table values were provided. Put the values for those outcomes in the corresponding table.

10. I did not see either your response or modification regarding the item-level analysis.

11. There is a misalignment between the title of the manuscript, the first statement on the abstract, the statement in the methods section (p. 6, lines 263-266), and the statement in the results section (p. 11, lines 448-450), the statements in the discussion section (p. 12, lines 485-487), and the conclusions (p. 14, lines 600-612). Take these misalignment problems into consideration.

In general, your writing style appears to be more focused on reporting on the differences. To ensure that the full study scenario and the collected results are correctly presented, the authors should be fair in communicating those results with no significant results, at least there should be a statement or two describing the insignificant results. Additionally, you should fairly state in the conclusion what has to be done for all enrolled students as well as specifically for those who did not complete the relevant tasks or activities.

Kind regards!

Comments on the Quality of English Language

On p. 5, line 243, I did not understand the word ‘hirst’? On the same page, line 245, change ‘the’ to ‘The’.

Additionally, on p. 5, line 229, ‘ini’ rewrite this word correctly.

Also, on p. 11, line 450, remove the word ‘influence’.

Author Response

Dear Reviewer #2,

Thank you for the detailed recommendations for the improvement of the manuscript. The detailed report that responds to each item is attached. Thanks again.

Author Response File: Author Response.pdf