Data Quality of Different Modes of Supervision in Classroom Surveys

Round 1

Reviewer 1 Report

1. Abstract

1) Unclear research objectives: The abstract must provide clearer information regarding the research objectives, the reason the research was conducted, and the importance of the topic for society or other research.

2) add a conclusion about the research's main findings and implications: The abstract should summarize the research findings and implications, helping readers understand the impact of the research.

2. Introduction:

1) Identify unanswered problems or gaps.

2) add significance or value to research in academic and practical contexts.

3) shorten the description of the research steps and move them to the methods section.

4) Add a theory or conceptual framework that is the basis of the research and connect it to the problem formulation and research objectives.

LIterature Review:

1) Try to make references more up-to-date and replace outdated literature.

2) Ensure that many previous researchers have yet to conduct this research. You can search for it with Scispace.

Method: okay

Results: (There is research data described in the discussion section)

Discussion: (I want to point out that your discussion presents too much data; reduce it and preferably move it to the results section).

1) Limit repetition of data in discussions

2) Compare the results of your research with previous research or with relevant literature.

3) Explain similarities, differences, or interesting patterns.

Conclusion: N/A

Limitation: N/A

Implication: N/A

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 2 Report

Dear Author, 

I have read and evaluated your manuscript. Overall, it needs major revisions. See the following points for your reference. Also, I have left some comments in the manuscript.

1. Introduction

The introduction serves as the initial presentation of the research topic, elucidating its development and highlighting existing gaps in the scholarly discourse. This phase necessitates the incorporation of pertinent references to substantiate the significance of the study, thereby convincing readers of its alignment with established literature and its essential contribution to the field. Regrettably, the introductory paragraphs in this manuscript lack appropriate referencing, thereby exposing a deficiency in the foundational framework underpinning this research. Hence, substantial revisions are imperative in order to address the aforementioned issues and adequately fulfill the requirements outlined in this critique.

2. Literature Review

To enhance the scholarly rigor of the manuscript, it is advisable for the author to dedicate a distinct section to the comprehensive review of relevant literature (Literature Review) instead of intermingling it within the introduction. Several critical points necessitate attention within this literature review segment. Furthermore, it is imperative to adhere to the conventions of academic writing, refraining from the use of the first person ("I") and reframing personal assumptions and subjective evaluations of the literature. A comprehensive revision of the literature review section is indispensable to align it with established academic standards.

3. Methodology

In this section, it is recommended that the authors adopt a more objective tone by employing terminology such as "this study" or "the research" instead of personal pronouns like "I." Detailed comments have been provided within the manuscript for the author's reference, with the expectation that these suggestions will guide the necessary revisions.

4. Discussion

Among the various sections in the manuscript, the discussion segment appears to be particularly lacking in depth and coherence. Moreover, the absence of implications, limitations, or conclusive insights derived from the research findings further diminishes the overall quality of this section. Detailed comments have been provided within the manuscript, and it is anticipated that substantial revisions will be undertaken to address these deficiencies.

I hope this feedback is helpful. 

Sincerely, 

Reviewer

Comments for author File: Comments.pdf

Moderate editing of English language is required.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Reviewer 3 Report

The manuscript titled ‘Data Quality of Different Modes of Supervision in Classroom Surveys’ has been carefully reviewed. It investigates how various forms of supervision (digital hybrid, in-person, or a combination) during standardized survey research in German secondary schools affect data quality, focusing on aspects like item non-response, interview duration, and dropout rates. The study, based on a quasi-experimental design with 923 adolescent participants during covid-19 pandemic, aims to offer insights for researchers on the efficiency and effectiveness of supervision modes in classroom surveys.

1. Content Contextualization: The manuscript contextualizes its content by succinctly summarizing relevant theoretical backgrounds and empirical research. It draws on previous studies to frame its investigation into the impact of different modes of supervision on data quality in classroom surveys, highlighting the novelty and significance of its approach in the evolving landscape of educational research methodologies.
2. Relevance of Cited References:  The references cited in the introduction section appear relevant and supportive of the conducted research. They cover a variety of previous studies, theories, and methodologies that are directly related to the main themes of survey methodology, adolescent research, and the impact of supervision on data quality. However, it is highly unusual and regrettable that the discussion section lacks references to previous literature, which undermines this study's contribution to existing scholarly work.
3. Clarity of Research Design and Methods:  

(1)   While the authors claim that the three factors of data quality (dropouts, item non-response, and interview duration) are reliable, it does not necessarily imply these factors adequately represent data quality. The authors should justify the selection of these three factors with references to the literature.

(2)   More information on the questionnaire items and interview questions is needed to clarify for readers the content being assessed.

(3)   Lines 301 and 305 should be moved to more appropriate paragraphs below.

(4)   The explanation regarding questionnaire items is somewhat limited; lines 309 and 310 have caused confusion. It is unclear which question was filtered out and why it was excluded from the count.

(5)   Lines 314 to 315 could benefit from rephrasing for better clarity. The current wording is confusing.

(6)   The research design of this study has certain limitations. Specifically, the study aims to compare data quality across online, hybrid, and in-person supervision formats. However, a notable discrepancy exists in the supervisor's presentation mode: the teacher is present in person, whereas external supervisors (e.g., researchers or research assistants) participate online. This discrepancy might introduce a significant confounding factor, conflating the effects of the supervision mode with the supervisor's presentation mode. Ideally, the research design would include additional groups with reversed roles, allowing teachers to administer tests online and external supervisors to be present in person. This limitation should be more explicitly acknowledged in the study's limitations section.

(7)   The study's data analysis involves several statistical methods (e.g., logistic regression). Although the assumptions for these methods have been lightly covered, a more thorough discussion on other critical assumptions would be beneficial. For example, logistic regression requires minimal to no multicollinearity.

4. Presentation of Empirical Results: The manuscript presents its results in a clear structure, effectively summarizing the data with tables and figures. However, clarity could be improved by offering more detailed explanations of the statistical analyses employed and interpreting the significance of the findings in the context of the study's research questions and hypotheses. Appendices A1 to A4 should be included in the main text, not relegated to the appendix. Additionally, the presentation of results should cross-reference key statistical figures to enhance the rigor of scientific communication.
5. Coherence and Balance in Arguments and Discussion: A discussion on how the research limitations were mitigated or their potential impact on the interpretation of the data would add depth. While the manuscript does a commendable job of presenting its findings, it could benefit from a more thorough comparison with existing studies. This should include a critical examination of how the findings align or diverge from previous research and the implications for the field. Furthermore, the discussion could be expanded to more critically explore the practical implications of the findings for educators, policymakers, and future research. This could involve a more detailed exploration of how different supervision modes can be effectively implemented in various educational settings.
6. Adequacy of References: The introduction section of the manuscript appears to be adequately referenced, drawing upon a diverse array of sources pertinent to its research themes. However, the discussion section could be enhanced by ensuring it engages with the most current and comprehensive range of references, which would further strengthen the manuscript. Assessing whether the latest studies or critical reviews have been included in the discussion section would ensure the research is positioned within the most current scholarly context.
7. Support for Conclusions: The conclusions in the manuscript are well-supported by the results presented, aligning closely with the data analysis and findings. They also reference secondary literature to contextualize the study's outcomes within the broader field. However, to provide more thorough support, the conclusions could benefit from deeper engagement with the literature, particularly by addressing any discrepancies or affirming consistencies with existing studies. Such an approach would enhance the robustness of the conclusions, ensuring they are not only derived from the study's results but also firmly grounded in the current understanding of the field.

It may be beneficial to consider hiring a proofreader or editor to improve clarity and reduce colloquial writing.

Author Response

Please see the attachment.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

I would like to point out that the article lacks a clear conclusion or implications of the findings presented. Strong conclusions and clear implications are essential to tie together all the information conveyed in the article. This helps readers understand comprehensively the significance and relevance of the findings within a broader context.

I suggest adding a conclusion section that summarizes the main findings and presents the implications of the research results. This will provide readers with a clearer picture of the significance of the findings and how they can be applied in practice or further research.

Thank you for your attention to this feedback, and I am confident that adding a conclusion and implications section will enhance the overall quality of the article.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 2 Report

Dear Authors,

I have carefully assessed your manuscript. Overall, you have made significant improvements. I have left some comments on the introduction, discussion, implications, and conclusion in the manuscript. I hope my comments can be helpful for your revision.

I wish you luck!

Reviewer

Comments for author File: Comments.pdf

Minor editing of English language is required.

Author Response

Please see the attachment.

Author Response File: Author Response.docx

Reviewer 3 Report

Thank you for taking the time to address my concerns and comments. Best wishes for the successful publication.

Author Response

Again, thank you for the time and effort you invested in reviewing my manuscript.