Validation of a Scale on University Teaching Quality in the Area of Mathematics

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Often, I hear students complain about professors who know their disciplines well but can not teach in a way that relates to a novice in the discipline.   I found your research relevant to this issue.  It has been some time since I've done any significant work with survey research.  That said I found not significant issues with your analysis.  

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

This study validated a scale to examine the quality of university teaching in Mathematics. The findings may improve our understanding of how to evaluate the quality of Mathematics teaching at the college level. However, the authors need to address several issues for potential publication.

First, in the introduction, I appreciate the authors’ review of related studies, which helps readers grasp the background of evaluating university teaching quality and why it is important to focus on the competencies of the instructor. However, I believe the arguments highlighting the significance of this study could be stronger. While the authors discuss the importance of assessing teaching quality in the major of Mathematics, they do not clearly articulate why it is necessary to create a teaching quality scale for the math area. There are many teaching quality scales at the university level for all subjects. Why are these scales not enough?

I suggest the authors review the existing scales, both the general scales on university teaching quality and the scales for the math area. Then, discuss what common challenges educators face using these scales and how this study will address these challenges.

The research questions are missing. I would suggest the authors include the research questions and make it clear how the literature review informs the research questions.

Second, in the Materials and Methods, convenience sampling is used, and the sample size is 229 (relatively small for validating a scale). The authors could discuss more about what they did to ensure the generalizability and transferability of the research results.

The authors could explain more about why we need to examine the five competency dimensions when evaluating the teaching quality in math. Is there any theoretical framework that supports the five dimensions? Also, according to the definitions the authors provided, it seems that the dimension of teaching performance covers the dimension of teaching effectiveness. Why do we need to keep teaching effectiveness as a separate dimension?

In addition, the authors could include more clarity on the process of developing the questionnaire items. Where do the items come from, from preexisting studies, or are they created by the authors? When they identified and selected the items and developed the wording of each item, what did they do to ensure the content validity?

In the Discussion, the authors could explain more about what the research results imply for researchers and educators. Why is this information important? The authors may wish to delve deeper into these points in the Discussion section to enhance the study's significance. Including additional studies in the discussion could help clarify the findings or illustrate the importance of this research. The authors could also explore whether their findings contribute new insights regarding specific theories or if there is evidence indicating advancements in those theories compared to previous research.

Comments on the Quality of English Language

I have no trouble understanding the content. Perhaps they could have a native speaker proofread the article.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

 Dear author,

Following a detailed review of your manuscript, I would like to provide some comments that I believe could enhance its rigour and clarity:

1. Introduction

This section presents highly interesting ideas; however, some parts are expressed ambiguously. It would be advisable to use more precise wording and focus the content specifically on the teaching of Mathematics in Higher Education. Furthermore, it is essential to clearly highlight the importance of validating an instrument that assesses the quality of teaching in this educational stage, providing a solid justification for this necessity.

Additionally, it would be beneficial to explicitly differentiate the general and specific objectives of the study, as well as to clearly state the research question and, where applicable, the hypotheses.

2. Materials and Methods

Methodological Design: The methodological design employed is not specified, nor is its justification. To enhance the scientific rigour of the study, it is essential to include a section that explains which design was used and why it was selected.

2.1. Sample: The order of presentation is important. It is recommended to first specify the type of sampling used and its justification before proceeding with a detailed description of the participants.

2.2. Data Collection Instruments: It would be important to include the Cronbach’s Alpha and McDonald’s Omega values for both each variable and the overall scale, to ensure a more precise analysis of the instrument’s reliability.

2.3. Data Collection and Analysis Procedure: This subsection is clearly presented for the reader.

3. Results

- Scale reliability: The calculated Cronbach’s Alpha is at the threshold of acceptability but not high. Therefore, it is advisable to calculate McDonald’s Omega for the overall scale, which would provide a more precise assessment of its reliability.

- Regression weights: While the data presented are correct, the robustness of the study would be strengthened by including a table showing the standardised regression weights. This raises the following question: Do the values presented in the Confirmatory Factor Analysis (CFA) correspond to these regression weights?

- CFA: It is suggested to redesign Figure 1 of the CFA, as in its current version, the numbers and connections are not clearly visible, making interpretation difficult.

4. Discussion

The discussion is coherent with the study’s findings. However, it would be advisable to include further references to previous studies that support the obtained results, as well as to provide a more in-depth justification for the importance of the validated scale in the study’s context.

5. Conclusions: The conclusions are well formulated.

6. References: It is recommended to adjust citations to APA format, ensuring compliance with the journal’s guidelines. It is also necessary to carefully review the formatting of the bibliographic references, as errors have been identified, such as the omission of italics in some elements and the absence of DOIs or URLs in certain references.

7. Annex A:

Dimension 1: The item "Treats students with respect" does not seem to align with this dimension. It is recommended to review or reassign it.

Dimension 2: The item "Shows concern for students’ understanding of the material" may not fit well within this dimension.

Dimensions 4 and 5: It is advisable to increase the number of items, as each currently contains only two, which may limit the reliability of their evaluation.

I hope these comments are helpful in improving the quality of the manuscript.

Best regards.

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

Overall, I believe the authors have addressed my comments. My only suggestion is to explicitly discuss in the introduction or literature review why validating a teaching quality scale focused on the Math area is important. Why are the existing general scales that apply to all subjects insufficient?  

Author Response

We sincerely appreciate your time and effort in reviewing this manuscript and your comments. Please find below a detailed response along with the corresponding revisions.

 

Comments 1: [Overall, I believe the authors have addressed my comments. My only suggestion is to explicitly discuss in the introduction or literature review why validating a teaching quality scale focused on the Math area is important. Why are the existing general scales that apply to all subjects insufficient?]

Response 1: We sincerely appreciate your thoughtful critique and valuable comments. In response to your comment, [we have ensured that the introduction explicitly justifies the importance of scale in the field of Mathematics. Specifically, we have highlighted how Mathematics presents unique didactic challenges that require a specialized approach to teaching assessment. The introduction now emphasizes the need for a validated instrument tailored to the specific characteristics of Mathematics teaching, ensuring a more accurate assessment of the quality of teaching and its impact on student learning. We trust that these clarifications adequately address your concerns and increase the clarity and relevance of the study. These changes can be found in the introduction from page 1, paragraph 3, line 24 onwards]

Reviewer 3 Report

Comments and Suggestions for Authors

Dear Author,

Thank you very much for taking the time and effort to improve the scientific rigour of your manuscript. After a careful reading and taking into account the journal's criteria, I would like to provide you with some comments:

In the introductory section, it lacks clarity and relevance. In my opinion, some ideas from several paragraphs are repeated. I also think that a much more solid theoretical framework and a more enlightening justification of the importance of carrying out this study could be provided. In addition, the specific objectives are still very generic and unclear.

From a methodological perspective, I would like to ask you, what is an instrumental methodology, it is the first time I have heard of it. Also, from my perspective, the methodological design should be clear and concise. It seems to me to be too long, which I think makes it difficult for the reader to understand in this section.

In relation to the instrument, I believe that the idea is that this scale is valid and can be used in the university context. There is still a persistent shortage of items in some of the factors/dimensions of the scale. This idea weakens this study and shows that the factor/dimension created does not measure what it is intended to measure, as with only 2 items it is complex. Therefore, this aspect has not been resolved, which significantly weakens this work and, therefore, the future use of this scale.

In relation to the references, I recommend that you strictly review the updated APA standards as some formatting errors persist.

Regards

Author Response

We sincerely appreciate your time and effort in reviewing this manuscript and providing your valuable feedback. Below, you will find detailed responses along with the corresponding revisions.   Comments 1: [In the introductory section, it lacks clarity and relevance. In my opinion, some ideas from several paragraphs are repeated. I also think that a much more solid theoretical framework and a more enlightening justification of the importance of carrying out this study could be provided. In addition, the specific objectives are still very generic and unclear.]   Response 1: We sincerely appreciate your criticism and constructive comments. In response to your comment, [we have carefully revised the introduction to eliminate redundant ideas, ensuring a more concise and coherent presentation of the study's rationale. Additionally, we have explicitly justified the importance of validating the evaluation instrument, highlighting its relevance in assessing the quality of Mathematics teaching. Furthermore, the objectives have been refined and articulated with greater clarity to enhance their precision and alignment with the study's scope. We trust that these revisions contribute to improving the manuscript’s clarity and methodological rigor. These changes can be found in the introduction from page 1, paragraph 3, line 24 onwards]   Comments 2: [From a methodological perspective, I would like to ask you, what is an instrumental methodology, it is the first time I have heard of it. Also, from my perspective, the methodological design should be clear and concise. It seems to me to be too long, which I think makes it difficult for the reader to understand in this section.]   Response 2: We sincerely appreciate your valuable feedback and the time you dedicated to reviewing this manuscript. In response to your comment, [we would like to clarify that previous reviewers recommended expanding this section to provide a more detailed explanation. To address this, we have restructured the content to enhance readability and ensure a clearer presentation of the methodological framework. We trust that these modifications contribute to a better understanding of the study’s design and methodological approach. These changes can be found in the methodological design section from page 3, paragraph 7, line 121 onwards]   Comments 3: [In relation to the instrument, I believe that the idea is that this scale is valid and can be used in the university context. There is still a persistent shortage of items in some of the factors/dimensions of the scale. This idea weakens this study and shows that the factor/dimension created does not measure what it is intended to measure, as with only 2 items it is complex. Therefore, this aspect has not been resolved, which significantly weakens this work and, therefore, the future use of this scale.]   Response 3: We appreciate your insightful feedback on this aspect of the study. [The final validated model presented in the manuscript is the result of rigorous statistical analysis, ensuring its methodological soundness. While it is true that some dimensions contain only two items, this does not pose a methodological issue, as previous literature supports the validity of such constructs when they demonstrate adequate psychometric properties. Nevertheless, we acknowledge the potential for further exploration and have included in the limitations section a recommendation to examine the scale with alternative dimensional structures in future research. We trust that this clarification addresses your concern and strengthens the comprehensiveness of the study. This change can be found on page 11, paragraph 4, line 397]   Comments 4: [In relation to the references, I recommend that you strictly review the updated APA standards as some formatting errors persist.]   Response 4: We sincerely appreciate your observation regarding the citation style. However, [we would like to clarify that the journal follows its own citation guidelines rather than the APA format. Nonetheless, we have carefully reviewed all citations throughout the manuscript and have corrected any inconsistencies to ensure full compliance with the journal's formatting requirements. The revisions have been implemented across the entire manuscript, and we trust that these adjustments enhance the accuracy and coherence of the references.]