Analysis of Gender Issues in Computational Thinking Approach in Science and Mathematics Learning in Higher Education

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Dear authors and editor,

I have completed the revision of the submission of your paper titled: "Analysis of gender in Computational Thinking approach Science and Mathematics learning in Higher Education”. I found the manuscript to be engaging and well-structured and the methods and results clearly and concisely presented. I consider the conclusions adequate, in relation to the analysis and methods that have been employed. In addition, the conclusions are well linked with the theoretical framework that has been presented.

Overall, this piece of research is very interesting and useful for educational research, but I note a few aspects that should be improved previous to publish the manuscript.

- As described in the manuscript, the study was conducted with university students. However, the test used to determine the level of Computational Thinking was the CTt by Roman-Gonzalez et al. (2017). According to the article, this test is suitable for students in grades 5th to 10th. The authors should justify why they thought this test was capable of measuring Computational Thinking skills in university students and why they did not choose another test more appropriate for the students' age.

- On page 11, to test the differences between the groups with low, medium, and high levels of computational thinking, the authors should use the Kruskal-Wallis test. The Kruskal-Wallis test compares several conditions when different participants take part in each condition and the resulting data violate an assumption of one-way independent ANOVA. As an statistical procedure, ANOVA (or the equivalent for no-parametric data, Kruskal-Wallis) compares several means, when those means have come from different groups of people,  as is the case presented in this section with the three-level samples. Please, redo the analysis in this section in terms of Kruskall-Wallis.

- In all cases, authors should provide the effect size measure for the statistical tests they are using. As stated in the literature, an effect size is an objective and usually standardized measure of the magnitude of observed effect. With this measure, we can compare effect sizes across different studies that have measured different variables, or have used different scales of measurement. I recommend that authors report these effect sizes in the results.

- In general, non-parametric statistical comparisons typically include the median value. In this article, the reported scores refer to the means (which have been provided in the tables), but it would be necessary to clarify this at the beginning of the results section or in the methodology. Reporting means is not incorrect (especially if they have been shown in the tables), but it is not usually the norm in non-parametric tests.

Thanks in advance for the opportunity to review this interesting piece of work.

Author Response

Dear Reviewer,

Thank you very much for your valuable and pertinent comments, we are happy to assist you. Below, I would like to highlight the relevant revisions point by point.

Comments 1: As described in the manuscript, the study was conducted with university students. However, the test used to determine the level of Computational Thinking was the CTt by Roman-Gonzalez et al. (2017). According to the article, this test is suitable for students in grades 5th to 10th. The authors should justify why they thought this test was capable of measuring Computational Thinking skills in university students and why they did not choose another test more appropriate for the students' age.

Response 1: The CTt, although mainly used for the analysis of Computational Thinking skills in elementary school students, has also been validated for students of later ages. In this sense, previous studies have used this questionnaire for the analysis of university students, and specifically, for pre-service teachers (Primary Education). Like these studies, the questions have been classified according to complexity, to adapt it to the age of the students, and subsequently validated with the statistics used. This has been added to the methodology section, as well as the corresponding references. In addition, the following has been added: Cronbach's alpha of internal consistency (α) was 0.79, which can be considered a good reliability”.

[52] Molina-Ayuso, Á.; Adamuz-Povedano, N.; Bracho-López, R., Torralbo-Rodríguez, M. Introduction to computational thinking with Scratch for teacher training for Spanish primary school teachers in mathematics. Education Sciences 2022, 12(12), 899. https://doi.org/10.3390/educsci12120899

[53] González-Martínez, J.; Peracaula i Bosch, M.; Meyerhofer-Parra, R. Impacto de una formación intensiva en programación en el desarrollo del Pensamiento Computacional en futuros/as maestros/as. Ried-Revista Iberoamericana de Educacion a Distancia 2024,, 27(1), 187-208.

Comments 2: On page 11, to test the differences between the groups with low, medium, and high levels of computational thinking, the authors should use the Kruskal-Wallis test. The Kruskal-Wallis test compares several conditions when different participants take part in each condition and the resulting data violate an assumption of one-way independent ANOVA. As an statistical procedure, ANOVA (or the equivalent for no-parametric data, Kruskal-Wallis) compares several means, when those means have come from different groups of people, as is the case presented in this section with the three-level samples. Please, redo the analysis in this section in terms of Kruskall-Wallis.

Response 2: Regarding Table 7, comment to the reviewer that the tests performed do not adjust for differences between groups, so the Kruskal-Wallis statistical test was not used. The Wilcoxon statistical test was performed, since as mentioned in the methodology section “the statistical differences between the averages of the participants belonging to each level in the pretest and the averages of the same participants after the intervention are analyzed, in order to determine whether the participants belonging to a specific level in the pretest belong to the same or to another level in the posttest”, as well as in the results: “shows the results corresponding to the Wilcoxon test on the statistical differences between the means of the participants belonging to each level in the pretest and the means of the same participants after the intervention, in order to appreciate whether the participants belonging to a specific level in the pretest, belong to the same or to another level in the posttest”.

Comments 3: In all cases, authors should provide the effect size measure for the statistical tests they are using. As stated in the literature, an effect size is an objective and usually standardized measure of the magnitude of observed effect. With this measure, we can compare effect sizes across different studies that have measured different variables, or have used different scales of measurement. I recommend that authors report these effect sizes in the results.

Response 3: The effect size statistics have been added to the statistical test tables, as well as specified in the methodology section.

Comments 4: In general, non-parametric statistical comparisons typically include the median value. In this article, the reported scores refer to the means (which have been provided in the tables), but it would be necessary to clarify this at the beginning of the results section or in the methodology. Reporting means is not incorrect (especially if they have been shown in the tables), but it is not usually the norm in non-parametric tests.

Response 4: The following comment has been added to clarify the indicated revision: “In addition, it should be clarified that in this study we used the means and not the medians of the statistical tests, as well as the standard deviations and effect sizes of the statistical tests”.

Thank you. King regards

Author Response File: Author Response.docx

Reviewer 2 Report

Comments and Suggestions for Authors

This study aims to analyze the Computational Thinking skills of pre-service teachers and the effect of gender factor, before and after an intervention based on Educational Robotics and a Science and Mathematics approach.

The topic addressed in this article is interesting and relevant. It justifies well the need of the research, with appropriate and current literature. The study conducted is well presented, concise, and clear.

However, by reducing the questions of the test, it cannot be considered to measure the same as the previous test. The fact of not using a validated and reliable test undermines the results of the research, as stated by the authors in the limitations of the study.

The questionnaire is weak. Since the KMO value is 0.6, this indicates that the sample is mediocre but still acceptable for conducting a factor analysis. However, it would be ideal to seek a higher value for greater reliability.

It is recommended that:

- Provide the cronbach's value

- Provide whether students had prior knowledge

- Improve the discussion with gender influence, highlighting the value of the results. The authors suggest about levels of predisposition and CT skills, but they aren’t measured in their research.

Author Response

Dear Reviewer,

Thank you very much for your valuable and pertinent comments, we are happy to assist you. Below, I would like to highlight the relevant revisions point by point.

Comments 1: Provide the cronbach's value

Response 1: The following comment has been added to clarify the indicated revision: “Cronbach's alpha of internal consistency (α) was 0.79, which can be considered a good reliability”.

Comments 2: Provide whether students had prior knowledge

Response 2: The following comment has been added to clarify the indicated revision: “Clarify that students had not had the opportunity to work previously with programming elements throughout their undergraduate degree”.

Comments 3: Improve the discussion with gender influence, highlighting the value of the results. The authors suggest about levels of predisposition and CT skills, but they aren’t measured in their research.

Response 3: The discussion of the results has been revised, and the following has been added, in view of the results described above, which allow complementing the discussion, as the reviewer mentioned: “This, in addition, is complemented by the results found regarding concepts and sub-concepts, since the results show that there is a higher mean for males in 11 of the 14 items of the questionnaire. However, in the post-test there is a greater disparity, with the male showing a higher mean in 7 items, while the female has a higher mean in 5 items; . However, as can be seen in the post-test, unlike the pre-test, the results of each item are closer between both sexes, with the exception of item 11”.

Thank you. King regards

Author Response File: Author Response.docx

Reviewer 3 Report

Comments and Suggestions for Authors

Dear Authors.

Please find my review of your article: Analysis of gender in Computational Thinking approach Science and Mathematics learning in Higher Education

Clarity, comprehensiveness and relevance to the field

The article titled “Analysis of gender in Computational Thinking approach Science and Mathematics learning in Higher Education” provides a comprehensive analysis of the impact of gender on Computational Thinking (CT) skills, particularly in science and mathematics learning in higher education. The focus on pre-service teachers, using educational robotics (ER), is timely and relevant given the increasing emphasis on integrating digital competences in teacher education and the well-known positive effect of including hand-on activities like robotics in education. The study clearly identifies a gap in the existing research related to gender disparities in CT skills and aims to address it. The paper is relevant to the broader fields of educational technology, computer science pedagogy, and teacher education.

Identification of a gap in knowledge

The article identifies a critical gap in knowledge: the lack of extensive research on gender differences in CT skills development among pre-service teachers, particularly in the context of educational robotics and STEM disciplines. While there are existing studies exploring this topic, including gender differences in CT and the role of ER in education, the article emphasizes that there is less focus on pre-service teachers specifically and in combining ER with CT in a STEM context with the emphasis on identifying the gender differences.

Recent and relevant citations

The references are mostly recent (within the last five years) and relevant to the field. The cited works support the background and methodological framework of the research effectively.

Coherence of statements and conclusions

The conclusions in the article generally align with the findings, but there are some areas that need more careful consideration. The conclusions about the increase in Computational Thinking (CT) skills post-intervention and the reduction of gender disparities are coherent with the data presented. The statistical analysis shows a measurable improvement in CT skills across both genders, and the results indicate a reduction in pre-existing gender differences. However, the extent of this equalization should be interpreted cautiously, as the statistical differences before and after the intervention might not be as conclusive across all CT subskills.

The results indicate that while there was improvement for both genders, the study did not explore whether these improvements would persist over time or whether the intervention had the same long-term effect for both male and female participants. Furthermore, although post-test scores for both genders were similar, the pre-existing higher scores for males before the intervention suggest that a more nuanced understanding of gender differences, beyond the immediate post-test, is required.

Figures, tables, and schemes

The figures and tables included are appropriate and well-designed. They present the data in a clear and interpretable manner, providing insights into both the pre- and post-intervention results. The box plots and statistical tables facilitate a better understanding of the changes in CT skills and gender differences.

Novelty

The research question—focusing on the integration of Educational Robotics (ER) in developing Computational Thinking (CT) skills among pre-service teachers while addressing gender differences—is relevant and timely but not entirely original. Several studies have already examined how ER can support the development of CT skills and how gender differences manifest in CT learning environments. However, the specific context of pre-service teacher training in STEM, combined with ER, adds a new dimension that contributes to the ongoing conversation, even if not groundbreaking in a broad sense.

Scope

The article fits well within the scope of the journal.

Significance

The results are significant, as they provide evidence of an effective intervention that addresses both pedagogical and gender-related challenges in STEM education.

Quality of writing

The article is generally well-written, with a clear structure and logical flow of arguments. The data presentation meets high standards, using appropriate statistical methods and clearly explaining the results but the level of English should be refined for clarity and quality.

Scientific soundness

The study design is appropriate, employing a pre-experimental design with quantitative analysis. The methodology is sound, with sufficient details on the intervention and measurement instruments. The statistical analyses are robust and appropriately detailed to allow for reproduction by other researchers.

Interest to readers

This article will likely attract a broad readership, including educators, researchers in STEM education, and those interested in gender studies in educational contexts. The findings have practical implications for teacher education programs, especially in how they can incorporate robotics and CT in their curricula to reduce gender disparities.

English language

The English language is readable but needs major refinements to be fluent and academically correct.

General recommendation

Accept after major revision.

Regards.

Comments on the Quality of English Language

The language needs to be refined.

Author Response

Dear Reviewer,

Thank you very much for your valuable and pertinent comments, we are happy to assist you. Below, I would like to highlight the relevant revisions point by point

Comments 1: The conclusions in the article generally align with the findings, but there are some areas that need more careful consideration. The conclusions about the increase in Computational Thinking (CT) skills post-intervention and the reduction of gender disparities are coherent with the data presented. The statistical analysis shows a measurable improvement in CT skills across both genders, and the results indicate a reduction in pre-existing gender differences. However, the extent of this equalization should be interpreted cautiously, as the statistical differences before and after the intervention might not be as conclusive across all CT subskills.

Response 1: Thank you for pointing this out. We agree with this comment. Therefore, the need to be cautious about these results and to carry out future studies to explore the influence of ER on all the subconcepts has been mentioned:

“Nevertheless, we need to consider as a handicap in this research the existence of a lack of learning in some CT skills, such as Repeat until and Simple Condition, due to the fact that both items of each computational subconcept present a statistical value far from being significant. Items 6 and 8 present a not very high average in the pretest that did not in-crease much after the intervention. For this reason, despite the great results obtained, it is necessary to improve the strategies for training these skills in future training programs, as well as to continue with further studies with the intention of understanding the influence of this resource in training all the subconcepts of CT”.

“However, although encouraging, the results should be viewed with caution. Despite the overall improvement in CT skills, some subconcepts, such as Repeat until and Simple conditionals, did not show a comparable degree of improvement. This observation suggests that while educational robotics in this approach has a positive impact on CT skill development, not all subconcepts show the same level of progress. This underscores the im-portance of supporting this strategy with complementary or specific approaches to strengthen those subconcepts that did not show a significant degree of improvement”.

“Nevertheless, it is important mention that some specific subconcepts still show some differences, suggesting that strategies need to be refined in order to ensure a uniform improvement in all areas of computational thinking, regardless of gender”.

Comments 2: The results indicate that while there was improvement for both genders, the study did not explore whether these improvements would persist over time or whether the intervention had the same long-term effect for both male and female participants. Furthermore, although post-test scores for both genders were similar, the pre-existing higher scores for males before the intervention suggest that a more nuanced understanding of gender differences, beyond the immediate post-test, is required.

Response 2: With respect to the differences obtained in terms of gender, the following question has been included in the section on limitations of studies and future lines of research: “Furthermore, studies are needed that examine how these improvements are preserved over time or whether the intervention has the same effect in the long term for both male and female participants. Interesting results that would add further evidence to the results of the present study could be obtained by longitudinal studies or by administering several post-intervention tests over time”.

Comments 3: The article is generally well-written, with a clear structure and logical flow of arguments. The data presentation meets high standards, using appropriate statistical methods and clearly explaining the results but the level of English should be refined for clarity and quality.

Response 3: The translation and editing of the manuscript have been revised.

Reviewer 4 Report

Comments and Suggestions for Authors

Review Report

Article title:

Analysis of gender in Computational Thinking approach Science and Mathematics learning in Higher Education

The authors present the results of a study that focused on the influence of Educational Robotics (ER) in the development of Computational Thinking (CT) skills through a teaching approach of scientific and mathematical contents. It has been found that the use of ER interventions allows increasing the development of CT, while mitigating gender differences, thus equalizing the level of skills developed. The topic of the study is relevant to the field. Based on the research carried out by Bers, M. et al. and Redmond, P. et al., it can be concluded that teachers have gaps in their knowledge, competences, self-efficacy and self-confidence about how to teach with technology and how to successfully integrate the technology in their practice. For these reasons, the research topic is very actual. The authors described in detail the research methods and explained the processing and evaluation of the results. Arguments and discussions of the findings are balanced. The results of the study are clearly presented. The article is adequately referenced. The conclusions are fully supported by the results presented in the article. The issues to be addressed are described below.

Specific comments:

1.        The abbreviation ER used in the Introduction (p. 2 line 91) is defined only in the subsection 1.4 (p. 3, line 122). Similarly, the acronym STEM is used without introduction. All abbreviations should be defined at their first mention in the manuscript.

2.           The text on Fig. 1 is unreadable. The font size should be increased.

3.        In Fig. 2, English axes titles instead of Spanish should be used. Furthermore, decimals should be separated by a dot instead of a comma. The same applies to Figs. 3, 4 and 5.

4.           In Figures 3, 4 and 5, the title of the vertical axis is not defined.

5.          English is readable; however, the authors often exclude nouns and prepositions. For example, the title of the article should be changed to the following: Analysis of gender issues in Computational Thinking approach in Science and Mathematics learning in Higher Education. A thorough proof-reading of the manuscript by a native speaker is required before resubmission.

Comments on the Quality of English Language

English is readable; however, the authors often exclude nouns and prepositions. For example, the title of the article should be changed to the following: Analysis of gender issues in Computational Thinking approach in Science and Mathematics learning in Higher Education. A thorough proof-reading of the manuscript by a native speaker is required before resubmission.

Author Response

Dear Reviewer,

Thank you very much for your valuable and pertinent comments, we are happy to assist you. Below, I would like to highlight the relevant revisions point by point.

Comments 1: The abbreviation ER used in the Introduction (p. 2 line 91) is defined only in the subsection 1.4 (p. 3, line 122). Similarly, the acronym STEM is used without introduction. All abbreviations should be defined at their first mention in the manuscript.

Response 1: The concept of Educational Robotics has been used to introduce the abbreviation ER (p. 2, line 91). Likewise, the concepts science, technology, engineering and mathematics have been used to introduce the acronym STEM (p. 1, line 42).

Comments 2: The text on Fig. 1 is unreadable. The font size should be increased..

Response 2: The programming sequences (Figure 1 and 2) have been divided to improve their visibility.

Comments 3: In Fig. 2, English axes titles instead of Spanish should be used. Furthermore, decimals should be separated by a dot instead of a comma. The same applies to Figs. 3, 4 and 5

Response 3: The corresponding figures shown below have been modified.

Comments 4: In Figures 3, 4 and 5, the title of the vertical axis is not defined.

Response 4: Vertical axis headings have been added in the figures indicated.

Comments 5: English is readable; however, the authors often exclude nouns and prepositions. For example, the title of the article should be changed to the following: Analysis of gender issues in Computational Thinking approach in Science and Mathematics learning in Higher Education. A thorough proof-reading of the manuscript by a native speaker is required before resubmission.

Response 5: The translation and editing of the manuscript have been revised. The title has also been modified, as mentioned by the reviewer.

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

Dear authors,

Thank you very much for the clarifications you have made in response to my comments.

In particular, I would like to thank you for the clarification regarding comment 2, where I suggested using the Kruskall-Wallis test to compare the means of the three groups. However, it is correct that the manuscript only establishes comparisons between pre and post measurements for each of the groups, making the use of the Wilcoxon test for paired samples appropriate.

I appreciate that you have added the effect sizes in the corresponding tables, but for a proper understanding of the article, it would be necessary to specify which method was used to obtain the effect size (e.g., Rosenthal's r, Cohen's d, etc.).

Therefore, my final comment for accepting the article is to include this information in the methodological section. This will allow subsequent studies (especially meta-analyses) to benefit from this piece of research.

Thank you again. Best regards.

Author Response

Comments 1: I appreciate that you have added the effect sizes in the corresponding tables, but for a proper understanding of the article, it would be necessary to specify which method was used to obtain the effect size (e.g., Rosenthal's r, Cohen's d, etc.).

Response 1: Dear Review, thank you for your comments and thank you again for the timely revisions. I have added the method used for the effect size (Rosenthal's r) in the indicated section. Best regards

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

Dear Authors.

Thank you for carefully addressing posed comments.

The article is now well prepared and in my opinion ready to be published.

 

Congrats.

Reviewer 4 Report

Comments and Suggestions for Authors

Dear Authors,

I appreciate the great efforts you have made in response to my previous questions and comments. The revision clarifies all points I raised. You have significantly improved the clarity of your writing and addressed most of my concerns. I recommend publishing the article in the current form.

Kind regards,

The Reviewer