Cookie-Jar Alarms: An Analysis of First-Grade Students’ Gendered Conceptions of Engineers following a Programming Design Task
Round 1
Reviewer 1 Report
This interesting study explores how an intervention involving coding affected first-grade students' engineering conceptions, notably how they might differ by gender. The two-week curriculum intervention improved students' conceptual understanding of engineering work regardless of gender. According to the study, the girls identified as engineers more often than the boys when asked to perform programming design tasks. In this paper, the authors discuss possible causal factors influencing the changes we observed in the curriculum and recommend gender-responsive curricula as a means of promoting equity in engineering education.
Dear Authors, please specify the objective of the study in the abstract.
How many researchers coded the interviews? How was grounded theory used? (What was the rationale for using it?) Please add a brief explanation in the methodological part.
Also, you could add the info that before you proceed with the non-parametric test, you carried out the normality test (such as the Shapiro-Wilk test) to identify whether your data were normally distributed or not.
Excerpts from interviews are sometimes written in italics (lines 368-389), sometimes not (just a small technical remark).
Author Response
- Please specify the objective of the study in the abstract.
Done. This purpose statement was added to the abstract:
“Increased emphasis on K-12 engineering education calls for the design of learning environments and curricula that increase interest and conceptual understanding of engineering work beginning in the early years of childhood. But what works, for whom, and in what contexts, how does it work, and how can engineering curricula be improved to promote social justice? Here, we evaluate the impact of a curricular intervention designed to promote equity in elementary engineering education.”
- How many researchers coded the interviews? How was grounded theory used? (What was the rationale for using it?) Please add a brief explanation in the methodology part.
Done. The first author did the coding of the interviews, which is a limitation of the study. This is now mentioned in both the Methods section and in the Limitations section. A justification for why grounded theory was used as a qualitative research method in the study is now highlighted in two places in the Methods section (see below).
Methodology:
“The study concludes with an integration of the QUAN and QUAL data to analyze the relative effectiveness of the curricular intervention for girls compared to boys. Specifically, we work towards the development of a theory grounded in the data (MM-GT) to explain why girls and boys might respond differently to the same curriculum. That is, we seek to answer a process-oriented, complex question: What works, for whom, in what contexts, how does it work, and how can conditions be improved to promote social justice?”
“Interview transcripts were coded using a grounded theory approach (MM-GT) for the purpose of developing a model that explains how or why boys and girls might respond differently to the same engineering curriculum (Creamer, 2018; Guetterman, Babchuk, Howell Smith, & Stevens, 2019; Johnson, McGowan, & Turner, 2010). Constant comparative methods with iterative coding and analysis were employed. Specifically, open, then axial, then selective coding was used by the first author to identify how and why the cookie jar alarm intervention may (or may not) have facilitated conceptual change and increased interest in engineering among first-grade students based on gender.”
Limitations:
“Finally, the first author alone did the coding of the qualitative interview data. Having multiple coders could have brought different perspectives to the data and enriched the interpretation.”
- You could add the info that before you proceed with the non-parametric test, you carried out the normality test (such as the Shapiro-Wilk test) to identify whether your data were normally distributed or not.
Done. The following sentence was added to the Methodology section:
“A Shapiro-Wilk test showed the data were not normally distributed, although both the Wilcoxon Signed-Rank test and the Mann-Whitney U test do not require a normal distribution.”
Reviewer 2 Report
The article contains a topic of interest to the academic community. It is well done with a rich scientific content with a well designed and implemented methodology with conclusive results and discussions.
Author Response
There were no suggested revisions by this reviewer.
Reviewer 3 Report
Thanks for the interesting manuscript. A few suggestions:
- It will be more appropriate to put the theoretical framework and research questions sections under methods which could be renamed as methodology section.
- It will be great if the results could emphasise further how the research question has been answered.
- It will be a stronger piece of work if the impacts of this proposed idea (i.e., a more equal representation of genders in STEM) could be highlighted in discussion section.
Look forward to the revised version :)
Author Response
- It will be more appropriate to put the theoretical framework and research questions sections under methods which could be renamed as methodology section.
Done. The Theoretical Framework section was moved to the Methods section (rather than the Literature Review section) as suggested. Accordingly, the Methods section was renamed Methodology.
- It will be great if the results could emphasize further how the research question has been answered.
Done. To emphasize the connection between the question and the results, more context was given to the purpose of the research question and subquestions in both the Introduction and Literature Review. Also, transition sentences were added to each subsection in the Results section to clarify how the results were addressing the research question. Figure 1 was also divided into two separate figures (see Figure 1 and Figure 2) to clarify how the data were answering different research questions.
- It will be a stronger piece of work if the impacts of this proposed idea (i.e., a more equal representation of genders in STEM) could be highlighted in discussion section.
Done. The following sentence has been added to the discussion section to highlight the potential impacts of a more equal representation of genders in STEM:
“A focus on the development of gender-responsive engineering curricula could be a means to promote equity in STEM education and perhaps lead to the increased participation of women in the engineering profession.”
Reviewer 4 Report
Thank you for the opportunity to revise this paper titled: “
Cookie-Jar Alarms: An Analysis of First-Grade Students’ Gendered Conceptions of Engineers Following a Programming De-3 sign Task”
First of all, I think the paper addresses an important topic, and it is generally well-written. I have, however, a few suggestions for the authors.
First, I would recommend expanding the introduction. This section should allow the reader to have a comprehensive view of what is done in the paper. I would recommend adding some details about what is done, which results are found (briefly) and how these results advance our understanding.
Similarly, I think the research question should be placed earlier, in the introduction, as this is the force that drives the rest of the paper. Unlike hypotheses, which are logically deducted from the arguments in the literature review, a research question is usually mentioned early to justify the choices in terms of theory and methodology, as well as the kind of contribution of gap filling the paper aims to do, and therefore placed early in the paper.
Please homogeneize the style (italics or not) of the excerpts from the interviews.
While the experiment is very interesting, your sample is quite small (15 students) and they all come from the same class. Therefore, generalizations are difficult to make. Obviously the best remedy would be to collect more information by conducting the experiment in other classes, but perhaps this is not possible. If not possible, this should be at the very least mentioned in the limitations of the paper and explained what future studies can do to enlarge it. In fact, it would be more appropriate to qualify the findings as exploratory or preliminary evidence throughout the paper (this not diminishing the value of your contribution, as this preliminary evidence is needed to expand knowledge and as required antecedent to larger-scale quantitative studies).
I wish you luck with your project!
Author Response
- First, I would recommend expanding the introduction. This section should allow the reader to have a comprehensive view of what is done in the paper. I would recommend adding some details about what is done, which results are found (briefly) and how these results advance our understanding. Similarly, I think the research question should be placed earlier, in the introduction, as this is the force that drives the rest of the paper. Unlike hypotheses, which are logically deducted from the arguments in the literature review, a research question is usually mentioned early to justify the choices in terms of theory and methodology, as well as the kind of contribution or gap filling the paper aims to do, and therefore placed early in the paper.
Done. The Research Question section was moved to the Introduction section (rather than at the end of the Literature Review section) as suggested to both expand the introduction and offer more details about what was done, which results were found, and how these results advance our understanding. Also, the research question was placed early in the paragraph as suggested by the reviewer. Specifically, the following paragraph was added to the Introduction to describe how the results of the study advance our understanding of equitable engineering education:
In this study, we explored how children’s conceptions of engineers are culturally constructed in ways that disadvantage girls. The research question we addressed was: How does a gender-neutral design task that integrates programming influence first-grade students’ conceptions of engineers and/or interest in engineering? Currently, many engineering curricula engage students in designing a vehicle (e.g., car, rocket) or structure (e.g., bridge, tower), often thought of as “boy stuff”. A cookie-jar alarm was chosen as the design task for this study because it was more gender-neutral. Using a mixed-methods approach, we analyzed students’ pre-/post-drawings of engineers, elicited their conceptions of engineering work in semi-structured interviews, and surveyed students about their interest in engineering (N=15). The data from this exploratory study were then integrated to evaluate the impact of the curricular intervention. We found that gendered conceptions of engineers were already present among first-grade students in this study. However, following the two-week curriculum intervention all girls drew female engineers in their post-drawings. Interestingly, girls (75%) also showed more interest in engineering than boys (14%) following the cookie-jar alarm design task (p = .041). Specifically, more girls than boys responded in the affirmative when asked, Do you think you are an engineer? The results provide evidence the curriculum supported girls in this study in developing an engineering identity. We discuss elements of the intervention that may have prompted this conceptual change. The preliminary findings will be useful to researchers interested in designing “gender-responsive” learning environments and engineering curricula that better support girls' identity work in STEM.
- Please homogeneize the style (italics or not) of the excerpts from the interviews.
Done. (Thank you for catching that.) All excerpts from interviews are consistently NOT italicized.
- While the experiment is very interesting, your sample is quite small (15 students) and they all come from the same class. Therefore, generalizations are difficult to make. Obviously the best remedy would be to collect more information by conducting the experiment in other classes, but perhaps this is not possible. If not possible, this should be at the very least mentioned in the limitations of the paper and explained what future studies can do to enlarge it. In fact, it would be more appropriate to qualify the findings as exploratory or preliminary evidence throughout the paper (this not diminishing the value of your contribution, as this preliminary evidence is needed to expand knowledge and as required antecedent to larger-scale quantitative studies).
Done. Moved the following sentences to the beginning of the Limitations section rather than the end to add emphasis to the point, and added the phrase “exploratory” here and throughout as suggested.
Before: The biggest limitation of the study is the small sample size. We hope to extend the cookie-jar alarm curriculum using KIBO robots to a larger number of students…in a future study.
After: Due to the exploratory nature of the study, a major limitation is the small sample size. We hope to extend the cookie-jar alarm curriculum using KIBO robots to a larger number of students in a future study.
Also, we provided a better justification for the qualitative methodology, which favors a smaller sample size in order to offer a rich description of participants’ lived experiences.
Round 2
Reviewer 4 Report
I have no further comments for the authors
