Construct Validation of the STEM Motivation Scale for Children: A Multi-Timepoint Study
Vadym Vasiutynskyi
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsDear Author(s)
Your article presents a thorough and scientifically well-grounded account of the development of the STEM Motivation Scale for Children and highlights the effectiveness of participation in a multi-day STEM program. The research findings have significant scientific and social relevance as well as remarkable prospects for practical implementation.
At the same time, I would like to offer a few comments.
In the list of references, the proportion of recent studies is insufficient.
As the author(s) note on p. 3, when determining the test–retest reliability, two measurement points were used to assess temporal stability. It would be appropriate to indicate what the time interval between the test and retest was.
Table 1 shows that no statistically significant differences in interest in the STEM field were observed between the control and experimental groups before the start of the Program. Here, firstly, it is unclear why only the lack of difference in interest in the STEM field is emphasized; secondly, the presence of a significant difference between the groups on item 16 is ignored.
Why is the number of participants in the intervention and control groups indicated in the text as NE = 272 and NC = 257, respectively, while in Tables S1 and S2 they are reported as 320 and 330?
The data presented in Table 4 should be supplemented with indicators of the significance of changes, or these should be provided in the text following the table.
It is necessary to remove repeated fragments in the paragraphs, one of which precedes Table 6, and the other follows it.
The presentation of correlation coefficients in Figure 3 as a heatmap appears somewhat excessive. However, this, of course, is a matter of the author(s)’ personal taste.
Author Response
Dear,
Thank you very much for your prompt and thoughtful review of our manuscript. We sincerely appreciate the time and effort you invested in providing such constructive feedback.
We are grateful for your positive assessment of the study’s design and the validation of the STEM Motivation Scale. Your comments regarding the potential educational value of the instrument are especially encouraging.
We fully acknowledge the points you raised and commit to revising the manuscript in accordance with all your suggestions.
Comment1: The proportion of recent studies in the reference list is insufficient.
Response 1: Thank you for this valuable observation. We have updated the reference list to include several recent studies published between 2022 and 2024, particularly those addressing STEM motivation and program effectiveness. These additions strengthen the theoretical foundation and reflect current developments in the field.
Comment 2: The time interval between the test and retest should be specified.
Response 2: We appreciate this suggestion. The manuscript has been revised to include the time interval between the test and retest, which was two weeks. This information is now provided in Section 3.3 under Test–Retest Reliability.
Comment 3: Table 1 emphasizes only the lack of difference in STEM interest, while a significant difference on item 16 is overlooked.
Response 3: Thank you for pointing this out. We have revised the text to clarify that while overall interest in STEM did not differ significantly between groups, item 16 (“I am interested in rainbows”) did show a statistically significant difference (p = 0.027). This nuance is now explicitly addressed in the Results section.
Comment 4: Participant numbers differ between the text (NE = 272, NC = 257) and Tables S1 and S2 (320 and 330).
Response 4: Thank you for drawing attention to this discrepancy. Upon review, we acknowledge that the participant numbers stated in the Supplementary Materials (Tables S1 and S2) are incorrect due to a formatting oversight. The correct numbers are those reported in the main text: NE = 272 and NC = 257. We will revise the Supplementary Materials accordingly to ensure consistency and accuracy across all sections of the manuscript.
Comment 5: Table 4 should include indicators of the significance of changes.
Response 5: Thank you for this important suggestion. We have added statistical significance indicators (p-values) for each dimension in Table 5 and elaborated on these results in the accompanying text to highlight the observed changes.
Comment 6: Repetitive fragments around Table 6 should be removed.
Respnse 6: We agree with this comment and have removed the redundant paragraphs surrounding Table 6. The revised text is now more concise and focused.
Comment 7: The use of a heatmap in Figure 3 may be excessive.
Response 7: Thank you for your thoughtful feedback. As stated in the manuscript, our intention was to provide a visual representation of the correlation matrix to enhance interpretability, particularly for readers less familiar with numerical tables. However, we understand your concern and agree that the heatmap may be perceived as excessive. In response, we will remove the heatmap.
We believe these revisions will significantly improve the clarity and impact of the manuscript. Thank you once again for your valuable input and for helping us enhance the quality of our work.
Kind regards
Author Response File: 
Author Response.docx
Reviewer 2 Report
Comments and Suggestions for AuthorsThis is a well-designed study on the effectiveness and validity of a questionnaire exploring motivation towards STEM in school aged children. The questionnaire validation is sound and executed well, and implications of using this scale as an additional educational tool will certainly be of interest to the readership of this journal.
I have a few minor suggestions for consideration by the authors:
Can the authors further explain what the rationale behind choosing 3 months as a post-test timepoint was? As three months are a fairly short amount of time, would a post-test further down the line (i.e. 6 months or at the end of the year) provide more meaningful insight into the scale's effectiveness?
Can the authors add effect size for correlations? Otherwise claims of meaningfulness are slightly tenuous.
The paragraph in lines 286-290 reads more like the legend for Fig. 3. Can this be integrated in the narrative?
The conclusion in lines 314-319 is fairly tenuous; it is not quite clear to me how the magnitude of the impact of hands-on, problem-centred learning is educationally. It is highly unlikely that deep engagement can be deduced from the reported differences in interest/motivation towards STEM across three fairly short timepoints as there are numerous extraneous factors that might influence the ongoing development of primary school students' affinity towards STEM. I would encourage the authors to add some clarification and elaboration here, lest the conclusion seem arbitrary. I do agree that the instrument will be a useful tool for educators but the possibility of student interest varying or eventually waning needs to also be acknowledged.
Provided the above are addressed, I would be happy to recommend publication of the manuscript.
Author Response
Dear,
Thank you very much for your prompt and thoughtful review of our manuscript. We sincerely appreciate the time and effort you invested in providing such constructive feedback.
We are grateful for your positive assessment of the study’s design and the validation of the STEM Motivation Scale. Your comments regarding the potential educational value of the instrument are especially encouraging.
We fully acknowledge the points you raised and commit to revising the manuscript in accordance with all your suggestions.
Comment 1: Can the authors further explain the rationale behind choosing 3 months as a post-test timepoint? Would a longer interval (e.g., 6 months or end of year) provide more meaningful insight?
Response 1: Thank you for this thoughtful observation. The three-month follow-up was selected based on practical constraints within the school calendar and to ensure participant retention. While we acknowledge that a longer interval could offer deeper insights into the scale’s long-term sensitivity, the three-month window was intended to capture short-term retention of motivational gains post-intervention. We have added a brief explanation of this rationale in the Methods section and noted the potential value of extended follow-up in the Limitations and Future Research section.
Comment 2: Can the authors add effect sizes for correlations? Otherwise, claims of meaningfulness are slightly tenuous.
Response 2: We appreciate this important suggestion. Effect sizes (Cohen’s r) have now been added to Table 6 and referenced in the Results section to support the interpretation of inter-factor relationships. This addition strengthens the claims regarding the meaningfulness of observed associations.
Comment 3: The paragraph in lines 286–290 reads more like the legend for Fig. 3. Can this be integrated into the narrative?
Response 3: Thank you for your observation. In response to a suggestion from another reviewer, Figure 3 has been removed from the manuscript. Consequently, the paragraph in lines 286–290, which originally served as a descriptive caption for the figure, has also been deleted to maintain coherence and avoid redundancy. We appreciate your attention to clarity and have ensured that the surrounding text remains consistent and informative.
Comment 4: The conclusion in lines 314–319 is fairly tenuous. The educational impact of hands-on learning is not clearly supported by the reported data. Please clarify and acknowledge the possibility of waning interest.
Response 4: We fully agree with your concern and have revised the conclusion to provide a more nuanced interpretation. While the data show statistically significant increases in motivation, we now acknowledge that these changes may not directly reflect deep engagement or long-term transformation. We have added a statement recognizing the influence of extraneous factors and the possibility of fluctuating interest over time. This clarification ensures that the conclusion remains grounded in the data and avoids overgeneralization.
We believe these revisions will significantly improve the clarity and impact of the manuscript. Thank you once again for your valuable input and for helping us enhance the quality of our work.
Kind regards
