Review Reports

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The manuscript addresses a relevant topic and uses a valuable dataset.

1. The outcome is a retrospective self-reported change collected after the intervention, not a genuine pre–post measurement. The authors should avoid causal terms such as “generated” or “demonstrated an increase” and use “students reported a perceived increase”.
2. Table 5 is not a true transition matrix because its columns represent perceived change rather than post-intervention interest. The title, labels, and interpretation should be corrected.
3. The claim that the intervention mitigated self-selection bias is not supported and should be removed or substantially qualified.
4. Without a control group, the observed change cannot be attributed directly to the programme. This limitation should be clearly acknowledged throughout the manuscript.
5. The authors should report the participant selection procedure and response rate, particularly because approximately 3,500 students attended the programme but only 584 were surveyed.
6. Students were nested within schools, visits, workshops, and programme editions. The analysis should account for this clustering, for example through multilevel modelling or cluster-robust standard errors.
7. Exposure to female mentors was confounded with workshop type and itinerary. Therefore, the manuscript cannot conclude that workshop content matters more than mentor gender.
8. Separate significant and non-significant results for male and female students do not prove gender differences. A formal interaction test is required.
9. Active participation was not statistically significant (p = 0.056 overall; p = 0.052 among males). It should not be described as a confirmed “main driver”, only as a possible trend.
10. The questionnaire items, scale construction, validity, missing-data treatment, and model diagnostics should be reported more clearly
11. Because most participants were minors, the ethics approval / waiver and consent procedures require fuller justification.
12. Reference 9 and Reference 10 appear to be duplicates

Author Response

Thank you very much for your comments and observations. Below you will find the response to each comment, numbered from 1 to 12 as indicated.

Response to comment 1: We completely agree that our data reflect a retrospective self-reported measure rather than a genuine pre–post experimental design. Following your recommendation, we have carefully reviewed the entire manuscript to remove causal language. Terms such as “generated”, “demonstrated”, or “caused” have been replaced with more precise expressions such as “associated with”, “indicated”, “self-reported”, and “perceived increase”. These adjustments have been made throughout the text, including the Abstract, Section 2.3, Section 3.2, Section 4, and the Conclusions, to explicitly acknowledge that the observed shifts in interest are based on students' self-reported perceptions

Response to comment 2: The comment is correct, and we have removed all mentions of “transition matrix”. Additionally, we have updated the title, footnote, and column headers of Table 5 (“Change 1” to “Change 5”) to clearly indicate that these values represent self-reported perceived change scores rather than post-intervention interest.   Response to comment 3:We have completely removed this term and claim from the Abstract and Section 4.1 to focus strictly on the observed perceived changes within our sample.  

Response to comment 4: To ensure this limitation is clearly acknowledged throughout the manuscript, we have made the following additions:

• In Section 2.3 (Participants and data collection procedure), we explicitly state that the study employed a single-group retrospective design, which evaluates perceived associations rather than direct causal effects.
• In Section 4.5 (Study limitations), we include a dedicated discussion formally acknowledging that without a baseline control group, the observed perceived changes cannot be causally attributed to the intervention

Response to comment 5: We thank the reviewer for requesting greater transparency regarding these figures, which we have now fully detailed in Section 2.3. The figure of approximately 3,500 students reflects total historical attendance across four academic years (September 2022 to May 2026, operating during the school term from September/October to May, with no sessions held during the summer recess). During the first academic year (September 2022 to May 2023), approximately 600 students attended before the evaluation protocol existed (supported by an official institutional press release [35]). The survey protocol was implemented across the subsequent three academic years (October 2023 to May 2026), during which institutional booking logs recorded 83 school visits totalling approximately 2,900 pre-registered students. We clarify in Section 2.3 that these attendance figures are approximate because they derive from pre-visit administrative registrations submitted by visiting secondary schools, and no individual headcount was taken on the day of the visits to account for occasional student absences. During this evaluation period, the anonymous, voluntary, and uncompensated paper questionnaire was offered in person to all students who attended at the conclusion of the morning activities. Because participation was strictly voluntary, many students chose not to complete the form (and no log of reasons for non-participation was kept), resulting in 584 valid responses. We explicitly report this 20.1% figure (584 out of approx. 2,900 pre-registered students) as an estimated response rate, or approximate participation proportion relative to administrative booking records, and acknowledge the resulting risk of volunteer/non-response bias in Section 4.5.   Response to comment 6: Clustering and independence: We fully agree that the programme data are structurally nested and that independence of observations cannot simply be assumed. However, a clustered analysis (such as multilevel modelling or cluster-robust standard errors) could not be implemented because school, visit date, programme edition, and rotation group identifiers were not collected in the strictly anonymous paper surveys and could not be reconstructed retrospectively:

1. 1. Schools, visits, and programme editions: To comply with strict anonymity requirements, no tracking identifiers for the visiting school, date, or academic year/edition were recorded on the paper questionnaires. Consequently, the potential contribution of school-level clustering or programme edition could not be reliably assessed in the anonymised dataset.
   2. Heterogeneous workshop itineraries: Students participated in rotating circuits of 4 to 5 workshops in smaller groups. When visits exceeded 50 students, morning time constraints meant that different groups from the same school often completed different workshop itineraries across the catalogue of 23 activities.
   3. Consequently, workshop evaluations were aggregated at the individual student level. In Section 4.5 (Study limitations), we now explicitly acknowledge that because cluster and edition identifiers were not retained, the individual-level models should be interpreted as exploratory and their standard errors may be underestimated if residual dependence was substantial.

Response to comment 7: We completely agree that because female mentors were tied to specific workshop topics, it is impossible to statistically isolate the effect of mentor gender from the intrinsic nature of those specific workshops. Consequently, we cannot make a definitive comparative claim that the content matters more than the mentor's gender. To correct this, we have substantially softened our claims. In Section 4.4, we added an explicit acknowledgment of this confounding effect, stating that we cannot definitively conclude that pedagogical content outweighs mentor gender. We have also modified the third bullet point in the Conclusions (Section 5) to remove the direct comparison between 'what is done' and 'who teaches it', focusing instead on the need for sustained contact for role models to be effective.   Response to comment 8: To correct this overstatement, we have thoroughly revised the manuscript to frame these findings strictly as 'exploratory observations' rather than proven 'gender differences'. Specifically, we have:

• Changed the headings of Sections 3.5 and 4.3 to remove the claim of 'gender differences'.
• Rewritten the final paragraph of Section 3.5 to explicitly state that separating these results does not prove a difference, and that without formal interaction tests, these are merely exploratory hypotheses.

Softened the Conclusions (Section 5) to reflect the exploratory nature of these nuances. We believe these changes accurately reflect the statistical reality of the models while preserving the value of these observations for future research hypotheses.   Response to comment 9: We completely agree that because the p-values for active workshop participation (p=0.056 overall; p=0.052 for males) lie just above the prespecified α=0.05 threshold, it is inaccurate to label this variable as a confirmed “main driver”. We have revised the Abstract, Results (Sections 3.4 and 3.5), Discussion (Sections 4.2 and 4.3), and Conclusions (Section 5) to remove this term, describing active workshop participation instead as showing a “positive predictive trend that did not reach the prespecified significance threshold”.

Response to comment 10: We have substantially expanded Section 2.4 (structured into Subsections 2.4.1 to 2.4.4) and Section 3.4 to address each of your points:

•  Questionnaire items & validity (Section 2.4.1): We provide the exact wording and anchors for the main items and clarify that this was an ad hoc instrument designed by the research team, with face validity established through review and approval by the school’s management team.
•  Missing-data treatment (Section 2.4.2): We detail that structurally missing data (unattended workshops) were handled by calculating the arithmetic mean strictly across the specific workshops each student actually attended, while missing baseline covariates were handled via listwise deletion.
• Model diagnostics (Sections 2.4.4 and 3.4): We detail our full diagnostic protocol in Section 2.4.4 and report the corresponding values in Sections 3.3 and 3.4, including the global likelihood ratio χ^2 test, McFadden's Pseudo-R^2, Variance Inflation Factors (VIF), the Box–Tidwell linearity test, Cook's distance, the Hosmer–Lemeshow test, the Brier score, AIC, and nested Likelihood Ratio Tests (LRT).

Response to comment 11: We have expanded both the “Institutional Review Board Statement” and the “Informed Consent Statement” at the end of the manuscript to fully detail the approval, exemption, and consent procedures:

• Institutional approval and ethics exemption: We clarify that the outreach programme and its evaluation questionnaire were reviewed and approved by the Management Team of the School of Industrial Engineering at the University of DELETED FOR REVIEW. Formal review by the university's Research Ethics Committee was waived (exempt) under institutional guidelines because the study evaluated a standard educational outreach practice using a completely anonymous, voluntary, and non-intrusive survey.
• Parental consent and student assent: Written parental or legal guardian consent for minors to attend the university outreach visit was obtained and managed directly by the participating secondary schools in accordance with their standard protocols for extracurricular educational activities. Formal written informed consent for the survey was waived under the supervision of the centre's Management Team to preserve complete anonymity (as signed forms would have compromised participant anonymity); prior to completing the form, students were verbally informed of the voluntary nature of the survey and their right not to participate, and their voluntary completion of the questionnaire constituted active student assent.

Response to comment 12: We have corrected the bibliography by replacing Reference [10] with a new, relevant citation in the literature, and we have slightly modified the corresponding sentence in the manuscript to align accurately with this new source.    

Reviewer 2 Report

Comments and Suggestions for Authors

This is a timely and potentially useful evaluation of a STEM outreach initiative. The manuscript is clearly organised and engages with recent literature; however, substantial revision is required before the reported findings can support the current conclusions. The principal issues concern study design, measurement, statistical specification, and over-interpretation of observational and non-significant results.

1. Please define the design more precisely. The work appears to be an observational programme evaluation rather than a conventional case study.

Because there is no control group and the outcome is an immediate self-report, causal terms such as “effect”, “impact”, “driver”, “activation mechanism”, “reversing”, and “transformative potential” should be replaced with language describing associations or perceived short-term changes.

Explicit research questions or hypotheses would also improve alignment between the Introduction, Methods, and Results.

2. Please report the number of eligible students during the survey period, the number invited, the number responding, the number of schools/visits, and the response rate. The analysed sample of 584 should be contextualised against the larger programme reach.

Because students are nested within schools, visits/editions, and workshop itineraries, independence cannot simply be assumed. Multilevel modelling or cluster-robust standard errors should be considered, with year/edition assessed where relevant.

3. Clarify whether “prior interest” was measured before the intervention or retrospectively in the post-session questionnaire. If retrospective, it should not be described as a prospective pre-test.

Provide the exact questionnaire wording and anchors, its development/source, and evidence of validity or reliability where applicable.

A single perceived-change item is a limited proxy for vocational change, and “overall satisfaction” should not be treated as equivalent to edutainment or playfulness.

4. Please justify dichotomising the five-category outcome and consider an ordinal model or sensitivity analysis.

McFadden’s pseudo-R-squared should not be interpreted as a percentage of variance explained.

If Model 2 is presented as an improvement over Model 1, support this using suitable fit comparisons, for example AIC/BIC and, where appropriate, a nested-model test.

Report multicollinearity diagnostics and clarify how averages across different sets of workshops were constructed.

5. The 53.2% figure is not evidence that the programme mitigated self-selection bias, because low/moderate-interest students already constitute 70.5% of the total sample. Compare within-group probabilities instead.

Active participation remains statistically uncertain (p = 0.056; 95% CI includes the null) and should not be called an established predictor or main driver.

Likewise, the mentor estimate is highly imprecise and non-randomly confounded with workshop content, so the appropriate conclusion is inconclusive evidence rather than no role-model effect.

6. A significant association in one gender stratum and a non-significant association in another does not demonstrate a statistically significant gender difference.

Please test gender-by-predictor interactions in the full sample or retain the stratified findings as exploratory.

Explanations involving stereotypes, STEM self-concept, and social barriers are theoretically plausible but were not directly measured and should therefore be clearly distinguished from empirical findings.

7. Expand the limitations to include:

• the absence of a control group and prospective pre-test;
• volunteer/non-response bias;
• common-method bias;
• clustering;
• heterogeneous workshop exposure;
• confounding of mentor gender with content;
• outcome dichotomisation; and
• limited subgroup precision.

Because participants are minors, identify the authority that granted the ethics/consent waiver and clarify parental/guardian consent and student assent.

The public data repository should also be reviewed for anonymity during peer review and for re-identification risk.

8. Rewrite the Conclusions so that they mirror the statistical evidence and clearly distinguish supported findings from hypotheses for future study.

Consider removing redundant figures where tables already report the same information.

The reference list should be audited: references 9 and 10 appear duplicated, and at least one DOI/reference entry appears incomplete.

A final language edit for concise British academic English and consistent terminology is recommended.

Comments on the Quality of English Language

The manuscript is generally intelligible, but it would benefit from editing for concision, idiomatic academic phrasing,
consistent British English usage, and more cautious statistical terminology.

Author Response

Thank you very much for your comments and observations. Below you will find the response to each comment, numbered from 1 to 8 as indicated.

Response to Comment 1: We thank the reviewer for these highly constructive suggestions, which have helped us frame our study much more accurately. We have addressed all three aspects of this comment:

Study design: We completely agree that “observational programme evaluation” is a much more accurate description than “case study”. We have updated the manuscript's Title and the methodology description accordingly.

Causal language: Throughout the entire manuscript (including the Abstract, Results, Discussion, and Conclusions), we have systematically removed causal terms such as “effect”, “impact”, “driver”, “activation mechanism”, “reversing”, and “transformative potential” when referring to our own findings and study objectives. We have replaced them with precise observational language describing “associations”, “perceived short-term changes”, and “positive predictive trends that did not reach the prespecified significance threshold”, carefully reflecting the single-group retrospective nature of our data.

Research questions and hypotheses: To improve the alignment between the Introduction, Methods, and Results, we have added a dedicated subsection at the end of the Introduction (Section 1.1. Theoretical hypotheses and research questions), formulating three explicit hypotheses (H1–H3) and three corresponding exploratory Research Questions (RQ1–RQ3).

 

Response to Comment 2: We completely agree with the need to contextualise the sample and address the nested nature of the data.

Sample contextualization: We thank the reviewer for requesting greater transparency regarding these figures, which we have now fully detailed in Section 2.3. The figure of approximately 3,500 students reflects total historical attendance across four academic years (September 2022 to May 2026, operating during the school term from September/October to May, with no sessions held during the summer recess). During the first academic year (September 2022 to May 2023), approximately 600 students attended before the evaluation protocol existed (supported by an official institutional press release [35]). The survey protocol was implemented across the subsequent three academic years (October 2023 to May 2026), during which institutional booking logs recorded 83 school visits totalling approximately 2,900 pre-registered students. We clarify in Section 2.3 that these attendance figures are approximate because they derive from pre-visit administrative registrations submitted by visiting secondary schools, and no individual headcount was taken on the day of the visits to account for occasional student absences. During this evaluation period, the anonymous, voluntary, and uncompensated paper questionnaire was offered in person to all students who attended at the conclusion of the morning activities. Because participation was strictly voluntary, many students chose not to complete the form (and no log of reasons for non-participation was kept), resulting in 584 valid responses. We explicitly report this 20.1% figure (584 out of approx. 2,900 pre-registered students) as an estimated response rate, or approximate participation proportion relative to administrative booking records, and acknowledge the resulting risk of volunteer/non-response bias in Section 4.5.

Clustering and independence: We fully agree that educational data is inherently clustered and independence should not simply be assumed. However, accounting for this clustering through multilevel modelling or cluster-robust standard errors was technically unfeasible for our study due to logistical and mathematical constraints, which we have now explicitly added to the Study Limitations (Section 4.5):

Schools and visits: To strictly comply with the ethical requirement of absolute anonymity, the paper surveys did not include tracking identifiers for the school or date. Once collected, they could not be reliably linked back to specific visits.

Heterogeneous itineraries: Students did not attend a single workshop but a rotating circuit of 4 to 5 activities. When a visit exceeded 50 students, logistical time constraints meant that not all groups could rotate through the exact same workshops. Consequently, students from the same school on the same day often completed entirely different itineraries. Developing a cross-classified multilevel model for 23 different workshops combined in such varied itineraries would require a significantly larger sample size to achieve mathematical stability.

Programme editions: The pedagogical design and the core team of instructors remained largely constant across the entire evaluation period. The only exceptions were two specific workshops led by university students (Fablab and XXX Motorsport) whose composition varied across academic years. Given this isolated variance, clustering by edition was not deemed statistically critical.

Given these structural constraints imposed by anonymous data collection, we aggregated the variables at the individual level. We have now clearly stated this assumption and the absence of cluster-robust standard errors as a formal limitation in Section 4.5.

Response to Comment 3: We have thoroughly revised the manuscript to address each of these points:

Retrospective measurement: We have clarified in Section 2.3 that “prior interest” was indeed measured retrospectively in the post-session questionnaire, asking students to recall their initial predisposition. To ensure methodological rigour, we have added this reliance on retrospective measurement, and the associated risk of recall bias, as a formal limitation in Section 4.5. It is no longer treated or described as a prospective pre-test.

Questionnaire wording and anchors: In Section 2.4, we have added the exact wording and numerical anchors for all the main variables (prior interest, change in interest, participation, and satisfaction). As stated in the manuscript, this was an ad-hoc instrument specifically designed for the logistical constraints of the visit, and its face validity was established by the centre’s management team.

Single-item proxy: We completely agree that a single perceived-change item is a limited proxy for actual vocational change. We have explicitly added this constraint to our Limitations (Section 4.5), acknowledging that it only captures an immediate, situational reaction rather than a sustained career choice.

Satisfaction vs. edutainment: We fully agree that a single-item measure of overall workshop satisfaction is not equivalent to the broader pedagogical construct of "edutainment" or playfulness. We have revised the manuscript throughout (including the Abstract, Introduction, Section 4.2, Section 4.5, and Section 5) to replace the term "dissociation" and avoid equating our variable with edutainment. We now strictly refer to our measured variables as "overall workshop satisfaction (self-reported enjoyment)" and "active workshop participation," describing their results as showing "different statistical patterns" while referencing edutainment solely as theoretical context from prior literature.

Response to Comment 4: We thank the reviewer for these rigorous statistical observations, which have substantially strengthened the methodological reporting of our study. We have addressed each diagnostic request as follows:

Dichotomisation vs. Ordinal model: In Section 2.4.3 (Coding and analytical sample), we explicitly justify the dichotomisation. An ordinal model was considered, but the very small number of responses in the decrease category (only 8 out of 584 students) resulted in unstable estimation and convergence problems. Therefore, the outcome was dichotomised for the primary analysis, and the resulting reduction in outcome granularity is explicitly acknowledged as a limitation in Section 4.5.

McFadden’s Pseudo- : We apologise for the inaccurate phrasing. We have corrected Section 3.3 to remove the reference to “variance explained”; it is now properly defined as representing a “relative improvement in model fit compared to the null model”.

Model fit comparison: In Section 3.4, we have added formal nested-model comparisons on the exact same analytical sample ( ). Because “general school attention” was dropped after Model 1, we compared Model 2 against the nested structural baseline model retaining student gender and prior interest. A nested Likelihood Ratio Test (LRT) confirmed that introducing the three pedagogical variables in Model 2 provided a statistically significant improvement in model fit ( ), further supported by a reduction in the Akaike Information Criterion (AIC) from 524.99 (structural baseline) to 507.79 (Model 2), alongside adequate calibration as indicated by the Hosmer–Lemeshow test ( ) and Brier score ( ).

Multicollinearity and assumption diagnostics: We have updated Sections 2.4.4 and 3.4 to report our full diagnostic checks. All predictor Variance Inflation Factors (VIF) fell below the recommended threshold of 5 (ranging from 1.01 to 4.42), the Box–Tidwell test indicated no statistically significant violations of logit linearity (  for active participation;  for overall satisfaction), and Cook's distance revealed no extreme influential outliers ( ).

Workshop averages construction: As detailed in Section 2.4.2 (Data cleaning and missing-data treatment), unvisited workshops in the catalogue were treated as structurally missing data, and the continuous variables (active workshop participation and overall workshop satisfaction) were calculated strictly as the arithmetic mean of only the specific 4 or 5 workshops that each student actually attended.

Finally, to ensure full methodological transparency and reproducibility, the Python script in the project's repository includes the complete code for all diagnostic metrics (VIF, Box–Tidwell, Cook's distance, Hosmer–Lemeshow, Brier score, AIC, and nested LRT; link anonymised during peer review).

Response to Comment 5: We have carefully recalibrated our claims throughout the manuscript to avoid over-interpreting the data:

Self-selection bias & 53.2% figure: We fully concede that because low/moderate-interest students constituted 70.5% of the baseline sample, their 53.2% share of the maximum-change group does not mathematically prove a mitigation of self-selection bias. We have completely removed the “mitigation of self-selection bias” claim from the Abstract and the Discussion (Section 4.1), contextualised the 53.2% figure against the 70.5% baseline share in Sections 3.2 and 4.1, and based our descriptive interpretation on the within-group probabilities reported in Table 3 (which show a positive perceived shift for 41.6% of the moderate group and 25.8% of the low group).

Active participation ( ): We agree that with a p-value just above the strict 0.05 threshold and a 95% CI including the null ( ), this variable cannot be labelled a confirmed main driver. We have revised the Abstract, Section 3.4, Section 4.2, and Section 5 to downgrade our language, explicitly describing active workshop participation as showing a “positive predictive trend that did not reach the prespecified significance threshold” rather than an established predictor.

Mentor confounding and imprecision: In Sections 2.4.3, 3.4, 4.4, 4.5, and 5, we now explicitly note that the mentor configuration estimate is statistically imprecise (exhibiting a wide 95% CI of [0.049, 2.802] due to low variability across multi-workshop rotations), combines female-only and mixed teams into a single indicator of any female mentor exposure, and is intrinsically confounded with specific workshop content. Consequently, we explicitly state that these constraints prevent a reliable interpretation of the parameter's direction or magnitude, yielding inconclusive evidence rather than demonstrating the absence of a role-model effect.

 

Response to Comment 6: We fully agree with this crucial statistical and theoretical correction. We have thoroughly revised the manuscript to ensure our reporting strictly aligns with these boundaries:

Exploratory findings vs. Proven differences: Because our sample size within the subgroups lacked the statistical power to conduct robust gender-by-predictor interaction tests in the full model, we have chosen to retain the stratified findings but strictly reframe them as exploratory. We have changed the headings of Sections 3.5, 4.3, and the Conclusions to describe these as "exploratory patterns" rather than gender differences. Furthermore, we added explicit disclaimers in Section 3.5, the Conclusions (Section 5), and the Study Limitations (Section 4.5), clearly stating that without formal interaction tests, these results do not demonstrate statistically significant gender differences, but merely serve as exploratory hypotheses for future research.

Theoretical vs. Empirical explanations: We agree that constructs such as gender stereotypes, STEM self-concept, and sociocultural barriers were not empirically measured in our survey. We have carefully revised Section 4.3 (Exploratory gender-stratified patterns in perceived vocational change) and Section 5 to explicitly state that these constructs were not directly measured in our dataset and to present them strictly as plausible theoretical interpretations grounded in prior literature (“From a theoretical standpoint, it is possible that...”, “As a theoretical implication consistent with prior research, rather than a direct empirical test from our data...”).

 

Response to Comment 7: We thank the reviewer for this comprehensive checklist. We have addressed all three areas as follows:

Expanded limitations: We have substantially expanded Section 4.5 (Study limitations) into a structured discussion that explicitly addresses all eight methodological constraints requested: (1) reliance on short-term self-reported measures; (2) the absence of a control group and reliance on a retrospective prior-interest measure (recall bias), single-item perceived change proxy, and single-item satisfaction measure; (3) the assumption of independent observations and inability to model clustering across schools, visits, editions, or heterogeneous rotating workshop itineraries; (4) potential volunteer and non-response bias from the approximate 20.1% participation proportion; (5) common-method bias from a single survey instrument; (6) the combination of female-only and mixed teams, wide confidence intervals, and confounding of mentor gender with workshop content; and (7) outcome dichotomisation and limited subgroup precision in the gender-stratified models.

Ethics and consent for minors: We have expanded the “Institutional Review Board Statement” and “Informed Consent Statement” at the end of the manuscript. We clarify that the outreach programme and its evaluation questionnaire were reviewed and approved by the Management Team of the School of Industrial Engineering at the University of DELETED FOR REVIEW, while formal review by the university's Research Ethics Committee was waived (exempt) under institutional guidelines because the survey evaluated a standard educational outreach practice and was completely anonymous and voluntary. Parental or legal guardian consent for minors to attend the university visit was obtained and managed directly by the participating secondary schools in accordance with their official protocols, and formal written survey consent was waived under the supervision of the centre's Management Team to preserve anonymity; students were verbally informed of the voluntary nature of the survey and their right not to participate, and voluntary completion of the questionnaire constituted active student assent.

Data repository and anonymity: First, to ensure author anonymity during double-blind peer review, we have replaced the GitHub link in Section 2.5 with a blinded placeholder. Second, in Section 2.5 we explicitly clarify that because the public dataset omits school, visit date, programme edition, and workshop group identifiers and includes only broad demographic variables, the risk of direct or indirect student re-identification is minimized.

 

Response to Comment 8: Conclusions reframing: We have thoroughly rewritten Section 5 so that the conclusions strictly mirror the statistical evidence, replacing causal terms (e.g., changing “main driver” to “a positive predictive trend that did not reach the prespecified significance threshold”), noting the inconclusive and confounded nature of the mentor variable, and explicitly distinguishing empirical observations from theoretical hypotheses for future longitudinal research.

Redundant figures: To avoid visual duplication and streamline the manuscript, we have removed all figures (formerly Figures 1 to 6), as both the descriptive distributions and the regression estimates (Odds Ratios and 95% CIs) were already reported in full detail in their corresponding tables (Tables 2, 4, and 6–9).

Reference audit: We have thoroughly revised and corrected the bibliography. Specifically, the sentence corresponding to Reference [10] has been slightly modified in the main text, and the duplicate entry has been replaced with a new, distinct reference. Additionally, the incomplete DOI has been fully completed.

Language and formatting: A final proofreading pass was conducted to ensure consistent terminology and strict adherence to concise British academic English.

 

 

Author Response File: Author Response.pdf

Reviewer 3 Report

Comments and Suggestions for Authors

The manuscript addresses a timely and practically relevant question concerning whether short immersive STEM outreach activities are associated with students’ vocational interest in engineering. The paper is generally well organized, the intervention is described in considerable detail, the sample is reasonably substantial, and the provision of anonymized data and analytical code is a welcome strength. The distinction between active engagement and general satisfaction is potentially valuable for STEM outreach research. However, in its current form, I have several major methodological and inferential concerns that substantially limit the conclusions that can be drawn.

1. The theoretical foundation should be strengthened and more explicitly connected to the empirical models. The manuscript would benefit from a dedicated subsection explaining why each predictor included in Models 1 and 2 is theoretically expected to relate to vocational change and, where appropriate, specifying directional hypotheses. The Discussion should then systematically compare the observed results, including null findings, with previous empirical evidence rather than primarily interpreting the present findings in isolation.

2. Section 2.3 states that the paper-based survey was administered immediately after completion of the activity. The same questionnaire appears to assess both “prior interest” and perceived “change in interest.” If so, this is not a conventional pre–post design: baseline interest was not prospectively measured before the intervention, and the dependent variable represents retrospectively perceived change rather than an observed pre-to-post difference. This distinction is crucial. Table 5 is also potentially misleading because columns labelled “Post 1–Post 5” apparently represent a change score rather than an actual post-intervention interest score. The design, measures, and terminology should therefore be revised throughout.

3. Sampling and dependence among observations need greater transparency. Approximately 3,500 students attended the programme, whereas 584 constitute the survey sample. The manuscript should report the number eligible during the actual survey period, participation/response rate, recruitment at the student level, and reasons for non-participation where available. Students also appear to be nested within schools, visits/sessions, workshop itineraries, and potentially mentors. Standard logistic regression assumes independent observations; therefore, the authors should assess this issue.
4. The logistic regression analyses require substantially more diagnostic reporting. The manuscript should report linearity of the logit for continuous predictors, influential observations, and appropriate model fit/calibration diagnostics. The manuscript should also state the prespecified significance threshold and analytical decision rules.
5. McFadden’s pseudo-\(R^2\) should not be interpreted as the percentage of variance explained; statements that values of .026 and .066 correspond to 2.6% and 6.6% of explained variance should be revised. Moreover, \(p=.056\) or \(p=.052\) does not constitute statistical significance under a conventional \(\alpha=.05\) criterion. Describing active participation as the “main driver” of positive change is therefore not supported by Model 2. Comparisons of female and male stratified models are also problematic: significance in one subgroup and non-significance in another does not demonstrate a gender difference. Formal gender × predictor interaction terms are required before claiming gender-differentiated mechanisms.
6. Terms such as “impact,” “effect,” “transformative potential,” “successfully stimulated,” “driver,” and statements implying that the intervention generated vocational change should be reconsidered. Without a prospective baseline assessment, comparison/control group, or randomized/quasi-experimental design, the data support associations with self-reported perceived change, not causal effects.

Author Response

Thank you very much for your comments and observations. Below you will find the response to each comment, numbered from 1 to 6 as indicated.

Response to comment 1: We have addressed this through two main actions:

Dedicated hypotheses subsection (Section 1.1): At the end of the Introduction, we have created a dedicated subsection (1.1. Theoretical hypotheses and research questions) that explicitly connects each predictor and control variable in Models 1 and 2 (baseline predisposition, student gender, general school attention, active workshop participation, overall workshop satisfaction, and female role models) to its theoretical foundation in prior literature and specifies our directional expectations (H1–H3) alongside our exploratory Research Questions (RQ1–RQ3).

Systematic comparison in the discussion (Sections 4.1-4.4): We have revised the Discussion to contextualise the principal findings, including the positive trend, the null association for satisfaction, and the inconclusive mentor estimate, in relation to previous literature.

Response to comment 2: Retrospective design clarification: We have thoroughly revised the manuscript to remove any implication of a conventional prospective pre-test/post-test design. Section 2.3 now explicitly defines the methodology as a "single-group retrospective design". We clearly state that prior interest was assessed retrospectively (asking students to recall their initial predisposition) and that our dependent variable exclusively captures "self-reported perceived change" rather than an observed true pre-to-post difference.

Table 5 Correction: We have corrected the column headers in Table 5. They now read "Change 1" through "Change 5" to accurately reflect that these values represent the perceived change score on the Likert scale, avoiding any confusion with an actual post-intervention continuous score.

Limitations updated: To ensure full transparency, we have explicitly added the reliance on retrospective measurement, and the associated risk of recall bias, as a formal constraint in Section 4.5 (Study limitations).

 

Response to comment 3: We have significantly expanded our methodological reporting to address both concerns:

• Sampling transparency: The figure of approximately 3,500 students reflects total historical attendance across four academic years (September 2022 to May 2026, operating during the school term from September/October to May, with no sessions held during the summer recess). During the first academic year (September 2022 to May 2023), approximately 600 students attended before the evaluation protocol existed (supported by an official institutional press release [35]). The survey protocol was implemented across the subsequent three academic years (October 2023 to May 2026), during which institutional booking logs recorded 83 school visits totalling approximately 2,900 pre-registered students. We clarify in Section 2.3 that these attendance figures are approximate because they derive from pre-visit administrative registrations submitted by visiting secondary schools, and no individual headcount was taken on the day of the visits to account for occasional student absences. During this evaluation period, the anonymous, voluntary, and uncompensated paper questionnaire was offered in person to all students who attended at the conclusion of the morning activities. Because participation was strictly voluntary, many students chose not to complete the form (and no log of reasons for non-participation was kept), resulting in 584 valid responses. We explicitly report this 20.1% figure (584 out of approx. 2,900 pre-registered students) as an estimated response rate, or approximate participation proportion relative to administrative booking records, and acknowledge the resulting risk of volunteer/non-response bias in Section 4.5.
• Dependence and clustering: We fully agree that standard logistic regression assumes independent observations. In Section 4.5, we now explicitly state that a clustered or multilevel analysis could not be implemented because school, visit date, programme edition, and group identifiers were not collected on the anonymous paper surveys and could not be reconstructed retrospectively. We also detail the heterogeneous workshop exposure across rotating groups and openly acknowledge that estimating the models at the individual level means that standard errors may be underestimated if residual dependence across clusters was substantial.

 

Response to comment 4: We have expanded Section 2.4.4 (Econometric modelling and diagnostics) to detail the diagnostic protocol and Section 3.4 to report the complete numerical results:

• Significance threshold and analytical decisions: We explicitly state in Section 2.4.4 that our prespecified significance threshold was set at , and that additional analytical decisions (such as the handling of small subgroups, outcome dichotomisation, and diagnostic procedures) are transparently documented in Sections 2.3 and 2.4.
• Linearity of the logit: Tested via the Box–Tidwell procedure ( interaction terms). Neither continuous predictor violated the linearity assumption at the  level: active workshop participation showed clear linearity ( ), and overall workshop satisfaction was also non-significant ( ; we prudently note this value approaching the threshold while retaining the linear specification to preserve model parsimony).
• Influential observations: Assessed using Cook's distance. The maximum Cook's across the sample was  (with  cases exceeding the critical threshold of , and only  cases slightly above the  heuristic cut-off of ), confirming that no extreme outliers distorted the parameter estimates.
• Model fit, calibration, and multicollinearity: In Section 3.4 and Table 7, we report the nested Likelihood Ratio Test comparing Model 2 against the nested structural baseline model on the exact same analytical sample ( ), the reduction in AIC (from to ), the Hosmer–Lemeshow calibration test ( , indicating adequate calibration), the Brier score ( ), and Variance Inflation Factors (all predictor , ranging from  to ).

 

Response to comment 5: We have rigorously corrected the manuscript to align with these methodological boundaries:

• McFadden’s Pseudo- : We apologise for the inaccurate phrasing. We have corrected Section 3.3 to remove the reference to “variance explained”; it is now properly defined as representing a “relative improvement in model fit compared to the null model”.
• Interpretation of and : We agree that under our prespecified  threshold, these results do not reach statistical significance. We have completely removed the term “main driver” from the Abstract, Results, Discussion, and Conclusions, and now strictly describe active workshop participation as showing a “positive predictive trend that did not reach the prespecified significance threshold”.
• Gender differences vs. Exploratory hypotheses: Because our sample size within the subgroups lacked the statistical power to conduct robust gender-by-predictor interaction tests in the full model, we have chosen to retain the stratified findings but strictly reframe them as exploratory. We added explicit disclaimers in Section 3.5, the Conclusions (Section 5), and the Study Limitations (Section 4.5), clearly stating that without formal interaction tests, these results do not demonstrate statistically proven gender differences, but merely serve as exploratory hypotheses for future research.

 

Response to comment 6: Acknowledging that our observational, single-group retrospective design precludes causal inference, we have conducted a comprehensive linguistic audit of the entire manuscript and systematically removed causal terminology when describing our findings. Specifically, terms such as “impact”, “effect”, “catalyst”, “activation mechanism”, and “transformative potential” have been replaced with appropriate observational language (e.g., “association”, “perceived changes”, “observed patterns”). Active verbs implying direct causality (such as “successfully stimulated” or “generated”) have been replaced with associative phrasing (“was associated with perceived increases in interest”), and terms like “driver” have been downgraded to “a positive predictive trend that did not reach the prespecified significance threshold”.

 

 

Author Response File: Author Response.pdf

Reviewer 4 Report

Comments and Suggestions for Authors

Brief summary

This manuscript examines the short-term change in pre-university students' interest in industrial engineering following participation in an immersive university outreach programme. Using survey data from 584 students and logistic regression models on a subsample of 408 students with low or moderate prior interest, the authors investigate whether active participation, satisfaction, mentoring configuration, prior interest, and gender are associated with reported increases in vocational interest. The paper addresses a relevant issue in STEM education and has the merit of moving beyond simple satisfaction measures by attempting to identify which characteristics of a short outreach intervention are associated with changes in vocational interest. The dataset is reasonably large for this type of intervention study, the authors make the analytical code and anonymised dataset available, and the distinction between enjoyment and active engagement could be useful for the design of STEM outreach programmes.

General concept comments

1. Programme

In the discussion and conclusions (Section 5.1), the authors refer to "well-structured immersive interventions" but do not provide precise and explicit descriptions of the programme itself, only of its effects, thereby reducing the potential for reproducibility in other contexts. Since students followed different combinations of four or five workshops selected from 23 possibilities, the authors should report how heterogeneous the actual exposure was. It would be useful to know whether certain workshops were substantially overrepresented and whether workshop type was associated with gender, prior interest, or outcome.

2. Method

The intervention involved approximately 3,500 students across multiple editions, schools, itineraries, and workshops. Each visit included four or five workshops selected from a catalogue of 23, depending on school preferences and staff availability. Consequently, students were almost certainly clustered within schools, visits, workshop itineraries, and potentially individual workshops/mentors.

This raises an important issue for the logistic regression analyses. The manuscript appears to treat all 408 students as independent observations. If students from the same school or visit share the same intervention conditions and institutional context, the independence assumption is questionable. Standard errors may therefore be underestimated.

The authors should clarify the sampling structure and report the number of schools, visits, cohorts, and workshop configurations represented in the analytical sample. They should consider a multilevel model or, at minimum, cluster-robust standard errors at an appropriate level if the number of clusters permits this. If such an analysis is not possible, the implications of the clustering need to be explicitly discussed as a limitation.

3. McFadden's pseudo-R²

The manuscript states that a McFadden's pseudo-R² of 0.026 means that the model "explain[s] 2.6% of the variance in interest change," and similarly interprets 0.066 as explanatory power in percentage terms. This is not an appropriate interpretation of McFadden's pseudo-R². It is not analogous to the variance-explained interpretation of ordinary R² in linear regression.

4. Gender

This is one of the more important statistical issues. The authors fit separate models for female and male students and then compare the significance of predictors across the two models. However, the fact that a predictor is statistically significant in one subgroup and not significant in another does not establish that the predictor differs significantly between the groups.

In my opinion, the weight given to the gender aspect is too strong (Sections 4.4 and 5.3). The discussion of gender stereotypes, social barriers, and technological competence should be presented as hypotheses consistent with previous literature, rather than as explanations established by this dataset. These constructs were not directly measured. The gender discussion attributes the observed pattern to social barriers, gender stereotypes, technological competence norms, and STEM self-concept. These are plausible hypotheses and are supported in general by the literature, but they were not measured in the present study. Consequently, they cannot explain the observed associations in an empirical sense.

5. Clarify the research object

The outcome is described as a five-point question concerning whether interest "decreased," "remained the same," or "increased" following the workshops. This is not equivalent to measuring interest before and after the intervention and calculating or modelling individual change. Rather, it appears to be a retrospective self-assessment of perceived change collected only after the intervention. In this perspective, I would also weaken the causal language used throughout the paper ("impact," "effect," "driver," "transformative potential," "activation mechanism," and "catalyst").

Specific comments

References 9 and 10: These appear to be duplicate references and should be checked.

Author Response

Thank you very much for your comments and observations. Below you will find the response to each comment, numbered from 1 to 5 as indicated.

Point 1 & 2 (Exposure Heterogeneity & Clustering): We fully agree that standard logistic regression assumes independent observations and that our participants were structurally nested. In Section 4.5 (Study limitations), we have clarified that a clustered or multilevel analysis could not be implemented because school, visit date, edition, and rotation group identifiers were not collected in the strictly anonymous paper surveys and could not be reconstructed retrospectively. Consequently, we explicitly warn that the individual-level models should be interpreted as exploratory and that standard errors may be underestimated if residual dependence was substantial. Furthermore, to directly address your Point 1, we explain that because students rotated in small groups through varying combinations of 4 to 5 workshops out of 23, and unattended cells do not distinguish between workshops not offered on a given day and those not completed, it is not possible to reconstruct the exact overall exposure distribution or determine whether certain workshops were overrepresented, which limits our ability to link specific workshop types to individual outcomes.

Point 3 (McFadden’s Pseudo-R²): We apologise for the inaccurate phrasing. In Section 3.3, we have removed the reference to "variance explained" and properly defined it as a "relative improvement in model fit compared to the null model."

Point 4 (Gender Differences & Theoretical Constructs): We fully accept this critical statistical boundary. In Sections 3.5, 4.3, and 5, we have reframed the gender-stratified findings strictly as "exploratory patterns" rather than proven differences, adding explicit disclaimers that formal interaction tests would be required to claim gender-differentiated mechanisms. Furthermore, in Section 4.3, we have explicitly separated our empirical findings from theoretical constructs (like STEM self-concept and social barriers), presenting the latter strictly as plausible theoretical hypotheses consistent with previous literature.

Point 5 (Retrospective Design & Causal Language): We have revised Section 2.3 to explicitly define the methodology as a “single-group retrospective design” measuring “self-reported perceived change”. Consequently, we conducted a full linguistic audit of the manuscript: causal terminology (e.g., “impact”, “effect”, “transformative potential”, “activation mechanism”, and “catalyst”) has been replaced with associative language, and terms such as “driver” have been downgraded to “a positive predictive trend that did not reach the prespecified significance threshold”.

Specific Comments (References): We have corrected the bibliography. Specifically, the sentence corresponding to Reference [10] has been slightly modified in the main text, and the duplicate entry has been replaced with a new, distinct reference.

 

 

 

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The authors have addressed the comments, and the manuscript can be accepted for publication

Reviewer 3 Report

Comments and Suggestions for Authors

The revised manuscript has improved and addresses the principal methodological concerns raised in the previous round. The study is now appropriately framed as a single-group retrospective observational programme evaluation, rather than as a prospective pre–post or causal design. The manuscript also reports the sampling process and approximate response proportion more transparently, acknowledges the inability to account for clustering, expands the logistic-regression diagnostics. I do not identify any remaining methodological or data-analysis problem that fundamentally invalidates the study. However, the practical implication drawn from active workshop participation should be moderated further because the association did not reach the prespecified significance threshold (p=.056). Overall, the manuscript is considerably stronger, and the major concerns from the previous review have been satisfactorily addressed.