Design and Preliminary Evaluation of an AI-Enabled Physical Systems Workshop

Abstract

Students’ use of artificial intelligence (AI) tools does not necessarily provide understanding of how AI-enabled systems are developed and evaluated. This study presents the design and preliminary evaluation of a guided experiential workshop on AI-enabled physical systems for secondary-school students. Students worked with programmable electronics, camera input, image datasets, machine-learning classification, and actuator responses. A one-group pre-test/post-test design examined immediate changes in workshop-specific foundational knowledge and exploratory associations with student-profile indicators. Pre-test data were available for 44 students, with 38 matched pre/post pairs. Mean scores increased from 7.21/16 to 11.55/16 (mean gain = 4.34 points; mean individual normalised gain = 0.50), t(37) = 8.35, p < 0.001, Cohen’s dz = 1.35. This standardised change describes performance on the workshop-aligned test. Post-test scores were higher in both electronics/programming and AI/ML/data domains. Exploratory analyses detected no significant associations between knowledge gains and initial STEM interest or ICT self-concept. Students reported positive perceptions of the workshop. The implementations provide preliminary evidence of practical feasibility with instructional support. These findings support further refinement and evaluation; they do not establish causal effects, broader AI literacy gains, or independent design competence.

1. Introduction

Students increasingly encounter artificial intelligence not only as a topic they explicitly study, but also as an often hidden layer within the digital tools they use to search for information, communicate, create content, generate ideas, and solve learning tasks (Ifenthaler et al., 2024; Licardo et al., 2025). This shift raises two related concerns. The first relates to responsible and critical AI use, including reliability, bias, privacy, fairness, and human oversight (Akgun & Greenhow, 2022; Ifenthaler et al., 2024). The second concerns whether students develop an understanding of the processes behind AI systems. This concern is central to AI literacy frameworks, which define AI literacy as more than tool use and include understanding, evaluating, creating with AI, and considering ethical implications (Ng et al., 2021; Almatrafi et al., 2024). From this perspective, an initial understanding of how data, model training, prediction, error, evaluation, and implementation shape AI behaviour provides a conceptual basis for discussing the capabilities and limitations of AI systems.

This broader framing is reflected in interdisciplinary, educational, and policy perspectives on AI literacy. ED-AI Lit and related competency frameworks emphasise that AI literacy includes understanding how AI systems are designed, trained, evaluated, and used in social contexts, rather than only operating AI-based tools (Allen & Kendeou, 2024; Chiu et al., 2024). From a European policy perspective, the AI Act similarly frames AI literacy as the knowledge, skills, and understanding needed for informed AI use and awareness of opportunities, risks, and possible harm (European Parliament and Council of the European Union, 2024). In technology education, this perspective is especially relevant because AI literacy extends beyond digital tool use to include understanding AI as part of integrated technological systems involving sensors, data, software, hardware, and the interaction between digital and physical components, as well as the role of human decision making in the design, evaluation, and responsible use of AI (Rupnik & Avsec, 2025; Stolpe & Hallström, 2024). These frameworks provide the broader educational context for the workshop; the present evaluation focuses on selected foundational technical and conceptual knowledge addressed in its activities, without directly assessing the full range of AI-literacy dimensions.

One implication of this perspective is that students should have opportunities to experience how AI systems are created, tested, and improved, rather than encountering AI only as a black-box feature of digital tools. Morales-Navarro and Kafai (2024) argue that K-12 machine-learning education should include opportunities for students to engage with datasets, model training, algorithms, and the distinction between model training and model use. Their discussion of black-box and glassbox approaches is especially relevant: educational activities can decide which parts of the machine-learning process remain hidden and which are made visible to learners. Accordingly, this study treats hands-on model building not as advanced AI training, but as a pedagogical approach for making selected AI-literacy-related concepts visible, testable, and discussable.

The emphasis on making otherwise hidden processes accessible to learners also has an established pedagogical basis. Cognitive apprenticeship makes expert problem-solving processes visible through modelling, coaching, scaffolding, articulation, and reflection (Collins et al., 1991). Applied to AI education, this perspective offers a rationale for demonstrating model evaluation and guiding learners to explain errors and consider revisions to data or models.

Hands-on and experiential approaches can make AI systems more understandable by connecting abstract concepts with observable behaviour. Förster et al. (2026) provide empirical support for experiential AI learning in K-12 education, while Kim and Kwon (2024) show that tangible computing tools can support students’ AI-related knowledge and perceptions. Robotics-based approaches to AI and machine-learning education also suggest that physical and interactive systems can help students connect AI concepts with system behaviour they can observe and modify (Karalekas et al., 2025). Similarly, work on Arduino and AI in STEM education highlights the value of affordable platforms that allow learners to program systems that respond to environmental stimuli and support active, practical learning (Bedoya-Cano et al., 2025). Low-cost Android/Arduino robots have also been developed as platforms for teaching applied computer vision, intelligent control, and machine learning (Lopez-Rodriguez & Cuesta, 2021).

At the same time, such interdisciplinary activities are demanding. A workshop that combines electronics, programming, data collection, machine learning, and physical system behaviour requires students to coordinate several types of knowledge. Cognitive load theory emphasises that learning is constrained by limited working-memory capacity, especially when students must process novel and highly interactive information (Sweller et al., 2019). Integrated STEM activities may therefore require careful instructional support, particularly when students must transfer concepts across representations, components, and physical systems. Kelly and Bugallo (2023), for example, found that students in an informal engineering programme experienced challenges related to abstraction, transfer, component functionality, pacing, and different rates of task completion. These considerations motivate a staged instructional sequence with guidance for technically demanding connections between electronics, data, models, and physical outputs.

Student characteristics may also shape how learners benefit from such activities. Research on motivation and achievement emphasises the role of expectancy beliefs, perceived competence, task value, motivation, and interest in shaping engagement and learning choices (Eccles & Wigfield, 2002). In technology-enhanced engineering education, ICT engagement and systems-thinking-oriented instruction have also been linked to students’ attitudes toward design and technology (Kurent & Avsec, 2025). In the context of AI literacy, Avsec and Rupnik (2025) further suggest that students’ profiles, agency, self-efficacy, and motivation are relevant when examining AI literacy development among technical secondary students. The present study therefore explores associations between these profile indicators and course-specific learning gains, alongside students’ perceptions of the workshop.

The workshop was designed to make relationships within an AI-enabled system observable through a guided sequence of practical activities. Students move through a sequence in which a physical input is captured and represented as data, the data are used to train or interact with a model, the model produces a prediction, program logic interprets that prediction, and a physical output responds. The sequence was intended to help students connect concepts such as data quality, classification, prediction error, confidence, and model evaluation with visible system behaviour. The contribution of the present study lies in documenting a guided sequence that connects dataset construction, model evaluation and revision, and programming physical responses, together with a preliminary evaluation of workshop-related knowledge.

This study focuses on the design and preliminary evaluation of an experiential AI and electronics workshop for secondary school students. The evaluation aimed to document implementation in three groups with varied educational backgrounds and to examine immediate changes in workshop-related knowledge as a basis for refining the workshop and planning subsequent research. The educational aim was to give students experience in developing simple AI-enabled physical systems with instructional support. The phrase “From Users to Creators” refers to students’ participation in developing these systems during the guided workshop, rather than to an independently assessed transition in design competence. Accordingly, the study addressed the following research questions:

• RQ1. What course-specific learning gains did students show after participating in the experiential AI and electronics workshop?
• RQ2. How did learning gains differ between electronics/programming concepts and AI/ML/data-related concepts?
• RQ3. Which knowledge-test items showed the largest increases in correct-response percentages, and which had the lowest correct-response percentages at post-test?
• RQ4. Were students’ learning gains associated with STEM interest, ICT self-concept, and selected STEM attitude indicators, and how did students perceive the workshop?

2. Materials and Methods

2.1. Course Design and Implementation

The educational intervention was designed as a guided, experiential workshop introducing secondary school students to artificial intelligence through programmable electronics, camera input, image data, and machine-learning-based classification. The design combined AI-literacy, technology-education, and physical-computing perspectives, with an emphasis on relationships among software, sensors, actuators, data, and observable system behaviour (Almatrafi et al., 2024; Bedoya-Cano et al., 2025; Rupnik & Avsec, 2025; Stolpe & Hallström, 2024). The workshop was developed and evaluated within a publicly funded educational initiative aimed at strengthening young people’s digital competences through practical learning about artificial intelligence and its responsible use.

The workshop comprised approximately 20 teaching periods of 45 min each (approximately 15 clock hours) across four sessions and was delivered to three groups between 2025 and 2026. Time devoted to additional project requirements was excluded from this estimate. Two implementations were delivered in secondary-school residence halls and one at a secondary school. The core instructional sequence and intended learning outcomes were comparable across groups, while students selected their final projects according to their interests. The same instructor taught all three groups using the same instructional materials and providing a comparable level of guidance and scaffolding.

The workshop followed a four-part progression from programmable electronics to the design and testing of simple AI-enabled physical systems. Figure 1 presents the progression, and Table 1 summarises the focus, activities, and AI-literacy-related emphasis of each part.

Figure 1. Four-part structure of the guided workshop, illustrating the intended progression towards developing AI-enabled physical systems.

Table 1. Overview of the workshop structure.

Workshop Part	Main Focus	Example Activities	AI-Literacy-Related Focus
1. Programmable electronics	ESP32-S3, Arduino IDE, digital and analog I/O, PWM, motors	LED control, push buttons, LDR readings, motor control, automatic barrier	Input/output, sensing, control logic, code-hardware relationships
2. Camera and connectivity	ESP32-S3 camera, OV2640 module, Wi-Fi, web interface, image capture	Camera setup, wireless connection, image capture, web-based monitoring	From physical environment to digital image data
3. Machine learning and models	Image datasets, Edge Impulse, image classification, model testing	Collecting images, defining classes, training simple models, testing predictions, interpreting errors	Data quality, model training, prediction, error, evaluation
4. AI-enabled project work	Integration of electronics, code, model outputs, and actuators	Small group projects using classification results to trigger LEDs, motors, or other responses	Designing, testing, and improving simple AI-enabled physical systems

The workshop used a guided format in which most instructional time was devoted to practical activities, supported by shorter explanations, demonstrations, and discussions. These segments introduced relevant concepts, connected students’ observations to broader ideas in AI and digital technologies and addressed aspects of responsible use. Students generally worked in pairs, enabling task sharing, peer discussion, collaborative troubleshooting, and mutual support. This structure was intended to combine learner activity and autonomy with the explicit guidance required when several unfamiliar STEM domains are integrated within the same task (Kelly & Bugallo, 2023; Sweller et al., 2019). Students were permitted to use generative AI for programming support. The instructor encouraged them to use it as a learning assistant and to understand and check its suggestions before applying them.

The workshop was designed as an experiential learning sequence rather than only as a collection of hands-on tasks. Students observed system behaviour, considered possible explanations, connected their observations to concepts such as input, output, data quality, prediction, and error, and then modified the circuit, code, dataset, or model before testing the system again. Practical activity therefore served as a starting point for interpretation, conceptualisation, and further experimentation.

The first part of the workshop introduced programmable electronics using ESP32-S3 microcontrollers (Espressif Systems, Shanghai, China) and components including LEDs, push buttons, light-dependent resistors, DC motors, and H-bridge motor drivers. Students connected hardware components, distinguished between digital inputs and outputs, read analog sensor values, and controlled actuators through the Arduino IDE (version 2.3.7; Arduino, Turin, Italy). Concepts such as software initialisation, repeated execution, conditional decision-making, timing, and digital control were introduced through their observable effects on physical components.

These foundations were extended through automation and camera-based input. Students developed systems in which sensor signals influenced physical behaviour, including an automatic light controlled by an LDR and a barrier mechanism responding to an input signal. They also configured an ESP32-S3 camera equipped with an OV2640 image sensor (OmniVision Technologies, Santa Clara, CA, USA), established wireless communication, and accessed captured images through a web interface. These activities introduced the transition from environmental conditions to sensor or image data and demonstrated that programmable systems can be inspected, modified, and explained through their component relationships.

The machine-learning component introduced image classification using Edge Impulse (Edge Impulse, San Jose, CA, USA) and the ESP32-S3 camera system. Edge Impulse was selected because its web-based interface supports an accessible workflow incorporating data collection, class definition, model training, evaluation, and deployment or API-based interaction (Banbury et al., 2023). Following an instructor demonstration, students collected image datasets, defined classification categories, trained simple models, and tested their predictions. The accompanying instruction addressed training data, class definitions, confidence scores, classification errors, data diversity, class balance, and model evaluation.

Direct on-device inference was initially explored but was not sufficiently reliable within the available hardware, time, and implementation constraints. The classroom version therefore used the Edge Impulse classification API. The ESP32-S3 captured JPEG images and transmitted them to the cloud-based classification endpoint. The returned prediction results were then processed by the program and used to activate an LED, move a motor, or trigger another physical response. Students adapted an existing camera example by configuring the pin assignments for their hardware and setting up Wi-Fi connectivity. The instructor supplied the more complex code required for communication with the Edge Impulse classification API. Students then focused on adding and adapting actuator-control commands in response to the classification results returned by the model. Although inference did not occur directly on the microcontroller, students could still examine the principal stages of the system: collecting image data, training and evaluating a model, generating predictions, interpreting model outputs in code, and connecting those outputs to physical behaviour. Similar educational implementations have used accessible online and open-source machine-learning platforms to introduce sensor-based ML workflows and engineering applications (Schulz et al., 2023).

During model testing, students encountered common limitations of image classification, including misclassification, sensitivity to lighting and background conditions, and problems associated with insufficient, unbalanced, or poorly varied training examples. These outcomes were not treated only as technical failures. They were used as opportunities to discuss data quality, model reliability, classification error, uncertainty, and the role of human evaluation. The instructor prompted students to inspect the confusion matrix to identify which classes were recognised successfully and which were frequently confused. They were then encouraged to examine individual correctly and incorrectly classified images and formulate hypotheses about possible sources of error. Discussion focused on features of the dataset, including variation in lighting, backgrounds and object orientation, as well as the representation of different classes. Students were encouraged to use these observations to revise their training datasets, for example by collecting additional, more varied images for poorly represented situations or classes. They then retrained the model in Edge Impulse and reviewed its performance again, comparing the updated confusion matrix and individual predictions with the earlier results. The purpose of this sequence was to connect data-collection decisions with observable changes in model behaviour and to make model evaluation and revision part of the learning activity.

In the final part of the workshop, students applied the acquired knowledge in small project-based tasks integrating electronics, programming, camera input, model classification, and actuators. Project work required them to plan a simple system, identify suitable inputs and outputs, test its behaviour, interpret errors, and make improvements. The workshop also included discussion of safe, responsible, and critical uses of digital technologies and AI. Instructional support prioritised helping students complete the practical tasks successfully. Final projects were used as learning activities and were not formally assessed using a project-quality rubric. Instructor-led discussions connected observations from the image-classification activities with generative AI tools familiar to students, including issues of data quality, errors, and the need to critically examine model outputs. These discussions formed part of the instruction but were not systematically recorded or assessed.

2.2. Participants and Ethical Considerations

The participants were secondary-school students who attended one of three workshop implementations between 2025 and 2026. Students were from mixed year levels, ranging from the first to the fourth year of secondary school. Group 1 consisted of female students enrolled in nursing and early childhood education programmes. Group 2 comprised male students attending a mechanical engineering-oriented secondary school. The baseline Group 3 cohort included students from computing, electrical engineering, economics, and automotive servicing programmes. Although the implementations differed in educational context and student composition, the implementation group was not treated as the main explanatory variable; the main analysis focused on individual profile indicators derived from supplementary questionnaires.

The dataset included 44 students with completed pre-tests, distributed across three implementation groups of 15, 12, and 17 students. Six students were excluded from the matched pre-test/post-test analysis because they were absent from the final assessment or had not attended the full workshop. Their pre-test data were retained in the baseline dataset. Overall, 44 valid pre-tests and 38 valid post-tests were available, yielding 38 matched pre-test/post-test pairs for the main learning-gain analysis. The matched sample comprised 23 male and 15 female students, with a mean age of 16.39 years (SD = 1.53; range = 14–19). Participation was voluntary. Before the workshops, participating students provided written informed consent for participation and the research use of their data. For minors, written consent was additionally obtained from their parents. The study was reviewed and approved by the Ethics Commission of the Faculty of Education of the University of Ljubljana (approval code: 38/2024; approval date: 10 December 2024). The distribution of participants across the three implementation groups, including the number of students with pre-test data, matched pre/post pairs, and age ranges, is presented in Table 2.

Table 2. Participant distribution by implementation group.

Group	Students with Pre-Test Data	Matched Pre/Post Pairs	Age Range in Matched Sample (Years)
1	15	14	14–15
2	12	12	18–19
3	17	12	14–18

2.3. Knowledge Test and Profile Measures

Students completed the same paper-based knowledge test in Slovenian immediately before and after the workshop. The same items, response options, and scoring criteria were used in all three implementations, with no changes to the instrument between groups. The test was developed by the first author as a course-specific, criterion-referenced achievement measure based on the workshop content and intended learning outcomes. It was reviewed by the second author, a professor with extensive experience in technology education. No separate pilot testing or independent validation was conducted before the workshop implementations. Within this preliminary evaluation, the test was intended to assess selected foundational concepts related to the workshop rather than general AI literacy or performance relative to a normative population.

The quantitatively scored section contained 16 closed-response items, with each correct answer contributing one point to a maximum score of 16. Seven items assessed electronics and programming fundamentals, including recognition of a camera-enabled microcontroller, understanding the purpose of a programming environment, distinguishing between digital inputs and outputs, recognising program instructions for controlling digital outputs, recognising conditional program logic, and understanding basic sensor behaviour. Nine items assessed AI-, machine-learning-, and data-related concepts, including training-data diversity, interpretation of classification errors, strategies for improving model performance, interpretation of feature-space visualisations and confusion matrices, changes in class-level performance following retraining, and class imbalance. The two-domain classification was defined a priori according to the principal concept assessed by each item. An English translation of the complete knowledge test, including all items, response options, original figures, and the answer key, is provided in Supplementary Material S1. The translation retains the content of the instrument administered to students. The test also included an open-ended question (Q11.3), which was not included in the quantitative scoring or statistical analyses. Table 3 documents the relationship between the assessed content and the workshop activities.

Table 3. Relationship between assessed content and workshop activities.

Assessed Content	Related Workshop Activities	Test Items
Recognition of camera-enabled hardware	Working with the ESP32-S3 camera system	Q1
Purpose of a programming environment	Using Arduino IDE to edit and upload code	Q2
Digital inputs and outputs	Connecting and using buttons and LEDs	Q3, Q4
Recognition of output-control instructions and conditional logic	Programming LED responses and conditional behaviour	Q5, Q6
Basic sensor behaviour	Observing light-sensor responses	Q7
Dataset diversity and class representation	Collecting images and considering dataset composition	Q8, Q9.2, Q12
Classification outcomes and errors	Examining predictions and confusion matrices	Q9.1, Q10.1, Q10.2
Interpretation of feature-space visualisations	Examining visual representations of image features	Q9.3
Changes in model performance following retraining	Revising datasets, retraining models, and comparing results	Q11.1, Q11.2

Item identifiers correspond directly to the question numbers and subquestion numbers in Supplementary Material S1 (e.g., Q9.1 denotes question 9.1). The test did not directly assess Wi-Fi configuration, PWM, or independent system integration.

Internal consistency of the 16-item knowledge test was assessed using Cronbach’s alpha in the matched sample (n = 38). Cronbach’s α was 0.64 at pre-test and 0.77 at post-test. Because the criterion-referenced test covered several distinct workshop learning outcomes rather than a single underlying construct, these coefficients were interpreted as supplementary evidence of reliability.

Supplementary questionnaires were used to characterise students’ profiles and perceptions of the workshop. Attitudes towards science and technology were measured using an existing questionnaire reported by Avsec and Ferk Savec (2021). The instrument provided indicators of STEM interest, negative attitudes towards science and technology, gender-stereotypical views of STEM, and the perceived importance of STEM. ICT self-concept was assessed using the digital competence self-concept instrument reported by Kurent and Avsec (2023), which measures students’ perceived competence in using, evaluating, creating, protecting, and troubleshooting digital technologies. Participant satisfaction was assessed using a short questionnaire developed for the present study. It included items addressing interest, fit with prior knowledge and experience, organisation, clarity of activities and instructions, instructor support, and perceived practical applicability.

Based on the scale structures reported in the source instruments and internal consistency analyses conducted in the present sample, STEM interest was calculated as the mean of eight science-and-technology interest items (Cronbach’s α = 0.90). Three additional exploratory attitude indicators were calculated for interpretive purposes: negative STEM attitudes (four items, α = 0.85), gender stereotypes regarding STEM (four items, α = 0.83), and perceived importance of STEM (four items, α = 0.75). ICT self-concept was calculated as the mean of 25 items scored on a six-point Likert scale (α = 0.95). Overall satisfaction was calculated as the mean of six items scored on a five-point Likert scale (α = 0.81). For the STEM and ICT measures, internal consistency was calculated using the baseline sample (n = 44); the satisfaction coefficient was calculated using the 36 available questionnaires.

2.4. Data Preparation and Analysis

The dataset was checked for incomplete entries, duplicates, incorrectly entered responses, and mismatches between pre-test and post-test responses. Closed-response answers were coded numerically, with A, B, C, and D corresponding to 1, 2, 3, and 4. Each item was scored as correct or incorrect according to the answer key. Total scores were calculated as the sum of correct responses across the 16 closed-response items. Learning-gain analyses were restricted to the 38 students with usable pre-test and post-test data. All 16 scored items were completed at both time points in this matched sample, and no knowledge-test responses were imputed. Absolute learning gain was calculated as the post-test score minus the pre-test score. Individual normalised gain was calculated relative to the maximum improvement possible from each student’s pre-test score:

$$g={\displaystyle \frac{\mathit{post}-\mathit{test}-\mathit{pre}-\mathit{test}}{\mathit{maximumscore}-\mathit{pre}-\mathit{test}}},$$

where the maximum possible score was 16. Class-level normalised gain was calculated from the group mean scores as (mean post-test score minus mean pre-test score) divided by (16 minus mean pre-test score). For domain-level analyses, the maximum scores were seven for electronics/programming and nine for AI/ML/data. Domain gains were compared within students using percentage-point gains and normalised gains. Normalised gain was treated as undefined when the pre-test score equalled the domain maximum; such cases were excluded only from analyses requiring that value. One student scored 9/9 on the AI/ML/data pre-test, leaving 37 paired observations for the comparison of normalised domain gains. Descriptive statistics were calculated for pre-test scores, post-test scores, learning gains, normalised gains, profile indicators, and satisfaction. Student-profile scale scores were calculated as the mean of the available item responses. Satisfaction analyses used the 36 available questionnaires. Correlations were calculated using cases with available values for both variables in each analysis. A paired-samples t-test assessed whether post-test scores differed from pre-test scores, and the Wilcoxon signed-rank test was used as a non-parametric robustness check. Cohen’s dz was calculated as the mean within-student score change divided by the standard deviation of the score differences. Domain-level and item-level analyses were conducted, and Spearman correlations were calculated between learning-gain measures and STEM interest, ICT self-concept, additional STEM attitude indicators, and satisfaction. All data preparation and statistical analyses were conducted in R version 4.6.1 using RStudio 2026.06.0+242. Implementation group and sex were inspected descriptively and through exploratory non-parametric tests, but they were not treated as primary explanatory variables because group composition and educational context were partly confounded. Correlational and subgroup analyses were exploratory. No adjustment for multiple comparisons was applied, and individual p-values from these analyses were not interpreted as confirmatory evidence.

2.5. Methodological Notes and Limitations

The study was conducted in authentic educational contexts, and several methodological considerations should therefore be noted. A one-group pre-test/post-test design was used without a control group, the sample size was relatively small, and the implementations varied in context and student composition. The observed score changes describe performance before and after participation in the workshop. The design does not allow the contribution of the workshop to be separated from other possible influences on test performance. The workshop was originally designed primarily for educational implementation rather than as a controlled research intervention. Consequently, the knowledge test was closely aligned with the workshop content and practical activities and should be interpreted as a course-specific measure rather than as a standardised AI-literacy instrument. The test had not undergone independent validation in the target population. Its scores therefore provide preliminary evidence of selected foundational knowledge related to the workshop, without directly assessing ethical reasoning or students’ ability to independently design an AI-enabled system. Using identical items immediately before and after the workshop may have contributed to score improvement through repeated exposure. The contribution of this testing effect cannot be separated from learning during the workshop in the present design.

The profile and satisfaction measures were self-report indicators, and the STEM attitude subscales were used to support interpretation rather than to diagnose students. Students selected final projects according to their interests, introducing some variation in the practical learning activities. Differences in age, educational background, and implementation context were partly confounded, limiting the interpretation of subgroup comparisons. Exclusion of students with incomplete attendance or unavailable post-test data may also have introduced selection bias in the matched sample. Consequently, the study does not provide a formal assessment of final project quality or students’ independent design competence. For these reasons, the results should be interpreted as a preliminary evaluation of selected workshop-related learning outcomes. Future studies should assess broader AI literacy and the retention and transfer of workshop-related knowledge using appropriate measures and delayed assessments.

3. Results

3.1. Overall Learning Gains

The maximum score for the closed-response section of the knowledge test was 16 points. In the matched sample (n = 38), the mean knowledge-test score increased from 7.21 to 11.55 out of 16, corresponding to a mean observed gain of 4.34 points. Post-test scores were higher for 35 students, unchanged for two, and lower for one. The sample size and overall learning-gain results are summarised in Table 4.

Table 4. Sample and overall learning-gain results.

Measure	Result
Valid pre-tests	44
Usable post-tests	38
Matched pairs	38
Pre-test mean (SD)	7.21/16 (3.02)
Post-test mean (SD)	11.55/16 (3.33)
Mean gain (points)	+4.34
Gain in percentage points	+27.1
95% CI for gain (points)	+3.29 to +5.40

Pre-test and post-test score statistics and gain estimates refer to the 38 matched pairs; the count of 44 pre-tests refers to the full baseline sample.

Post-test scores were significantly higher than pre-test scores in the matched sample, with a large standardised within-student score change (Cohen’s dz = 1.35; Table 5). This value describes change on the workshop-aligned knowledge test; it does not quantify a causal effect of the instructional approach or improvement in general AI literacy.

Table 5. Statistical tests for overall learning gain.

Analysis	Result
Paired-samples t-test	t(37) = 8.35
t-test significance	p < 0.001
Wilcoxon signed-rank test	V = 653.5
Wilcoxon significance	p < 0.001
Cohen’s dz	1.35
Mean individual normalised gain	0.50
Class-level normalised gain	0.49

3.2. Learning Gains by Knowledge Domain

The 16 closed-response items were grouped a priori into two domains: electronics/programming fundamentals (seven items) and AI, machine-learning, and data-related concepts (nine items). Mean test scores were significantly higher at post-test in both knowledge domains (Table 6). The mean percentage-point gain was 34.2 in electronics/programming and 21.6 in AI/ML/data. The within-student comparison showed a larger percentage-point gain in electronics/programming, t(37) = 2.64, unadjusted p = 0.012. The difference in normalised gains did not reach the 0.05 significance threshold, t(36) = 1.97, unadjusted p = 0.057. The normalised-gain comparison included 37 students because one student had already achieved the maximum AI/ML/data score at pre-test, making that domain’s normalised gain undefined.

Table 6. Pre-test and post-test performance by knowledge domain.

Knowledge Domain	Items	Pre (%)	Post (%)	Gain (pp)	p	dz
Electronics/programming fundamentals	7	35.3%	69.5%	+34.2 pp	<0.001	1.37
AI/ML/data-related concepts	9	52.6%	74.3%	+21.6 pp	<0.001	0.88

Pre and Post represent mean percentages of correct responses within each domain. The p-values and Cohen’s dz refer to paired pre-test/post-test comparisons within each domain (n = 38). Gains were calculated from unrounded values.

3.3. Item-Level Results

Item-level performance was examined descriptively to identify patterns across the full set of assessed concepts. Table 7 presents the percentage of correct responses before and after the workshop, together with the corresponding percentage-point gains for all 16 quantitatively scored items.

Table 7. Item-level pre-test and post-test performance.

Item	Domain	Assessed Concept	Pre-Test (%)	Post-Test (%)	Gain (pp)
Q1	Electronics/programming	Recognition of camera-enabled hardware	34.2	92.1	+57.9
Q2	Electronics/programming	Purpose of a programming environment	63.2	84.2	+21.1
Q3	Electronics/programming	Selection of a digital input for a button	26.3	50.0	+23.7
Q4	Electronics/programming	Selection of a digital output for an LED	26.3	55.3	+28.9
Q5	Electronics/programming	Recognition of a digital-output control instruction	26.3	84.2	+57.9
Q6	Electronics/programming	Recognition of conditional program logic	36.8	68.4	+31.6
Q7	Electronics/programming	Basic sensor behaviour	34.2	52.6	+18.4
Q8	AI/ML/data	Importance of diversity in image-training data	52.6	63.2	+10.5
Q9.1	AI/ML/data	Interpretation of incorrectly classified examples	50.0	71.1	+21.1
Q9.2	AI/ML/data	Dataset-improvement strategy involving diverse training examples	28.9	73.7	+44.7
Q9.3	AI/ML/data	Interpretation of feature-space visualisations	50.0	73.7	+23.7
Q10.1	AI/ML/data	Interpretation of misclassification in a confusion matrix	60.5	78.9	+18.4
Q10.2	AI/ML/data	Reading a correct-classification rate from a confusion matrix	34.2	68.4	+34.2
Q11.1	AI/ML/data	Identifying improvement following model retraining	76.3	76.3	+0.0
Q11.2	AI/ML/data	Identifying deterioration following model retraining	60.5	84.2	+23.7
Q12	AI/ML/data	Class imbalance	60.5	78.9	+18.4

Percentages are based on the 38 matched pairs. Gains were calculated from unrounded percentages.

The percentage of correct responses increased on 15 of the 16 items. For Q11.1, which required identifying the class with the greatest improvement following retraining, the percentage remained unchanged at 76.3%. The largest percentage-point increases were observed for recognising camera-enabled hardware (Q1; +57.9 pp), recognising a digital-output control instruction (Q5; +57.9 pp), and selecting a dataset-improvement strategy involving more diverse training examples (Q9.2; +44.7 pp). Correct-response percentages also increased for recognising conditional program logic (Q6; +31.6 pp) and reading a class-specific correct-classification rate from a confusion matrix (Q10.2; +34.2 pp).

The lowest post-test correct-response percentages were observed for selecting a digital input for a button (Q3; 50.0%), understanding basic sensor behaviour (Q7; 52.6%), and selecting a digital output for an LED (Q4; 55.3%). Within the AI/ML/data domain, Q8, concerning diversity in training images, had the lowest post-test correct-response percentage (63.2%). These descriptive item-level patterns do not establish a stable ranking of concept difficulty.

3.4. Student Profiles and Learning Gains

STEM interest and ICT self-concept were analysed as the main student-profile indicators. Learning gains were not significantly associated with either STEM interest or ICT self-concept (Table 8). Pre-test scores were positively associated with post-test scores and negatively associated with absolute gain. Their association with normalised gain was not statistically significant. Because of the small sample size and the exploratory nature of these correlations, the results should be interpreted descriptively.

Table 8. Correlations between main profile indicators and learning outcomes.

Indicator	Outcome	n	ρ	p
STEM interest	Absolute gain	38	0.049	0.769
STEM interest	Normalised gain	38	0.193	0.246
ICT self-concept	Absolute gain	38	−0.010	0.953
ICT self-concept	Normalised gain	38	0.164	0.325
Pre-test score	Post-test score	38	0.502	0.001
Pre-test score	Absolute gain	38	−0.418	0.009
Pre-test score	Normalised gain	38	0.065	0.699

All correlations are based on the 38 matched pairs; p-values are unadjusted.

Additional STEM attitude indicators were inspected exploratorily (Table 9). Negative STEM attitude scores were negatively correlated with pre-test and post-test scores, whereas their correlations with absolute and normalised gain were not statistically significant. Gender stereotypes and perceived STEM importance showed weak-to-moderate, non-significant associations with gain measures.

Table 9. Exploratory correlations involving additional STEM attitude indicators.

Indicator	Outcome	n	Spearman’s ρ	p
Negative STEM attitude	Pre-test	38	−0.423	0.008
Negative STEM attitude	Post-test	38	−0.372	0.021
Negative STEM attitude	Absolute gain	38	0.076	0.652
Negative STEM attitude	Normalised gain	38	−0.180	0.278
Gender stereotypes	Absolute gain	38	0.311	0.057
Gender stereotypes	Normalised gain	38	0.290	0.078
STEM importance	Absolute gain	38	0.124	0.456
STEM importance	Normalised gain	38	0.266	0.106

These correlations are exploratory; p-values are unadjusted for multiple comparisons.

3.5. Descriptive Results by Implementation Group

Results for each implementation group are reported descriptively to show variation in baseline scores and observed pre-test/post-test changes (Table 10). Mean pre-test and post-test scores varied across groups. Mean absolute gains ranged from 3.86 to 5.00 points, whereas mean individual normalised gains ranged from 0.34 to 0.61. These group-level differences should be interpreted cautiously because age, sex, educational background, prior knowledge, and setting varied together across implementations.

Table 10. Pre-test and post-test scores and learning gains across the three implementation groups.

Implementation Group	n	Pre → Post (Points)	Mean Absolute Gain (Points)	Mean Individual Normalised Gain
Group 1	14	5.29 -> 9.14	+3.86	0.34
Group 2	12	7.25 -> 12.25	+5.00	0.58
Group 3	12	9.42 -> 13.67	+4.25	0.61

Values refer to students with matched pre-test/post-test data in each group (n = 14, 12, and 12, respectively).

3.6. Student Perceptions of the Workshop

Student satisfaction ratings were treated as self-reported perceptions of the workshop rather than as evidence of learning gains. Mean overall satisfaction was 4.19 (SD = 0.57) on a five-point scale (n = 36). Instructor support (M = 4.69) and clarity of activities and instructions (M = 4.61) received the highest mean ratings (Table 11). The lowest mean rating concerned perceived practical applicability (M = 3.61, SD = 1.13).

Table 11. Student satisfaction and perceived usefulness.

Item	Focus	n	M (SD)
UP1	Interest	36	4.00 (0.79)
UP2	Fit to prior knowledge/experience	36	4.03 (0.81)
UP3	Organisation	36	4.22 (0.80)
UP4	Clarity of activities/instructions	36	4.61 (0.55)
UP5	Instructor support	36	4.69 (0.58)
UP6	Perceived practical applicability	36	3.61 (1.13)
Overall	Mean of six satisfaction items	36	4.19 (0.57)

Ratings were recorded on a five-point scale; the overall score is the mean of the six items.

Satisfaction was positively correlated with STEM interest (ρ = 0.568, p < 0.001). Its correlations with absolute gain (ρ = −0.038, p = 0.825) and normalised gain (ρ = 0.150, p = 0.384) were not statistically significant. These exploratory analyses included 36 students; p-values were unadjusted.

4. Discussion

This study documented the implementation of a guided AI and electronics workshop and examined immediate changes in workshop-related knowledge and their exploratory associations with selected student-profile indicators. A large standardised increase in workshop-related knowledge-test scores was observed, with higher post-test scores in both electronics/programming and AI/ML/data. The magnitude of this change should be interpreted in the context of approximately 15 clock hours of focused instruction, a workshop-aligned test, and repeated administration of identical items immediately before and after the workshop. It therefore represents an immediate change in performance on the selected course-specific items rather than an equivalent increase in broader AI literacy. The present study contributes a documented instructional sequence linking programmable electronics, image data, guided model evaluation and revision, and actuator responses, together with a preliminary evaluation of workshop-related knowledge gains.

A central design feature was that students could examine AI within an observable physical and technical system. By working with sensors, image data, model training, prediction results, program logic, and physical outputs, students could trace how environmental inputs became data, how models generated predictions, and how those predictions were translated into actuator responses. This visible functional chain may have helped connect abstract AI and programming concepts with directly observable system behaviour. This emphasis can be interpreted through cognitive apprenticeship, which makes problem-solving strategies accessible through modelling, coaching, scaffolding, articulation, and reflection (Collins et al., 1991). Instructor demonstrations and guided examination of classification errors align with selected elements of this framework. Encouraging students to explain possible sources of error, revise their datasets, and inspect the results of retraining created opportunities to articulate and reconsider their reasoning. Whether these processes contributed to the observed score gains remains a hypothesis for future research. These design aims are also consistent with AI-literacy frameworks that emphasise understanding, evaluation, creation, and ethical participation (Almatrafi et al., 2024; Rupnik & Avsec, 2025), with technology-education perspectives that connect AI literacy with data, hardware, software, and systems thinking (Stolpe & Hallström, 2024), and with K-12 machine-learning approaches that seek to make selected stages of the ML process visible to learners rather than leaving them entirely within a black box (Morales-Navarro & Kafai, 2024).

The three implementations provide preliminary evidence that the workshop is practically feasible, with instructional support, for a heterogeneous secondary-school sample. The diversity of participants’ educational backgrounds, STEM interests, and ICT self-concepts supports continued development and more systematic evaluation across learner profiles.

In this sample, no statistically significant correlations were detected between knowledge gains and STEM interest or ICT self-concept. These exploratory correlations do not establish that learning gains were independent of student profiles or equal across profiles. Higher STEM interest was associated with more positive satisfaction ratings, but its correlations with knowledge gains were not statistically significant. This distinction is pedagogically important because subjective appeal and measured learning were not equivalent outcomes.

Initial knowledge was positively associated with post-test achievement, which is unsurprising because students who entered the workshop with stronger prior knowledge also tended to finish with higher scores. Pre-test scores were negatively associated with absolute gain but not statistically significantly associated with normalised gain. Since absolute gain is calculated by subtracting the pre-test score from the post-test score, the negative association alone does not demonstrate that the workshop was more effective for students with lower initial knowledge. More negative STEM attitude scores were associated with lower pre-test and post-test scores, while their correlations with gain measures were not statistically significant. Given the small sample and unadjusted multiple comparisons, these findings provide exploratory questions for follow-up rather than confirmatory evidence of profile-related learning differences.

The domain-level findings provide a further insight. Gains were significant in both domains, but observed percentage-point gains were larger in electronics/programming than in the AI/ML/data domain. Several explanations for this pattern are plausible, but the present design cannot distinguish between them. Lower baseline scores in electronics/programming left more room for improvement in that domain. The physical activities offered opportunities to observe code–hardware relationships; their contribution to the difference between domains was not isolated. The difference between domains should therefore be interpreted cautiously, as the two domains differed in baseline performance, number of items, and conceptual demands. The difference in normalised gains was not statistically significant.

Post-test correct-response percentages were comparatively lower on items addressing digital inputs and outputs and training-image diversity. These patterns identify content to revisit in future implementations, while allowing for the influence of item wording and the limited coverage of each concept. Visualisation-based items assess interpretations of particular representations rather than broader competence in evaluating machine-learning models. Students had to coordinate multiple interacting elements, including electronics, programming, sensor signals, image data, model behaviour, and physical outputs. Cognitive load theory suggests that learning becomes more difficult when learners must process many novel and interdependent elements simultaneously (Sweller et al., 2019). Similar challenges have been reported in integrated STEM and engineering contexts, where students may struggle to transfer concepts across representations, components, and physical systems (Kelly & Bugallo, 2023).

The workshop sought to support learner autonomy by using Edge Impulse for model development and keeping actuator-control code within the scope of introductory programming. More demanding API communication code was supplied by the instructor.

The priority for future iterations is to simplify technically demanding integration so that students can assume greater responsibility for constructing and revising their systems. This should preserve opportunities to inspect and explain the relationship between data, model predictions, and actuator responses. A future iteration could explore on-device inference using an exported Edge Impulse library on hardware with sufficient memory and processing capacity for the selected image model (Edge Impulse, n.d.). This could remove the cloud API communication step, while camera handling and actuator integration would still need to be made accessible to students. Future iterations could place greater emphasis on actuator control in the introductory electronics and programming component, while reducing coverage of additional sensors not required for the camera-based project. With the camera serving as the principal input, students could focus on interpreting model predictions and programming the corresponding physical responses.

Student satisfaction results provide a complementary perspective. Students evaluated the workshop positively overall, with particularly high ratings for the clarity of activities and instructor support. The comparatively lower rating for practical applicability identifies an aspect of students’ perceptions that warrants further exploration. It does not establish whether they could transfer the acquired knowledge to unfamiliar tasks. Future implementations should make links to schoolwork, personal projects, and everyday technological systems more explicit. This does not necessarily require additional content; rather, it requires clearer examples of how the same principles of sensing, data collection, prediction, decision-making, and system response can be applied in other contexts.

The implementation experience and preliminary assessment suggest practical considerations for educators adapting similar workshops. First, learners should be able to trace the functional chain from environmental input and data collection to model prediction, program decision, and physical response. Second, practical activities should be followed by structured interpretation, reflection, and iterative revision rather than ending when a functioning system is produced. Third, reusable examples and prepared communication code should make the technical integration manageable, while students retain responsibility for selecting training data, examining errors, and programming physical responses. Finally, learners should be shown how the same principles can be transferred to new projects and familiar technological contexts. These considerations provide a basis for adapting the workshop, while their effectiveness and transferability require further evaluation. The workshop discussions also connected observations about classification errors and data quality with students’ experiences of generative AI. This provided an instructional link to familiar AI tools, although transfer of understanding between these contexts was not directly assessed.

The results should be interpreted in light of several limitations. The study used a one-group pre-test/post-test design without a comparison or control group, and the observed gains therefore cannot be attributed exclusively to the workshop. Repeated exposure to identical test items may also have contributed to higher post-test scores. The sample was relatively small, and the workshop was implemented in authentic educational settings that differed in student composition and context. The knowledge test was course-specific and measured selected workshop-related outcomes rather than general AI literacy. The instrument was developed and reviewed within the author team; evidence supporting its validity in the target population remains limited. The student-profile and satisfaction measures were used mainly as contextual indicators, and the exploratory correlational findings should be interpreted cautiously. In addition, the immediate post-test does not indicate whether the observed learning gains were retained over time or transferred to unfamiliar tasks.

Overall, the three implementations provide preliminary evidence of the practical feasibility of this guided workshop in a heterogeneous secondary-school sample, alongside immediate gains on workshop-related knowledge items. Its principal contribution lies in integrating electronics, image data, machine-learning classification, programming, and physical system responses within a coherent learning sequence. The findings suggest that such systems may help make selected AI processes visible and open to learner investigation. The assessment shows that higher overall scores can coexist with uneven item-level performance. A priority is to simplify the technically demanding parts of the workflow so that students can take greater responsibility for system development while retaining opportunities to explain how its components interact. Subsequent research should assess broader AI literacy, retention, and transfer using measures with evidence supporting their use with secondary-school students and delayed tasks in unfamiliar contexts. Project assessment should record the quality of students’ design decisions and the amount of support required, allowing practical participation to be distinguished from independent competence. Larger samples and, where the research question concerns causal effects, comparison conditions would strengthen the evaluation. Transfer to generative AI could be examined through tasks requiring students to identify errors, verify outputs, and justify revisions to AI-generated code or explanations.

5. Conclusions

This study evaluated a pilot experiential workshop designed to enable secondary-school students to develop simple AI-enabled physical systems with instructional support. The workshop was associated with a large pre-test to post-test increase in course-specific knowledge-test scores in both electronics/programming and AI/ML/data. Particularly strong gains were observed in recognising camera-enabled hardware, output-control instructions and conditional logic, and selecting a dataset-improvement strategy involving diverse training examples. Lower correct-response percentages on selected input/output and training-data items identify priorities for further instructional and assessment refinement.

The workshop was delivered, with instructional support, across three groups of students with varied educational backgrounds and profiles. This provides preliminary evidence of practical feasibility in the implemented settings; the exploratory correlations do not establish equivalent learning gains across profiles. Students with greater STEM interest evaluated the workshop more positively. Positive satisfaction ratings provide additional evidence of students’ perceived acceptability of the workshop.

Overall, the findings support continued development and more systematic evaluation of the workshop. The workshop provides an empirically informed starting point for refining the learning sequence, support strategies, and technical integration. A practical priority is to simplify the workflow and focus introductory programming on camera input, conditional logic, and actuator control, creating more opportunities for students to develop and revise their systems. Future research should assess broader AI literacy, independent design competence, retention, and transfer using appropriate measures, project assessments documenting the support required, and delayed tasks. Larger samples and comparison conditions suited to the research questions would strengthen this next stage of evaluation.