From Guided Hardware–Software Experiments to an Autonomous Robot Competition: A Six-Year Repeated Cross-Sectional Mixed-Methods Analysis of Student Evaluations

Abstract

Introductory engineering courses must balance guidance with opportunities for independent design. This retrospective repeated-cross-sectional mixed-methods study examined perceptions of a two-semester course progressing from guided Arduino-based activities to an autonomous robot competition. It integrated 87 group-level evaluation records from 2020–2025 with 415 archived comment units and their thematic assignments. The common-period semester comparison (2021–2023) did not reach statistical significance (p = 0.126; rank-biserial correlation = 0.24). A secondary full-period comparison favored Semester 1 (p = 0.010), but year coverage and overall-item wording differed. Scores varied across years, H(5) = 15.18, p = 0.010, although no Holm-adjusted pairwise contrast was significant. Feedback addressed practical activity, instructor support, workload, and organization. Only lecture–laboratory alignment differed between semesters after Holm adjustment, including in the common-period analysis. Original-score comparisons by theme presence detected no Holm-significant differences; their confidence intervals allowed effects in either direction. A conflicting source score was examined by substitution and exclusion. The findings describe perceptions and recorded feedback, subject to changing measures, aggregated records, and unestablished intercoder reliability. They identify course-design priorities but do not demonstrate learning gains, development of autonomy, or an independent competition effect.

1. Introduction

Engineering education is expected to prepare students not only to understand scientific and mathematical principles but also to apply them in designing, constructing, testing, and improving technological systems. Engineering design is an iterative process that involves generating possible solutions, evaluating alternatives, integrating disciplinary knowledge, managing constraints, and communicating decisions (Dym et al., 2005). Introducing these practices during the first year provides opportunities for students to apply foundational scientific and technical knowledge to design problems while gaining experience in prototyping, teamwork, and engineering decision-making (Salyards et al., 2022; Klein et al., 2026). However, first-year students often enter engineering programs with different levels of prior experience in programming, electronics, mechanics, and collaborative project work. Consequently, introductory engineering courses must balance accessibility and instructional support with opportunities for authentic design, experimentation, and problem-solving.

Active learning approaches, including project-based learning (PjBL), are widely used in engineering education to connect foundational knowledge with design practice and to promote integration of technical knowledge, teamwork, and problem solving (Freeman et al., 2014; Prince & Felder, 2006). Project-based engineering activities, however, also create instructional challenges for novice students. As projects become more complex and open-ended, students must coordinate multiple technical domains while assuming greater responsibility for design decisions and problem solving. Introductory courses must therefore balance sufficient instructional support with opportunities for increasing independence and authentic engineering practice. This tension between guidance and autonomy is particularly relevant in courses designed as a progression from structured activities to increasingly autonomous projects.

Educational robotics offers a particularly suitable environment for this form of progression. A robot is a tangible system through which students can observe the relationship between programmed instructions and physical behavior. Robotics activities can integrate programming, electronics, control, mechanics, mathematics, and engineering design while providing immediate feedback through the robot’s performance (Alimisis, 2013; Benitti, 2012). Robotics projects can also develop teamwork and communication because students must distribute responsibilities, coordinate subsystems, explain technical decisions, and collectively troubleshoot system failures (Taylor & Baek, 2018). Although Arduino platforms make physical computing accessible, integrated projects still require students to coordinate programming, electronics, sensing, actuation, and mechanical design, making instructional guidance and technical support important.

Competition can add a further dimension to project-based robotics education. A competition supplies a concrete performance criterion and an authentic deadline, potentially increasing motivation, persistence, and team identification. It also requires students to move beyond demonstrating isolated technical functions toward producing a reliable, integrated system. At the same time, competitive projects may introduce pressure, unequal participation, workload concerns, and an excessive focus on performance outcomes. Robotics competitions should therefore be considered not merely as motivational events but as learning environments whose educational effects depend on project requirements, assessment practices, team organization, and instructional support.

Although PjBL and educational robotics have received considerable research attention, much of the literature examines individual interventions, projects, or competitions rather than sustained instructional progressions across successive cohorts (Anwar et al., 2019; Benitti, 2012; Spolaôr & Benitti, 2017). Reviews of PjBL in higher education have reported generally favorable student outcomes but have also identified substantial variation in implementation and an emphasis on self-reported perceptions rather than learning processes and project products (Guo et al., 2020). Within engineering education, project-based approaches are particularly valued for connecting disciplinary knowledge with design, collaboration, and authentic problem solving, although their effectiveness depends strongly on appropriate structure and facilitation (Mills & Treagust, 2003; C.-H. Chen & Yang, 2019). Less is known about how students perceive a sustained transition from guided hardware–software experimentation to increasingly autonomous and integrative engineering project work. Examining this progression across successive cohorts can distinguish recurring course-design issues from patterns associated with particular years or semesters.

The course examined here uses a two-semester progression from guided hardware–software experimentation and a bounded line-following robot project in Semester 1 to a more autonomous, integrative robot project culminating in a competition in Semester 2. This structure provides a natural context for examining students’ perceptions as instructional guidance decreases and design autonomy and system complexity increase.

Student evaluations can provide useful evidence for this purpose, but they must be interpreted cautiously. Numerical teaching-evaluation scores summarize students’ perceptions efficiently, yet they cannot fully explain why students viewed a course positively or negatively. Ratings may also be influenced by workload, expected grades, class characteristics, course requirements, and contextual factors beyond instructional quality (Spooren et al., 2013; Uttl et al., 2017). Consequently, evaluation scores should not be treated as direct or comprehensive measurements of learning.

Open-ended comments provide a complementary source of evidence because students can identify specific aspects of course organization, workload, practical activities, instructional clarity, feedback, project support, and instructor responsiveness. Written comments may reveal concerns or valued experiences that are obscured when responses are reduced to a single score (Nasser-Abu Alhija & Fresko, 2009; Brockx et al., 2012). They are therefore particularly useful for identifying which aspects of the course experience underlie broader quantitative evaluation patterns.

Integrating quantitative evaluation scores with qualitative comments is therefore particularly appropriate for examining a complex course experience. Mixed-methods analysis can determine whether changes in numerical ratings correspond with changes in students’ written descriptions and can clarify why similar scores may reflect different experiences (Creswell & Plano Clark, 2018; Fetters et al., 2013). In the present context, this integration can help identify how students perceive the progression from structured experiments to increasingly autonomous project work and whether course- and instructor-related themes vary across semesters and academic years.

The present study addresses the intersection of first-year engineering education, educational robotics, PjBL, and repeated cross-sectional student evaluation. Its contribution lies in integrating numerical ratings with course- and instructor-related feedback from a course organized as a two-semester progression. The observation window spans six academic years, but both semesters are represented only in 2021–2023. The analysis identifies recurring perceived strengths and implementation concerns relevant to scaffolding, workload, organization, resources, and project support. It does not assess individual development across the progression.

Research Objective and Questions

The objective was to describe and compare students’ reported perceptions of a two-semester introductory engineering course using aggregated evaluation records and written feedback. The analysis examined variation across available academic years and semesters and the co-occurrence of feedback themes with overall group-level evaluation scores. Learning gains, engineering competence, learner autonomy, and design skills were not directly measured. The semester comparison concerns a bundle of instructional differences, including project scope, technical complexity, opportunities for independent decisions, teamwork demands, and the competition; it cannot isolate the effect of any one component.

The study addressed the following research questions:

RQ1. How did overall teaching-evaluation scores and comparable evaluation dimensions vary across academic years and between the two semesters?

RQ2. What course-related and instructor-related strengths and areas for improvement were recorded in the archived coding of students’ open-ended feedback?

RQ3. How did the prevalence of the principal feedback themes differ between Semester 1 and Semester 2?

RQ4. How did the prevalence of the principal qualitative themes vary among the available academic-year cohorts?

RQ5. How did original overall evaluation scores differ between records with and without selected feedback themes?

To ground the analysis of these research questions, Section 2 presents the theoretical framework guiding this study. This framework connects experiential and project-based learning with cognitive apprenticeship, scaffolding, and authentic engineering practice to interpret students’ perceptions of the progression from guided activities to increasingly autonomous engineering project work.

2. Theoretical Framework

2.1. Experiential and Project-Based Learning

Experiential and project-based learning connect conceptual knowledge with practical application and design activity. Project-based learning organizes instruction around extended, meaningful tasks (Krajcik & Blumenfeld, 2006), whereas experiential learning emphasizes cycles of experience, reflection, conceptualization, and experimentation (D. A. Kolb, 1984). These perspectives concern possible learning processes; they do not make favorable course evaluations a direct measure of learning.

In engineering education, PjBL has been associated with the integration of technical knowledge and professional competencies, including design thinking, teamwork, communication, problem solving, and self-directed learning (J. Chen et al., 2021). A recent systematic review of engineering-education research further characterizes PjBL as an experiential pedagogical methodology that can support the development of real-world engineering skills by engaging students with multidisciplinary, technological, laboratory, simulation, and professional contexts (Lavado-Anguera et al., 2024). Rather than viewing PjBL and experiential learning as separate approaches, they can therefore be considered complementary: PjBL describes the instructional structure through which students engage with extended tasks, whereas experiential learning helps explain the learning processes that occur through engagement with those tasks.

Recent research has also examined design experiences specifically in first-year engineering education. Bieryla et al. (2024) studied the timing of a design sprint in relation to engineering identity and entrepreneurial mindset. Such studies illustrate the value of measuring educational constructs directly, rather than inferring them from overall course ratings.

Experiential learning theory provides a useful framework for understanding how knowledge develops through engagement with practical activity. D. A. Kolb (1984) conceptualized learning as a cyclical process involving concrete experience, reflective observation, abstract conceptualization, and active experimentation. Rather than treating experience alone as sufficient for learning, the model emphasizes the transformation of experience through reflection and subsequent application of emerging ideas. A. Y. Kolb and Kolb (2005) further emphasized experiential learning as a process in which learners move between experiencing, reflecting, thinking, and acting within supportive learning environments. This perspective is particularly relevant to engineering education, where students repeatedly move between designing, implementing, testing, interpreting outcomes, and modifying their solutions. Contemporary reviews of engineering PjBL similarly emphasize the value of learning environments that connect project activity with authentic or real-world engineering practice and the development of both technical and transferable competencies (Lavado-Anguera et al., 2024; Ramírez de Dampierre et al., 2024).

Educational robotics provides a concrete setting for experiential learning because students can formulate a solution, implement it, observe its behavior, identify errors, and revise both hardware and software (Alimisis, 2013; Benitti, 2012). In undergraduate engineering education, project-based robotics activities have also been used to engage students in integrating programming, design, and practical problem solving (Bada et al., 2013). These iterative cycles of action, feedback, reflection, and modification are consistent with experiential approaches to learning and closely resemble iterative engineering design and experimentation. Within project-based settings, such cycles allow students to integrate conceptual knowledge with practical decision making rather than encountering theory and application as separate activities. In the course examined in this study, this experiential process occurs first within relatively structured hardware–software activities and later within a more autonomous and integrative engineering project.

2.2. Scaffolding, Fading, and Learner Autonomy

The benefits of project-based and experiential learning depend not only on the authenticity and complexity of the task but also on the level and timing of instructional support. This issue is particularly important for novice learners, who may be required to coordinate several unfamiliar domains while simultaneously making design decisions and solving technical problems. In hardware–software projects, for example, students may need to integrate programming logic, electronic connections, sensor behavior, actuator control, mechanical construction, troubleshooting, and system-level reasoning. Problem-based and inquiry-oriented learning should therefore not be understood as minimally guided learning; rather, effective implementations typically incorporate substantial scaffolding to support learners as they engage with complex tasks (Hmelo-Silver et al., 2007).

The theoretical basis for such support is closely related to Vygotsky’s (1978) concept of the zone of proximal development, which distinguishes between what learners can accomplish independently and what they can accomplish with appropriate assistance. Building on this idea, Wood et al. (1976) introduced the concept of scaffolding to describe temporary support that enables learners to perform aspects of a task that would initially exceed their independent capabilities. In software-supported and complex learning environments, scaffolding operates both by structuring tasks to reduce unnecessary complexity and by problematizing key concepts to direct student attention toward core disciplinary practices (Quintana et al., 2004; Reiser, 2004). Scaffolding is therefore intended to support participation while competence is developing rather than to replace learners’ own cognitive and practical activity.

During this transition, the instructor’s role changes from demonstrating procedures and structuring task execution toward coaching students’ reasoning and monitoring when assistance is needed. As students assume greater responsibility, instructors can use questions, feedback, and prompts to help them articulate design decisions, diagnose difficulties, and evaluate alternatives while leaving substantive decisions to the students (Collins et al., 1989, 1991). Fading therefore involves adapting the form and amount of support to learners’ developing capabilities and task demands, rather than withdrawing assistance according to a fixed schedule. Support may remain available or be temporarily increased when students encounter unfamiliar integration problems (Quintana et al., 2004; Reiser, 2004). From this perspective, instructional clarity and responsiveness remain relevant as autonomy increases, but assistance should enable students to advance their own reasoning rather than take over the design process. Emberley and Moore (2023) provide an empirical illustration of this role in a multiple-case study of three instructors teaching introductory engineering design. Instructors used dialogue to elicit students’ ideas, assess their progress, and provide prompts adapted to their immediate needs. However, support for problem scoping, testing, troubleshooting, and reflection was less comprehensive, revealing potential gaps between instructors’ educational intentions and their classroom interactions.

Lee and Benten (2023) describe an electrical engineering curriculum in which students progressively gain greater choice over their projects and how they demonstrate learning, while retaining access to expert guidance, weekly coaching, and formative feedback. Their implementation account also identifies challenges: some students found increased freedom unsettling, and instructors required time and support to adapt to their coaching and assessment roles. This example illustrates that expanding learner autonomy requires preparation and continuing support for both students and instructors.

This progression is especially relevant to introductory engineering education, where students are expected to engage with increasingly complex and open-ended design tasks before they have developed extensive disciplinary experience. Research on engineering design education emphasizes the importance of scaffolding students’ reasoning and design activity as they move from novice toward more independent performance, while recent reviews of experiential learning in engineering similarly identify scaffolding and progressive integration across levels as important features of effective engineering learning environments (Gómez Puente et al., 2013; Tembrevilla et al., 2024). Insufficient guidance at an early stage may leave students unable to resolve persistent technical difficulties or coordinate multiple interacting aspects of a system, potentially interfering with productive design activity. Conversely, highly prescriptive or prolonged guidance may restrict opportunities for independent problem solving, design judgment, and responsibility for engineering decisions. Effective instructional design therefore requires not simply more or less guidance but an appropriate alignment between the level of support, learners’ developing competence, and the complexity of the task. This issue is particularly important in introductory engineering design, where instructors must scaffold students toward increasingly advanced design performance while allowing them progressively greater responsibility for their work (Emberley & Moore, 2023).

Recent first-year engineering course redesigns illustrate how these principles can be implemented. Salyards et al. (2022) describe a shared scaffolding structure across interdisciplinary design projects and course sections. Klein et al. (2026) organize a multidisciplinary project around staged technical design briefs, supported by common instructional materials and instructor coordination. These examples show how introductory design courses can structure opportunities for students to apply and integrate technical knowledge while providing support for complex project work.

In the course examined in this study, this principle is reflected in a staged transfer of responsibility: instructional structure is progressively reduced as students assume greater responsibility for problem solving and design decisions.

2.3. Authentic Engineering Practice

A central aim of engineering education is to provide students with opportunities to engage in activities that approximate the practices of professional engineers. Engineering design involves more than applying isolated technical procedures; it requires identifying and defining problems, generating and evaluating possible solutions, integrating knowledge from multiple domains, working within constraints, making trade-offs, testing performance, and revising designs in response to evidence (Dym et al., 2005). Design-based learning in higher engineering education similarly emphasizes open-ended, hands-on, authentic, and multidisciplinary tasks that resemble the work of professional engineering communities (Gómez Puente et al., 2013). Authentic engineering learning can therefore be understood as participation in tasks that require students to make meaningful design decisions and integrate technical knowledge in situations where procedures and solutions are not completely predetermined.

Robotics provides a particularly suitable context for such practice because system performance depends on interactions among software, electronics, sensing, actuation, control, and mechanical construction. First-year robotics projects can therefore engage students in programming, fabrication, testing, system integration, teamwork, and iterative improvement (Akintewe & Silva Sotillo, 2022; Foust, 2021). Importantly, difficulties encountered during integration may not have a single obvious cause: an observed failure may originate in programming, wiring, sensing, mechanical alignment, power supply, or interactions among subsystems. Students must therefore diagnose, test, revise, and retest rather than simply execute predetermined procedures, reproducing an important feature of engineering design practice (Abas, 2026).

The authenticity of an engineering learning experience can also increase as tasks become less prescribed and students assume greater responsibility for system-level decisions. Guided experiments may expose students to genuine engineering components and procedures while constraining the range of decisions they are required to make. A bounded integration project requires students to coordinate previously learned elements within a defined problem, whereas a more open-ended project transfers greater responsibility for selecting solutions, integrating subsystems, troubleshooting failures, managing constraints, and achieving performance goals. Such progression is consistent with design-based approaches in which students increasingly engage with the uncertainty and open-endedness characteristic of engineering practice (Gómez Puente et al., 2013).

The course was designed to offer progressively more integrative and open-ended engineering activities. Students moved from guided hardware–software experimentation to a bounded robot-building task and subsequently to a more complex project involving integration, independent decision making, troubleshooting, and responsibility for system performance. The Semester 2 competition provided common performance constraints for the robot. This description concerns intended task demands and opportunities for participation, rather than measured development of students’ engineering competence or autonomy.

Taken together, these perspectives describe a coherent developmental progression. Experiential and project-based learning explain how students develop knowledge through cycles of action, reflection, and experimentation; scaffolding and cognitive apprenticeship explain how responsibility for increasingly complex activity can be progressively transferred to learners; and authentic engineering practice explains the educational significance of increasing system integration, open-endedness, constraints, and design responsibility. The framework, therefore, directs attention not only to the educational value of robotics-based project learning but also to how students perceive the changing balance among instructional support, learner autonomy, project complexity, workload, and responsibility for engineering decisions.

3. Materials and Methods

3.1. Research Design

This study employed a retrospective repeated-cross-sectional mixed-methods design to examine aggregated student-evaluation records from successive course groups across six academic years (2020–2025). The design combined quantitative analysis of group-level evaluation scores with descriptive summaries of archived thematic assignments to students’ open-ended comments. The two strands were first analyzed separately and subsequently integrated to identify complementarity and divergence between numerical evaluation patterns and students’ written feedback (Creswell & Plano Clark, 2018; Fetters et al., 2013). As individual students were not tracked over time, annual comparisons describe variation among available cohort records rather than longitudinal change within students.

The unit of quantitative analysis was the group-level evaluation record rather than the individual student response. Each record represented the aggregated evaluation results for one course group during a particular semester and academic year. The qualitative dataset consisted of discrete comment units extracted from the open-ended feedback associated with these records.

3.2. Course Context

The course was a required two-semester introductory course in an electrical and computer engineering department at a higher-education institution in Israel. It was designed for the first year and progressed from guided hardware–software experimentation to more independent project work. The archive does not identify each respondent’s year of study; some 2021 Semester 2 comments explicitly refer to second-year workload. Accordingly, first-year refers to the intended curricular placement, not a verified characteristic of every respondent.

Semester 1 comprised 13 weeks, with two hours of lectures and two hours of laboratory sessions per week. During Semester 1, students completed guided laboratory activities involving Arduino microcontrollers, electronic components, sensors, and actuators. The activities introduced microcontroller programming, input–output operations, electronic interfacing, sensor measurement, actuator control, testing, and troubleshooting. At the end of the semester, students constructed and presented a simple line-following robot. This project required students to integrate the hardware and software concepts introduced through the guided laboratory activities within a comparatively bounded task.

Despite the accessibility of the Arduino platform, the laboratory and project activities required students to coordinate multiple hardware and software components. Technical difficulties could arise from wiring errors, unstable power supplies, incorrect sensor calibration, mechanical limitations, programming faults, or interactions among subsystems. Addressing these difficulties required testing, troubleshooting, and, where necessary, instructional and technical support.

During Semester 2, students worked in teams of four to five students. Seven two-hour team-based sessions were held every two weeks; during each session, each team met with the instructor for approximately 30 min of consultation and guidance. Students collaborated to design, construct, program, test, and refine a more complex autonomous mobile robot. The robot was required to follow a predefined track, overcome obstacles, and complete the course in the shortest possible time. The semester culminated in a competition conducted under common track and obstacle conditions. The project required the integration of programming, sensing, actuation, navigation, electronic interfacing, mechanical construction, teamwork, and project management.

The broad instructional progression was retained across the observation period, but delivery conditions were not uniform. The course account identifies remote or hybrid teaching during the COVID-19 period, and 2021 Semester 2 comments explicitly describe remote-learning constraints. The retained records do not support a complete year-by-semester classification of face-to-face, remote, and hybrid delivery. We therefore do not assign a delivery mode to each year or attribute rating differences to a specific disruption. Appendix A, Table A1 and Table A2, distinguish the documented course structure and record coverage from contextual information unavailable in the archive. Year labels are retained as stored in the evaluation records; precise calendar teaching dates are unavailable.

3.3. Data Sources and Dataset Construction

The data came from the institution’s routine student-evaluation system. The teaching-development unit administered each semester’s evaluations separately through an electronic link near the end of teaching, usually during weeks 10–12, as described in the course account. Exact opening and closing dates and their relationship to each final competition were not retained. The responses therefore concern the experience available when students answered and are not treated as confirmed post-competition evaluations.

The quantitative archive contained 87 distinct group identifiers, each linked to an academic year, semester, and overall score. The reanalysis checked identifiers, semester classification, score availability, and agreement between the group-score and item-level files. Semester 1 contributed 63 records and Semester 2 contributed 24. Semester 2 records were available only for 2021–2023; that common period contained 35 Semester 1 records and all 24 Semester 2 records. The reason for the missing Semester 2 periods and the completeness of the original institutional export could not be established from the archive.

A group-level record was the exported aggregate identified by year, semester, and instructional group. It was not a student project team or an individual respondent. Each group mean contributed one observation, regardless of the number of respondents. The archive does not provide student or instructor linkage sufficient to identify overlapping respondents across records, so statistical independence cannot be verified beyond distinct record identifiers.

Each group-level record included, where available:

• Academic year;
• Course semester;
• Overall teaching-evaluation score;
• Aggregated scores for comparable questionnaire items; and
• Open-ended comments concerning the course and its instructors.

The evaluations were anonymous with respect to students. The analytical records contained no direct student identifiers. Instructor names appearing in free text were not reproduced in published excerpts.

3.4. Quantitative Measures

The primary outcome was the exported overall instructor-evaluation score for each group, reported on the institutional six-point scale, with larger values indicating more favorable ratings. The summary-item wording changed across periods: “Summary evaluation—lecturer teaching,” “Summary evaluation—lecturer guidance,” and “Overall satisfaction with the lecturer’s teaching.” Appendix B, Table A4, maps these labels to periods. These related items are analyzed as archived ratings, not as a validated invariant measure of course quality or a composite questionnaire scale. Full questionnaire forms and exact verbal response anchors were not available in the analytical archive.

The designated overall-score field in the group-level file was used consistently for the primary analyses. It agreed with the mapped summary item in 86 of 87 records. For 2023, Semester 2, Group 3, the group file recorded 4.8 and the item file recorded 3.2. Neither value could be adjudicated from an original institutional form. Rather than selectively replace a value in the designated analysis field, we retained 4.8 and repeated the principal score analyses with 3.2 and with the record excluded (Appendix B, Table A6). This source discrepancy is a data-quality limitation; agreement of sensitivity results does not establish which value is correct.

Secondary analyses used related item-level dimensions for periods with available observations. The export labels support thematic mapping but do not establish identical wording or measurement invariance. These comparisons are exploratory and are qualified by the periods and semesters represented.

Enrollment, invitation, and respondent counts were absent from the analytical archive. Response rates, mean respondents per record, unique-student totals, and respondent-weighted estimates could therefore not be calculated. The 87 records and 415 comment units must not be interpreted as numbers of students.

3.5. Preparation of the Open-Ended Feedback

The original evaluation system organized written feedback into four predefined categories:

• Course strengths;
• Course areas for improvement;
• Instructor strengths; and
• Instructor areas for improvement.

These categories were retained as source metadata. They identify the response field in which a comment was submitted, rather than independently adjudicated sentiment. A strengths response could also contain criticism, and an improvement response could include praise.

The qualitative archive contained 415 already-delimited comment units with stable identifiers. These units were retained without further splitting so that all frequencies remained traceable to the source audit. Some units contain multiple clauses or topics; they are not assumed to be uniformly sized or to represent different students.

Each unit retained its Hebrew source text, original feedback category, academic year, semester, and group-record link. The 415 units came from 81 of the 87 records; six records contained no comment units. Published English excerpts were checked against their linked Hebrew text and omitted direct names. The analysis did not reconstruct the number of original respondents or make a claim about an independently verified historical segmentation procedure.

3.6. Quantitative Data Analysis

Descriptive statistics for group-level overall scores included n, mean, sample standard deviation, median, range, and annual 95% confidence intervals. Annual intervals were the 2.5th and 97.5th percentiles of the archived 5000-resample bootstrap means (reported seed 2026). The stored distributions were used consistently for interval estimation and Figure 1. The three-record 2020 interval provides particularly limited information about cohort uncertainty.

Figure 1. Mean overall teaching-evaluation score by academic year, with 95% confidence intervals.

Rank-based tests were selected because these were bounded aggregated ratings with unequal record counts and small subgroups. They reduce distributional assumptions but do not correct measurement differences, dependence between records, or confounding.

Annual variation was examined using the tie-corrected Kruskal–Wallis test. Dunn pairwise comparisons used tie correction and Holm adjustment across the 15 year contrasts. The omnibus effect size was ε² = (H − k + 1)/(N − k), where k is the number of year groups.

Semester comparisons used the Mann–Whitney U test, with rank-biserial correlation as the effect-size measure. Interpretation emphasizes the restricted 2021–2023 analysis because both semesters were represented in that period. The comparison across all available years is reported as a secondary, temporally unbalanced contrast. Restriction to common years reduces the coverage imbalance but does not match individual students, ensure identical year composition, or control other differences between course groups. The common-period restriction addresses temporal coverage only. Because overall-item wording varies, a further within-2022 comparison uses the same exported lecturer-guidance item in both semesters; it remains an exploratory comparison of different group records.

Record-level theme presence was compared between semesters using Pearson’s chi-square test without continuity correction when every expected count was at least five; otherwise, a two-sided Fisher exact test was used. Cramér’s V was computed from the Pearson statistic, including comparisons assessed by Fisher’s test. Each record contributed at most once per theme.

Holm adjustments were applied separately to the eight full-period semester-theme comparisons, the eight common-period comparisons, the seven original-score theme comparisons, and the six item-level tests. Adjustment used full-precision p-values and the cumulative-maximum step. These exploratory families were kept separate because they addressed distinct questions. Tests of theme presence count records without comments as having no recorded theme; this is a reporting indicator, not evidence that the experience was absent.

Scores were analyzed without categorization because no substantively validated thresholds were available and categorization would discard information. For seven selected themes, original scores were compared between records with and without the theme using two-sided Mann–Whitney tests. Rank-biserial correlation was r_rβ = 2 U/(n₁n₂) − 1, positive for higher scores in the theme-present group. Pointwise percentile 95% confidence intervals used 10,000 independent within-group bootstrap resamples, NumPy default_rng was initialized with seed 2026, and the themes were processed in a fixed order.. These intervals are not simultaneous. Analyses pooled response-field orientations and did not adjust for year, semester, or questionnaire version. Sensitivity checks substituted or excluded the conflicting score and restricted analysis to the 81 records containing comments (Appendix B).

Tests were two-sided, with α = 0.05. Calculations used Python and SciPy 1.17.0. Mann–Whitney p-values used a tie-corrected asymptotic normal approximation with continuity correction; U corresponds to the first-listed group. Non-significance was not interpreted as equivalence. The reported precision and p-values are conditional on treating exported records as independent; respondent-level linkage was unavailable to estimate clustering. OpenAI ChatGPT/Codex, powered by GPT-6, assisted revision and analytical-code preparation; numerical results were calculated from the archived data using the stated statistical procedures.

3.7. Qualitative Analysis

The qualitative component was a secondary analysis of an archived coded-comment corpus. Its codebook comprised eight course themes and six instructor themes. Appendix A, Table A3, provides all 14 operational definitions. The reanalysis retained the archived assignments and summarized the original comments and code indicators, rather than claiming a newly validated inductive thematic analysis.

For each comment, the audit contained assigned theme names, binary theme indicators, and a coding-evidence field with lexical or regular-expression matches. These fields document the recorded decisions but do not establish the relative roles of manual and automated coding, the chronology of code development, or how historical coder disagreements were resolved. No new claim of inductive theme generation or independent dual coding is made.

Multiple coding was permitted. A theme was counted once within a comment even if repeated and once within a record regardless of how many linked comments contained it. The binary matrix contained 589 comment–theme assignments: 146 units had multiple codes and 19 had no assigned code. Unassigned units remained in the 415-unit and source-category denominators, and percentages across themes need not sum to 100%. Theme frequencies were recalculated from the binary indicators.

The codebook distinguishes domain, theme name, and operational definition. These definitions document the scope of the archived codes; they are not presented as a reconstructed history of inclusion/exclusion decisions.

Themes were summarized within the original course/instructor and strength/area-for-improvement fields. These fields indicate where a comment was submitted; they do not independently resolve the evaluative orientation of every clause. A comment in a strengths field can contain criticism. Theme presence in the record-level tests pools these source categories, and the source categories were preserved separately in the descriptive summaries.

3.8. Coding Reliability and Trustworthiness

The archive contained one final code matrix, with no independently paired coder ratings, identifiable double-coding sample, or reconciliation record. Intercoder agreement could not be estimated from that matrix. Accordingly, no Cohen’s κ or independently double-coded percentage is reported. A single consensus or final coding set cannot establish reliability between coders.

Traceability was assessed through comment identifiers, source text, category labels, record links, and binary indicators. Theme totals were reproduced from those indicators, and the English excerpts were compared with the original Hebrew. This verifies counting and excerpt fidelity, not all coding judgments. Lexical cues can miss context: for example, an uncoded unit requests more patient delivery. Frequencies therefore describe the archived assignments and may underestimate or misclassify some meanings. We retained all assignments consistently rather than selectively correcting examples after inspecting results. The qualitative strand is used to describe reported experiences and generate course-design questions; it does not establish the completeness or validity of the coding framework. The reported themes are descriptive patterns in the archived assignments, not a comprehensive or independently validated representation of students’ comments.

3.9. Theme-Frequency and Academic-Year Analyses

Theme frequencies were examined at two analytical levels. Comment-level frequency represented the number and percentage of comment units assigned to a theme. Record-level prevalence represented the number and percentage of group-level evaluation records containing at least one instance of that theme.

Record-level prevalence was emphasized in statistical comparisons because multiple comment units associated with the same group-level score were not statistically independent. Counting a theme only once within each record reduced the risk that groups submitting more comments would exert disproportionate influence on comparisons.

Themes were compared descriptively across years using each year’s record count as denominator. Figure 2 displays seven themes occurring in at least 20 records overall. Annual percentages are not estimates of student prevalence, and no trend test was conducted. Semester coverage and willingness to comment may affect the annual patterns.

Figure 2. Record-level prevalence of the principal feedback themes by academic year.

3.10. Integration of Quantitative and Qualitative Findings

The quantitative and qualitative findings were integrated after the initial separate analyses. A joint display was developed to connect patterns in overall and item-level scores with corresponding qualitative themes (Fetters et al., 2013; Guetterman et al., 2015).

Integration focused on two principal relationships:

• Complementarity, when the qualitative findings provided context or explanation for a quantitative pattern; and
• Divergence, when favorable numerical evaluations coexisted with critical comments or when written feedback did not correspond with an expected numerical pattern.

Integration considered co-occurrence between archived ratings and coded topics, including practical activity, instructional clarity, support, workload, organization, and resources. Because comments were linked to group means rather than individual ratings, the joint display cannot establish that the same respondent supplied a rating and a particular comment or explain a causal mechanism.

3.11. Ethical Considerations

The study used retrospective, anonymized data collected through the institution’s routine course-evaluation process. The researchers did not have access to direct student identifiers, and the analysis was conducted at the aggregated group-record level. Written comments were anonymized before reporting, and quotations that could reveal the identity of a student, instructor, or course group were excluded or minimally edited without changing their substantive meaning.

The study was determined to be exempt from full ethical review because it involved the secondary analysis of anonymized institutional evaluation data. Data were stored and analyzed in accordance with institutional data-protection requirements.

4. Results

4.1. Dataset Characteristics

The dataset comprised 87 group-level records: 63 from Semester 1 and 24 from Semester 2. The 415 comment units were linked to 81 records; six records contained no comment units. These counts represent exported records and comments, not unique students. Enrollment and respondent counts were unavailable, so participation rates and respondent-weighted estimates are not reported. Table 1 summarizes the two analytical levels, and Table A2 reports year-by-semester coverage.

Table 1. Composition of the quantitative and qualitative datasets.

Dataset Component	Category	n	%
Group-level records	Semester 1	63	72.4
Group-level records	Semester 2	24	27.6
Group-level records	Total	87	100.0
Comment units	Course strengths	66	15.9
Comment units	Course areas for improvement	129	31.1
Comment units	Instructor strengths	149	35.9
Comment units	Instructor areas for improvement	71	17.1
Comment units	Total	415	100.0

The four original feedback categories were retained during analysis. Instructor strengths constituted the largest category, with 149 comment units. The distribution of the remaining 266 units across course strengths, course areas for improvement, and instructor areas for improvement is presented in Table 1.

Semester 1 covered 2020–2025, whereas Semester 2 covered only 2021–2023. The shared-period score comparison is emphasized; full-period comparisons are secondary and temporally unbalanced. Table A5 reports the corresponding common-period theme comparisons.

4.2. Evaluation Scores Across Academic Years

Overall teaching-evaluation scores differed significantly across the six academic years; Kruskal–Wallis H(5) = 15.18, p = 0.010, ε² = 0.126. Descriptive statistics indicated variation in the mean and median scores across years, as presented in Table 2 and Figure 1.

Table 2. Overall teaching-evaluation scores by academic year.

Academic Year	n	Mean	SD	Median	Minimum	Maximum	95% CI
2020	3	5.67	0.21	5.60	5.50	5.90	5.50–5.90
2021	14	5.28	0.45	5.35	4.40	6.00	5.04–5.51
2022	17	4.88	0.51	5.00	3.30	5.50	4.62–5.10
2023	28	4.75	0.77	4.85	2.40	5.90	4.45–5.01
2024	15	5.16	0.70	5.30	3.00	6.00	4.77–5.45
2025	10	5.26	0.49	5.15	4.60	6.00	4.98–5.54

However, none of the pairwise comparisons between individual academic years remained statistically significant after Holm correction. Accordingly, the significant omnibus result indicates overall variation across the six-year period but does not establish a statistically reliable difference between any particular pair of years.

4.3. Evaluation Scores by Semester

For the common 2021–2023 period, Semester 1 comprised 35 records (M = 5.01, SD = 0.70) and Semester 2 comprised 24 records (M = 4.78, SD = 0.59). The comparison did not reach statistical significance, U = 519.50, p = 0.126, r_rβ = 0.24 (Table 3). This result does not establish equivalence or exclude a potentially meaningful difference. The records are aggregated group evaluations, and the comparison does not represent within-student change or a year-adjusted estimate. It also pools different summary-item wordings; the common period is not a measurement-equivalence restriction.

Table 3. Overall teaching-evaluation scores by semester and comparison period.

Comparison Period	S1, n	S1, M (SD)	S2, n	S2, M (SD)	U	p	rrβ
Common years, 2021–2023	35	5.01 (0.70)	24	4.78 (0.59)	519.50	0.126	0.24
All available years	63	5.12 (0.66)	24	4.78 (0.59)	1026.50	0.010	0.36
Same item, 2022	10	5.05 (0.33)	7	4.64 (0.65)	49.50	0.166	0.41

Note: S1 = Semester 1; S2 = Semester 2; n = group-level records; M = mean; SD = standard deviation; r_rβ = rank-biserial correlation. The common-period comparison is emphasized for interpretation; the full-period comparison is temporally unbalanced.

In the secondary comparison across all available years, Semester 1 had a higher overall score (n = 63, M = 5.12, SD = 0.66, median = 5.20) than Semester 2 (n = 24, M = 4.78, SD = 0.59, median = 5.00), U = 1026.50, p = 0.010, r_rβ = 0.36. Because Semester 1 spans 2020–2025 and Semester 2 only 2021–2023, this contrast cannot establish a semester effect independent of year composition. Neither analysis isolates the contribution of the competition from the other instructional differences between semesters.

Within 2022, both semesters used the same exported lecturer-guidance item: Semester 1 n = 10, M = 5.05, and Semester 2 n = 7, M = 4.64. This smaller comparison did not reach statistical significance, U = 49.50, p = 0.166, r_rβ = 0.41. It does not remove other semester differences or establish equivalence. Substituting 3.2 for the conflicting 4.8 score gave common-period p = 0.090; excluding the record gave p = 0.138. The substantive caution remains unchanged (Table A6).

4.4. Comparable Evaluation Dimensions

Table 4 presents exploratory comparisons of mapped evaluation dimensions using the available item-level records. The labels concern related content but are not evidence of a fully invariant instrument. Both unadjusted and Holm-adjusted p-values are reported across the six tests.

Table 4. Comparable evaluation dimensions across academic years and semesters.

Evaluation Dimension	Comparison	Groups: Year/Semester, n, M (SD)	Test	p	Effect Size	Holm p
Instructional clarity	Academic year	2020: 3, 5.53 (0.32); 2021: 8, 5.40 (0.28); 2023: 28, 4.79 (0.82); 2024: 15, 5.12 (0.75); 2025: 10, 5.20 (0.58)	H(4) = 8.83	0.065	ε2 = 0.082	0.196
Instructional clarity	Semester (2023)	S1: 17, 4.86 (0.81); S2: 11, 4.68 (0.87)	U = 103.00	0.671	rrβ = 0.102	0.671
Course organization	Academic year	2021: 8, 5.57 (0.19); 2023: 11, 4.30 (0.61); 2024: 15, 4.59 (0.83); 2025: 10, 5.11 (0.46)	H(3) = 19.70	<0.001	ε2 = 0.418	0.001
Practical contribution	Academic year	2020: 3, 5.73 (0.23); 2023 S1: 17, 4.78 (0.86);	U = 46.00	0.034	rrβ = 0.804	0.169
Project contribution	Academic year	2021 S2: 6, 4.52 (0.69); 2022: 17, 4.93 (0.61)	U = 29.50	0.140	rrβ = −0.422	0.280
Project contribution	Semester (2022)	S1: 10, 5.18 (0.33); S2: 7, 4.57 (0.77)	U = 56.50	0.040	rrβ = 0.614	0.169

Note: S1/S2 = Semester 1/2; M = mean; SD = sample standard deviation; H = Kruskal–Wallis statistic; U = Mann–Whitney statistic; ε² = epsilon-squared; r_rβ = rank-biserial correlation. Holm adjustment covers all six tests. Positive r_rβ favors the first-listed group. Item mappings and semester coverage limit comparability.

Only the course-organization comparison remained significant after Holm adjustment (adjusted p = 0.001). That comparison also mixes year and semester, because its 2023 records are from Semester 2 and the other years from Semester 1. Practical-contribution and within-2022 project-contribution differences had unadjusted p-values below 0.05 but adjusted p = 0.169; neither is treated as a confirmed contrast. The practical-contribution comparison additionally includes only three records from 2020.

4.5. Course- and Instructor-Related Themes

The archived codebook and binary matrix contained 14 themes, eight course-related and six instructor-related. Table 5 reports their recalculated frequencies within the original response fields. Counts describe recorded code assignments, subject to the coding limitations in Section 3.8; they are not a complete census of meanings expressed by students.

Table 5. Comment-unit frequencies and group-record prevalence of course- and instructor-related themes.

Domain	Theme	Strengths, n (%)	Areas for Improvement, n (%)	Total Comment Units, n (%)	Records with Theme, n (%)
Course	Practical and experiential learning	20 (30.3%)	8 (6.2%)	28 (6.7%)	23 (26.4%)
	Hardware–software integration	6 (9.1%)	12 (9.3%)	18 (4.3%)	13 (14.9%)
	Engineering relevance	14 (21.2%)	9 (7.0%)	23 (5.5%)	17 (19.5%)
	Project and robotics experience	10 (15.2%)	19 (14.7%)	29 (7.0%)	20 (23.0%)
	Workload and time demands	1 (1.5%)	60 (46.5%)	61 (14.7%)	31 (35.6%)
	Lecture–laboratory alignment	3 (4.5%)	19 (14.7%)	22 (5.3%)	17 (19.5%)
	Organization and clarity	14 (21.2%)	50 (38.8%)	64 (15.4%)	35 (40.2%)
	Equipment and resources	7 (10.6%)	31 (24.0%)	38 (9.2%)	24 (27.6%)
Instructor	Instructional clarity	64 (43.0%)	24 (33.8%)	88 (21.2%)	48 (55.2%)
	Availability and responsiveness	38 (25.5%)	18 (25.4%)	56 (13.5%)	37 (42.5%)
	Project and laboratory support	14 (9.4%)	19 (26.8%)	33 (8.0%)	24 (27.6%)
	Professional knowledge	30 (20.1%)	12 (16.9%)	42 (10.1%)	29 (33.3%)
	Patience and interpersonal climate	61 (40.9%)	16 (22.5%)	77 (18.6%)	46 (52.9%)
	Coordination and consistency	0 (0.0%)	10 (14.1%)	10 (2.4%)	10 (11.5%)

Note: Strength and improvement percentages are calculated within the corresponding original feedback category (course strengths, n = 66; course improvements, n = 129; instructor strengths, n = 149; instructor improvements, n = 71). Total-comment percentages use N = 415, and record-level percentages use N = 87. Because multiple coding was permitted, percentages across themes do not sum to 100%. The 19 units with no recorded theme remain in the comment denominators, and the six records with no comments remain in the record denominators. The categories denote original response fields, not independently recoded sentiment.

Table 5 presents the frequency of each theme at two levels. Comment-unit frequency represents the number of discrete comments assigned to a theme, whereas record-level prevalence represents the number of group-level records in which the theme appeared. Record-level prevalence was used for inferential comparisons to avoid treating multiple comments associated with the same evaluation score as independent observations.

4.5.1. Course Strengths

Within the archived assignments, the most frequent course-strength themes were practical and experiential learning (20 comments), engineering relevance (14), organization and clarity (14), and project and robotics experience (10). Comments described valuing opportunities to apply theoretical knowledge through physical construction, programming, testing, and troubleshooting. Hardware–software integration was also identified as a valued aspect of the course, although it occurred less frequently.

An illustrative comment stated:

“Physical construction and practical experience with the material improve understanding of the subject.”

(C0101; 1 Semester 2022; translated from Hebrew)

Other course strengths included hardware–software integration, which emphasized connections between programming and physical components. These illustrative course-strength comments described valuing hands-on application, engineering relevance, and opportunities to integrate hardware and software within authentic tasks.

4.5.2. Course Areas for Improvement

The principal course-related areas for improvement were workload and time demands (60 comments), organization and clarity (50), equipment and resources (31), lecture–laboratory alignment (19), and project and robotics experience (19). Workload comments described excessive demands, insufficient time, inconvenient scheduling, and difficulty combining course requirements with other academic responsibilities. Organization comments concerned task sequencing, instructions, deadlines, and the relationship between lectures, laboratories, and project requirements.

An illustrative comment stated:

“The course workload combined with the workload of the second year was inhuman.”

(C0080; 2 Semester 2021; translated from Hebrew)

These comments show that positive statements about practical activity and concerns about its implementation coexist in the archive. They do not establish that every group or individual held both views. The reference to second-year workload also limits an interpretation of the sample as exclusively first-year students.

4.5.3. Instructor Strengths

Instructor strengths accounted for 149 of the 415 comment units (35.9%). The most frequent instructor-strength themes were instructional clarity (64 comments), patience and interpersonal climate (61), availability and responsiveness (38), and professional knowledge (30). Comments described instructors as clear, supportive, patient, and willing to provide assistance.

An illustrative comment stated:

“The material was presented clearly.”

(C0009; 1 Semester 2020; translated from Hebrew)

4.5.4. Instructor Areas for Improvement

The most frequent instructor-related areas for improvement were instructional clarity (24 comments), project and laboratory support (19), availability and responsiveness (18), patience and interpersonal climate (16), professional knowledge (12), and coordination and consistency (10). Coordination comments referred to inconsistent requirements, instructions, or expectations across instructors and groups.

An illustrative comment stated:

“More coordination is needed between the instructors of the different groups; the rules are not the same for both groups.”

(1 Semester 2023)

This distinction was retained because inconsistent instructions, expectations, or support across instructors could reflect both an instructor-related perception and a broader course-implementation issue.

4.6. Differences in Feedback Between Semesters

Semester 1 feedback more frequently addressed instructional clarity, organization and clarity, and lecture–laboratory alignment, whereas Semester 2 feedback more frequently addressed project and robotics experience. At the record level, project and robotics experience appeared in 17.5% of Semester 1 records and 37.5% of Semester 2 records. In contrast, instructional clarity appeared in 61.9% and 37.5%, organization and clarity in 47.6% and 20.8%, and lecture–laboratory alignment in 27.0% and 0.0% of Semester 1 and Semester 2 records, respectively.

After accounting for multiple comparisons, these differences should primarily be interpreted as descriptive differences in recorded thematic salience; lecture–laboratory alignment provided the strongest evidence of a semester-related difference. However, Semester 1 records span 2020–2025, whereas Semester 2 records cover only 2021–2023. These differences may therefore reflect academic-year composition as well as semester-specific experiences. Moreover, theme prevalence represents the proportion of group-level evaluation records containing a theme, rather than the proportion of individual students expressing that view. These contrasts are partly expected from the course design and reflect different opportunities to comment on particular activities. More robotics-related comments in a robotics-centered semester, or fewer lecture-related comments where lecture exposure differs, do not establish superior quality, learning, or autonomy.

Before adjustment, project and robotics experience, instructional clarity, organization and clarity, and lecture–laboratory alignment differed between semesters. After Holm correction across the eight comparisons, only lecture–laboratory alignment remained statistically significant, with adjusted p = 0.019. The remaining semester differences should be interpreted as descriptive differences in thematic salience.

In the common 2021–2023 period, lecture–laboratory alignment remained the only Holm-significant contrast: 11/35 Semester 1 records versus 0/24 Semester 2 records, Fisher p = 0.0017, adjusted p = 0.014. Organization and clarity occurred in 19/35 versus 5/24 records (adjusted p = 0.071); other adjusted p-values were at least 0.537 (Table A5). This restriction reduces unequal coverage but does not remove differences in activities, opportunities to comment, or coding limitations.

4.7. Patterns in Feedback Themes Across Academic Years

Instructional clarity, availability and responsiveness, organization and clarity, and workload and time demands appeared across all six academic years, indicating that these were recurring dimensions of the course experience. Project and robotics experience was concentrated in 2021–2023 and was absent from the coded records for 2024 and 2025. Equipment- and resource-related feedback was also most frequent during 2021–2023. Practical and experiential learning appeared in every year but varied substantially in prevalence. Interpretation is constrained by semester coverage: Semester 2 records occur only in 2021–2023. The concentration of project-related themes in those years therefore cannot be separated from the composition of the available records. Absence from the coded comments does not establish absence of the experience.

The figure includes seven analytically central themes that occurred in at least 20 group-level records overall. Apparent annual differences should be interpreted cautiously when yearly denominators are small.

Figure 3 summarizes semester-level thematic prevalence from Table 6. It is a descriptive display of the full, temporally unbalanced dataset.

Figure 3. Group-record prevalence of principal feedback themes by semester.

Table 6. Group-record prevalence of principal feedback themes by semester across all available years.

Theme	S1 n (%)	S2 n (%)	p	Holm p	V
Practical and experiential learning	18 (28.6%)	5 (20.8%)	0.464	1.000	0.078
Project and robotics experience	11 (17.5%)	9 (37.5%)	0.047	0.245	0.213
Instructional clarity	39 (61.9%)	9 (37.5%)	0.041	0.245	0.219
Availability and responsiveness	27 (42.9%)	10 (41.7%)	0.920	1.000	0.011
Workload and time demands	23 (36.5%)	8 (33.3%)	0.782	1.000	0.030
Organization and clarity	30 (47.6%)	5 (20.8%)	0.023	0.159	0.244
Lecture–laboratory alignment	17 (27.0%)	0 (0.0%)	0.002	0.019	0.304
Equipment and resources	18 (28.6%)	6 (25.0%)	0.739	1.000	0.036

Note: Percentages use semester-specific denominators (Semester 1, n = 63; Semester 2, n = 24). Pearson’s chi-square test was used unless expected frequencies required Fisher’s exact test. A record is counted once per theme, regardless of the number or evaluative orientation of its comments. Theme prevalence is not a percentage of individual students. The academic-year coverage differs between semesters.

Values reproduce Table 6. A record contributes at most once to each theme; the bars do not represent percentages of individual students or distinguish praise from criticism. Unequal academic-year coverage and differences in instructional exposure limit interpretation. Only lecture–laboratory alignment remained significant after Holm adjustment (adjusted p = 0.019).

4.8. Original Evaluation Scores by Theme Presence

Table 7 compares original group-level scores between records with and without each of seven selected themes. The archived score field is retained without lower/moderate/higher categorization. Each record contributes one score and one binary indicator per theme, irrespective of its number of comments.

Table 7. Original overall evaluation scores by record-level theme presence.

Theme	n1/n0	M1/M0	U	p	rrβ [95% CI]
Practical and experiential learning	23/64	4.94/5.05	624.0	0.282	−0.152 [−0.408, 0.113]
Instructional clarity	48/39	5.03/5.01	973.5	0.752	0.040 [−0.201, 0.287]
Availability and responsiveness	37/50	4.90/5.11	803.0	0.296	−0.132 [−0.379, 0.116]
Workload and time demands	31/56	5.00/5.03	861.0	0.954	−0.008 [−0.274, 0.266]
Organization and clarity	35/52	5.04/5.01	943.0	0.778	0.036 [−0.219, 0.283]
Lecture–laboratory alignment	17/70	5.11/5.00	602.0	0.944	0.012 [−0.286, 0.313]
Equipment and resources	24/63	5.01/5.03	752.0	0.973	−0.005 [−0.294, 0.284]

Note: n₁/n₀ and M₁/M₀ = record counts and mean scores with/without a recorded theme; U uses theme-present records first. Positive r_rβ indicates higher scores with the theme. Intervals are pointwise bootstrap 95% confidence intervals from 10,000 within-group resamples. All seven Holm-adjusted p-values equal 1.000. The absent group includes records without comments. Theme indicators pool evaluative orientations.

Differences are exploratory because overall-item wording varies and record independence is not established. Neither the test nor the effect interval is adjusted for year, semester, respondent number, or response-field orientation.

None of the seven tests detected a Holm-significant difference. Rank-biserial correlations ranged from −0.152 to 0.040, and every bootstrap interval included zero. For workload, U = 861.00, p = 0.954, r_rβ = −0.008, 95% CI [−0.274, 0.266]. These intervals allow effects in either direction and do not establish equivalence or no association.

Substitution of the conflicting score, exclusion of its record, and restriction to the 81 records with comments each left all seven Holm-adjusted p-values at 1.000. These checks address selected data-handling decisions; they do not correct measurement or coding error, clustering, or ecological inference (Appendix B).

4.9. Integrated Interpretation of Quantitative and Qualitative Findings

The mixed-methods integration examined how the qualitative themes converged with, complemented, or diverged from the quantitative patterns. Table 8 presents the joint display of the quantitative findings, qualitative context, and integrated interpretations.

Table 8. Joint display of quantitative and qualitative findings.

Quantitative Finding	Qualitative Context	Integrated Interpretation
Overall evaluation scores varied across academic years, but no pairwise difference remained statistically significant after Holm adjustment.	Instructional clarity, instructor availability and responsiveness, organization and clarity, and workload appeared in every academic year. Project/robotics and equipment/resource themes were concentrated in 2021–2023.	Complementarity: The qualitative themes identify recurring aspects of the course experience and provide context for the variation across years. They do not establish which themes caused the quantitative variation.
The full-period comparison favored Semester 1, whereas the common 2021–2023 comparison did not reach statistical significance.	Project and robotics experiences were more prevalent in Semester 2. Instructional clarity, organization and clarity, and lecture–laboratory alignment were more prevalent in Semester 1.	Complementarity: Feedback identifies different topics in the two semesters. It does not explain an overall score difference because year coverage, questionnaire wording, and course activities differ.
Instructional-clarity ratings were generally favorable and did not differ significantly across comparable years or semesters.	Instructional clarity was the most frequent theme, appearing in 88 comments and 48 of 87 records. Strength comments outnumbered improvement comments, 64 versus 24.	Complementarity: The predominance of positive clarity comments is consistent with the favorable ratings, while the improvement comments indicate that clarity was not uniformly experienced.
Organization scores varied across the included periods (Holm p = 0.001), but year and semester were partly confounded.	Organization and clarity appeared in 64 comments and 35 of 87 records. Improvement comments substantially outnumbered strength comments, 50 versus 14.	Complementarity: The predominance of organization-related improvement comments contextualizes the weaker ratings, although the year comparison was partly confounded by semester coverage.
Overall evaluation scores were generally favorable across the examined periods.	Workload codes occurred in 61 comments and 31 records; 60 comments came from improvement fields. Original-score comparison: U = 861.00, p = 0.954, Holm p = 1.000; rrβ = −0.008, 95% CI [−0.274, 0.266].	Divergence: Favorable group-level evaluations coexisted with workload-related feedback. The pooled theme indicator does not establish a student-level relationship or the absence of an association with ratings.
Practical-contribution ratings were favorable in the available comparison: 2020, M = 5.73; 2023 Semester 1, M = 4.78.	Practical and experiential learning appeared in 28 comments and 23 of 87 records. Strength comments outnumbered improvement comments, 20 versus 8.	Complementarity: The emphasis on hands-on learning is consistent with favorable practical-contribution ratings, while the improvement comments indicate opportunities to refine its implementation.

Note: Complementarity indicates that the qualitative findings contextualized or extended the quantitative pattern. Divergence indicates that the qualitative and quantitative findings revealed contrasting aspects of the student experience. The integrated findings are associational and should not be interpreted as evidence of causation. The qualitative component summarizes archived code assignments and does not provide independent validation of either the themes or their coverage of students’ comments.

The integrated analysis identified perceived strengths alongside implementation concerns. All 31 records containing workload codes included a workload comment submitted in an improvement field. Their overall scores ranged from 2.4 to 5.9, demonstrating coexistence across a broad range of ratings without defining arbitrary rating categories. This does not establish that the same students both praised the course and criticized workload or that workload had no influence on ratings.

The semester-specific themes complemented the quantitative comparison by distinguishing the descriptively greater Semester 2 emphasis on project and robotics experiences from the greater Semester 1 emphasis on instructional clarity, organization, and lecture–laboratory alignment. These integrated findings remain associational because the unit of analysis was the aggregated group-level record and academic-year coverage differed between semesters. These thematic contrasts are partly expected from the different course activities and do not establish educational advantage or differences in teaching quality.

5. Discussion

This study examined six years of group-level student evaluations from a two-semester introductory engineering course. It identifies perceived strengths and implementation concerns rather than demonstrating learning gains or development of autonomy. The common-period semester comparison did not reach statistical significance; the significant full-period contrast remains confounded by year coverage and changing overall-item wording. Numerical scores varied among academic-year cohorts, although no Holm-adjusted pairwise contrast was significant. Written feedback added information about instructional clarity, interpersonal climate, responsiveness, organization, workload, and practical learning that was not captured by an overall score.

The common-period semester comparison does not establish that one semester was more effective than the other. Thematic differences were consistent with differences in course activities and opportunities for comment: robotics-related feedback was more prevalent in Semester 2, whereas lecture–laboratory alignment was more prevalent in Semester 1. Such patterns are useful descriptions of what appeared in the feedback, but lower prevalence does not indicate lower instructional quality or reduced need for support. Moreover, autonomy, technical complexity, teamwork demands, project duration, and competition vary together. Their separate contributions cannot be estimated from these data.

The descriptive semester patterns are consistent with the rationale for robotics and design-based learning, which engage students in coordinating hardware and software through iterative design, testing, and refinement (Alimisis, 2013; Benitti, 2012; Abas, 2026). Viewed through cognitive apprenticeship, the course progression represents a shift from explicit coaching toward progressive fading (Collins et al., 1989). As responsibility increases, support may need to shift from structuring task execution toward assisting planning, integration, and troubleshooting (Hmelo-Silver et al., 2007; Quintana et al., 2004; Reiser, 2004). This provides a framework for interpreting the course design, although the evaluation data do not directly establish the effectiveness of progressive autonomy or scaffolding.

Workload and time demands accounted for 61 coded units, 60 from improvement fields. The 31 associated records had overall scores ranging from 2.4 to 5.9. For course improvement, it may be useful to consider productive engineering effort, such as testing, integration, and troubleshooting, separately from potentially avoidable organizational demands, such as unclear requirements, scheduling problems, or limited resources. This is an interpretive consideration, not an empirically established classification in this study. The archived workload code does not allow these categories to be systematically distinguished or quantified. One 2022 Semester 1 comment linked excessive preparation time to unclear laboratory goals and reports (C0107), illustrating a possible organizational concern; it does not establish how representative such concerns are among workload-related comments. The data therefore do not establish the relative prevalence of productive and avoidable demands or any learning benefits of workload. The original-score comparison did not detect a difference and cannot demonstrate that workload is unrelated to ratings.

Using original scores preserves information that categorization would discard, but the broad intervals and pooled comment orientations still limit inference. Written feedback can identify concerns obscured by overall ratings (Nasser-Abu Alhija & Fresko, 2009; Brockx et al., 2012). More informative future analyses require linked individual ratings and comments, consistent measures, and independently assessed coding quality.

The perceptions suggest several course-design priorities for prospective evaluation; the present study did not test their effectiveness. First, the transition to the autonomous project should retain explicit scaffolds for system decomposition, interface management, testing, and troubleshooting rather than relying on general availability of help. Second, the sequence and relationship among lectures, laboratories, and project milestones should be made visible so that students encounter prerequisite knowledge before it is required in integration tasks. Third, workload should be reviewed at the level of weekly deliverables, team responsibilities, access to equipment, and opportunities for iterative testing. Intermediate design reviews and staged performance criteria may preserve authenticity while reducing late project overload. Finally, coordination among instructors should include shared expectations, consistent technical guidance, and clear escalation routes for project difficulties. These measures are consistent with scaffolding as a temporary structure that supports increasingly independent performance rather than reducing the intellectual challenge of the task (Wood et al., 1976; Collins et al., 1991).

For instructors and laboratory teaching teams, these priorities provide a basis for reviewing where students need clearer explanations, timely feedback, and help with integration tasks. Programme leaders and course coordinators can use the recurring organizational and workload concerns to review the scheduling of prerequisites, shared assessment expectations, equipment access, and staffing for project consultations. Students are the intended beneficiaries of clearer milestones and more accessible support, particularly when they enter with uneven technical experience. These are proposed benefits to be evaluated, rather than outcomes demonstrated by the present data.

For designers of robotics competitions and other introductory engineering projects, the implication is to plan the surrounding teaching and support alongside the final performance task. The findings justify reviewing how project demands fit available time and resources; they do not justify adopting or removing competition on the basis of overall ratings. Beyond this course, the same questions may inform other programmes that move from guided exercises to open-ended team projects, although local needs and effects require assessment.

The joint display illustrates how ratings and written feedback can provide complementary descriptive evidence (Fetters et al., 2013; Guetterman et al., 2015). Prospective work should use consistent evaluation items, record participation and delivery conditions, and directly assess design artifacts, code, and robot performance. Comparisons across courses and institutions could then test whether the proposed changes improve learning or the student experience.

For engineering-education researchers and teaching-development units, the study partially addresses the gap identified in the literature review: limited evidence on students’ perceptions of a sustained progression from guided hardware–software activities to integrative project work across successive cohorts. It complements reviews of robotics and project-based learning (Spolaôr & Benitti, 2017; Guo et al., 2020) with a six-year description of one course, including a three-year period with records from both semesters. Its contribution is to identify recurring implementation concerns and show how comments qualify the interpretation of aggregate ratings. This provides specific priorities for prospective research on alignment, support, and workload, while the effectiveness of the instructional progression remains an open question.

6. Conclusions

This repeated-cross-sectional mixed-methods study describes student perceptions recorded across six academic years. Descriptive patterns in the archived code assignments included practical activity and instructor support alongside workload and organizational concerns; their completeness and independent coding reliability were not established. Annual scores varied, but no adjusted pairwise year comparison was significant. The common-period semester comparison did not establish a difference; the full-period contrast was temporally unbalanced and pooled changing item wordings. Only lecture–laboratory alignment differed in both full- and common-period theme comparisons after Holm adjustment. Original-score tests did not detect theme-related differences, with uncertainty allowing effects in either direction. These patterns identify priorities for reviewing guidance, alignment, resources, and workload. They do not demonstrate learning gains, development of autonomy, or an independent effect of the competition.

Returning to the literature review, the reported value of practical activity is consistent with the experiential and project-based learning rationale described by D. A. Kolb (1984) and Krajcik and Blumenfeld (2006). The recurring attention to instructor support and clarity likewise accords with the emphasis on scaffolding in complex tasks (Hmelo-Silver et al., 2007; Reiser, 2004). These findings support the relevance of these perspectives to students’ reported experiences; they neither confirm their predicted learning benefits nor negate them, because learning processes and outcomes were not measured. More directly, the coexistence of workload concerns with a broad range of overall scores reinforces the value of written comments as a complement to ratings (Nasser-Abu Alhija & Fresko, 2009; Brockx et al., 2012). Consistent with Spooren et al. (2013) and Uttl et al. (2017), the ratings cannot establish educational effectiveness.

The study therefore contributes contextual evidence about how a sustained instructional progression is perceived across successive cohorts, rather than a test of the superiority of autonomous or competitive projects. For instructors and programme leaders, its practical contribution is a set of priorities for reviewing alignment, workload, resources, and project support. For researchers, it identifies issues to examine using stable measures, linked records, and direct assessments of design reasoning and performance. Students may benefit if such reviews lead to better-supported project experiences, but those benefits require prospective evaluation.

7. Limitations

The study concerns one introductory course in one department at one institution. Without contemporaneous comparison courses, course-specific patterns cannot be separated from institutional or contextual changes. Record availability differed by semester and year, and 2020 had only three records. Although the instructional schedule and Semester 2 team size are described in Section 3.2, year-specific delivery histories, team-formation rules, numbers of project teams, and survey dates relative to each competition were unavailable. These limitations prevent attributing annual or semester differences to COVID-19, other disruptions, or particular instructional changes.

Group means were equally weighted because enrollment and respondent counts were unavailable. Unique student totals, response rates, and possible overlapping respondents across records could not be established. Nominal tests treat records as independent; unmeasured clustering can affect p-values and intervals. The data do not track individual development, and group-level theme–rating associations cannot be transferred to students. Voluntary comments may also be affected by self-selection (Nasser-Abu Alhija & Fresko, 2009).

Semester comparisons combine differences in technical complexity, project scope, teamwork, instructional exposure, and competition. Common-period analyses reduce unequal year coverage without isolating these components. Modest record counts and wide effect intervals limit precision; non-significance does not establish equivalence. Theme presence pools response-field orientations, and no recorded theme may reflect silence rather than absence of the experience.

Overall-item wording changed across periods, and measurement invariance was not established. The related-item comparisons therefore describe archived scores rather than a validated common construct. One score conflicted between source fields; substitution and exclusion checks did not alter the principal interpretation but did not resolve the source error. The coded archive also lacked independent ratings and documented procedures for code development and disagreement resolution. Reproducible counts, lexical evidence, and verified quotations cannot substitute for intercoder reliability or comprehensive contextual validation.

The course was intended for the first year, but some comments refer to second-year study. With no respondent-level year-of-study data, the findings concern the course’s available evaluation records rather than a verified exclusively first-year population.

Finally, the institutional instrument was designed for routine evaluation, not to measure autonomy, scaffolding, hardware–software integration, or engineering competence. Ratings and comments are evidence of perceptions rather than direct learning outcomes (Spooren et al., 2013; Uttl et al., 2017). Prospective evaluation should combine aligned perception measures with assessments of design reasoning, artifacts, code quality, and system performance.