Enhancing Engagement in Learners with Special Educational Needs Through Physical and Immersive Musical Toys: A Mixed-Methods Comparative Study Protocol

Abstract

Musical play offers learners with Special Educational Needs (SEN) an immediate way to act on, perceive, and reorganize a learning environment through sound. This study protocol proposes a randomized, counterbalanced, within-participant mixed-methods comparison of two functionally matched forms of musical interaction: physical/tangible musical toys and virtual/immersive musical environments controlled through touch, gesture, or movement. The study is diagnosis-inclusive and is intended for school-aged learners with documented SEN across heterogeneous neurodevelopmental, sensory, motor, communication, and combined support profiles; eligibility is defined by functional access rather than diagnosis alone. Engagement is treated as a multidimensional process encompassing behavioral participation and persistence, emotional involvement, and cognitive-agentic investment in exploration, choice, and task completion. The methodological novelty lies in matching the two conditions at the level of musical function and adult mediation while translating both tangible interactions and digital traces into a common event ontology. This design reduces the measurement asymmetry that often favors digitally instrumented environments and enables within-learner comparison across media. Mixed-effects models will estimate the primary condition contrast, whereas access-profile moderation will be treated as exploratory. Educator ratings and qualitative Framework Method analysis will contextualize individual response patterns. No empirical results are reported at the protocol stage. The protocol is intended to support the collection of pedagogically meaningful and computationally traceable evidence for the design of accessible serious games, digital musical instruments, and hybrid physical–digital learning experiences in inclusive education.

1. Introduction

A learner reaches for a drum, a resonant object, or a luminous digital surface and waits for the environment to respond. In that brief interval, engagement is already more than attention: the learner initiates an action, anticipates a sensory consequence, experiences an affective response, and decides whether to continue, vary the action, invite another person, or withdraw. For learners with SEN, these moments are especially informative because engagement cannot be reduced to compliance, stillness, or time spent in front of a device. It emerges from the relationship among the learner, the task, the material or digital medium, adult mediation, and the possibilities for acting in ways that remain intelligible and personally meaningful. Recent work therefore distinguishes behavioral, emotional, cognitive, and agentic forms of engagement, while evidence from SEN populations indicates that participation conditions and relationships shape engagement alongside learner characteristics [1,2].

Music offers an unusually dense setting in which these dimensions can be observed together. Producing a sound requires an action; sustaining a musical exchange requires persistence and temporal coordination; variation requires discrimination and decision-making; shared music invites mutual attention and turn-taking; and the affective qualities of sound can render success, surprise, anticipation, and frustration immediately perceptible. Within inclusive education, musical activity has therefore been approached not merely as curricular content but as a participation environment in which technological mediation can widen the range of feasible actions and expressive forms [3]. A recent systematic review of music lessons for pupils with SEN nevertheless shows that pedagogical support remains unevenly organized and that research still provides limited guidance on how support practices can be matched to heterogeneous learner needs [4]. This methodological deficit matters because a technology may be technically accessible while still failing to produce educationally meaningful participation. The educational value of the interaction is generated by the fit among learner, affordance, task, and mediation, a view consistent with pedagogical accounts that treat inclusion as adaptive regulation rather than as the simple addition of assistive resources [5].

The protocol compares two action-feedback architectures: physical/tangible musical toys and virtual/immersive musical environments. Here, the term musical toy denotes an interactive object or environment whose musical rules can be explored through play while retaining an explicit educational purpose. Tangible systems rely on manipulable objects and physical interfaces; virtual or immersive systems map touch, gesture, or bodily movement onto programmable musical feedback. The comparison therefore does not position digital media as a technological successor to physical media. It asks whether functionally matched action-feedback architectures are associated with different engagement profiles and whether those profiles vary according to learners’ access needs.

1.1. Engagement as a Multidimensional and Relational Outcome

Engagement is frequently operationalized through visible participation, yet observable activity alone is an incomplete indicator of educational involvement. A learner can remain physically active while repeating actions without exploration, can appear behaviorally quiet while listening with sustained cognitive investment, or can complete a task under dense prompting while exercising little agency. A multidimensional model allows these possibilities to be distinguished. Behavioral engagement refers in this protocol to active participation, persistence, initiation, turn-taking, and task completion. Emotional engagement concerns observable enjoyment, approach, frustration, avoidance, and willingness to remain in or return to the activity. Cognitive-agentic engagement concerns purposeful exploration, decision-making, strategy variation, error recovery, self-initiated choice, and attempts to shape the activity rather than only respond to it [1]. The agentic component is retained within the cognitive domain for parsimony in the primary three-domain analysis, while its indicators are also reported separately in secondary analyses.

For learners with SEN, the measurement problem is intensified by heterogeneity in communication, motor control, sensory processing, attention regulation, and familiarity with conventional classroom response formats. Measures that privilege speech, rapid manual responses, or a single normative expression of positive affect can convert access differences into apparent engagement differences. The protocol therefore defines engagement through multiple observable channels and interprets each indicator in relation to the learner’s documented access profile. This relational orientation is also consistent with evidence that teacher–student relationships and opportunities for participation are associated with engagement and disengagement among learners with SEN [2]. Consequently, educator mediation is treated as a standardized component of the experimental situation and as an interpretive variable, rather than as background noise to be ignored.

1.2. Musical Interaction, Embodiment, and Multisensory Access

Musical interaction is particularly suited to an embodied analysis because sound unfolds as a consequence of action while simultaneously reorganizing subsequent action. Rhythm invites temporal prediction; changes in intensity or timbre make the effects of force and movement perceptible; repeated patterns allow anticipation; and shared pulse can support interpersonal coordination. Recent empirical work with children who have special needs has associated embodied musical engagement with attention control, emotion regulation, persistence, and cooperative behavior [6], while studies in autism have linked active rhythm engagement to social interaction and expressive communication pathways [7]. Randomized and feasibility studies likewise indicate that structured music-based activity can support engagement, initiation, attention to communication, and language-related outcomes for some autistic learners [8,9], and a recent meta-analysis reports positive effects of music therapy on communication and social skills, while also documenting substantial heterogeneity across interventions [10]. These findings do not establish that any musical technology will produce the same effects; rather, they justify closer examination of the interaction mechanisms through which music becomes accessible and motivating.

Evidence from the broader creative-arts literature provides an additional basis for examining musical interaction in inclusive settings. Recent systematic reviews of art therapy and music therapy with children who have autism, ADHD, language disorders, and learning disabilities report potential benefits across social, communicative, cognitive, emotional, and behavioral domains, while also emphasizing substantial heterogeneity in intervention goals, methods, and outcome measures [11]. Reviews focused on music therapy and social communication in autistic children similarly identify promising relational and communicative outcomes [12], whereas art-therapy research highlights the value of multimodal, expressive, and non-verbal routes to participation alongside the need for more rigorous and clearly specified designs [13]. These findings support attention to agency, sensory fit, expression, and relational participation, but they do not imply that the present educational protocol constitutes therapy or that therapeutic effects can be assumed. The present study instead draws on these studies to strengthen the rationale for comparing accessible forms of creative musical interaction under controlled educational conditions.

The multisensory character of musical play is educationally relevant because auditory information can be coupled with tactile, visual, proprioceptive, and spatial feedback. This coupling can create redundant or complementary channels, but it can also increase sensory load. Accessible design therefore requires more than maximizing stimulation. It requires controllable intensity, predictable contingencies, clear mappings between action and consequence, opportunities for withdrawal, and the possibility of adapting the interface to motor and sensory preferences. Recent reviews of natural and digital musical instruments for autistic children similarly emphasize both the expressive potential of technology and the need to account for individual sensory and interaction profiles [14]. The proposed comparison treats sensory controllability and action-feedback intelligibility as measurable affordance properties rather than as assumed benefits of one medium.

1.3. Tangible Musical Toys: Materiality, Direct Manipulation, and Shared Visibility

Physical musical toys provide information through material properties such as weight, resistance, texture, vibration, and spatial position. When action and sound are tightly coupled, learners can vary force, timing, location, or movement and immediately hear or feel the consequence. These qualities can make action-feedback relations concrete and socially visible. However, materiality is not inherently inclusive: conventional instruments may presuppose particular ranges of motion, grip, timing precision, bilateral coordination, or strength. The relevant question is therefore whether the physical affordances of a device fit the learner’s access profile.

Work on Accessible Digital Musical Instruments (ADMIs) demonstrates why the physical–digital distinction should not be treated as binary. Many ADMIs are tangible interfaces whose physical surfaces are digitally mapped, allowing the form of the gesture and the resulting sound to be decoupled and reconfigured. A social-ecological design framework developed in SEN school settings shows that accessibility emerges from the interaction among instrument design, learners, educators, institutional routines, and musical goals [15]. The I-Ork project similarly situates accessible instrument design within music technology, human–computer interaction, music education, therapy, and community music, underscoring the value of adaptable mappings and collaborative participation [16]. Participatory design research with autistic children further indicates that collaborative accessible instruments can be developed around movement and shared musical action when teacher knowledge and learner experience inform design choices [17]. These studies motivate the protocol’s emphasis on functional affordances rather than on hardware category alone.

1.4. Virtual and Immersive Musical Toys: Programmability, Adaptive Mapping, and Traceability

Virtual and immersive musical environments place part of the interaction logic in a programmable mapping layer. Touch, gesture, or body movement can be translated into pitch, timbre, spatialized sound, visual animation, or haptic response, while sensitivity and task complexity can be adjusted without requiring the same motor action from every learner. Prior systems have used elastic displays, whole-body sound mapping, and combined movement–sound–visual traces to support playful and collaborative interaction with autistic people and learners with complex needs [18,19,20,21]. Their value lies not in immersion itself but in the possibility of adapting mappings while preserving a recognizable musical task.

Digital systems also provide native time-stamped traces of interaction. Such logs can characterize initiation latency, action frequency, temporal spacing, choice sequences, navigation, and response to prompts. This analytic richness can create an unfair comparison if the tangible condition is measured less precisely. To reduce that asymmetry, the protocol represents both conditions through a common event ontology, with video-coded tangible events translated into the same action classes used for digital logs. Computational traceability is therefore treated as a methodological property of the study rather than as evidence of greater educational value in the digital condition.

1.5. Inclusive Multimedia Design: Agency, Accessibility, and Educator Mediation

Inclusive multimedia design should not equate access with successful input registration. An interface may detect a gesture reliably while leaving the learner little meaningful control over musical structure; conversely, a slower or less precise action can represent strong engagement if it reflects deliberate choice, experimentation, or social negotiation. Disability-oriented analyses of musical interfaces have shown that design assumptions can either reproduce normative expectations about performance or enable alternative forms of musical authorship [22]. Research on game-based learning for learners with disabilities likewise emphasizes the joint role of technology, rules, community, division of labor, and target outcomes, arguing against interpretations that isolate the digital artifact from its activity system [23]. Reviews of artificial intelligence and virtual reality in inclusive education further indicate that personalization and immersive access must be evaluated alongside usability, accessibility, teacher readiness, and the possibility of new forms of exclusion [24,25].

For this reason, the protocol treats agency as a design and measurement principle. Agency is operationalized through self-initiated actions, meaningful choice, successful modification of the environment, voluntary continuation, and the capacity to influence the pace or direction of the activity. Educator behavior is standardized through a prompt hierarchy so that differences in adult assistance do not masquerade as differences between media. At the same time, educator observations are preserved qualitatively because educators possess contextual knowledge about communication modes, fatigue, sensory regulation, and atypical expressions of preference that may not be inferable from logs alone. This dual treatment of mediation—as both a controlled component and a source of interpretive evidence—reflects inclusive pedagogical perspectives in which teaching action is adaptive, situated, and oriented toward making participation possible through proportionate regulation rather than through uniform demands [26,27]. Virtual environments can be especially useful when they allow perspective, movement, and response conditions to be manipulated systematically, as shown in recent work using a virtual classroom videogame to study perspective-taking in children [28].

Figure 1 summarizes the conceptual model used in the study. The two media conditions are first matched at the level of musical goal, choice structure, duration, and adult prompting. Their different affordances are then expected to influence proximal sensorimotor, relational, and access processes, which in turn may be expressed through behavioral, emotional, and cognitive-agentic engagement. Learner access profile and educator mediation are treated as moderators across the interaction cycle.

Figure 1. Affordance-to-engagement framework guiding the matched comparison of tangible and immersive musical play. The model does not assume that either condition is globally superior; it specifies candidate processes through which different interaction architectures may support or constrain engagement.

1.6. Research Gap, Objectives, and Research Questions

Existing research provides strong reasons to study musical technologies in inclusive education, but the evidence base remains difficult to compare across systems. Studies often examine one instrument, one platform, one diagnostic group, or one therapeutic goal, and they use engagement as a broad descriptor rather than as a multidimensional outcome. Moreover, physical and digital musical tools are rarely compared under equivalent task rules within the same learners. This makes it difficult to distinguish a medium effect from novelty, task difficulty, adult support, musical content, or differences in measurement density. The present study is designed to isolate the interaction medium more carefully by holding the musical task architecture constant while varying the physical versus virtual/immersive mode of action and feedback.

The primary objective is to estimate within-learner differences in behavioral, emotional, and cognitive-agentic engagement across matched tangible and immersive musical activities. A second objective is to examine whether specific affordance properties and functional access characteristics help explain heterogeneous responses. A third objective is methodological: to test a common event representation that combines systematic observation and digital logs without treating one data source as inherently more valid than the other. The confirmatory component is limited to the primary within-learner condition contrast and the pre-specified secondary outcome families. Analyses involving individual access profiles are exploratory because the planned sample is not powered for a large set of condition-by-moderator interactions. This distinction will be preserved in preregistration and reporting.

The study addresses five research questions: (RQ1) How does behavioral engagement differ between tangible and immersive musical play when musical goals, choice structure, duration, and educator prompting are matched? (RQ2) How does emotional engagement differ between conditions, particularly in enjoyment, frustration/avoidance, recovery, and willingness to continue? (RQ3) How does cognitive-agentic engagement differ in purposeful exploration, decision-making, strategy variation, error recovery, and self-initiated choice? (RQ4, exploratory) Do a limited set of functional access variables—sensory accommodation needs, motor access method, and prior digital/immersive familiarity—show condition-by-profile patterns that warrant further study? Communication mode and prior musical experience will be described and may be examined only in sensitivity or hypothesis-generating analyses when cell support is adequate. (RQ5) How do educator observations and event-level interaction traces explain cases in which quantitative engagement indicators converge or diverge? No directional superiority hypothesis is imposed; the principal expectation is heterogeneity across affordance configurations and learners (Table 1).

Table 1. Theoretical comparison of the two musical interaction conditions.

Analytical Dimension	Physical/Tangible Musical Toys	Virtual/Immersive Musical Environments	Comparative Implication
Action-feedback coupling	Direct material manipulation; acoustic and tactile consequence may be physically co-located.	Programmable mapping of touch, gesture, or movement to sound and visual/haptic feedback.	Latency and mapping transparency must be documented in both conditions.
Embodiment	Weight, resistance, texture, vibration, reach, and spatial manipulation.	Body movement can be amplified, filtered, or remapped through sensing and software.	Motor demand should be matched to the learner rather than standardized only by device.
Agency	Choice may be constrained by physical layout and available objects.	Choice sets, sensitivity, and response rules can be dynamically personalized.	Agency is measured through effective control, not merely number of options.
Social visibility	Actions and objects are directly visible in a shared space.	Actions may be distributed across screen, sensor field, avatar, or projected space.	Turn-taking and shared attention are coded explicitly.
Measurement	Requires structured observation and, where available, sensor instrumentation.	Produces native time-stamped logs in addition to observation.	Both are translated into a common event ontology.
Accessibility risk	Mechanical effort, reach, grip, or conventional performance assumptions.	Visual complexity, tracking failure, latency, sensory load, interface abstraction.	Barriers are logged as interaction events rather than attributed to learner deficit.

2. Materials and Methods

2.1. Study Design and Comparative Logic

The study is designed as a randomized, counterbalanced, within-participant crossover protocol with an embedded qualitative strand. Each learner experiences both the tangible condition (A) and the immersive condition (B). Participants are assigned to sequence AB or BA using computer-generated block randomization with balanced allocation (Supplementary Code S1). The within-participant structure is preferred because SEN populations are heterogeneous and between-group comparisons can be dominated by stable differences in communication, motor access, sensory profile, prior musical experience, or support needs. Each learner therefore serves as their own primary comparator, while sequence and period effects are modeled explicitly.

The quantitative strand estimates condition effects on pre-specified engagement outcomes. The qualitative strand examines interaction episodes, educator observations, access barriers, and learner-specific responses that help interpret the quantitative pattern. Integration occurs at the participant-by-condition level through joint displays rather than by treating qualitative notes as anecdotal supplements. The design is comparative but not evaluative in the sense of selecting a universal winner (Figure 2). Its target is conditional fit: which medium-affordance configuration is associated with which form of engagement, for which learner, under which access conditions.

Figure 2. Planned randomized counterbalanced crossover sequence. AB and BA orders distribute novelty and period effects across conditions; the same outcome framework is applied after each condition.

2.2. Setting, Participants, and Eligibility

The intended sample comprises school-aged learners with documented SEN enrolled in inclusive or specialized educational provision. Recruitment is planned through cooperating educational settings. The protocol is diagnosis-inclusive: participants may present neurodevelopmental, sensory, motor, communication, or combined educational support needs, but eligibility is determined by functional access rather than by diagnostic category. Inclusion criteria are: (a) current enrollment in a participating educational setting; (b) formal documentation of SEN or equivalent educational support eligibility; (c) capacity to participate in at least one supported cause-and-effect musical interaction and to indicate assent, preference, continuation, or withdrawal through speech, augmentative communication, gesture, gaze, body movement, or another individually recognized response; and (d) availability of a parent/legal guardian and an educator who can provide contextual information. Exclusion criteria are limited to temporary conditions that make participation unsafe on the scheduled day, sensory conditions that preclude access to both planned forms of feedback even after reasonable adaptation, or clinical restrictions that contraindicate the movement or sensory exposure required by the protocol.

The sample will be characterized descriptively by age, sex, educational setting, SEN category where ethically and educationally relevant, communication mode, mobility/access method, sensory accommodations, prior musical experience, prior exposure to digital or immersive systems, and level of routine educational support. Diagnostic labels will not be used as explanatory shortcuts. Wherever subgroup analyses are feasible, they will be framed around functional access variables, because two learners sharing a diagnostic label may require very different mappings, sensory intensities, or prompting strategies.

2.3. Sample-Size Planning and Sensitivity

The primary inferential contrast is the within-participant condition effect. Because directly comparable estimates for matched tangible-versus-immersive musical interaction in heterogeneous SEN samples are not available, the initial calculation uses a pragmatic small-to-moderate standardized within-participant effect of d_z = 0.45. This value is a planning assumption rather than an expected effect or a minimally important difference. Under a two-sided alpha of 0.05, d_z = 0.45 requires 54 complete participant pairs for 90% power in a paired-condition approximation. Allowing approximately 15% incomplete data due to absence, fatigue, withdrawal, unusable video, or technical failure yields an initial recruitment target of 64 participants. This calculation supports the primary within-learner contrast only; it does not provide adequate power for numerous moderator interactions. RQ4 is therefore explicitly exploratory, and nonsignificant interaction terms will not be interpreted as evidence of absence of heterogeneity. Figure 3 illustrates the paired-approximation sensitivity curve underlying this provisional sample-size calculation.

Figure 3. Sample-size sensitivity guide based on a two-sided paired-condition approximation. The final power statement will be updated using simulation from the mixed-effects model after protocol calibration; the figure is intended as a transparent planning reference rather than as a substitute for model-based power analysis.

Before confirmatory recruitment is finalized, a calibration phase will inform a simulation-based power analysis for the planned mixed-effects model. The simulation will examine plausible within-participant correlations (0.30–0.70), alternative distributions of the active-engagement proportion, boundary mass at 0 or 1 (0–20%), between-participant random-intercept variability, and incomplete-session rates of approximately 10–20%. The provisional recruitment target will remain 64 if at least 80% simulated power is retained for the primary condition contrast across the preregistered core scenarios. If power falls below 80%, the target will increase in increments of four participants, up to a provisional ceiling of 80. The final rule, parameter values estimated from calibration, and recruitment ceiling will be frozen before confirmatory data collection. The calibration sample will not contribute to the primary confirmatory analysis unless this is specified in a preregistered amendment. Moderator analyses will not be used to increase the sample adaptively after outcomes have been inspected.

2.4. Experimental Conditions and Functional Matching

Both conditions will implement the same four musical functions: (1) cause-and-effect sound triggering; (2) controlled variation of at least one musical parameter such as pitch, timbre, intensity, or tempo; (3) construction or continuation of a short rhythmic or sonic sequence; and (4) a shared or turn-based musical exchange with an educator or peer when appropriate. The tangible condition will use a small set of manipulable sound-producing devices representing at least three access modes: strike-based interaction, shake/rotation-based interaction, and pressure/touch-based interaction. The immersive condition will provide equivalent musical functions through touch and/or camera-based gesture or body-movement tracking connected to a programmable sound engine. At the present protocol stage, final commercial models and software versions have not yet been frozen. Their exact names, versions, input sensors, output devices, audio chain, measured action-to-feedback latency, and accessibility settings will be entered in the technical specification and condition-matching audit (Supplementary Table S1) before preregistration and before any confirmatory data collection. This dependency is now made explicit so that equipment selection cannot be altered after outcome data are examined.

Functional equivalence is prioritized over visual similarity. A physical shaker and a gesture-controlled digital object need not look alike, but they must instantiate comparable opportunities to initiate sound, vary a parameter, repeat an action, stop, and make a choice. Before participant recruitment, the technical matching audit in Supplementary Table S1 will verify the same number of task phases, comparable choice cardinality, equivalent nominal duration, the same musical material where feasible, the same adult prompt hierarchy, and a preregistered feedback-latency tolerance. When a learner requires an access adaptation, the aim is not to impose identical motor gestures across conditions but to preserve the same educational function through the least restrictive access route (Table 2).

Table 2. Condition-matching matrix to be completed and frozen before preregistration.

Feature	Tangible Condition	Immersive Condition	Matching Rule
Session duration	30 min planned	30 min planned	Equal nominal duration; learner-led early stop permitted.
Familiarization	5 min	5 min	Same explanation and prompt hierarchy.
Free exploration	8 min	8 min	Same number of available musical functions.
Goal-guided tasks	12 min	12 min	Same musical goal, choice cardinality, and success criterion.
Shared/turn-based play	5 min	5 min	Same partner role and turn structure.
Feedback latency	Measured at device level	Measured at system level	Report median and 95th percentile; keep below preregistered tolerance.
Adult prompting	Five-level prompt hierarchy	Five-level prompt hierarchy	Prompt level time-stamped/coded in both conditions.
Data capture	Video + coded events (+ sensors if available)	Native logs + video + coded events	Common event ontology and synchronized clock.

2.5. Session Procedure

Each participant will first complete a baseline access-profile session with an educator and researcher. This session will document preferred communication, motor access, relevant sensory accommodations, previous experience with musical or interactive technologies, and any individualized stop signals. A brief familiarization with both media will occur before randomization so that the main contrast is not simply first contact with one device. No performance criterion is required during familiarization; the purpose is to establish a minimal understanding that an action can produce a musical consequence.

Each experimental session is planned for approximately 30 min and follows the same temporal structure: 5 min of supported familiarization/reorientation, 8 min of learner-led free exploration, 12 min of goal-guided musical tasks, and 5 min of shared or turn-based play. A learner may pause or stop at any time. The educator follows a five-level prompt hierarchy: (0) no prompt, (1) environmental cue, (2) gestural/model prompt, (3) verbal or symbolic prompt, and (4) individualized physical or access support already used in the learner’s educational plan. Prompts are delivered only after a preregistered response interval unless safety or distress requires immediate intervention. The interval between the two condition blocks will be at least one school day. Where feasible, sessions will occur at similar times of day and in the same room to limit context-related variation.

2.6. Engagement Outcomes and Operational Definitions

Engagement is operationalized through a multi-source measurement matrix. No single indicator is treated as the construct itself. The behavioral domain combines active interaction time, self-initiated actions, persistence after unsuccessful attempts, task completion, turn-taking, and prompt dependence. The emotional domain combines observable positive affect, approach behavior, frustration/avoidance, recovery after difficulty, and willingness to continue or repeat the activity. The cognitive-agentic domain combines purposeful variation, choice differentiation, rule discovery, strategy change, error recovery, independent task sequencing, and attempts to modify the activity. Because affect can be expressed idiosyncratically, educator interpretation is recorded separately and does not overwrite the primary observational code.

The primary outcome will be the proportion of scorable session time classified as actively engaged according to the behavioral coding protocol. This outcome was selected because it is time-normalized, observable across diverse communication modes, available in both conditions, and relatively low in interpretive inference, making it suitable for the principal within-learner comparison. It is not treated as a complete representation of engagement. Emotional and cognitive-agentic domains remain essential secondary outcome families, and the mixed-methods integration is designed to identify cases in which active interaction time diverges from affective quality, agency, or purposeful exploration. Domain composites will be created only if reliability and dimensionality are acceptable in calibration data; otherwise, pre-specified indicators will be analyzed separately with multiplicity control. Learner refusal, withdrawal, self-regulation breaks, and sensory avoidance will be coded as meaningful events rather than automatically categorized as negative engagement because voluntary stopping may itself express agency (Table 3).

Table 3. Planned engagement measurement matrix.

Domain	Indicator	Operationalization	Primary Data Source
Behavioral	Active interaction time	Proportion of scorable time in task-relevant manipulation, exploration, listening-with-orienting, or shared musical action.	Video coding; digital task state/logs.
Behavioral	Initiation and persistence	Latency to first self-initiated action; number and duration of sustained action sequences; return after unsuccessful action.	Common event ontology.
Behavioral	Prompt dependence	Highest and mean prompt level; proportion of actions following prompts versus self-initiation.	Observer codes + facilitator event marker.
Emotional	Approach/enjoyment	Observable approach, positive affect, voluntary continuation, request to repeat.	Video coding + educator rating.
Emotional	Frustration/avoidance	Observable distress, withdrawal, repeated rejection, dysregulation; recovery time coded separately.	Video coding + educator contextual note.
Cognitive-agentic	Purposeful exploration	Systematic variation in action or parameter rather than undifferentiated repetition.	Video/log sequence analysis.
Cognitive-agentic	Choice and control	Distinct choices, self-initiated changes, successful environment modification, voluntary stopping.	Common event ontology.
Cognitive-agentic	Strategy and error recovery	Change in action after mismatch/failure; successful completion after strategy revision.	Video/log event sequence.
Relational secondary	Turn-taking/shared attention	Reciprocal alternation, partner-oriented action, coordinated start/stop or shared sonic goal.	Video coding; facilitator event marker.

2.7. Common Event Ontology and Multimodal Data Architecture

A central methodological component is a common event ontology that represents interactions from both conditions in the same analytic language (Supplementary Table S3). Every coded or logged event will contain, at minimum, participant pseudonym, condition, session, timestamp, task state, action class, feedback class, prompt level, success/mismatch flag where applicable, and annotation confidence. Action classes will include initiate, repeat, vary, choose, stop, request/help, partner-oriented action, and recovery/change strategy. Feedback classes will include auditory, visual, haptic/tactile, combined, delayed/failed, and educator-mediated feedback. The ontology will be frozen before confirmatory analysis and published with the analysis code.

Digital events will be exported directly from the immersive system using a synchronized clock. Tangible events will be annotated from video with event-marking software and, where feasible, supplemented by simple sensors. Synchronization checks will be performed at the beginning and end of each session using a visible/audible marker. Derived features will include event rates, inter-event intervals, initiation latency, exploration breadth, sequence depth, prompt-adjusted autonomy, and choice diversity. Exploration entropy may be estimated for participants with sufficient event counts, but it will remain a secondary descriptive feature because high entropy can reflect productive exploration or disorganized responding depending on task context. Multimodal learning analytics research has repeatedly cautioned that data fusion is informative only when modalities are theoretically aligned and temporally synchronized [29,30,31]. The protocol therefore prioritizes interpretable event features over opaque high-dimensional prediction (Figure 4).

Figure 4. Planned synchronized data architecture. Tangible observations and immersive digital logs are transformed into a shared event representation before feature derivation, statistical modeling, and qualitative integration.

2.8. Observation Coding, Training, and Reliability

A structured coding manual will define each engagement event, boundary rule, ambiguous case, and condition-independent example. Coders will complete training on recordings not included in the confirmatory dataset, followed by calibration until acceptable agreement is reached. At least 25% of sessions, stratified by condition and participant support level, will be independently double-coded. Continuous and proportion outcomes will be evaluated using intraclass correlation coefficients with an a priori target of at least 0.75 for confirmatory use; ordinal codes will use weighted kappa, and event occurrence will additionally be examined through tolerance-window matching when timestamp agreement matters. If reliability falls below the threshold, the code definition will be revised and affected material recoded before unblinding the primary condition comparison.

Complete blinding to condition is impossible because the physical form of the interface is visible on video. Bias will therefore be reduced through other means: coders will not be informed of the study’s expected direction, condition labels will be replaced with neutral codes during annotation, the primary analysis script will be written before decoding the condition key where feasible, and automated log-derived features will be computed from pre-specified functions.

2.9. Educator Measures and Qualitative Strand

Immediately after each session, the educator will complete a brief researcher-developed contextual rating form covering overall engagement, sensory fit, motor access, communication opportunity, autonomy/choice, recovery from difficulty, and independence from adult assistance. Each item will use a five-point behaviorally anchored response scale. The form is not presented as a validated psychometric engagement scale and will not replace the primary observational outcomes; it is intended to provide structured contextual information from an adult who knows the learner. The planned items, anchors, and two symmetrical open prompts—one episode in which the system supported access and one in which it constrained access—are provided in Supplementary Table S2. This structure is intended to reduce technology-positive reporting bias and to make the educator contribution auditable rather than anecdotal.

The qualitative strand will use the Framework Method [32], combining a deductive starting framework with inductive category development. A purposive subsample of approximately 12–16 cases will be selected after quantitative quality checks and before final interpretation. Selection will seek representation of five explanatory patterns: consistently high engagement, consistently low engagement, strong tangible preference, strong immersive preference, and divergence between quantitative indicators and educator ratings. Sampling will be considered adequate when each major pre-specified pattern is represented by at least two information-rich cases and two consecutive additional cases fail to add a substantively new explanatory category, subject to expansion within the 16-case ceiling when a negative case requires it. Two researchers will independently familiarize themselves with the selected material, develop and refine the analytic framework, chart data into a participant-by-domain matrix, and maintain reflexive memos and an audit trail. Disagreements will be resolved through documented analytic discussion rather than reduced to a single agreement coefficient. Negative and contradictory cases will be used to revise, qualify, or reject emerging interpretations. The initial framework and charting structure are provided in Supplementary Table S4.

2.10. Statistical Analysis Plan

Analyses will be conducted in R (version to be frozen at preregistration) and, where -appropriate, reproduced in an open analysis notebook. The primary outcome—proportion of active engagement time—will be modeled using a mixed-effects framework appropriate to its distribution. If values are sufficiently interior to (0,1), beta mixed regression will be preferred; if a substantial mass occurs at 0 or 1, a zero/one-inflated or binomial-time formulation will be considered according to the preregistered decision rule. The principal fixed effect is condition (tangible versus immersive). Period, randomized order, session block, and measured task duration will be included as design covariates. Participant will be modeled as a random intercept; educational site/class will be added as a random effect if the number of clusters permits stable estimation. The primary estimand is the average within-learner condition contrast with a 95% confidence interval.

Count outcomes such as self-initiated actions or strategy changes will use Poisson or negative-binomial mixed models based on dispersion diagnostics. Binary outcomes will use logistic mixed models. Approximately continuous ratings or standardized composites will use linear mixed models with residual diagnostics and robust alternatives if assumptions are materially violated. Confirmatory inference will focus on the primary condition effect and the pre-specified secondary outcome families. RQ4 moderation will be exploratory and restricted to sensory accommodation needs, motor access method, and prior digital/immersive familiarity, with interaction estimates and uncertainty intervals emphasized rather than binary significance claims. Communication mode and prior musical experience will be retained as descriptive variables and may enter sensitivity analyses only when adequate cell support and a preregistered rationale are present.

Multiplicity will be controlled hierarchically. The primary outcome will be tested at two-sided alpha = 0.05. Secondary indicators within behavioral, emotional, and cognitive-agentic families will be adjusted using a false-discovery-rate procedure, while effect sizes and confidence intervals will be emphasized over dichotomous significance labels. Period-by-condition and order-by-condition terms will be inspected for carryover. If substantial carryover is detected, a sensitivity analysis restricted to the first condition period will be reported, acknowledging the loss of within-participant efficiency. All model specifications, exclusions, transformations, and convergence remedies will be documented (Table 4).

Table 4. Pre-specified analysis family and estimands.

Outcome Type	Example	Planned Model	Primary Interpretation
Proportion	Active engagement time	Beta mixed model or preregistered boundary alternative	Adjusted within-learner condition difference/marginal contrast.
Count	Self-initiated actions; strategy changes	Poisson or negative-binomial mixed model	Condition rate ratio with 95% CI.
Latency/time	Time to first initiation; recovery time	Log-transformed LMM or survival/mixed model if censoring is substantial	Condition difference or time ratio.
Ordinal	Educator ratings; prompt level	Cumulative-link mixed model where feasible	Shift in probability of higher/lower category.
Binary	Task completion; voluntary repeat request	Logistic mixed model	Condition odds ratio and marginal probabilities.
Sequence feature	Choice diversity; exploration entropy	LMM/GLMM after minimum-event threshold	Secondary descriptive/associational contrast.
Moderator	Functional access variable × condition	Interaction in corresponding mixed model	Heterogeneity of within-learner condition effect.

2.11. Missing Data, Protocol Deviations, and Robustness

Every incomplete session will be assigned a reason code: learner withdrawal/refusal, absence, fatigue or health, sensory distress, equipment failure, recording failure, facilitator deviation, or other documented cause (Supplementary Table S5). Mixed-effects models will use available repeated observations under a missing-at-random working assumption, but missingness associated with fatigue, refusal, or sensory difficulty will be treated as plausibly outcome-related rather than assumed to be random. Sensitivity analyses will therefore include reason-stratified complete-case comparisons and pattern-mixture/tipping-point analyses in which the unobserved active-engagement proportion for outcome-related dropouts is shifted from the model-based prediction by prespecified delta values of 0.10 and 0.20 in both directions, bounded to the 0–1 scale. Technical and recording failures will be examined separately. If the substantive interpretation changes across plausible missing-not-at-random scenarios, conclusions will be explicitly qualified. Single-value imputation will not be used for primary outcomes.

Novelty is a particular concern when comparing immersive technology with familiar physical objects. Familiarization, counterbalancing, repeated exposure where scheduling permits, and explicit modeling of session block are therefore built into the design. A secondary analysis will estimate whether the condition contrast changes from first to later exposure. If the interaction attenuates sharply, the study will interpret the initial difference as partly novelty-related rather than as an enduring media effect.

2.12. Mixed-Methods Integration

Quantitative and qualitative evidence will be integrated using participant-level joint displays. For each learner, the display will juxtapose condition-specific quantitative indicators, event-sequence features, educator ratings, access adaptations, and coded qualitative affordance episodes. Integration will classify evidence as convergent, complementary, divergent, or silent. A digital system, for example, may produce more logged actions but lower educator-rated autonomy if many actions follow prompts; conversely, a tangible condition may produce fewer actions but longer sustained sequences and more partner-oriented engagement. These are not measurement contradictions to be averaged away; they are different descriptions of how engagement is organized.

The final interpretation will therefore distinguish intensity from quality, activity from agency, and successful task completion from independent participation. This structure is intended to produce design knowledge that can inform multimedia systems for education rather than a single rank ordering of media.

2.13. Ethics, Accessibility, and Data Governance

The study involves minors and potentially vulnerable participants. This manuscript describes one prospective research protocol only. At the time of this revision, neither ethics approval nor an official determination of exemption has been obtained. No participants have been recruited, no experimental procedures have been conducted, and no study-related data have been collected. Before undertaking any recruitment or research procedures involving learners, the authors will seek a written decision from the competent institutional ethics authority identified through the University of Salerno and, where applicable, the participating educational settings. Recruitment, experimental procedures, and data collection will not begin unless and until the required written ethical approval or formal exemption determination has been obtained. The competent authority, decision date, and reference number will be reported when available. Written informed consent will be obtained from parents or legal guardians, and learner assent will be sought through communication methods accessible to each participant. Assent will be treated as ongoing: withdrawal behavior, an established stop signal, or sustained refusal will pause or terminate the session even when guardian consent has been provided. Participation or withdrawal will have no effect on educational provision.

Sensory safety will be addressed through pre-session profiling, adjustable sound level and visual intensity, removal of unnecessary flashing effects, accessible seating/positioning, planned breaks, and immediate access to the learner’s usual regulation supports. Audio output will remain within safe listening levels. No interface will require a participant to tolerate a stimulus that they reject merely to complete the experimental sequence. Physical assistance, if part of the learner’s routine support, will be documented as a prompt event rather than concealed.

All participant data will be pseudonymized at source. Raw video will be stored on encrypted institutional storage, with the re-identification key held separately and access limited to named members of the research team whose roles require identifiable data. Transfer of video or logs between authorized devices will use encrypted channels; local temporary copies will be deleted after verified transfer. Public release will be limited to de-identified derived event tables, codebooks, analysis scripts, and aggregated outcomes. Raw identifiable video will not be released openly. Digital logs will exclude names, email addresses, precise location, and unnecessary device identifiers. The approved data-management plan will specify the institutional retention period, role-based access schedule, lawful basis for processing, and secure disposal procedure. At the end of the approved retention period, identifiable video and linkage files will be securely deleted in accordance with institutional policy and European data-protection requirements.

2.14. Preregistration, Reproducibility, and Protocol Amendments

At the time of this revision, the study has not yet been preregistered. Preregistration will occur after the required written ethical approval or formal exemption determination and technical calibration, but before confirmatory participant recruitment and outcome data collection. The Open Science Framework record will include the research questions, confirmatory-versus-exploratory analysis distinction, final sample-size rule, randomization procedure, condition-matching audit, primary and secondary outcomes, exclusion rules, coding manual, power simulation, missing-data sensitivity plan, and statistical models. The persistent identifier and public link will be inserted in the manuscript as soon as the record is created. Any subsequent protocol amendment will be timestamped, justified, and clearly distinguished from the original plan in the final report.

The multimedia system description will include enough technical information for reproduction: hardware and sensor specifications, software version, input mapping, output modality, task-state logic, event-log schema, latency measurement procedure, and accessibility parameters. If proprietary components prevent code release, the manuscript will document the functional behavior and interfaces needed for independent replication.

3. Planned Analytical Outputs and Falsifiable Interpretive Logic

Because this manuscript is a study protocol, no empirical results are reported. The study is designed, however, so that several possible outcome patterns carry distinct interpretations. A consistent condition advantage across behavioral, emotional, and cognitive-agentic domains would support a broad media-affordance effect, provided that order and novelty effects are small. A domain-specific advantage—for example, higher exploratory diversity in the immersive condition but longer sustained interaction in the tangible condition—would support the view that media organize engagement differently rather than globally. Strong condition-by-access-profile interactions would indicate that personalization and functional fit explain more variance than the medium category alone. Finally, weak or negligible condition effects accompanied by substantial between-learner heterogeneity would argue against universal claims about physical or immersive superiority and would redirect design attention toward individualized mappings and educator mediation.

Qualitative integration is specifically intended to test interpretations that cannot be inferred safely from event volume. More actions may indicate active exploration, but they may also arise from rapid unsuccessful attempts. Longer duration may indicate persistence, but it may also reflect difficulty leaving a task. Fewer prompts may reflect autonomy, but they may also occur because the educator recognizes a learner’s need for processing time. The joint-display analysis will therefore search for explanatory cases that challenge the most convenient quantitative narrative.

4. Discussion

4.1. Scientific Contribution to Inclusive Multimedia Research

The protocol contributes to inclusive multimedia research by making the interaction architecture, rather than the device label, the unit of comparison. Physical and immersive musical toys are defined through affordances—how action is sensed, mapped, fed back, shared, and adapted—so that the comparison remains meaningful even as specific commercial systems change. This is particularly important in music technology, where tangible interfaces may be digitally mediated and virtual systems may depend on highly embodied movement. Recent accessible-instrument research has already moved toward social–ecological and participatory models of design [15,16,17,22,33,34]. The proposed study extends that orientation into comparative measurement by asking whether matched affordances produce distinct engagement profiles within the same learners.

A second contribution is computational. The common event ontology gives the physical condition a trace representation comparable to native digital logs. This reduces a recurring methodological asymmetry in which digital systems appear analytically richer simply because they record more events. The ontology also makes it possible to connect educational observation with multimedia analytics while retaining interpretable constructs. Rather than using multimodal data because more channels are technologically available, the protocol fuses only channels that correspond to explicit engagement or affordance hypotheses [29,30,31].

A third contribution concerns inclusive pedagogy. The design recognizes that adaptation is not a methodological contaminant to be eliminated. If a learner uses eye gaze in one condition and a large-area touch surface in another, the motor actions differ, but the educational function—making a meaningful musical choice—can remain equivalent. This principle is compatible with inclusive approaches that seek proportional, context-sensitive regulation of learning demands and with perspectives on music, technology, and the learner’s broader life project [26,27,35]. The comparison therefore standardizes purpose and opportunity while allowing access routes to vary.

4.2. Methodological Strengths

The within-participant randomized crossover structure reduces confounding by stable learner differences; counterbalancing distributes first-exposure effects; a common task architecture limits differences in musical goal and adult prompting; the common event ontology prevents the immersive condition from benefiting solely from native logging; and multi-source outcomes reduce reliance on a single expression of engagement. Reliability procedures, explicit missing-data codes, preregistration, and public analysis code are intended to make analytic decisions inspectable.

The protocol also separates access from performance. A tracking failure, unreachable object, sensory overload, or misunderstood mapping is recorded as an interaction-system event before it is interpreted as disengagement. This distinction is especially important when studying learners with disabilities because system failure can otherwise be misclassified as learner failure. Participatory and disability-oriented instrument research has shown the importance of taking such design assumptions seriously [17,22,33].

4.3. Anticipated Limitations and Boundary Conditions

Several limitations are anticipated. First, functional matching cannot make physical and immersive systems identical, and some affordances are inherently medium-specific. The protocol addresses this by reporting those differences explicitly rather than implying perfect equivalence. Second, novelty may favor the immersive condition for some learners and increase anxiety for others; repeated exposure and order modeling can reduce but not remove this effect. Third, an educationally heterogeneous SEN sample improves ecological relevance but limits diagnostic subgroup inference. Functional access moderators are therefore prioritized, although statistical power for interactions may remain limited.

Fourth, engagement is context-sensitive. A condition that supports exploration in a 30 min study session may not support sustained curriculum participation over a semester. The study should therefore be regarded as evidence about proximal interaction and engagement, not as proof of long-term learning effects. Fifth, educator ratings provide valuable contextual knowledge but can be influenced by expectations about technology or familiarity with the learner. The protocol mitigates this through behaviorally anchored ratings, symmetrical prompts about benefits and constraints, and comparison with independently coded events. Finally, the planned event ontology is intentionally interpretable and may omit subtle qualities of musical expression. Future studies may extend it with acoustic, movement, physiological, or gaze features when those modalities answer a clear question and can be collected without disproportionate burden.

The protocol also defines several priorities for subsequent research. Longitudinal studies should examine whether engagement profiles observed during brief experimental sessions persist across repeated classroom use, curriculum participation, or a full school term. Replication in more narrowly characterized functional profiles could test whether particular sensory, motor, or communication access patterns show reproducible condition-specific responses. A further line of work should compare hybrid musical technologies that combine tangible action with programmable digital mapping, because such systems may preserve material predictability while extending personalization and traceability. These studies would help determine which findings generalize beyond proximal engagement and which remain specific to short-term interaction conditions.

4.4. Implications for System Design and Educational Practice

If the study reveals heterogeneous engagement profiles rather than a single media advantage, the practical implication will be to design musical systems as configurable repertoires of action-feedback relations. Educators and designers would then select among tangible resistance, large-movement control, touch precision, haptic feedback, visual support, simplified choice sets, or adaptive mappings according to the engagement function being targeted. A learner who benefits from material predictability may require a different configuration from a learner whose motor range is better amplified by motion tracking. Hybrid environments may become especially valuable when they preserve the physical clarity of tangible action while using software to personalize sound mappings and collect interpretable traces.

For educators, the measurement framework offers a way to discuss engagement without reducing it to attention or compliance. A system that increases independent choice but produces fewer total actions may still represent an educational gain; a system that produces excitement but requires continuous prompting may not support autonomy. The proposed behavioral, emotional, and cognitive-agentic matrix encourages decisions based on a profile of participation and on the learner’s capacity to influence the activity.

5. Conclusions

This study protocol proposes a controlled but ecologically informed comparison of physical and immersive musical toys for learners with SEN. No empirical results are reported because the study has not yet begun. The protocol asks not whether digital interaction is globally better than material interaction, but how different action-feedback architectures may organize participation, emotion, exploration, and agency for different learners. By combining a randomized within-participant design, functional task matching, systematic observation, common event coding, native digital logs, educator interpretation, and mixed-effects analysis, the protocol is intended to support the collection of evidence that is both pedagogically meaningful and computationally traceable.

The anticipated value of the study lies in the resolution of its comparison rather than in a predetermined media ranking. Instead of treating engagement as a single score or technology as a uniform intervention, the design examines multiple forms of engagement and the access conditions under which they may emerge. If implemented as preregistered, the protocol can provide a transparent basis for developing and evaluating accessible digital musical instruments, serious games, immersive environments, and hybrid physical–digital experiences in which learner agency and educational purpose remain visible within the multimedia system itself.