Performing-Arts-Based ICH-Driven Interaction Design Framework for Rehabilitation Game

Abstract

The lack of deep engagement strategies that include cultural contextualization in the current rehabilitation game design can result in limited user motivation and low adherence in long-term rehabilitation. Integrating cultural semantics into interactive rehabilitation design offers new opportunities to enhance user engagement and emotional resonance in digital rehabilitation therapy, especially in a deeper way rather than visually. This study introduces a framework comprising a “Rehabilitation Mechanism–Interaction Design–Cultural Feature” triadic mapping model and a structured procedure. Following the framework, a hand function rehabilitation game is designed based on Chinese string puppetry, as well body rehabilitation games based on shadow puppetry and Tai Chi. The hand rehabilitation game utilizes Leap Motion for its gesture-based input and Unity3D for real-time visual feedback and task execution. Functional training gestures such as grasping, wrist rotation, and pinching are mapped to culturally meaningful puppet actions within the game. Through task-oriented engagement and narrative immersion, the design improves cognitive accessibility, emotional motivation, and sustained participation. Evaluations are conducted from rehabilitation professionals and target users. The results demonstrate that the system is promising in integrating motor function training with emotional engagement, validating the feasibility of the proposed triadic mapping framework in rehabilitation game design. This study provides a replicable design strategy for human–computer interaction (HCI) researchers working at the intersection of healthcare, cultural heritage, and interactive media.

1. Introduction

The accelerated aging of the global population has led to a rising demand for rehabilitation, underscoring its growing significance in contemporary healthcare systems. Meanwhile, the advancement of digital health technologies is enabling transformative changes in rehabilitation approaches [1]. Among various digital interventions, rehabilitation games that integrate entertainment, interactivity, and therapeutic functionality demonstrate substantial potential across multiple domains, including motor function recovery, cognitive training, and psychological therapy [2,3,4]. Particular attention has been paid to their efficacy in enhancing patient engagement, alleviating treatment-related anxiety, and improving adherence to rehabilitation protocols [5,6,7]. Accordingly, rehabilitation games are becoming an important complement to conventional therapeutic methods. However, current research on rehabilitation games has primarily focused on human–computer interaction and virtual reality (VR), revealing a clear tendency to emphasize functionality while neglecting cultural dimensions. This absence of cultural consideration is particularly pronounced in rehabilitation contexts and is generally prevalent across the domain of serious games. Numerous studies have demonstrated that the integration of cultural elements into rehabilitation games can significantly enhance product affinity and appeal, addressing the existing shortcomings of “monotonous usage environments and a lack of cultural connotation” and fulfilling patients’ psychological and emotional needs, including “emotional care,” “cultural identity,” and “empathic resonance.” On the one hand, through the attributes of “low-threshold participation and high emotional resonance,” familiar cultural symbols, narratives, and rituals are provided to users, generating resonance with their prior experiences and social context, thereby increasing motivation to engage in rehabilitation activities. On the other hand, cultural elements create richer and more immersive ecological interaction scenarios, transforming repetitive therapeutic tasks into meaningful and enjoyable experiences and stimulating users’ emotional identification and cognitive involvement. For example, traditional folk culture has been applied in the rehabilitation of children with autism, and ethnic culture has been utilized to establish rehabilitation mechanisms for post-disaster populations and to rebuild social connections [8,9]. Although these studies provide preliminary evidence of the potential for integrating culture with rehabilitation, limitations remain in terms of cultural element extraction and the ways in which these elements are embedded into rehabilitation functions. Specifically, cultural components should not merely serve as superficial symbolic decorations but should instead be deeply integrated into rehabilitation mechanisms, ensuring both clinical efficacy and deeper emotional engagement.

As an essential component of traditional Chinese culture, intangible cultural heritage (ICH) offers novel cultural pathways and creative support for the interactive design of rehabilitation games. However, the integration of ICH and rehabilitation games remains at an exploratory stage. Preliminary studies and design practices have attempted to incorporate ICH elements such as Paper Cutting, New Year paintings, dragon dances, and string puppetry into game interfaces or character modeling. This approach aims to enhance cultural recognizability and improve user engagement [10]. Nevertheless, these efforts largely remain at a superficial level of cultural symbolization, failing to establish deeper connections between the cultural logic of ICH and the interactive mechanisms of rehabilitation games. The absence of coherent alignment between cultural context and interactive logic weakens the immersive quality and emotional resonance of the game environment. This disjunction further disrupts the intended synergy between user experience and rehabilitation objectives. Simultaneously, the current research tends to prioritize functional integration between rehabilitation goals and interaction design, focusing on the technical balance between playability and therapeutic effectiveness. However, limited attention has been given to systematically mapping the interrelations among cultural attributes of ICH, rehabilitative demands, and interactive structures from a culturally driven perspective [11,12,13,14,15,16,17,18]. As a result, the deep integration of ICH logic into rehabilitation games continues to suffer from the absence of a clear theoretical framework, as well as a lack of guiding design paradigms and actionable methodologies.

The performing-arts-based intangible cultural heritage (ICH) features highly codified bodily movements, rhythmic patterns, and narrative scenarios. Its embedded bodily practice mechanisms, immersive contexts, and capacity for emotional resonance endow it with intrinsic rehabilitative value. Integrating such elements into rehabilitation games is considered an approach that combines both effectiveness and innovation. Concurrently, with the development of digital technologies, techniques such as motion capture, 3D modeling, and VR can precisely record and capture dynamic processes of bodily movements, dance gestures, and musical performance. These methods not only achieve the preservation of authenticity and the living transmission of heritage [19] but also provide immersive and operational interactive resources for rehabilitation training, thereby promoting physiological recovery while enabling users to gain a more comprehensive and multidimensional understanding of performing-arts-based ICH.

In summary, to address the prevailing issue of “emphasis on functionality over cultural integration” in the interactive design of rehabilitation games, a ternary mapping framework comprising rehabilitation mechanisms, performing-arts-based ICH features, and interaction strategies is proposed. The primary goal of the framework is to improve user experience and engagement, thereby enhancing adherence to rehabilitation programs. Focusing on hand function rehabilitation, core mechanisms are systematically identified, and matched strategies are developed based on representative ICH elements. On this basis, a rehabilitation game featuring string puppet culture and implemented using Leap Motion technology is developed. The user testing and questionnaire results confirm its effectiveness in enhancing user engagement and rehabilitation experience.

2. Related Work

Current research on rehabilitation game interaction design achieves progress mainly in functional implementation, user needs, and system adaptation optimization. However, an overall functional orientation remains dominant, with limited integration of cultural elements, narrative structures, and immersive experiences representing cultural and emotional design dimensions. Although preliminary exploration exists on the integration of rehabilitation games with traditional culture, research on the fusion of ICH with rehabilitation games remains scarce. This situation reveals an urgent need for a paradigm shift from functional orientation to cultural integration, especially toward ICH-oriented design. Corresponding design strategies and interaction models require further systematic investigation.

2.1. Current Status of Interaction Design in Rehabilitation Games

Although the current rehabilitation game interaction design emphasizes functionality, it lacks systematic integration of narrative and immersive experiences. Multiple exploratory pathways have been established, particularly in optimizing user experience and system adaptation. Distinct design trends have gradually emerged within relevant research.

Firstly, it centers on device design with a functional orientation, emphasizing the enhancement of motor rehabilitation through diverse interaction configurations and technological methods. For instance, Ghobadi et al. propose a user-friendly finger–thumb coordination device that enables patients to control games via purposeful hand movements, thereby promoting hand injury rehabilitation [11]. Xu et al. develop a dual-hand-operated ellipsoidal device that facilitates interactive grasp training [12]. Liu et al. introduce an interactive game mat coupled with an intelligent gait analysis system, aiming to improve leg muscle function in children with cerebral palsy [13]. Additionally, Rowe et al. design a cluster technology-based motion capture rehabilitation platform for intensive training of stroke patients [14]. Such studies predominantly approach the problem from hardware and functional compatibility perspectives, aiming to improve rehabilitation efficiency and precision. Secondly, they prioritize interaction experience optimization through contextual design, emphasizing not only rehabilitation outcomes but also motivational, immersive, and contextual interaction aspects. Perez Sanpablo et al. propose a persuasive system design model targeted at pediatric patients to enhance adherence in short-term gait training [15]. Binti Mohd Hashim et al. design a dual cognitive task challenge game integrated with VR for stroke rehabilitation scenarios [16]. Lau et al. develop the MCI-GaTE framework to investigate therapeutic experiences of game-based interventions for mild cognitive impairment [17]. Thirdly, intelligent systems oriented towards adaptive and evaluative mechanisms are emphasized. System adaptability and intelligent feedback are focused on enhancing personalized and data-driven rehabilitation. Kira et al. propose an automatic adaptation scheme for serious games and validate it through proof-of-concept experiments in clinical settings. The research incorporates dual perspectives of therapists and users, reinforcing the alignment between game mechanics and rehabilitation demands [18].

Taken together, current rehabilitation game interaction design reveals multifaceted explorations and preliminary achievements across three dimensions: functional recovery, user experience, and system adaptation. Targeted design strategies have been proposed concerning interaction devices, contextual construction, and intelligent feedback. Nevertheless, most explorations remain confined to partial functions or single objectives. Systematic integration of deeper experiential dimensions, including cultural context, storytelling, and narrative coherence, within the rehabilitation process remains insufficient. Furthermore, a unified design framework that addresses comprehensive rehabilitation experiences has yet to be developed.

2.2. Current State of Cultural Integration in Rehabilitation Games

The application of ICH in the field of rehabilitation, particularly in the design of rehabilitation games, has not yet been systematically examined, and relevant studies remain at an early stage. When the scope is extended to traditional culture more broadly, it is found that the integration of cultural elements into rehabilitation games has begun to emerge in preliminary practices According to the current literature review, only one study specifically investigates how ICH contributes to mental rehabilitation. In this research, tie-dye techniques are combined with art therapy to construct a multi-sensory intervention model, which effectively facilitates emotional regulation, cognitive restructuring, and social connection for individuals with mental disorders. This preliminary evidence suggests the potential of ICH in therapeutic applications. Nevertheless, relevant research remains scarce, primarily due to the wide disciplinary gap and limited interdisciplinary collaboration. ICH is typically studied within the humanities, while rehabilitation games are primarily developed within medical or engineering domains. Substantial differences in research methodologies, expression frameworks, and the underdeveloped digital infrastructure of ICH hinder its effective integration and direct implementation in rehabilitation game design. More importantly, a mutual lack of understanding regarding the intrinsic value and potential contributions of each domain leads to underexploration and underutilization of valuable cultural resources in rehabilitation contexts. Nonetheless, the limited attention to this area highlights its significant potential for future research. The integration of ICH into rehabilitation games represents a promising and underexplored direction that warrants systematic investigation.

At present, several studies attempt to integrate traditional cultural elements into rehabilitation games. These efforts enhance users’ cultural identity and engagement, improve rehabilitation outcomes and emotional experiences, and highlight the significant value and potential of traditional culture in rehabilitation design, offering key directions for future research. For instance, Turikumana et al. develop a rehabilitation program based on a Rwandan traditional board game, utilizing familiar game rules to guide hand function training and enhance cultural familiarity during rehabilitation [20]. Singla et al. construct a neurorehabilitation exergame within the framework of users’ personal belief systems, embedding religious cultural content into the training process [21]. Almousa et al. create a VR training scenario under the framework of Colombian local culture, embedding traditional imagery and customary elements into task structures to strengthen user immersion and engagement [22]. The application value and development potential of rehabilitation culture in interactive game design are preliminarily demonstrated through these practical cases. In addition to rehabilitation games, traditional culture also shows positive effects in other forms of rehabilitation. For example, Chan et al. design a culturally themed escape-room rehabilitation game, using familiar cultural cues to stimulate memory and increase participation among elderly stroke patients [23]. Gurgenidze et al. organize cultural activities that provide individuals with special needs opportunities for emotional expression and social interaction [24]. Pereverzeva et al. integrate folk music elements into gamified art therapy, enabling children to engage with familiar melodies and rhythms during rehabilitation [25]. Although preliminary exploration into the integration of traditional culture and rehabilitation games has been conducted, research on the in-depth integration of ICH into rehabilitation games remains limited and still at an early stage. Therefore, systematic investigation into the integration of ICH and rehabilitation games is urgently needed.

2.3. Current State of Frameworks Regarding with Cultural Attributes, Narrative, and Immersive Experiences

There are quite a few design frameworks trying to integrate cultural attributes, narrative, and immersive experiences currently. But they either focus primarily on immersive technologies, or treat cultural adaptation as symbolic enrichment. These aspects are often regarded as supplementary rather than as central logics, leaving them at the periphery of the design process instead of core work. Moreover, most frameworks lack a systematic theoretical structure to explain how cultural elements and narrative logic can be organically mapped onto interaction design in ways that directly serve functional needs. This limitation highlights the necessity of developing a comprehensive framework. To address this gap, our study proposes the integrated triadic mapping framework

Current design framework research in this area mainly explores how to integrate cultural heritage and artistic elements into narratives and immersive interactions, thereby creating user experiences with cultural depth and emotional resonance. For example, Liu et al. incorporate historical stories and folk activities into game narratives and contexts [26]. Baker et al. propose a framework that integrates educational goals, game design, narrative, and cultural relevance [27]. Chu et al. develop the Tangible and Embodied Narrative Framework (TENF), which emphasizes the significance of cultural attributes and narrative design in immersive experiences [28]. In addition, some studies explore this issue from a methodological perspective. However, these frameworks mostly remain conceptual and lack specific approaches to systematically integrate cultural attributes, narrative logic, and interaction design. For instance, Antemate et al. propose a conceptual framework for rehabilitation games based on User-Centered Design (UCD) and the GameFlow model, integrating user experience, emotional motivation, and game mechanics into the game process [29]. Colorado et al. combine UCD, structured activities from software engineering, and gamification elements while also paying attention to cultural adaptation, narrative design, and visual enhancement [30]. Moreover, other frameworks highlight the role of immersive technologies in shaping user experiences, but they often over-rely on technological performance and lack theoretical support for mapping such technologies onto narrative logic and situational design. For example, Zhang et al. construct the Clinical–Functional–Interest design framework in a VR environment [31], while Avola et al. propose a full-body rehabilitation framework using 3D serious games with head-mounted displays [32]. Similarly, Jin employs VR to transform traditional cultural elements into multimodal design languages [33]. Dogan et al. propose a conceptual framework employing VR or AR and haptic technologies as key tools for immersion [34].

To address this gap, we propose an integrated interaction design framework that positions cultural value transmission, rehabilitation goal orientation, and interaction behavior facilitation as core driving forces, working in concert with design strategies to provide systematic guidance for shaping the humanistic dimensions of rehabilitation game experiences. This integration not only enriches patients’ emotional engagement and cultural identity but also establishes both a theoretical foundation and practical pathways for advancing interdisciplinary rehabilitation game design.

In order to express the features of our “Triadic Mapping Framework” more clearly, a comparison between present studies and our framework has been conducted. Three perspectives have been included: the Cultural and Narrative Dimension, the Experience and Technology Dimension, and the Methodological and Practical Dimension, as shown in

Table 1. Comparison of similar frameworks (Yes/No).

		[26]	[27]	[28]	[29]	[30]	[31]	[32]	[33]	[34]	This Study
Cultural and Narrative Dimension	Cultural Attributes	Yes	Yes	Yes	No	No	No	No	Yes	Yes	Yes
	Narrative Logic	Yes	Yes	Yes	Limited	Limited	Limited	No	Yes	Yes	Yes
	Culture–Interaction Mapping	Limited	Limited	Yes	No	No	No	No	Limited	Yes	Yes
Experience and Technology Dimension	Immersive Technologies	Yes (VR/AR)	No	No	No	Limited	Yes (Leap motion)	Yes (HMD)	Yes (VR)	Yes (VR/AR)	Yes (Leap motion)
	Technology Integrated with Cultural Features	Limited	No	Limited	No	No	No	No	Yes	Yes	Yes
	Immersive Experience	Yes	Yes	Yes	Limited	Yes	Limited	Yes	Yes	Yes	Yes
Methodological and Practical Dimension	Function-oriented Culture Utilization	Limited	Yes	Limited	No	No	No	No	Yes	Yes	Yes
	Systematic and Reusable Design Path	Yes	Yes	Yes	Yes	Yes	Limited	Yes	No	Yes	Yes
	Practical Application	Yes	Yes	Yes	No	Yes	Yes	Yes	No	Yes	Yes
	Evaluation or Validation	Yes	No	Yes	No	No	Limited	Yes	No	Limited	Yes

. The Cultural and Narrative Dimension primarily examines whether a framework emphasizes cultural elements, narrative logic, and the mapping between culture and interaction. The Experience and Technology Dimension considers the use of immersive devices and technologies, whether these technologies are integrated with cultural mechanisms and immersive experiences. The Methodological and Practical Dimension examines whether it translates cultural attributes into functional rehabilitation practices, whether the framework demonstrates systematic and reusable, and whether it has undergone empirical validation through expert review or user assessment. The Function-oriented Culture Utilization Dimension emphasizes whether cultural elements are not only preserved at the narrative or symbolic level but also effectively mapped into operable design functions.

The results of the comparison are indicated in Table 1. Existing frameworks often focus on one particular dimension but lack cross-dimensional integration such as Culture–Interaction Mapping and Function-oriented Culture Utilization. Our framework achieves comprehensive coverage and integration across all three dimensions. At the same time, it shows a great advantage as regards the aspects of trying to map the inherent features of culture with interaction routes, as well as regarding efforts to drag culture out of the superficial sense and instead dig into its function and utilization in serious scenarios, such as clinical settings. The inner correlation among the characteristics of culture, the mechanism of rehabilitation, and also the interaction routes has been closely interlocked. More serious concerns would be raised such as in which way and which aspects of the culture could be powerful to support the serious functional use. These considerations have made our framework outstanding, leading innovative exploration in the new field and from new perspectives of traditional culture.

3. Framework Construction

3.1. Existing Frameworks: MDA

To position the proposed approach within the broader context of game design theory, this study first reviews the Mechanics–Dynamics–Aesthetics (MDA) framework, which remains one of the most widely applied models for analyzing the relationship between game components and player experiences.

The original MDA framework analyzes the relationship between game components and player experience. Mechanics refers to the underlying rules and systems; Dynamics describes the behavior and interactions that emerge during gameplay; and Aesthetics focuses on the sensory and emotional experiences of the player. In the context of rehabilitation games driven by performing-arts-based ICH, however, the player’s role shifts from a general user to a rehabilitation subject. The focus moves from entertainment toward the restoration of physical function. Within this framework, we construct a framework consisting of rehabilitation mechanism–interaction design–cultural experience, which serves as the theoretical basis for subsequent mapping relationships and system design (see Figure 1). The rehabilitation mechanism refers to the embedded therapeutic strategies and training methods in the game. Interaction design covers the way users engage with the system and how input is translated into feedback. Cultural experience refers to the emotional and symbolic resonance brought by the integration of performing-arts-based ICH elements.

This adapted framework serves as a conceptual bridge between general game design theory and culturally integrated rehabilitation systems, providing a theoretical basis upon which this study develops its own triadic mapping framework in Section 3.2.

3.2. Proposed Framework: Rehabilitation Mechanism–Cultural Feature–Interaction Design

Building upon the adapted MDA framework reviewed in Section 3.1, a rehabilitation game interaction design framework that integrates performing-arts-based ICH is proposed. This framework introduces a triadic mapping model of “rehabilitation mechanism-cultural feature-interaction design” (see Figure 2). Its purpose is to provide a structured pathway for transforming rehabilitation requirements into culturally meaningful, operable, and feedback-responsive game experiences.

In this framework, the rehabilitation mechanism serves as the logical starting point, defining the functional goals such as muscle strengthening, joint mobility enhancement, or fine motor coordination. The cultural feature dimension focuses specifically on performing-arts-based ICH, which possesses rich bodily, symbolic, and narrative expressions that can be mapped to rehabilitation movements. The interaction design dimension translates these mapped cultural features into concrete game mechanics, integrating usability principles to ensure clarity, consistency, and cultural relevance.

3.2.1. Analysis of Physical Rehabilitation Mechanisms and Classification of Training Approaches

In rehabilitation game design, it is fundamental to identify the types and internal structure of rehabilitation mechanisms when constructing a function-oriented design framework. The common rehabilitation goals and their corresponding mechanisms are systematically reviewed so that the functional integration of cultural content and interaction methods can be informed.

1. 

Physical rehabilitation goals and mechanisms

Physical rehabilitation primarily targets individuals with motor function decline due to illness, injury, or aging. Its goal is to restore and rebuild the function of specific body parts through structured training. Common rehabilitation goals can be categorized into four types. Corresponding to these goals, the rehabilitation process involves coordinated engagement of the neuromuscular and skeletal systems, including muscle strengthening, joint mobilization and stabilization, promotion of neuroplasticity, and restoration of sensorimotor feedback pathways [35]. Muscle strengthening focuses on resistance-based training to enhance muscular control. Joint mobilization relies on stretching and flexion exercises to prevent stiffness and contracture [36]. Neuroplasticity plays a key role in coordination training, especially for repetitive task-oriented rehabilitation after neurological damage [37,38]. Reconstructing sensory–motor feedback loops helps restore proprioception and tactile feedback, which is essential for functional movement. These mechanisms not only provide clear functional targets for rehabilitation game design but also lay the groundwork for mapping cultural mechanisms and interaction strategies.

2. 

Physical rehabilitation approaches

Current rehabilitation approaches generally include Physical Therapy (PT), equipment-assisted training, and functional training. While the first two approaches have recognized clinical value, they often lack active user engagement and contextual feedback, which can hinder long-term motivation. In contrast, functional training emphasizes goal-directed activities in daily-life scenarios. Through imitation-based and task-based practice, it helps rebuild motor memory and real-world function. Functional training offers advantages such as low dependency, high adaptability, and strong user initiative. Rehabilitation games align well with this logic by transforming real-life tasks into digital interactive experiences. Through interactive devices, such games can meet therapeutic objectives while enhancing user immersion and motivation. They are particularly suitable for embedding cultural elements to create contextualized and meaningful experiences.

Based on this analysis, various training approaches within the functional training framework can be systematically mapped to specific rehabilitation goals and mechanisms.

Table 2. Correspondence of mechanisms, goals, and training approaches in functional rehabilitation.

Rehabilitation Goal	Rehabilitation Mechanism	Training Approaches
Muscle strength enhancement	Neuromuscular activation	Progressive resistance training
		Activities of daily living
Joint mobility improvement	Joint mobilization and stabilization	Joint mobility exercises
		Dynamic movement sequences
Fine motor coordination training	Task-specific neuroplasticity	Goal-directed practice
		Coordination tasks
Sensorimotor integration training	Restoration of sensorimotor pathways	Sensory stimulation tasks
		Biofeedback-based training

summarizes the four core mechanisms, their corresponding goals, and representative training paradigms, providing a foundation for mapping cultural features and interaction strategies in subsequent sections.

Despite the advantages of functional training, traditional rehabilitation still faces several challenges, such as long durations, high repetition, and lack of engagement. These issues are especially prominent during early stages of motor function decline, when patients often lack sustained motivation. Effective rehabilitation outcomes rely heavily on training frequency, intensity, and precision, and the rebuilding of functional mechanisms requires multi-system coordination. Therefore, a deep understanding of the pathways among rehabilitation mechanisms is essential for designing effective interventions. It also provides a theoretical basis for developing a structured rehabilitation framework and strategies for integrating cultural elements into the system.

3.2.2. Analysis of the Typological Dimensions and Expressive Mechanisms of Performing-Arts-Oriented ICH

As the second core dimension of the proposed triadic mapping framework, performing-arts-oriented ICH serves not only as the cultural foundation of rehabilitation game content but also plays key roles in symbolic cognition, emotional stimulation, and cultural identity formation. Compared with the broad category of ICH, the performing arts domain focuses on embodied performance, staged narratives, and multimodal expression, making it highly compatible with interactive rehabilitation scenarios. Clarifying the typological dimensions and expressive mechanisms of performing-arts-oriented ICH is, therefore, essential for ensuring the cultural adaptability and user identification of rehabilitation games. Drawing on UNESCO’s performing arts classification, representative Chinese practices, and recent research in cultural interaction design, this study categorizes the cultural features of performing-arts-based ICH into three main expressive dimensions [39], as shown in

Table 3. Classification of cultural features in performing-arts-based ICH.

Feature Type	Representative Forms	Rehabilitation Potential
Physical and Kinesthetic	Chinese string puppetry, shadow puppetry, Peking opera gestures, traditional dance routines	Structured movement templates; rhythm guidance; fine–motor training
Craft-Oriented and Material-Linked	Puppet making, costume crafting, stage prop manipulation	Sequential task guidance; tactile feedback; phased goal completion
Symbolic and Narrative	Opera plots, folk drama scripts, character archetypes, ritualized stage conventions	Emotional regulation; cultural awakening; identity formation

.

1. 

Physical and Kinesthetic Features

These features are grounded in bodily experience and emphasize the transmission of cultural meaning through physical actions [40]. For example, the finger manipulations in Chinese string puppetry (“lifting–pulling–rotating”), the codified gestures in Peking opera (“point–wave–press”), and the arm–wrist coordination in shadow puppetry demonstrate both symbolic expressiveness and procedural motion. Such embodied movement patterns offer highly structured templates for embedding motor sequences and rhythm-based interventions in rehabilitation tasks, supporting both gross and fine motor function recovery.

2. 

Craft-Oriented and Material-Linked Features

Many performing-arts-based ICH practices are embedded in specific materials and procedural techniques [41]. Puppet making, costume embroidery, and stage prop assembly involve sequential craft operations and tactile engagement with materials. These processes closely mirror rehabilitation workflows, which often require step-by-step execution, sensory feedback, and phased accomplishment, thereby reinforcing user motivation and satisfaction.

3. 

Symbolic and Narrative Features

Performing-arts-based ICH often conveys national history and collective memory through symbolic motifs, totems, and legends. For instance, the hero–villain dichotomy in opera narratives, symbolic color schemes in costumes, and ceremonial stage movements all contribute to emotional resonance and cultural identification. These features can evoke emotional resonance and cultural affiliation, providing positive psychological regulation during the rehabilitation process.

Moreover, performing-arts-based ICH frequently integrates multimodal expression mechanisms such as sound (drumming and vocal styles), visuals (costume patterns and stage props), and rhythm (act structure and scene transitions). These elements can serve as important perceptual cues for game pacing, interactive prompts, and feedback mechanisms.

3.2.3. Interaction Design Strategies for Rehabilitation Games

In rehabilitation games, interaction methods serve not only as input channels for completing training tasks but also as carriers for cultural experience. Effective interaction design should ensure rehabilitation efficacy while enhancing user motivation and immersive engagement. Based on current research in interaction design, this study classifies common interaction types in rehabilitation games into four categories: data interaction, visual interaction, voice interaction, and behavioral interaction [38]. To ensure the practical applicability of these interaction types, three core principles from Nielsen’s usability heuristics are incorporated into the framework. It includes system visibility, match with real-world context, and consistency [42]. The corresponding relationships are summarized in

Table 4. Interaction design in rehabilitation games.

Interaction Types	Usability Principle	Key Design Considerations in Rehabilitation Games
Data Interaction	System visibility, consistency	Real-time feedback for action outcomes; consistent layout and button design to improve operational efficiency
Visual Interaction	System visibility, match to real world	Real-time visualization of gestures or motions; simulate ICH workflows and align with cultural behaviors
Voice Interaction	System visibility, consistency	Clear and timely responses to voice commands; maintain consistent language style for prompts
Behavioral Interaction	Match to real World, consistency	Simulate real cultural actions; strengthen the mapping between gesture and cultural meaning; ensure consistent feedback logic

.

Data interaction is the most common form of human–computer interaction. It includes mouse clicks, keyboard input, and slider control. In rehabilitation games, this type of interaction is primarily used for setting task parameters, presenting training outcomes, and controlling the game interface. According to the principle of system visibility, training status should be presented through real-time progress bars, numerical indicators, or graphic animations, enabling users to clearly perceive their rehabilitation progress. To fulfill the consistency principle, interface interactions of the same type should maintain consistent button design, feedback style, and operation flow, reducing cognitive load and enhancing usability.

Visual interaction utilizes computer vision and image recognition technologies to translate users’ actions, facial expressions, or body states into control signals within the game. It is an effective method to enhance cultural immersion. In the context of ICH, visual interaction provides an intuitive medium for simulating and conveying culturally significant motor skills, including the intricate hand movements involved in paper cutting, Tai Chi, and embroidery. These interactions can transform abstract cultural elements into perceptible operation paths. In line with the system visibility principle, real-time image feedback and visualized motion trajectories can improve users’ awareness of their operating status. To support a match between the system and real world, interface prompts should be designed using language and symbols from the corresponding ICH context, lowering the threshold for cultural understanding.

Voice interaction provides a natural language input–output mechanism that supports operational control, status feedback, and cultural narration in rehabilitation games. Through speech recognition and synthesis technologies, the system can interpret user commands and provide voice-based training prompts or cultural explanations. Based on the system visibility principle, voice responses should be clear, timely, and intuitive to prevent user confusion. Furthermore, to satisfy the consistency principle, the style and structure of prompts should remain uniform throughout the game, helping users develop stable voice interaction habits during repeated training sessions.

Behavioral interaction is an advanced interaction mode based on body motion sensing. It includes gesture recognition and posture detection and is the most physically embodied and culturally linked form of interaction in rehabilitation games. This method is particularly suitable for embedding symbolic cultural actions—such as pulling, pinching, or swinging gestures inspired by puppet control—directly into the training process. In line with the match between system and real-world principle, behavioral interaction should align with users’ physical experience and cultural cognition, using cultural semantics to guide training tasks. To ensure consistency, similar gestures (e.g., opening/closing and swinging) should follow the same input logic and feedback mechanisms across different tasks, enhancing operational stability and learning efficiency.

In summary, interaction design in rehabilitation games is not only about input and feedback mechanics but also a crucial bridge linking cultural meaning with functional training. By classifying interaction types and applying targeted usability principles, designers can enhance both the scientific rigor of rehabilitation and the immersive quality of cultural experience, thereby supporting the effective integration of ICH into game-based rehabilitation systems.

3.2.4. Triadic Mapping Framework

To systematically guide the integration of performing-arts-based ICH into rehabilitation games and ensure a closed-loop logic among cultural elements, rehabilitation objectives, and interaction mechanisms, this study proposes a triadic mapping framework: rehabilitation mechanism–cultural feature–interaction design (see Figure 3). Based on the prior analysis of the three dimensions, the framework integrates rehabilitation goals, cultural features from performing-arts-based ICH, and interaction design into a mutually constructed system. Rehabilitation goals serve as the primary orientation, while interaction design mediates between functional requirements and the symbolic, procedural, and embodied elements of ICH. This structure enables the development of operable and culturally meaningful interaction strategies that are grounded in both therapeutic objectives and cultural expression. In this triadic relationship, interaction design serves as a dynamic intermediary, bridging functional objectives with cultural semantics by translating symbolic meanings, procedural logic, and embodied actions into engaging, operable experiences. The framework comprises three core dimensions.

1. 

Rehabilitation Mechanism

Rehabilitation mechanism refers to the functional objectives embedded in the game, including four categories: muscle strengthening, joint mobility enhancement, motor coordination training, and sensory feedback reconstruction. These correspond to distinct physical rehabilitation pathways.

2. 

Cultural Feature

Cultural feature refers to the types of performing-arts-based ICH expressions embedded in the game, categorized as physical/action-oriented, craft/process-oriented, and symbolic/narrative. These features provide structured action templates, multi-step task feedback, and emotional/cultural resonance mechanisms, respectively.

3. 

Interaction Design

Interaction design denotes the implementation strategies, including data interaction, behavioral interaction, and visual interaction. These are guided by design principles such as system visibility, real-world compatibility, and operational consistency, ensuring that the interaction aligns with both the physical logic of rehabilitation and the immersive nature of cultural experience.

In practical application, rehabilitation mechanisms are explored after the functional problem is addressed through user study (e.g., muscle strength or fine motor control). Cultural features serve as situational and symbolic resources. For example, the “lift–pull–rotate” sequences from Chinese string puppetry provide rhythm and structural guidance, while ritualized stage movements in opera can evoke emotional connections. Interaction methods integrate these features into perceptible, operable, and feedback-responsive user experiences. For example, if the goal is to improve joint mobility, performance-based cultural elements such as puppetry arm manipulation or Peking opera sleeve movements can be adapted into dynamic action sequences, supported by visual interaction and real-time feedback to enhance execution fluency and engagement. Figure 4 illustrates how each rehabilitation mechanism aligns with specific cultural features and interaction strategies. Muscle strengthening, for instance, can be paired with high-motion ICH forms like shadow puppetry, simulating gripping or lifting actions; emotional regulation goals can be addressed through ritualistic or narrative performing-arts-based ICH forms, supported by symbolic interaction designs or story-driven gameplay for cultural and psychological engagement.

The framework exhibits strong adaptability and scalability. Cultural features and rehabilitation mechanisms are not limited to one-to-one mappings; they can form many-to-many flexible associations, allowing tailored combinations of content and tasks for diverse user groups (e.g., children, older adults, or cognitively impaired users). On the technical side, the framework is platform-agnostic: it supports interaction via Leap Motion, or Kinect, or can be extended to multi-modal systems such as VR, touchscreens, and voice interfaces, enabling cross-platform deployment and personalized rehabilitation workflows. In summary, this framework builds a structured bridge between cultural semantics, rehabilitation objectives, and interaction strategies. It provides a unified logic system of function–context–behavior for the design of rehabilitation games. By embedding cultural features into the concrete structure of training behaviors, the framework goes beyond superficial integration, enabling the development of rehabilitation systems that are functional, culturally resonant, and experientially immersive.

3.2.5. Design Practice

Once the triadic mapping relations are established, the framework is implemented through a design practice phase. During this phase, the conceptual mappings are translated into functional modules through the development of low- and high-fidelity prototypes. Based on initial try-outs, the prototype is iteratively refined, focusing on cultural representation, interaction mechanisms, and feedback logic.

3.2.6. Evaluation and Validation

In this stage, the refined prototype is assessed through a combination of expert evaluation and user testing. Domain experts (rehabilitation specialists, performing-arts-based ICH practitioners, and interaction design researchers) are invited to examine the accuracy of rehabilitation mechanisms, the appropriateness of cultural representations, and the usability of interaction methods. In addition, user tests with representative participants are carried out to evaluate functional effectiveness, user experience, and overall cultural engagement during actual training sessions. Based on the insights obtained from both expert feedback and user testing, the design is further refined and iteratively improved, which results in the final version of the rehabilitation game system.

4. Design Practice

Based on the proposed “Rehabilitation Mechanism–Cultural Feature–Interaction Design” triadic mapping framework, a digital rehabilitation game inspired by Chinese string puppetry is conducted. This practice involved the translation of key rehabilitation gestures into culturally meaningful interaction tasks, the integration of multimodal feedback mechanisms, and the construction of an interaction environment. Through this implementation, we aimed to examine the feasibility and effectiveness of embedding cultural semantics into functional rehabilitation scenarios.

4.1. Target Users

The target users of this project are primarily young and middle-aged individuals with hand dysfunction caused by neurological injuries or repetitive strain conditions, including wrist pain, tenosynovitis, finger–wrist discomfort, and carpal tunnel syndrome. To gain a detailed understanding of the functional limitations, rehabilitation expectations, and interaction preferences of the target population, in-depth interviews are conducted with patients and rehabilitation professionals at rehabilitation centers and community clinics in Beijing (China). Additionally, a randomized online survey is distributed to collect broader feedback.

The results of the interviews reveal that over 80% of respondents are working-age individuals. These users commonly experience hand dysfunctions caused by extended use of keyboards, mice, and mobile devices, leading to chronic strain and reduced functional capacity. Among them, approximately 60% are in the early to middle stages of rehabilitation and require repetitively active training. The survey results show that users strongly prefer rehabilitation approaches characterized by greater engagement and interactivity. Moreover, more than 70% of rehabilitation professionals highlight the importance of incorporating systems with visual feedback and measurable assessment to improve the scientific validity of training and promote patient adherence.

Based on the survey results, this study develops a digital rehabilitation game that integrates hand rehabilitation mechanisms with the cultural features of string puppetry. The game incorporates the gesture-based control and immersive experience elements inherent in string puppetry into rehabilitation training, aligning with clinical goals such as improving range of motion, flexibility, and coordination. This rehabilitation game is highly applicable, with intuitive feedback and strong interactivity, suitable for repeated training, and provides a sustainable rehabilitation solution for users with varying degrees of hand function impairment.

4.2. Analysis of Hand Rehabilitation Mechanism

The target user group includes individuals with hand motor impairments characterized by limited range of motion, diminished muscle strength, and coordination deficits of varying severity. Therefore, the hand rehabilitation mechanism is primarily designed to enhance muscle strength, improve fine motor coordination, and restore joint flexibility.

To achieve the aforementioned rehabilitation goals, this study employs functional training as the underlying rehabilitation logic for the interactive mechanism. Functional training helps patients perform fine motor exercises by simulating hand movements required in daily life, such as making a fist, pinching, placing, and manipulating objects. This ensures that the training process is highly relevant to the patient’s actual life [43]. We have reviewed and analyzed widely used hand rehabilitation methods in current clinical practice. At present, functional training for hand dysfunction rehabilitation mainly includes occupational therapy, task-oriented training, and functional motor training, all of which emphasize the need for rehabilitation exercises to reflect real-life situations. By completing specific operational tasks with clear goals, these methods enhance patient initiative and rehabilitation effectiveness [44,45].

As a convenient and accessible daily intervention, functional training can be seamlessly integrated into the lives of patients, thereby fostering long-term adherence. It improves the functional transferability of trained movements and increases the practical value of rehabilitation. Therefore, this approach is well suited to meet the diverse needs of patients across varying levels of impairment severity. Based on these rehabilitation methods, we have summarized several common and representative hand movements in hand rehabilitation, as illustrated in

Table 5. Rehabilitation gestures in hand function training.

Rehabilitation Gestures	Illustration	Training Area	Rehabilitation Goals
Finger flexion and extension		Finger flexor muscles	Enhance finger flexor and extensor muscle strength and overall grip ability
Pinch grip Wrist joint mobility		Thumb and index or middle finger	Improve fine coordination and thumb opposition function
Wrist joint mobility		Wrist rotator muscles and wrist flexor muscles	Enhance wrist control and forearm rotational flexibility
Finger abduction		Interosseous muscles	Strengthen finger separation control and improve independent dexterity
Finger–thumb opposition		Thumb and other fingers	Restore multi-finger coordinated control in complex fine motor tasks
Thumb abduction and extension training		Thenar muscles	Strengthen thumb abduction ability and improve adaptability for grasping large objects

. These rehabilitation gestures are grounded in established clinical practice and widely adopted in institutions such as the Shanghai Rehabilitation Center and Peking University Third Hospital.

After comprehensively considering the functional transferability of rehabilitation gestures, the difficulty of movement execution, and the adaptability of interactive feedback, this study identified three representative hand rehabilitation gestures. Specifically, finger flexion–extension exercises, active wrist flexion–extension, and fingertip pinching are selected, as illustrated in Figure 5. These gestures cover multiple levels of functional recovery, ranging from large joint movements to fine finger dexterity. They enable users to progressively restore the autonomous operational abilities required for daily living, thereby meeting essential activities such as grasping, rotating, and picking up objects [46,47,48].

The finger flexion–extension exercise primarily targets the finger flexor and extensor muscles. In this movement, users are required to alternately clench and release their fists, thereby repetitively stimulating finger flexion and extension. This exercise helps increase the range of motion in finger joints, restore grip strength, and establish a foundation for performing more precise and stable hand movements.

Similarly, the wrist flexion–extension exercise primarily engages the wrist joint and proximal forearm muscles. This motion aids in reducing joint adhesions and relieving tension in the proximal region, contributing to improved joint flexibility and mobility.

In addition, the finger–thumb opposition gesture focuses on enhancing the oppositional function between the thumb and other fingers. It requires users to perform highly precise and coordinated movements, stimulating distal finger nerves and activating intrinsic small muscles. This exercise promotes fine motor coordination and supports the reconstruction of dexterity and task-oriented manipulation abilities. Consequently, it prepares users for essential yet challenging daily activities such as grasping, pinching, and handling objects.

4.3. Cultural Characteristics of ICH of String Puppets

As a traditional form of Chinese intangible cultural heritage, string puppetry is a cultural art form centered on motion control and narrative performance, characterized by high precision in movement and strong contextual expressiveness, as illustrated in Figure 6. Specifically, its unique control logic and performance structure reflect a high dependency on fine hand coordination. In addition, it embodies key cultural attributes such as embodiment, symbolism, and anthropomorphic expression. These characteristics offer valuable paradigms for the design of rehabilitation systems, particularly in modeling physical training and constructing immersive, culturally resonant interactive environments.

4.3.1. Motion Control

One of the defining cultural characteristics of string puppetry lies in its refined manipulation techniques, which are embodied in the complex stringing mechanisms and the highly skilled art of string coordination. The operation of the puppet relies on the bimanual control of puppeteer, requiring subtle coordination between the fingers and wrists to precisely adjust the tension and movement rhythm of each string [49]. As a result, the puppet is able to exhibit smooth, lifelike movements and expressive gestures, highlighting the embodied and artisanal nature of the performance.

The manipulation of puppets through fingertip movements shares the same behavioral logic as hand rehabilitation training, requiring flexible wrist joints and highly independent finger movements. This operational similarity closely aligns with rehabilitation goals aimed at improving fingertip control and movement precision. Moreover, repetitive practice in such fine motor tasks can enhance finger flexibility and the coordination of individuated movements.

Therefore, the cultural characteristics of fine motor manipulation provide a meaningful foundation for integrating puppet-inspired gestures into rehabilitation mechanism. Embedding these gestures within a culturally resonant framework not only enriches the interactive mechanism but also enhances emotional engagement of users and intrinsic motivation, thereby supporting more sustained and meaningful participation in rehabilitation activities.

4.3.2. Performative Narratives

Beyond its refined manipulation techniques, string puppetry is also characterized by strong symbolic and narrative dimensions. Puppeteers often incorporate complete storylines and character representations into their performances. Behaviorally, they animate the puppets through hand movements to deliver anthropomorphic performances [50]. Psychologically, they engage in a form of projection, viewing the puppet’s narrative as an extension of the self. This process enables deep emotional involvement, allowing the performer to identify with the character and its context, thus creating a profound sense of emotional resonance and immersive experience.

Conventional rehabilitation training often lacks narrative engagement, making repetitive motor exercises feel monotonous and mechanical. However, by integrating task-driven traditional narratives and meaningful culture actions into repetitive rehabilitation gestures, the training process gains narrative depth, playfulness, and immersive quality. Thereby, each movement is no longer seen as an isolated point of action but rather as constructing a continuous storyline in which users manipulates the virtual puppet to complete tasks rich in symbolic context. These performative tasks enable users to experience rehabilitation as a form of participatory storytelling. As a result, training gains emotional depth, playfulness, and immersive quality. The integration of narrative not only enhances the sense of agency and engagement but also improves adherence by fostering a stronger emotional connection and intrinsic motivation to complete the training.

From the dual perspectives of movement control and narrative performance, the cultural attributes embedded in traditional string puppetry strongly resonate with the kinematic foundations of hand rehabilitation. The integration of cultural storytelling and motor tasks provides a rich, culturally grounded framework for rehabilitation game design. This approach significantly enhances immersion, identification, and intrinsic motivation of users, ultimately contributing to more holistic and embodied rehabilitation outcomes.

4.4. Interaction Design

In this interaction system design, the “Rehabilitation Goals-Cultural Context-Interaction Mechanism” triadic mapping framework is incorporated into the functional logic. The framework facilitates the construction of a multimodal interaction system specifically tailored to functional rehabilitation scenarios. Within this system, the interaction mode serves as a mediating mechanism connecting the physical rehabilitation logic with cultural semantic expression. To achieve this integration, a multimodal interaction framework is developed from the dual perspectives of “functional rehabilitation training” and “cultural context construction.” The framework encompasses four key interaction modalities, including data interaction, image interaction, behavioral interaction, and voice interaction. These modalities form a systematic interaction strategy that supports the achievement of rehabilitation goals, enhances user cognitive understanding, promotes physical engagement, and fosters cultural immersion. Ultimately, this framework enables a high level of integration between rehabilitation training needs, cultural behavioral content, and interaction feedback systems.

4.4.1. Data Interaction

Data interaction serves as the foundational support for functional rehabilitation. In this system, it undertakes tasks such as visualization of the training process, path tracking, and feedback adjustment. The system records and analyzes parameters such as frequency, accuracy, and reaction time during the execution of rehabilitation movements. Based on these metrics, it can dynamically evaluate and provide real-time feedback on the rehabilitation process. To assess the completion of movements, the system continuously collects and extracts feature data based on the spatial trajectory of key joints such as the fingers, palms, and wrists, forming the data logic foundation for the interaction system. As a result, this data-driven mechanism enhances both the scientific rigor and adaptive personalization of the rehabilitation experience.

4.4.2. Image Interaction

Image interaction plays a crucial role in visually linking user rehabilitation movements with cultural content. This visual coupling allows users to intuitively perceive the consequences of their actions, thereby reinforcing movement cognition and sensory feedback. Specifically, each rehabilitation gesture is dynamically mapped to corresponding puppet behavior on the screen, enabling users to observe real-time visual responses that reflect their physical input. Furthermore, the system interface integrates visual elements such as icons, animations, and completion prompts throughout the process. These visual cues allow users to continuously perceive their status within the system and to understand the quality and progress of their rehabilitation movements. Image interaction effectively connects the rehabilitation process, cultural semantics, and physical behavior, enhancing operation clarity, emotional stability, and training focus.

4.4.3. Behavioral Interaction

Behavioral interaction is the core mechanism of this system, and its primary goal is to transform functional rehabilitation movements into culturally meaningful operational behaviors. The system constructs interaction gestures based on the logical movement paths aligned with rehabilitation processes, ensuring the rehabilitation value of the training tasks by employing natural movement modes. Additionally, these gestures are mapped to the manipulation of string puppets, assigning them symbolic functionality. This enables users to experience the role of “puppeteer” during training and enhances their sense of situational involvement. Furthermore, the system tracks and provides feedback in real time, forming a closed loop between movement execution and perception. This transforms the rehabilitation process from passive execution to active participation, achieving an organic integration of physical training and cultural immersion.

4.4.4. Voice Interaction

Voice interaction serves as an auxiliary channel for information delivery, effectively enhancing the guidance of rehabilitation tasks and the experience of cultural immersion. Through voice prompts and feedback at key stages, users can clearly identify task phases and movement execution statuses, improving the fluency and comprehensibility of the interaction. On the cultural level, the system incorporates the tones and sound effects of traditional opera. Through anthropomorphic and ritualistic design, it creates an auditory atmosphere that reflects intangible cultural heritage, thereby triggering emotional resonance in users. As a non-visual interaction channel, voice interaction complements visual and behavioral modalities to construct a multimodal perceptual environment. This integration strengthens the system’s cultural expressiveness and the integrity of the immersive experience.

4.5. Design Description

This study focuses on hand rehabilitation goals and develops an interactive hand function training game based on the cultural context of string puppetry. The system utilizes the Leap Motion gesture recognition device for precise motion capture, combined with the Unity3D engine to construct an interactive environment. Within this environment, users execute specific hand training gestures to control a virtual puppet, whose corresponding performance actions are rendered in real time. In this way, rehabilitation tasks are completed within a culturally immersive and contextually meaningful setting.

To support this interaction, we develop a practical rehabilitation game centered on standardized hand semantics and visual feedback. As illustrated in Figure 7, the design includes three core rehabilitation gestures and four game design dimensions corresponding to gamification tasks. The gesture-to-action correspondence highlights the alignment between rehabilitation movements and the cultural logic of string puppet manipulation. Furthermore, the integration of narrative-driven tasks and ritualized visual elements enhances user immersion and training engagement. Consequently, it achieves a semantic transformation across the three levels of movement, function, and culture.

4.5.1. Hardware Selection and Development Platform

To achieve the rehabilitation interaction design, Leap Motion (Ultraleap Ltd., Bristol, UK) and Unity3D (version 2022.3.0f1c1) are selected to ensure precise gesture capture, smooth system operation, and immersive cultural representation. The combination of gesture recognition hardware and a real-time interaction engine enables the seamless transformation of physical movements into culturally meaningful digital interactions that align with rehabilitation goals.

Leap Motion functions as a key HCI interface by connecting to a computer via USB and accurately tracking the three-dimensional positions and orientations of the user’s fingers and palm, as shown in Figure 8. This enables seamless technical integration between the reproduction of cultural gestures and rehabilitation movement training. As a non-wearable device, Leap Motion supports short-range, non-invasive interaction, greatly enhancing user comfort and acceptance during use. This makes it well suited for natural hand movement training in rehabilitation contexts.

Meanwhile, the Unity engine is a powerful cross-platform real-time interaction platform. It establishes a mapping logic between the hand movement data captured by Leap Motion and the puppeteer’s manipulation of the puppet. In coordination with an animation-driven system, it continuously updates the puppet’s state, inheriting and reconstructing the hand-controlled essence of string puppetry culture. This provides a solid technical foundation for transforming cultural behaviors into effective rehabilitation training pathways.

4.5.2. Interaction Design and User Scenarios

In this rehabilitation game, real-time visual feedback corresponding to the user’s motion perception is integrated into the design. Specifically, the user manipulates multiple fine strings controlling the virtual puppet’s bodily movements in the game interface through wrist swinging and independent or coordinated finger motions, as illustrated in Figure 9a. Each hand movement performed by the user is immediately reflected in the puppet’s dynamic behavior. The game interface provides visual and auditory feedback based on the accuracy, completion, and rhythm of the rehabilitation movements. During rehabilitation training, users are seated in front of a laptop, with their hands positioned above the Leap Motion device. By performing different gestures, they control the puppet character on the screen to complete specific rehabilitation tasks, as shown in Figure 9b. This seamless interaction between gesture input and system response enhances user engagement and supports the therapeutic process by offering timely feedback and goal-oriented motivation.

In the game interaction logic, three core gesture interaction types are designed to correspond to key movement requirements during rehabilitation training. First, the fist clenching and opening gestures target the flexor and extensor muscles of the fingers. Users control the basic vertical movement and gait fluctuations of the puppet character in the game by opening and closing their fists. When the hand opens, the puppet performs an upward leap; when the fist clenches, the puppet lowers in position, as illustrated in Figure 10a.

Second, the wrist swinging gesture trains wrist flexion–extension ability and flexibility. The directional swing of the wrist is translated into lateral movement and orientation changes of the puppet. For instance, swinging the wrist to the left causes the puppet to move leftward, and similarly for the right direction, as shown in Figure 10b.

Lastly, the fingertip pinching gesture focuses on the coordinated movement between the thumb and other fingers. Within the game context, it drives the puppet to perform anthropomorphic fine motor actions, as shown in Figure 10c.

These three gesture types constitute a comprehensive rehabilitation hierarchy, ranging from large joint movements to fine finger control. While emphasizing the synchrony and visual linkage between gesture inputs and game responses, the design leverages the puppet manipulation logic embedded in the cultural context. This approach achieves an organic integration of motor training and immersive interaction. This establishes an effective alignment among interactive behavior, cultural gestures, and rehabilitation goals.

4.5.3. Game Tasks and Interface Design

In the game mechanism design, a task-driven approach is incorporated into the rehabilitation training, requiring users to manipulate the puppet strings to complete designated tasks within each level. The gameplay is structured around three typical hand rehabilitation gestures, which are mapped to three progressive training levels, as illustrated in Figure 11. Each level is designed by integrating specific rehabilitation goals with corresponding puppet character actions.

• Level 1: Finger Flexion–Extension Exercise—Avoid the Darts

In the first level, users drive the puppet to jump to avoid the darts by performing the fundamental hand gestures of fist clenching and opening. During the task, dart obstacles continuously appear on the game interface, requiring users to control the puppet’s jumps through rhythmically coordinated hand opening and closing motions to accurately avoid the darts, as shown in Figure 12. When the puppet is hit by a dart, visual effects of bleeding and corresponding sound effects are triggered to enhance the realism and tension of the feedback, further stimulating the user’s attention and movement precision. The primary goal of this level is to activate the finger flexor and extensor muscles and establish a foundational active movement pattern. Moreover, this gesture closely corresponds to the “leaping” and “jumping steps” performance forms in string puppetry. The game establishes a correspondence between hand rhythm and puppet actions, allowing users to naturally immerse themselves in the puppetry control context during rehabilitation, thereby reinforcing the cognitive linkage between physical movement and cultural behavior.

2.

Level 2: Wrist Joint Activity—Kick the Wine Jar

In the second level, users control the puppet’s lateral movement and perform striking actions through left and right wrist swinging to kick the wine jars, as shown in Figure 13. The game interface features continuously appearing “wine jar” props, which users must accurately “kick away” by directing the puppet toward the target. The training objective of this level is to improve the range of motion and directional control of the wrist joint and enhance the coordination between the hand and forearm. From a cultural perspective, this mechanic draws inspiration from the lateral string manipulation techniques employed in string puppetry, where the puppeteer controls sideward tension to simulate dynamic footwork. By incorporating wrist swinging as the input modality, the game reproduces this traditional control method, thereby embedding rehabilitation training within an authentic cultural framework.

3.

Level 3: Fingertip Pinching Training-Place the Offerings

In the third level, users need to perform fingertip pinching between the thumb and different fingers to manipulate the puppet in replacing props and placing them in designated positions to trigger the “success” feedback, as shown in Figure 14. This task primarily trains the user’s opposition ability and movement precision, enhancing fine motor skills essential for daily activities such as pinching, picking up, and placing objects. To enhance contextual immersion, the interface adopts a puppet theater stage backdrop, reinforcing a sense of ritual and cultural atmosphere during training. The game incorporates the prop replacement action by embedding rehabilitation movement logic into the cultural story scenarios of string puppetry. These scenarios involve anthropomorphic actions such as “holding objects,” “drinking wine,” and “dancing with a sword.” These anthropomorphic interactions establish a situational linkage between operational movements and cultural narratives, thereby strengthening the narrative significance and immersion of user participation.

To further enhance the user experience, each level includes task objective guidance and feedback animations throughout the training process. This progressively strengthens the user’s control over training rhythm and movement accuracy. Successful operations trigger animated effects and auditory feedback to enhance the sense of achievement. This forms a progressive task structure aligned with rehabilitation pathways, improving the systematic and staged nature of movement recovery. This task mechanism and interface design enhance the engagement and immersion of rehabilitation training through a three-layer linkage of rehabilitation movements, game challenges, and cultural behaviors. Meanwhile, it ensures scientific rigor and practical applicability in task design, providing users with a culturally embedded functional training environment.

5. Other Design Practices

5.1. A Full-Body Physical Rehabilitation Game Based on Shadow Puppetry

Guided by the triadic mapping framework, this study also develops a full-body physical rehabilitation game leveraging shadow puppetry as the cultural mechanism. Implemented using the Unity game engine and Kinect motion-sensing technology, the project digitalizes the ICH of shadow puppetry. Through body-based interaction, users engage in cultural experiences while performing exercises targeting muscle strength, joint flexibility, and coordination, thereby achieving an immersive rehabilitation process. This case study demonstrates the framework’s capacity to systematically guide the design process from concept to implementation.

5.1.1. Rehabilitation Mechanism

The design targets full-body physical rehabilitation while providing cultural immersion. It focuses on three training functions: muscle strength, coordination, and perceptual feedback. Muscle strength is enhanced through large upper-body and torso movements manipulating digital shadow puppets. Coordination is trained via hand–eye and bimanual actions, improving multi-limb control. Specifically, the rehabilitation mechanism primarily employs the star pose from full-body rehabilitation [51]. Users stand with their feet shoulder-width apart, arms extended laterally at shoulder height, and the body slightly stretched outward, forming an “X”-shaped posture, as illustrated in Figure 15. This posture engages upper and lower limbs and core muscles, improving trunk stability, balance, joint flexibility, and overall coordination, supporting comprehensive rehabilitation with a culturally enriched experience.

5.1.2. Cultural Features

This design centers on shadow puppetry, a national intangible cultural heritage, integrating its unique performance form with rehabilitation exercises, as illustrated in Figure 16a. A distinctive feature of shadow puppetry is that the limbs, head, and torso of the figures are connected through articulated joints, allowing for flexible and lifelike movement, as illustrated in Figure 16b. The traditional manipulations of shadow puppetry are transformed into rehabilitation movement templates, allowing users’ training to be naturally integrated with the performance form, combining functionality with cultural expression [52]. The integration of puppet characters and story scenarios enables users to participate in role-playing and story enactment while performing rehabilitation exercises, fostering emotional involvement and strengthening cultural identification and a sense of belonging.

5.1.3. Interaction Method

At the interaction level, the design maps cultural actions to rehabilitation exercises through multi-layered mechanisms. Using Kinect to capture full-body skeletal points, the system enables real-time motion recognition and control, as shown in Figure 17a. Traditional shadow puppetry techniques are linked to therapeutic movements, such as pushing, lifting, and rotating, allowing users to participate in culturally meaningful movements while performing rehabilitation exercises, as shown in Figure 17b. A virtual puppet stage in Unity translates user movements into real-time performances, enhancing feedback, intuitiveness, and immersion. By combining high-precision motion capture with culturally informed action mapping, the design integrates rehabilitation functionality with cultural engagement.

5.1.4. Game and Interface Design

The game adopts a task-driven approach, where users control virtual shadow puppets through body movements synchronized with in-game actions such as bowing, offering, and dancing. Storylines are adapted from classic shadow puppetry scenes, and the interface features traditional stage backdrops, strengthening the link between rehabilitation and cultural narrative. Additionally, to enhance social interaction and motivation, a two-player cooperative mode allows simultaneous participation, as shown in Figure 18. This cooperative mode extends the single-player experience with social elements, encouraging synchronized body movement and interpersonal coordination, supporting long-term adherence and improving rehabilitation compliance.

5.2. A Physical Rehabilitation Game Based on Tai Chi

This design practice is grounded in Wu-style Tai Chi as a cultural foundation and, under the guidance of the triadic mapping framework, develops a rehabilitation training game through the unreal engine and Kinect-based motion-sensing technology. By embodying Tai Chi movements, users map their physical actions onto the virtual character, thereby transforming culturally meaningful practices into structured rehabilitation tasks. In this process, Tai Chi’s cultural embodiment and symbolic expressiveness not only provide therapeutic relevance for enhancing muscle strength and physical coordination but also enrich the user’s immersive engagement and cultural identification.

5.2.1. Rehabilitation Mechanism

This design integrates Wu-style Tai Chi into rehabilitation modules, targeting muscle strengthening and whole-body coordination. Muscle strength is enhanced through movements such as Tui Shou (Push Hands) and Ma Bu (Horse Stance), reinforcing lower-limb and trunk stability [53], as illustrated in Figure 19. Coordination training focuses on bilateral alternation, limb synchronization, hand–eye coordination, and whole-body continuity. In addition, the system employs a real-time feedback mechanism within the game to guide users in continuously correcting their postures during movement execution, progressively rebuilding motor perception and body control.

5.2.2. Cultural Feature

Wu-style Tai Chi emphasizes using softness to overcome hardness, achieving motion through stillness, and integrating body and mind [54]. Its codified movement patterns constitute highly structured motor sequences, including circular arm trajectories, coordinated shifts of body weight, and precise transitions between stances, seamlessly integrating balance, rhythm, and breath control. Rooted in Daoist philosophy, each posture conveys symbolic meaning. Practicing Tai Chi requires coordinated full-body movement, where advances and retreats, rises and falls, and motions in all directions align with breathing and mental focus, providing psychological comfort and cultural affirmation throughout the rehabilitation process [55].

5.2.3. Interaction Method

In terms of interaction design, this game employs the Kinect device to achieve real-time full-body skeletal tracking, mapping the movements of users onto a virtual character, as shown in Figure 20. Players follow a virtual coach in the game to practice Wu-style Tai Chi movements. The system captures key trajectories of the user’s arms, legs, and torso and compares them with standard postures to evaluate movement accuracy. Upon completion of the movement, the game provides positive visual feedback to reinforce motivation.

5.2.4. Game Tasks and Interface Design

In terms of interface design, the game incorporates a clear and concise introductory page prior. This page introduces users to the basic operation methods and training procedures while also providing background information on the history and cultural value of Wu-style Tai Chi, as shown in Figure 21. This not only allows users to quickly grasp the operational logic of the game but also enables them to appreciate the health benefits and intangible cultural significance of Wu-style Tai Chi from the very beginning. Once the training begins, the main interface simultaneously displays a virtual coach and a virtual user, as shown in Figure 22a. Users are required to imitate and synchronize with the coach’s movements, while the system employs real-time posture comparison, combined with visualized motion trajectories and interaction cues, to provide intuitive guidance and instant feedback, as shown in Figure 22b. This design enhances both the accuracy of execution and the enjoyment of interaction.

6. Evaluation

To evaluate the practical effectiveness of the proposed “Rehabilitation Mechanism–Intangible Cultural Feature–Game Interaction Design” triadic mapping framework in rehabilitation game design, and to assess the functional performance and user experience of the puppet-themed hand rehabilitation game developed based on this framework, a mixed-method validation is conducted through expert interviews and user testing.

6.1. Expert Assessment

First, to assess the medical validity and functional relevance of the rehabilitation game, two associate chief physicians from the China Rehabilitation Research Center are invited for structured in-depth interviews: one specialist is from the Department of Rehabilitation Assessment, and the other is from the Department of Neurology. Both experts have over ten years of clinical experience in rehabilitation evaluation and intervention. The interviews focus on the three core interactive movements used in the game: finger extension–flexion, wrist swinging, and fingertip pinching. The experts assess the game from three aspects: alignment with rehabilitation theory, anatomical motion rationality, and substitutability for standard functional exercises.

The results confirm that the in-game movements are highly consistent with standard hand rehabilitation practices. Finger extension–flexion effectively activates flexor and extensor muscle groups and supports muscle strength and endurance training. Wrist swinging promotes joint flexibility and improves the range of motion and motor control. Fingertip pinching contributes to fine motor control and inter-finger coordination and is a key movement in daily functional training. Both experts agree that these game tasks demonstrate clear rehabilitation relevance and are feasible as partial substitutes for traditional therapy, especially in early-stage active training without equipment. Moreover, the integration of cultural context and playful mechanics is recognized as beneficial for enhancing patient engagement and compliance, providing a valuable complement to conventional rehabilitation methods.

6.2. User Test

Following the expert validation of the functional suitability, user experience testing is conducted. A total of 39 participants (aged 18–25, mean age = 22.5) are recruited, with demographic profiles summarized in

Table 6. Demographic profiles of participants.

Question	Traditional Training
Gender	Male (18), Female (21)
Age	M = 22.5, SD = 2.24
Education degree	Undergraduate (11), Master (21), Doctor (7)
Previous gaming experience	M = 3.15 (moderate experience), SD = 0.83
Familiarity with rehabilitation training	M = 2.15 (slightly familiar), SD = 1.00

. All are non-professional gamers experiencing minor hand fatigue or sub-health conditions, with no severe motor impairments. The experiment is conducted in a laboratory setting using Leap Motion sensors and a standard laptop platform. After receiving a brief explanation of the system, users complete three training levels independently, each session lasting approximately 15 min. After gameplay, participants complete a structured questionnaire including subjective evaluation and open feedback.

To establish a comparative baseline, the user experience testing also incorporates a control experiment involving conventional equipment-based training methods commonly used in clinical rehabilitation settings. This control setup includes finger spring grips, wrist rollers, and elastic resistance bands, targeting similar muscle groups and movement patterns as our game-based system. The control group performs the same three categories of hand movement tasks as in the game but in a non-digital, mechanical format, without narrative, visual feedback, or cultural immersion.

A 5-point Likert scale questionnaire (1 = strongly disagree, 5 = strongly agree) is designed based on standard user experience evaluation frameworks. It includes three dimensions and six questions. The Usability Layer includes the following: (1) “The way of this training are simple, intuitive, and easy to understand;” (2) “The training tasks are appropriately designed.” The Experience Layer includes the following: (3) “The training experience is enjoyable;” (4) “The training process has rhythm and responsive feedback.” The Emotional Layer includes the following: (5) “This training allows me to immerse in an attractive scene;” (6) “Using this training makes me more willing to continue training.” A paired-sample t-test is performed to analyze differences between the Leap Motion system and traditional training. The results are shown in

Table 7. User experience results.

Dimension	Question	Leap Motion System	Traditional Training	t (38)	p
Usability Layer	Q1	4.08	3.82	1.885	0.067
	Q2	3.59	3.28	2.629	0.012 *
Experience Layer	Q3	3.92	3.67	2.039	0.048 *
	Q4	3.82	3.33	2.349	0.024 *
Emotional Layer	Q5	3.33	2.82	2.857	0.007 *
	Q6	4.31	3.51	4.594	<0.001 *

.

The results indicate that, except for Q1, all other differences between the two systems are statistically significant (p < 0.05), with Q5 and Q6 showing particularly strong effects in favor of the Leap Motion system. This indicates generally positive user attitudes toward the Leap-Motion-based rehabilitation game. The Leap Motion system scored significantly higher than traditional training in terms of usability and fun, indicating that the gesture control method provides intuitive interaction with low learning barriers. In contrast, conventional training methods are viewed as more monotonous and less emotionally engaging. Notably, emotional-related scores for the proposed system are significantly higher, indicating that the integration of cultural context can enhance immersion and foster intrinsic motivation for sustained participation.

To provide a clearer view of the score distribution between the two training modalities, six box plots are generated (Figure 23). Overall, the Leap Motion system shows slightly higher medians and narrower interquartile ranges on most items.

More specifically, both approaches show an equal median on Q1, but the Leap Motion system has a slightly higher upper quartile. For Q2–Q4, the Leap Motion system maintains a median of 4, while the traditional training drops to 3 (notably with a low outlier in Q4). On the emotional items (Q5 and Q6), the Leap Motion system again shows more concentrated and higher responses, whereas the traditional condition presents a wider range of scores and lower outliers.

These visual results are consistent with the statistical analysis and further confirm that the proposed cultural-based interactive system tends to generate more stable and more positive user responses, particularly in terms of engagement and motivation.

In summary, both expert and user evaluations demonstrate that the proposed system performs well in terms of both function and experience. Functionally, the interactive movements align with core rehabilitation mechanisms and show strong relevance to training goals. Experientially, the cultural context and responsive feedback mechanisms effectively increased user motivation and engagement. The preliminary validation supports the feasibility of the proposed triadic mapping model in rehabilitation game design and provides empirical evidence for the “cultural embedding-functional training-interaction-driven” design approach.

7. Discussion

7.1. Practical Guidance of the Mapping Model for Interaction Design in Rehabilitation Games

The proposed “Rehabilitation Mechanism–Intangible Cultural Feature–Game Interaction Design””triadic mapping model offers a structured and function-oriented design approach for rehabilitation games. Traditional rehabilitation games often rely on subjective judgment and designer experience when integrating cultural content. In contrast, this model begins with rehabilitation mechanisms and maps them to cultural features, such as physicality, symbolism, and materiality, based on their functional compatibility. This process ensures that cultural content is selected on the basis of “supporting rehabilitation goals,” thus reducing the arbitrariness and superficiality commonly observed in cultural integration. Moreover, the model translates rehabilitation needs into concrete operational tasks through interaction strategies. By leveraging action patterns, rhythmic cues, and symbolic schemas found in ICH, it provides clear guidance for designing game interaction methods. The coordinated logic, from rehabilitation mechanisms to interaction strategies to cultural features, establishes a replicable and verifiable design model for rehabilitation game development.

On the user experience level, the inclusion of cultural mechanisms brings stronger contextual engagement and emotional regulation. For example, in the puppet-themed rehabilitation game, users not only perform training movements but also engage in character-based role-play. This transforms repetitive exercises into meaningful cultural behavior, enhancing motivation, reducing training fatigue, and improving immersion and adherence throughout the rehabilitation process.

7.2. Potential for Scaling and Adapting the Model in Future Rehabilitation Game Design

The triadic mapping model demonstrates strong generalizability and scalability. On one hand, its logic, starting from rehabilitation goals and using cultural mechanisms for functional support, is not restricted to a specific cultural genre or rehabilitation scenario. It can be adapted to diverse user groups (e.g., children, the elderly, and stroke patients) by flexibly matching cultural resources with game strategies based on users’ physical and psychological needs.

On the other hand, the three categorized cultural features and the six common rehabilitation goals established in the model can serve as modular design units. This modularity supports the future development of a “Cultural + Rehabilitation Content Library,” which can promote both scalable and personalized content generation. Furthermore, with the advancement of AIGC (Artificial-Intelligence-Generated Content) technologies, the model can serve as a logical backbone for automated content generation, enabling the creation of culturally adaptive rehabilitation game solutions. This opens new possibilities for developing intelligent, personalized, and culturally sensitive rehabilitation systems.

7.3. Limitations and Future Work

Despite the framework’s theoretical contribution and practical effectiveness, several limitations remain. First, cultural semantics are inherently subjective. Interpretations may vary across different cultural backgrounds, and the emotional resonance of such semantics is difficult to standardize, which presents challenges for the framework’s repeatability and empirical validation. Second, the current evaluation is limited to a single prototype, the puppet-themed hand rehabilitation game. The framework’s applicability to other forms of ICH, rehabilitation scenarios, and user groups has not yet been tested extensively. Kapp has systematically summarized the application of game mechanics in education and training, highlighting from theoretical, empirical, design, and practical perspectives how gamification effectively enhances learning motivation, engagement, and outcomes. He provides a detailed analysis of the roles of goals, rules, feedback, and narrative as core elements in constructing learning experiences and explains how these mechanisms stimulate intrinsic motivation by integrating flow theory and self-determination theory. Furthermore, through numerous cross-industry cases, including medical simulations, corporate training, and educational courses, Kapp demonstrates the broad applicability of gamification in knowledge acquisition, skill development, emotional regulation, and behavior change. In terms of design methodology, he emphasizes creating tiered incentive systems and personalized learning paths based on user types and advocates using rapid prototyping, iterative testing, and outcome evaluation to continuously optimize gamified solutions, providing a practical framework for translating serious games from concept to implementation [56,57].

Based on these theoretical and practical insights, future research can further incorporate timely feedback, progressive challenges, narrative structures, and culturally relevant reward mechanisms into rehabilitation training games to explore their specific effects on enhancing motivation, sustaining training engagement, and improving rehabilitation outcomes. Additionally, personalized training paths and tiered task objectives can be designed according to the abilities of different rehabilitation participants, while prototyping, iterative optimization, and user experience evaluation are used to continuously refine the usability and long-term effectiveness of serious games in rehabilitation. Moreover, integrating multimodal interaction technologies, such as motion capture, virtual reality, or haptic feedback, can further enhance immersion and engagement, providing a more systematic, scientific, and compelling intervention approach for rehabilitation training.

8. Conclusions

This study addresses the integration of ICH into rehabilitation game design by proposing a triadic mapping framework encompassing “Rehabilitation Mechanism–Cultural Feature–Interaction Design.” Starting from rehabilitation goals, the framework identifies compatible ICH mechanisms and transforms them into interactive elements through logical interaction pathways. It establishes an effective connection among rehabilitation actions, cultural behaviors, and user experience.

At the theoretical level, this framework introduces a function-oriented cultural integration approach. It bridges the gap between rehabilitation goals, cultural semantics, and interaction strategies, overcoming the limitations of superficial cultural integration and fragmented interaction in previous rehabilitation games. At the practical level, the framework was implemented in the development of a hand rehabilitation game inspired by traditional string puppetry. Using Leap Motion for gesture tracking and Unity for interactive development, the game translated culturally meaningful actions into rehabilitation tasks, creating a training system that combines functional recovery, cultural immersion, and engaging interaction.

User testing and questionnaire evaluations confirmed the framework’s feasibility and application value. Overall, this research presents an innovative, operable, and scalable triadic strategy framework that builds a systematic pathway from cultural semantics to rehabilitation mechanisms. It fills the methodological gap in existing rehabilitation game design by offering a structured, culturally driven design framework. Furthermore, it responds to the broader shift in rehabilitation practice from function-centered to experience-centered approaches. By embedding ICH deeply into the design, this study explores a pathway to connect rehabilitation motivation with cultural identity, enriching training with emotional and cultural value. It holds potential to transform rehabilitation products from being merely “usable” to “desirable” and “engaging,” offering valuable guidance for future culturally adaptive rehabilitation game design.

Future research may advance this framework from a structured heuristic framework to a data-driven quantitative system. Large-scale user behavior data and rehabilitation outcome feedback can be used to build an evaluation system for matching cultural features, rehabilitation goals, and interaction strategies. By incorporating expert scoring and the analytic hierarchy process (AHP), a preliminary feature-weighting framework could be developed. On this basis, machine learning algorithms may be introduced to extract key features and optimize matching, transforming the process of cultural selection, rehabilitation content configuration, and interaction design into an intelligent AI-driven workflow. This shift is expected to enhance both efficiency and accuracy in design and fundamentally reshape the traditionally experience-based workflow of rehabilitation game development. Additionally, the approach could be extended to a broader range of rehabilitation tasks and cultural resources, facilitating the evolution of rehabilitation games toward greater personalization, modularity, and intelligence.