Next Article in Journal
Automating an Encoder–Decoder Incorporated Ensemble Model: Semantic Segmentation Workflow on Low-Contrast Underwater Images
Previous Article in Journal
A Train Timetable Optimization Method Considering Multi-Strategies for the Tidal Passenger Flow Phenomenon
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

A Two-Step Neurorehabilitation Program Utilizing Extended Reality and Telerehabilitation for Children with Cerebral Palsy: A Pilot Study on Effectiveness, Adherence, and Technical Feasibility

1
Associazione “La Nostra Famiglia”—IRCCS “E. Medea”—Scientific Hospital for Neurorehabilitation—Unit for Severe Disabilities in Developmental Age and Young Adults (Developmental Neurology and Neurorehabilitation), 72100 Brindisi, Italy
2
Scientific Institute IRCCS “E. Medea”, Scientific Direction, 23842 Bosisio Parini, Italy
*
Author to whom correspondence should be addressed.
Appl. Sci. 2024, 14(24), 11961; https://doi.org/10.3390/app142411961
Submission received: 30 October 2024 / Revised: 17 December 2024 / Accepted: 19 December 2024 / Published: 20 December 2024
(This article belongs to the Section Computing and Artificial Intelligence)

Abstract

:
In recent years, extended reality (XR) and telerehabilitation (TR) technologies have increasingly been used in the neurorehabilitation of motor dysfunctions in patients with cerebral palsy (CP). The Khymeia Virtual Reality Rehabilitation System (K-VRRS) is a medical device specifically designed for neuromotor rehabilitation, and it can also be used in TR mode. This pilot study aims to evaluate the effectiveness and adherence to a “two-step neuromotor program” (TS-NP) approach using K-VRRS to enhance upper limb motor functions in children with CP. The TS-NP protocol consists of two phases. In the first phase, patients undergo intensive motor training with K-VRRS during a period of hospitalization. In the second phase, initiated after discharge, patients continue K-VRRS treatment at home through TR, building upon the progress made during their hospital stay. A total of seven children with unilateral spastic CP (ages 4–10 years) were assessed at three time points: baseline (T0), after the first phase of in-person hospital treatment (T1), and following the second phase of TR treatment at home (T2). Standardized outcome measures were used, with the primary measure being the Melbourne Assessment 2. Preliminary data support the hypothesis that intensive K-VRRS treatment during hospitalization enhances motor function in the affected upper limb of children with CP. Furthermore, continuing K-VRRS treatment at home through TR appears crucial for maintaining the motor gains achieved during the hospital phase.

1. Introduction

1.1. Background

Cerebral palsy (CP) is defined as a group of permanent disorders of movement and/or posture resulting from non-progressive injury to the developing brain, occurring either during fetal development or in early childhood [1,2,3]. These neurological impairments result in lifelong challenges that affect motor control, leading to limitations in daily activities and social engagement. In addition to mobility difficulties, individuals with CP often experience secondary complications, including musculoskeletal problems, epilepsy, cognitive impairments, and behavioral challenges, all of which can further impact their quality of life [1]. Neurorehabilitation treatments for children with cerebral palsy (CP) target all aspects of the International Classification of Functioning, Disability, and Health (ICF) framework, using a motor learning approach [4,5]. The primary goal is to enhance motor function by prioritizing child-initiated movement. The approach incorporates strategies such as task specificity, goal-directed training, high-intensity training, parental engagement (aligned with family-centered care), and environmental modifications [6,7]. These interventions are often supported by pharmacological treatments, such as oral baclofen and botulinum toxin injections, as well as surgical options. The combined approach seeks to optimize motor function during key periods of neuroplasticity, ultimately improving the quality of life for individuals with CP [6,8,9,10]. In recent years, the digital age has introduced transformative technologies and e-health tools, paving the way for groundbreaking treatments across diverse healthcare domains. Among these, “extended reality” (XR) has emerged as a comprehensive term that encompasses a range of immersive technologies, including augmented reality (AR), mixed reality (MR), and virtual reality (VR). XR applications, combined with advancements in telerehabilitation (TR), have demonstrated substantial potential in the field of neurorehabilitation, showing promise particularly in addressing motor dysfunctions in individuals with cerebral palsy [11,12,13,14]. Beyond CP, these technologies are increasingly applied to address both motor and cognitive impairments across a wide spectrum of neurological and neuropsychological conditions, including stroke, neurodegenerative diseases, and neurodevelopmental disorders [15,16,17].
Virtual reality (VR) is an advanced human–computer interface that allows users to engage with and immerse themselves in a simulated environment, often crafted to mirror elements of the real world [18]. VR enables individuals to feel as though they are in a different location from their physical surroundings, creating a “sense of presence”—the feeling of genuinely being within the virtual setting. This immersive experience responds naturally to stimuli, involving not only visual perception but also engaging the user’s physiological and neural responses. This effect, known as “embodiment”, describes the sensation of having a body within the virtual space that behaves as it would in reality. This embodied experience can serve as a rehabilitation tool, promoting and enhancing functional recovery [19,20].
VR systems can be broadly categorized into immersive, semi-immersive, and non-immersive types [21]. The term “immersion” refers to the degree to which a VR system provides a convincing sensorimotor experience by simulating reality through sensory input [22]. Immersive virtual reality (VR) is designed to transport users into computer-generated three-dimensional environments, offering a sensation of total immersion. Aiming to enhance the sense of presence in a digital world, immersive VR goes beyond mere visual engagement, creating an experience that blurs the boundaries between reality and simulation. In this context, the term “immersive” refers to VR’s ability to fully envelop the user in a three-dimensional (3D) world, stimulating the senses to enable natural interaction with the virtual environment. To achieve this effect, immersive VR uses key components, including the headset, or Head-Mounted Display (HMD), which the user wears like a helmet and contains screens for each eye, producing a 3D stereoscopic effect. These headsets are often equipped with motion sensors, allowing users to look around and visually interact with the simulated environment.
Another essential component is haptic feedback devices, such as gloves, vests, or controllers, which allow users to physically perceive objects within the virtual world. This type of sensory feedback adds a tactile dimension to the experience, intensifying realism and making the immersion even more engaging. Some examples of devices used in immersive VR are: Cave Automatic Virtual Environment (CAVE), Nirvana-BTS Bioengineering, Quincy, MA 02169, USA, GRAIL VR (Motekforce Link, Amsterdam, The Netherlands), and Virtual Reality Rehabilitation System (VRRS, Khymeia srl, Padua, Italy) integrated with the HMD.
In contrast, non-immersive VR systems do not rely on a headset. Instead, they present virtual environments on standard screens or touch monitors, allowing users to engage using familiar devices such as mice, keyboards, controllers, or consoles. A prime example is the VRRS system, which records movements and displays them in a virtual setting on a large screen. The level of sensorimotor realism in VR systems varies along a continuum, depending on the underlying technology [23].
Advances in computing have paved the way for blending VR with the physical world, resulting in augmented reality (AR). AR systems incorporate a display, such as a head-mounted display (HMD) or monitor, a position-tracking device, and a camera to enable users to see their surroundings. This real-world view is enhanced by computer-generated objects and context-specific information, overlaying digital elements onto the natural environment precisely when and where they are needed. Unlike VR, which fully immerses the user in a virtual environment separate from the real world, AR centers the real world in the experience, enriching it with virtual elements that allow users to engage with both digital and physical objects (Figure 1).
Tele-rehabilitation (TR), a subset of telemedicine, uses telecommunications to remotely deliver various rehabilitation services, including VR-based neuromotor treatments [24]. Both VR- and TR-based approaches offer several potential advantages over traditional rehabilitation methods [18,25,26]. For instance, VR treatments may provide greater ecological validity than traditional neurorehabilitation by simulating real-world environments within virtual settings. This resemblance may improve the transfer of skills learned in VR to real-life tasks [18,27]. However, the evidence for skill transfer remains mixed, with some reviews highlighting the need for further research [25,28].
Moreover, VR and TR can boost patient motivation and adherence to rehabilitation programs, which is essential for promoting long-term neural plasticity and functional recovery [29,30]. Achieving sustained neurofunctional recovery requires extended treatment periods with frequent, intensive, and repetitive exercises that are both engaging and rewarding. Compared to conventional neurorehabilitation exercises, VR tasks offer a more enjoyable and stimulating experience, resulting in greater user engagement and increased repetition rates [25,31].
The lengthy nature of neurorehabilitation programs can pose significant logistical challenges, such as time, travel, and the need for specialized staff during each session. These factors often create barriers that reduce patient adherence and limit access to consistent treatment (i.e., fewer treatment sessions). TR addresses these challenges by minimizing travel requirements and reducing dependence on continuous supervision. TR-based exercises can be thoughtfully tailored, enabling patients to perform them independently while still benefiting from personalized and effective remote care [26,32].
The Khymeia Virtual Reality Rehabilitation System (K-VRRS) is a medical device designed for neuromotor and cognitive rehabilitation, with the added flexibility of being used in telerehabilitation (TR) modality. K-VRRS has been extensively applied in the treatment of various neurological and neuropsychological conditions. Research indicates that K-VRRS-based interventions, whether applied alone or in combination with traditional therapies, hold potential for neuromotor rehabilitation in adults. However, there is a significant lack of studies exploring its use for neuromotor rehabilitation in children and adolescents [33] (Figure 2).

1.2. Objectives and Hypotheses

The objective of this pilot study is to examine, for the first time in the literature, the potential effectiveness, technical feasibility, and adherence to a ‘two-step neurorehabilitation program’ (TS-NP) using K-VRRS in combination with standard physiotherapy to enhance upper limb motor function in children with cerebral palsy (CP), as well as the usability of the technology employed (i.e., VRRS).
The TS-NP in this study involves two distinct phases. In the first phase, patients undergo standard physiotherapy combined with intensive motor training with K-VRRS during their hospital stay. The second phase begins post-discharge, where patients continue the K-VRRS therapy initiated in the hospital from home through telerehabilitation. This study intends to evaluate both the immediate effects of intensive K-VRRS therapy during hospitalization (Phase One) and the impact of sustained motor rehabilitation through telerehabilitation following discharge (Phase Two). We hypothesize that the TS-NP may lead to an improvement in motor function both after intensive treatment and following the treatment in TR.

2. Material and Methods

2.1. Participants

Participants were consecutively recruited from children admitted to a comprehensive in-patient neurorehabilitation program at the Unit for Severe Disabilities in Developmental Age and Young Adults (Developmental Neurology and Neurorehabilitation), Association “La Nostra Famiglia”—IRCCS “E. Medea”, Scientific Hospital for Neurorehabilitation, Brindisi.
Participants were eligible for inclusion if they met the following criteria: age between 5 and 15 years, a diagnosis of unilateral spastic cerebral palsy (CP) and classification at levels I or II on the Gross Motor Function Classification System (GMFCS) and the Visual Function Classification System (VFCS). Exclusion criteria were as follows: GMFCS level > II, recent treatment with Botulinum toxin, significant attention deficits, or an IQ below 50. Attention deficits were measured using the visual attention subtest of the NEPSY-Italian version [34,35], and intellectual functioning was evaluated with the Wechsler Intelligence Scale for Children and the Wechsler Preschool and Primary Scale of Intelligence [36,37].
This study was approved by the Ethic Committee of Istituto Tumori “Giovanni Paolo II”—IRCCS, with the reference number 15/CE Medea 19 October 2021, and informed consent was obtained from all subjects involved in the study and their parents.

2.2. K-VRRS Technologies and Intervention

K-VRRS (depending on the technological components used) allows the implementation of both immersive and non-immersive virtual reality-based treatments. In this study, we implemented non-immersive virtual reality-based treatments. The first phase of the treatment was conducted during the participants’ hospitalization at our facility, using the K-VRRS-EVO system. K-VRRS-EVO consists of a PC station connected to a high-resolution touchscreen LCD monitor, which displays virtual activities, and a 3D motion tracking system.
This motion tracking system includes a series of sensors placed on the subject to detect and reproduce their movements in real time within a three-dimensional virtual environment. The sensor technologies used include K-sensors, inertial sensors for measuring acceleration and angular velocity, and passive 3D magnetic sensors. These sensors are attached to anatomical bands or gloves and, in conjunction with a low-intensity electromagnetic field generator, allow for real-time kinematic tracking and motion detection. The VRRS software (version 5.2.32.16055) processes data collected by magnetic sensors, which capture the spatial coordinates (x, y, z) of a joint segment, and by inertial sensors (IMUs), which measure angular velocities and linear accelerations. These data are analyzed using algorithms for signal integration and processing. This computational pipeline extracts kinematic descriptors (e.g., velocity, path length), movement smoothness indices (e.g., jerk), and temporal metrics (e.g., movement execution time), providing information concerning the spatiotemporal dynamics of motion.
During the first phase of the treatment, participants underwent both standard physiotherapeutic treatment and intensive K-VRRS-based neuromotor rehabilitation. Standard physiotherapy intervention primarily focused on functional training of one or both arms, emphasizing activities oriented toward specific functions and guided by principles of motor learning and control. The K-VRRS-based treatment included two in-person rehabilitation sessions per day, six days a week, with each session lasting approximately 45 min, throughout the hospitalization period.
Each session involved the patients performing 3D virtual games and tasks, such as controlling a diver to collect fish and seahorses or moving an object vertically and horizontally along simple or complex trajectories within the virtual environment. Additionally, participants performed Catching (gripping) and Reaching (reaching) exercises, which required them to execute flexion/extension movements, shoulder internal/external rotation, and elbow flexion/extension across three planes: frontal, sagittal, and transverse.
The second phase of the treatment was conducted via TR. The K-VRRS system facilitates TR through two distinct workstations: one located at the patient’s home and the other at the rehabilitation hospital. The home workstation, referred to as the VRRS-Home Kit, includes a tablet, a motion detection camera, and wearable sensors for movement tracking. Two types of sensors were used in the Home Kit: the K-Sensor and the K-Wand, which together enable tele-motor rehabilitation.
The K-Wand is an innovative device designed for the tele-motor rehabilitation of the upper limb and trunk. It features exclusive light recognition technology, allowing it to detect movements in depth. The K-Sensor consists of a set of inertial sensors placed on the upper limb using anatomical bands.
The hospital workstation, known as the TeleCockpit, is a technological setup equipped with an integrated videoconferencing system that allows therapists to control the home device and monitor the patient’s actions via the Home Kit. TR sessions can be conducted in two ways: (i) online, with real-time interaction between the patient and therapist via videoconference, or (ii) offline, where the patient performs exercises independently with the guidance of a Smart Virtual Assistant.
During this phase, participants attended at least one and up to three online sessions per week with a therapist. In addition, they could choose to complete any number of offline sessions. Each session lasted approximately 45 min. During each TR session, the patient performed gripping (Catching) and reaching (Reaching) exercises (as described above).

2.3. Evaluation Timeline and Outcome Measures

Participants were evaluated longitudinally at three time points (T0, T1, and T2) using standardized outcome measures (detailed below). The first assessment (T0—baseline) was conducted prior to the start of treatment. The second assessment (T1—before discharge) occurred at the conclusion of the initial phase of face-to-face treatment administered during hospitalization. The final assessment (T2) was conducted at the end of the second phase of treatment delivered via telemedicine (TR) (Figure 3).
The primary outcome measure was the Melbourne Assessment 2 (MA2) [38], a tool designed to assess four key aspects of upper limb movement quality in children aged 2.5 to 15 years with neurological impairments. The four assessed aspects are: (i) Range of Movement (MA2-RM), (ii) Accuracy of Reach and Placement (MA2-ARP), (iii) Dexterity in Grasp, Release, and Manipulation (MA2-DEX), and (iv) Fluency of Movement (MA2-FM). The MA2 allows for the calculation of four separate scores, each reflecting one of these key aspects of upper limb movement quality.
The secondary outcome measures included:
  • The Besta Scale [39];
  • A subtest measuring manual dexterity from the Movement Assessment Battery for Children—MABC-2 [40];
  • The Pediatric Quality of Life Inventory—Cerebral Palsy Module (PedsQL) [41].
The Besta Scale (BS) consists of three subscales that measure the following: grasp function of the paretic hand (Besta A), spontaneous use of the paretic hand in bimanual activities (Besta B), and the use of the paretic hand in daily living activities (Besta C). Each subscale score ranges from 0 to 12, with higher scores indicating better motor performance.
The MABC-2 is a standardized test used to assess children’s motor skills, including manual dexterity, aiming and catching, and balance. In this study, we used the manual dexterity subtest, specifically the “posting coins” task, where children are asked to place 12 coins into a container through a slot. Both hands are tested independently. Although this subtest is designed for children aged 3–6 years to assess manual dexterity, in our study, it was used to measure the speed of the paretic limb during the task. Children performed the task only with their paretic arm, and the score was determined by the number of seconds required to complete the task.
The PedsQL measures health-related quality of life (HRQoL) specific to cerebral palsy (CP) and has two formats: a parent-report format (P-PedsQL) that assesses parents’ perceptions of their child’s quality of life, and a child self-report (S-PedsQL). The PedsQL consists of seven scales: (1) Daily Activities, (2) School Activities, (3) Movement and Balance, (4) Pain and Hurt, (5) Fatigue, (6) Eating Activities, and (7) Speech and Communication. Each scale is scored from 0 to 100, with higher scores indicating a better quality of life.
Finally, at T2, the usability of the K-VRRS was assessed using the System Usability Scale (SUS) [42]. The SUS is a ten-item scale that measures users’ subjective evaluation of the usability of technological systems and devices. The total score ranges from 0 to 100, with higher scores indicating better usability. In this study, the SUS was administered to the parents of the participating children. All the scales and tests used as outcome measures have good psychometric properties [38,39,40,41,42]. Each of the different tests was administered by a different professional (neuropsychomotor specialists, physiotherapists, psychologists, and occupational therapists), all of whom routinely conduct the specific tests in standard clinical practice. However, the same test was administered to all the children by the same professional. The person administering the tests was not blind to the research protocol.

2.4. Statistical Analysis

We conducted all statistical analyses using Jamovi Statistical Software (Version 1.6.23.0) (https://www.jamovi.org, accessed on 10 May 2024). To assess whether our data followed a normal distribution, we used the Shapiro–Wilk test. While some of the outcome variables were normally distributed, others were not. Given the very small sample size and the non-normal distribution of some of the analyzed variables, we opted for nonparametric methods. To evaluate changes in outcome measures over time, we applied the Friedman test for repeated measures. For post hoc pairwise comparisons, we used the Durbin-Conover test. The significance level was set at p < 0.05. Eta-square (η2) values were calculated to determine the magnitude of the change in outcome measures over time.

3. Results

A total of seven children (five males and two females, aged 4–10 years) were included in the study (Table 1 presents participant characteristics). All seven children completed the first phase of the study. However, during the second phase, one participant withdrew and did not undergo TR treatment. In the first phase, the median number of treatment sessions completed was 34 (range: 29–37 sessions), and the median length of hospitalization was 29 days (range: 18–33 days). In the second phase, the median number of sessions completed was nine (range: 6–13 sessions).
Regarding the primary outcome measure, there was a significant change over time in the MA2-Range of movement score (χ2(2) = 11.2, p = 0.004; η2: 0.2) and in the MA2-Fluency of movement score (χ2(2) = 6.74, p = 0.034; η2: 0.2). As shown in Table 2 (see also Figure 4 and Figure 5; box plots related to the outcome measures at different assessment points are shown in the Supplementary Materials), the Durbin-Conover pairwise comparison test revealed a statistically significant increase in the MA2-Range of movement score over time between the assessment at T0 and T1, as well as between T0 and T2, while the change from T1 to T2 was not statistically significant. The Durbin-Conover test also indicated a statistically significant increase in the MA2-Fluency of movement score from T0 to T1 and from T0 to T2, but no significant change from T1 to T2.
The increase in other MA2 scores, such as the MA2-Accuracy of Reach and Placement and MA2-Dexterity in Grasp, Release, and Manipulation scores, was only marginally statistically significant (MA2-ARP: χ2(2) = 5.70, p = 0.058; MA2-DEX: χ2(2) = 5.44, p = 0.06).
For the secondary outcome measure, we observed a statistically significant increase over time in certain Parent-PedsQL scores: Parent-PedsQL Daily Activities score (χ2(2) = 9.33, p = 0.009; η2: 0.3), Parent-PedsQL 3.0 Movement and Balance score (χ2(2) = 7.28, p = 0.026; η2: 0.3), and Parent-PedsQL Fatigue score (χ2(2) = 9.29, p = 0.010; η2: 0.2). The Durbin-Conover pairwise comparison test indicated that the P-PedsQL Daily Activities score increased significantly from T0 to T2, as well as from T1 to T2. Additionally, the Parent-PedsQL Movement and Balance score showed a statistically significant increase only from T0 to T2, while the Parent-PedsQL Fatigue score increased significantly from T0 to T1 and from T0 to T2. No significant changes were observed over time for the other secondary outcome measures.
A marginally significant increase in the Besta B score was found (χ2(2) = 5.60, p = 0.06). Finally, the mean SUS score was 72.5.

4. Discussion

In this pilot study, we implemented a TS-NP combined with K-VRRS in children with unilateral spastic cerebral palsy (CP) to examine its effects on upper limb motor functions. Specifically, we investigated the impact of intensive motor treatment with K-VRRS, administered alongside standard physiotherapy during hospitalization, and the importance of continuing maintenance treatment with K-VRRS via tele-rehabilitation (TR) at home after discharge. To date, only a few case studies have explored the effectiveness of K-VRRS for younger populations. One study reported improvements in posture and balance in a 17-year-old female with myopathy and moderate intellectual disability following K-VRRS treatment via telerehabilitation [43]. More recent findings demonstrated enhanced motor functions in a 13-year-old girl with spinal muscular atrophy after K-VRRS-based telerehabilitation [44]. Additionally, another investigation documented a reduction in upper limb motor impairments in children with congenital hemiplegia when combining traditional physiotherapy with K-VRRS interventions during in-person rehabilitation sessions [45]. In summary, only two single-case studies have used K-VRRS in telerehabilitation mode [43,44] and only one study implemented a rehabilitation program that used VRRS during in-person sessions conducted at the rehabilitation center [45].
Our findings suggest that intensive K-VRRS treatment during hospitalization, combined with standard physiotherapy, improves motor functions of the affected upper limb in children with unilateral spastic CP. These improvements, observed in measures such as range of movement and movement fluency (as assessed by MA2), indicate that such an inpatient treatment regimen positively influences motor function. However, the continuation of K-VRRS treatment via TR post-discharge did not result in further functional improvements beyond those achieved during hospitalization. Nonetheless, TR was crucial for maintaining the gains made during inpatient treatment, as evidenced by better motor function outcomes at the T2 assessment compared to the initial T0 assessment.
While the improvement in motor function from T0 to T1 highlights the benefits of inpatient K-VRRS treatment, the lack of significant changes between T1 and T2 suggests that post-discharge TR may primarily serve a maintenance role rather than driving further enhancements. It is worth noting that the number of TR sessions in our study was relatively low, which may have limited its effectiveness. Future studies might explore whether increasing the frequency of TR sessions could yield further motor improvements in children with CP, as other research has indicated [43].
Regarding the impact of TS-NP on quality of life, we found a statistically significant increase in certain PedsQL measures, suggesting that the TS-NP approach used in this study improved parents’ perceptions of their child’s quality of life. Additionally, the usability of the K-VRRS system was rated positively, with a mean SUS score of 72.5, indicating good usability in line with Bangor et al. [46].
We did not observe statistically significant changes in secondary motor outcome measures such as the “posting coins” sub-test and the BESTA scale. The lack of improvement in the “posting coins” sub-test could be due to the fact that this task involves several fine motor, visuospatial, and visual-motor coordination skills, which were not directly targeted by our intervention. Although the BESTA scale scores did not show significant changes over time, we found a marginally significant increase in the BESTA B score, suggesting that K-VRRS-based treatment may enhance the spontaneous use of the paretic hand in bimanual activities. Further research with larger sample sizes is needed to confirm these findings and potentially identify significant improvements in BESTA scale scores following K-VRRS treatment.
The neuromotor improvements associated with TS-NP using K-VRRS that we found may be attributed to several intrinsic characteristics of the treatment itself. Firstly, interventions that involve high doses of treatment tend to produce superior therapeutic effects [29,30]. Therefore, our protocol emphasized an intensive and prolonged approach. Secondly, the implementation of TS-NP in this study, utilizing K-VRRS, incorporated two additional factors that could significantly enhance the effectiveness of neuromotor rehabilitation: motivation for treatment and error feedback. Indeed, the enjoyable and stimulating nature of VR tasks, along with enhanced feedback on errors, are key reasons why VR training may facilitate neuromotor recovery [25,30,31]. In summary, TS-NP with K-VRRS is likely to improve neuromotor functions due to its foundation in extended treatment periods that involve frequent, intensive, and repetitive exercises that are both engaging and rewarding.
Our study has certain limitations. Performing a power analysis using G*Power software (www.psycho.uni-duesseldorf.de/abteilungen/aap/gpower3, accessed on 10 May 2024), we found that our study should have involved a sample of 43 subjects to achieve an effect size of 0.25 and a test power of 0.95. The very small sample size restricts the generalizability of our findings. Moreover, the absence of a control group limits the study’s internal validity. To evaluate the effect of the treatment, we conducted a longitudinal comparison of changes in outcome measures between baseline and subsequent assessments. Some patients (though not all) underwent additional treatments after discharge. In the absence of a control group, it is not possible to determine the effectiveness of the VRRS treatment without accounting for other variables that might have contributed to improvements in motor performance. Future research should incorporate larger sample sizes and control groups to more accurately evaluate the effects of K-VRRS-based motor treatment in children with CP.

5. Conclusions

TS-NP with K-VRRS has been implemented in previous studies [33], but to our knowledge, it has not yet been applied in the neuromotor rehabilitation of children with cerebral palsy (CP). Our study suggests the hypothesis that intensive TS-NT with K-VRRS administered during hospitalization could induce an improvement in motor function in the impaired upper limb of children with unilateral spastic CP. Additionally, our data suggest that TR with K-VRRS can provide continuity or extend the benefits of treatments initiated in rehabilitation centers, which may be critical for maintaining the motor improvements achieved during the hospitalization period. However, randomized, double-blind controlled studies with larger sample sizes are necessary to thoroughly assess the efficacy of K-VRRS-based neuromotor treatments in enhancing upper limb motor function in children with CP. Moreover, the use of K-VRRS relies on various technologies (e.g., internet access), which may hinder accessibility to rehabilitation programs based on this system. Therefore, further studies are needed to fully understand the challenges of implementing K-VRRS-based treatments. Finally, comprehensive cost-effectiveness and economic analyses should be conducted to determine whether rehabilitation with the VRRS, either during hospitalization or through the TR modality, reduces costs for healthcare providers and patients compared to conventional (face-to-face) treatments. The limitations of the present study prevent the generalization of the results observed in our sample and do not allow conclusions to be drawn regarding the effectiveness of the TS-NP with K-VRRS. We are aware that further studies are needed to evaluate the hypotheses raised by the present study. Nevertheless, we consider the data we have collected in the present study encouraging and potentially motivating for further studies concerning the application and the effectiveness of TS-NP with K-VRRS in improving motor function in patients with cerebral palsy.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/app142411961/s1. Figure S1: Box plots related to Range of movement scores as measured by Melbourne Assessment 2 at T0. Figure S2: Box plots related to Range of movement scores as measured by Melbourne Assessment 2 at T1. Figure S3: Box plots related to Range of movement scores as measured by Melbourne Assessment 2 at T2. Figure S4: Box plots related to Fluency of movement score as measured by Melbourne Assessment 2 at T0. Figure S5: Box plots related to Fluency of movement score as measured by Melbourne Assessment 2 at T1. Figure S6: Box plots related to Fluency of movement score as measured by Melbourne Assessment 2 at T2.

Author Contributions

Concept/idea: A.T. and L.M.; Research design: L.M., A.T., G.B., M.D.R., C.F., V.N. and G.P.; Writing: L.M. and A.T.; Data collection: I.F., M.C.O., I.G., M.D.R., G.A., V.N. and G.P.; Data analysis: L.M., A.T., C.F. and G.P.; Project management: A.T. and L.M.; Consultation (including review of manuscript before submitting): L.M., A.T., C.F., I.F., M.C.O., I.G., M.D.R., G.A. and G.P.; Final approval of the Manuscript: A.T. and L.M. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the “5 per mille” 2021–funds for biomedical research-Italian Health Ministry–by Antonio Trabacca.

Institutional Review Board Statement

The study was approved by the Ethic Committee of Istituto Tumori “Giovanni Paolo II”—IRCCS, with the reference number 15/CE Medea 19 October 2021.

Informed Consent Statement

Informed consent was obtained from all subjects involved in this study.

Data Availability Statement

Data will be available on request to the corresponding author.

Acknowledgments

The authors extend their heartfelt thanks to the hospital staff, parents, and children, whose support enables our research activities.

Conflicts of Interest

The authors declare no conflicts of interest.

References

  1. Rosenbaum, P.; Paneth, N.; Leviton, A.; Goldstein, M.; Bax, M.; Damiano, D.; Dan, B.; Jacobsson, B. A report: The definition and classification of cerebral palsy April 2006. Dev. Med. Child Neurol. Suppl. 2007, 109, 8–14. [Google Scholar] [PubMed]
  2. Christine, C.; Dolk, H.; Platt, M.J.; Colver, A.; Prasauskiene, A.; Krägeloh-Mann, I. Recommendations from the SCPE collaborative group for defining and classifying cerebral palsy. Dev. Med. Child Neurol. Suppl. 2007, 109, 35–38. [Google Scholar] [CrossRef] [PubMed]
  3. Sadowska, M.; Sarecka-Hujar, B.; Kopyta, I. Cerebral Palsy: Current Opinions on Definition, Epidemiology, Risk Factors, Classification and Treatment Options. Neuropsychiatr. Dis. Treat. 2020, 16, 1505–1518. [Google Scholar] [CrossRef] [PubMed]
  4. World Health Organization. International Classification of Functioning, Disability, and Health: Children & Youth Version: ICF-CY; World Health Organization: Geneva, Switzerland, 2007. [Google Scholar]
  5. Schiariti, V.; Longo, E.; Shoshmin, A.; Kozhushko, L.; Besstrashnova, Y.; Król, M.; Neri Correia Campos, T.; Náryma Confessor Ferreira, H.; Verissimo, C.; Shaba, D.; et al. Implementation of the International Classification of Functioning, Disability, and Health (ICF) Core Sets for Children and Youth with Cerebral Palsy: Global Initiatives Promoting Optimal Functioning. Int. J. Environ. Res. Public Health 2018, 15, 1899. [Google Scholar] [CrossRef]
  6. Morgan, C.; Darrah, J.; Gordon, A.M.; Harbourne, R.; Spittle, A.; Johnson, R.; Fetters, L. Effectiveness of motor interventions in infants with cerebral palsy: A systematic review. Dev. Med. Child Neurol. 2016, 58, 900–909. [Google Scholar] [CrossRef] [PubMed]
  7. Molinaro, A.; Fedrizzi, E.; Calza, S.; Pagliano, E.; Jessica, G.; Fazzi, E. Family-centred care for children and young people with cerebral palsy: Results from an Italian multicenter observational study. Child Care Health Dev. 2017, 43, 588–597. [Google Scholar] [CrossRef] [PubMed]
  8. Trabacca, A.; Russo, L.; Losito, L.; Rinaldis, M.D.; Moro, G.; Cacudi, M.; Gennaro, L. The ICF-CY perspective on the neurorehabilitation of cerebral palsy: A single case study. J. Child Neurol. 2012, 27, 183–190. [Google Scholar] [CrossRef] [PubMed]
  9. Bekteshi, S.; Monbaliu, E.; McIntyre, S.; Saloojee, G.; Hilberink, S.R.; Tatishvili, N.; Dan, B. Towards functional improvement of motor disorders associated with cerebral palsy. Lancet Neurol. 2023, 22, 229–243. [Google Scholar] [CrossRef]
  10. Trabacca, A.; Vespino, T.; Di Liddo, A.; Russo, L. Multidisciplinary rehabilitation for patients with cerebral palsy: Improving long-term care. J. Multidiscip. Healthc. 2016, 9, 455–462. [Google Scholar] [CrossRef]
  11. Chen, Y.; Fanchiang, H.D.; Howard, A. Effectiveness of Virtual Reality in Children with Cerebral Palsy: A Systematic Review and Meta-Analysis of Randomized Controlled Trials. Phys. Ther. 2018, 98, 63–77. [Google Scholar] [CrossRef] [PubMed]
  12. Nguyen, T.; Choromanski, L.; Kreuzer, T.; Stroppini, J. Exploring the Feasibility of a Virtual, Home-Based MusicGlove® Protocol for Children with Hemiparetic Cerebral Palsy. Open J. Occup. Ther. 2022, 10, 1–15. [Google Scholar] [CrossRef]
  13. Rathinam, C.; Mohan, V.; Peirson, J.; Skinner, J.; Nethaji, K.S.; Kuhn, I. Effectiveness of virtual reality in the treatment of hand function in children with cerebral palsy: A systematic review. J. Hand Ther. 2019, 32, 426–434.e1. [Google Scholar] [CrossRef]
  14. Tamboosi, M.E.; Al-Khathami, S.S.; El-Shamy, S.M. The effectiveness of tele-rehabilitation on improvement of daily living activities in children with cerebral palsy: Narrative review. Bull. Fac. Phys. Ther. 2021, 26, 40. [Google Scholar] [CrossRef]
  15. Kim, O.; Pang, Y.; Kim, J.H. The effectiveness of virtual reality for people with mild cognitive impairment or dementia: A meta-analysis. BMC Psychiatry 2019, 19, 219. [Google Scholar] [CrossRef] [PubMed]
  16. Laver, K.E.; Lange, B.; George, S.; Deutsch, J.E.; Saposnik, G.; Crotty, M. Virtual reality for stroke rehabilitation. Cochrane Database Syst. Rev. 2017, 11, CD008349. [Google Scholar] [CrossRef]
  17. Wang, M.; Reid, D. Virtual reality in pediatric neurorehabilitation: Attention deficit hyperactivity disorder, autism and cerebral palsy. Neuroepidemiology 2011, 36, 2–18. [Google Scholar] [CrossRef] [PubMed]
  18. Schultheis, M.T.; Rizzo, A.A. The application of virtual reality technology in rehabilitation. Rehabil. Psychol. 2001, 46, 296. [Google Scholar] [CrossRef]
  19. Maselli, A.; Slater, M. The building blocks of the full body ownership illusion. Front. Hum. Neurosci. 2013, 7, 83. [Google Scholar] [CrossRef] [PubMed]
  20. Dewe, H.; Gottwald, J.M.; Bird, L.A.; Brenton, H.; Gillies, M.; Cowie, D. My Virtual Self: The Role of Movement in Children’s Sense of Embodiment. IEEE Trans. Vis. Comput. Graph. 2022, 28, 4061–4072. [Google Scholar] [CrossRef] [PubMed]
  21. Riva, G.; Mancuso, V.; Cavedoni, S.; Stramba-Badiale, C. Virtual reality in neurorehabilitation: A review of its effects on multiple cognitive domains. Expert Rev. Med. Devices 2020, 17, 1035–1061. [Google Scholar] [CrossRef]
  22. Slater, M.; Wilbur, S. A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments. Presence Teleoperators Virtual Environ. 1997, 6, 603–616. [Google Scholar] [CrossRef]
  23. Weiss, P.L.; Kizony, R.; Feintuch, U.; Katz, N. Virtual reality in neurorehabilitation. Textb. Neural Repair Rehabil. 2006, 51, 182–197. [Google Scholar]
  24. Piron, L.; Turolla, A.; Agostini, M.; Zucconi, C.; Cortese, F.; Zampolini, M.; Zannini, M.; Dam, M.; Ventura, L.; Battauz, M.; et al. Exercises for paretic upper limb after stroke: A combined virtual-reality and telemedicine approach. J. Rehabil. Med. 2009, 41, 1016–1102. [Google Scholar] [CrossRef] [PubMed]
  25. Howard, M.C. A meta-analysis and systematic literature review of virtual reality rehabilitation programs. Comput. Hum. Behav. 2017, 70, 317–327. [Google Scholar] [CrossRef]
  26. Nuara, A.; Fabbri-Destro, M.; Scalona, E.; Lenzi, S.E.; Rizzolatti, G.; Avanzini, P. Telerehabilitation in response to constrained physical distance: An opportunity to rethink neurorehabilitative routines. J. Neurol. 2022, 269, 627–638. [Google Scholar] [CrossRef] [PubMed]
  27. Rizzo, A.A.; Buckwalter, J.G.; van der Zaag, C. Virtual environment applications in clinical neuropsychology. In Handbook of Virtual Environments; CRC Press: Boca Raton, FL, USA, 2002; pp. 1067–1104. [Google Scholar]
  28. Levac, D.E.; Huber, M.E.; Sternad, D. Learning and transfer of complex motor skills in virtual reality: A perspective review. J. Neuroeng. Rehabil. 2019, 16, 121. [Google Scholar] [CrossRef]
  29. Cheung, K.L.; Tunik, E.; Adamovich, S.V.; Boyd, L.A. Neuroplasticity and virtual reality. In Virtual Reality for Physical and Motor Rehabilitation; Springer: New York, NY, USA, 2014; pp. 5–24. [Google Scholar]
  30. Kleim, J.A.; Jones, T.A. Principles of experience-dependent neural plasticity: Implications for rehabilitation after brain damage. J. Speech Lang. Hear. Res. JSLHR 2008, 51, S225–S239. [Google Scholar] [CrossRef]
  31. Lewis, G.N.; Rosie, J.A. Virtual reality games for movement rehabilitation in neurological conditions: How do we meet the needs and expectations of the users? Disabil. Rehabil. 2012, 34, 1880–1886. [Google Scholar] [CrossRef]
  32. Seelman, K.D.; Hartman, L.M. Telerehabilitation: Policy issues and research tools. Int. J. Telerehabilit. 2009, 1, 47. [Google Scholar] [CrossRef] [PubMed]
  33. Macchitella, L.; Amendola, S.; Barraco, G.; Scoditti, S.; Gallo, I.; Oliva, M.C.; Trabacca, A. A narrative review of the use of a cutting-edge virtual reality rehabilitation technology in neurological and neuropsychological rehabilitation. NeuroRehabilitation 2023, 53, 439–457. [Google Scholar] [CrossRef] [PubMed]
  34. Korkman, M.; Kirk, U.; Kemp, S. NEPSY-II, Italian Version; Giunti OS Organizzazioni Speciali: Firenze, Italy, 2011. [Google Scholar]
  35. Urgesi, C.; Campanella, F.; Fabbro, F. NEPSY-II. Contributo Alla Taratura Italiana; Giunti OS: Firenze, Italy, 2011. [Google Scholar]
  36. Orsini, A.; Pezzuti, L.; Picone, L. WISC-IV: Contributo Alla Taratura Italiana, WISC-IV Italian ed.; Giunti OS: Firenze, Italy, 2011. [Google Scholar]
  37. Wechsler, D. Wechsler Preschool and Primary Scale of Intelligence, 4th ed.; The Psychological Corporation: San Antonio, TX, USA, 2012. [Google Scholar]
  38. Randall, M.; Johnson, L.; Reddihough, D. The Melbourne Assessment 2; Royal Children’s Hospital: Melbourne, Australia, 1999. [Google Scholar]
  39. Rosa-Rizzotto, M.; Visonà Dalla Pozza, L.; Corlatti, A.; Luparia, A.; Marchi, A.; Molteni, F.; Facchin, P.; Pagliano, E.; Fedrizzi, E. A new scale for the assessment of performance and capacity of hand function in children with hemiplegic cerebral palsy: Reliability and validity studies. Eur. J. Phys. Rehabil. Med. 2014, 50, 543–556. [Google Scholar] [PubMed]
  40. Henderson, S.E. Movement Assessment Battery for Children, 2nd ed.; Pearson: London, UK, 2007. [Google Scholar]
  41. Varni, J.W.; Burwinkle, T.M.; Berrin, S.J.; Sherman, S.A.; Artavia, K.; Malcarne, V.L.; Chambers, H.G. The PedsQL in pediatric cerebral palsy: Reliability, validity, and sensitivity of the Generic Core Scales and Cerebral Palsy Module. Dev. Med. Child Neurol. 2006, 48, 442–449. [Google Scholar] [CrossRef] [PubMed]
  42. Brooke, J. SUS: A quick and dirty usability scale. In Usability Evaluation in Industry; Taylor & Francis: Abingdon, UK, 1996. [Google Scholar]
  43. Menici, V.; Barzacchi, V.; Filogna, S.; Beani, E.; Tinelli, F.; Cioni, G.; Sgandurra, G. Tele-Rehabilitation for Postural Control by Means of Virtual Reality Rehabilitation System in an Adolescent with Motor Disorder: A Case Study. Front. Psychol. 2021, 12, 720677. [Google Scholar] [CrossRef]
  44. Barraco, G.; Macchitella, L.; Accogli, G.; Pirani, G.; Nicolardi, V.; Trabacca, A. Exploring the Application and Usability of Emerging Technologies in Neuromotor Rehabilitation for a Patient with 5Q-Spinal Muscular Atrophy Type 2 Receiving a Gene-Based Therapy: A Single Case Study. In Proceedings of the International Conference on Extended Reality, Lecce, Italy, 4–7 September 2024; pp. 50–57. [Google Scholar]
  45. Olivieri, I.; Chiappedi, M.; Meriggi, P.; Mazzola, M.; Grandi, A.; Angelini, L. Rehabilitation of children with hemiparesis: A pilot study on the use of virtual reality. BioMed Res. Int. 2013, 2013, 695935. [Google Scholar] [CrossRef] [PubMed]
  46. Bangor, A.; Kortum, P.T.; Miller, J.T. An empirical evaluation of the system usability scale. Intl. J. Hum.-Comput. Interact. 2008, 24, 574–594. [Google Scholar] [CrossRef]
Figure 1. Types of extended reality (XR) modalities.
Figure 1. Types of extended reality (XR) modalities.
Applsci 14 11961 g001
Figure 2. Khymeia Virtual Reality Rehabilitation System (K-VRRS)—Hospital-based treatment and home-based treatment via telerehabilitation.
Figure 2. Khymeia Virtual Reality Rehabilitation System (K-VRRS)—Hospital-based treatment and home-based treatment via telerehabilitation.
Applsci 14 11961 g002
Figure 3. Study design and TS-NP.
Figure 3. Study design and TS-NP.
Applsci 14 11961 g003
Figure 4. Graphical representation of the average MA2_RM score observed during each of the three assessments conducted at different time points: before starting inpatient treatment (T0), after completing inpatient treatment (T1), and after finishing tele-rehabilitation (TR). The figure visually shows the changes in mean scores over time across the three assessment points.
Figure 4. Graphical representation of the average MA2_RM score observed during each of the three assessments conducted at different time points: before starting inpatient treatment (T0), after completing inpatient treatment (T1), and after finishing tele-rehabilitation (TR). The figure visually shows the changes in mean scores over time across the three assessment points.
Applsci 14 11961 g004
Figure 5. Graphical representation of the average MA2-FM score observed during each of the three assessments conducted at different time points: before starting inpatient treatment (T0), after completing inpatient treatment (T1), and after finishing tele-rehabilitation (TR). The figure visually shows the changes in mean scores over time across the three assessment points.
Figure 5. Graphical representation of the average MA2-FM score observed during each of the three assessments conducted at different time points: before starting inpatient treatment (T0), after completing inpatient treatment (T1), and after finishing tele-rehabilitation (TR). The figure visually shows the changes in mean scores over time across the three assessment points.
Applsci 14 11961 g005
Table 1. Participant characteristics.
Table 1. Participant characteristics.
Sex N (%)
Male5 (71.4%)
Female2 (28.6%)
Age
Mean age (Standard Deviation)6.82 (2.08)
Age range4.25–10.6
QI Mean (Standard Deviation)88 (16.2)
Side of motor impairment N (%)
Right4 (57.1%)
Left3 (42.9%)
GMFCS N (%)
Level 13 (42.9%)
Level 24 (57.1%)
MACS N (%)
Level 11 (14.3%)
Lecel 26 (85.7%)
VFCS N (%)
Level 17 (100%)
Level 20 (0%)
GMFCS: Gross Motor Function Classification System; VFCS: Visual Function Classification System; MACS: Manual Ability Classification System; QI was assessed via Wechsler Intelligence Scale for Children and the Wechsler Preschool and Primary Scale of Intelligence. The GMFCS and MACS are classification systems that describe mobility and manual function in individuals with cerebral palsy (CP), respectively. In the GMFCS, individuals who walk without support are classified as Level 1, while those who walk with some limitations are classified as Level 2. In the MACS, individuals who ‘handle objects easily and successfully’ are classified as Level 1, whereas those who ‘handle most objects but with somewhat reduced quality and/or speed of achievement’ are classified as Level 2. The VFCS is a classification system designed to describe the visual functioning of children with CP in their daily lives. Level 1 of the VFCS refers to individuals with CP who use visual functions easily and effectively during vision-related activities. Level 2 describes individuals with CP who use visual functions effectively but require compensatory strategies that can be spontaneously employed. The term ‘side of motor impairment’ refers to the hemibody affected by spasticity.
Table 2. Statistically significant results showed by Durbin-Conover pairwise comparison test in primary outcome measures.
Table 2. Statistically significant results showed by Durbin-Conover pairwise comparison test in primary outcome measures.
Mean (Standard Deviation) Mean (Standard Deviation) p
MA2-RM_T021.3 (3.40)MA2-RM_T124.6 (2.51)6.55<0.001
MA2-RM_T021.3 (3.40)MA2-RM_T224.3 (2.81)5.24<0.001
MA2-RM_T124.6 (2.51)MA2-RM_T224.3 (2.81)1.310.215
MA2-FLU_T011.3 (3.15)MA2-FLU_T114.6 (3.82)2.7990.016
MA2-FLU_T011.3 (3.15)MA2-FLU_T214.9 (3.44)2.9740.012
MA2-FLU_T114.6 (3.82)MA2-FLU_T214.9 (3.44)0.1750.864
The table shows all the possible changes in different primary outcome measures that occurred longitudinally between the three different assessment points: changes between T0 and T1, changes between T0 and T2, and changes between T1 and T2. T0: Assessment conducted at baseline; T1: Assessment conducted before discharge; T2: Assessment conducted at the end of TR; MA2: Melbourne Assessment 2; RM: Range of movement score; FLU: Fluency of movement score. MA2_RM: Range of movement scores as measured by Melbourne Assessment 2; MA2_FLU: Fluency of movement score as measured by Melbourne Assessment 2.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Macchitella, L.; Accogli, G.; Barraco, G.; Nicolardi, V.; Pirani, G.; Ferrante, C.; Oliva, M.C.; Fanizza, I.; Gallo, I.; De Rinaldis, M.; et al. A Two-Step Neurorehabilitation Program Utilizing Extended Reality and Telerehabilitation for Children with Cerebral Palsy: A Pilot Study on Effectiveness, Adherence, and Technical Feasibility. Appl. Sci. 2024, 14, 11961. https://doi.org/10.3390/app142411961

AMA Style

Macchitella L, Accogli G, Barraco G, Nicolardi V, Pirani G, Ferrante C, Oliva MC, Fanizza I, Gallo I, De Rinaldis M, et al. A Two-Step Neurorehabilitation Program Utilizing Extended Reality and Telerehabilitation for Children with Cerebral Palsy: A Pilot Study on Effectiveness, Adherence, and Technical Feasibility. Applied Sciences. 2024; 14(24):11961. https://doi.org/10.3390/app142411961

Chicago/Turabian Style

Macchitella, Luigi, Giuseppe Accogli, Giulia Barraco, Valentina Nicolardi, Greta Pirani, Camilla Ferrante, Maria Carmela Oliva, Isabella Fanizza, Ivana Gallo, Marta De Rinaldis, and et al. 2024. "A Two-Step Neurorehabilitation Program Utilizing Extended Reality and Telerehabilitation for Children with Cerebral Palsy: A Pilot Study on Effectiveness, Adherence, and Technical Feasibility" Applied Sciences 14, no. 24: 11961. https://doi.org/10.3390/app142411961

APA Style

Macchitella, L., Accogli, G., Barraco, G., Nicolardi, V., Pirani, G., Ferrante, C., Oliva, M. C., Fanizza, I., Gallo, I., De Rinaldis, M., & Trabacca, A. (2024). A Two-Step Neurorehabilitation Program Utilizing Extended Reality and Telerehabilitation for Children with Cerebral Palsy: A Pilot Study on Effectiveness, Adherence, and Technical Feasibility. Applied Sciences, 14(24), 11961. https://doi.org/10.3390/app142411961

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop