The TCTRS Project: A Holistic Approach for Telerehabilitation in Rett Syndrome

Abstract

Telerehabilitation (TR) has been proven to be an effective tool in improving the adaptive skills of children and young adults with Multiple Disabilities (MDs). The application of a comprehensive set of new technologies reveals new opportunities for both physical and cognitive telerehabilitation, but there is no holistic approach in the case of genetic syndromes. In this paper we present reflections and early results of the TCTRS project that aims at implementing a telerehabilitation system capable of offering complete coverage of rehabilitation needs for people with Rett Syndrome, from both the physical and cognitive points of view. Moreover, the data acquired through the system can also represent a basis for machine learning applications to remotely support therapists and physicians. Our first tests on the system application show the great potential of our approach, in terms of feasibility and applicability, for both rehabilitation centers and families.

1. Introduction

The US Individuals with Disabilities Education Act (IDEA, 2004) defines multiple disabilities (MDs) as “concomitant impairments (such as intellectual disability-blindness or intellectual disability-orthopedic impairment), the combination of which causes such severe educational needs that they cannot be accommodated in special education programs solely for one of the impairments”. People affected by MDs have severe difficulty in communicating their needs, in freely moving their body to access and engage their world, and in learning abstract concepts and ideas. They also need support in major life activities (e.g., domestic, leisure, community use, vocational tasks). For these reasons, MDs require the intervention of multiple professionals, one from every identified disability area. Additionally, both families and caregivers have a crucial function in rehabilitative tasks [1].

With reference to rehabilitation of persons with MDs, most interventions are focused on four areas: medical, physical, learning, and affective. Consequently, responses to MDs range from interventions for improving body functions or cognitive abilities, to more comprehensive measures designed to promote adaptive skills. Recently, new telecommunication-based practices have been developed in the field of rehabilitation, known as telerehabilitation (TR) [2]. TR is defined as the use of information technologies (IT) to provide distant support, assessment and information to people with a disability [3]. Specifically, TR refers to the delivery of rehabilitation services using means of telecommunication in the consumer’s normal home or work environment [4].

Hence, it is a valuable clinical service delivery model that includes assessment, therapy, and follow-up services, using phone and/or video technologies as remote monitoring [5]. In the course of time, the implementation of TR has included different, sophisticated technologies to connect with patients: from telephone, radio and closed-circuit television to video conference and, today, internet based communication software (e.g., Skype) with headsets, webcams, speakers, handheld cameras, microphones and internet [4]. It is expected that advancements in technology can increase the usability and effectiveness of TR and, consequently, improve adaptive skills of patients with MDs [6].

Most TR systems use three categories of technology: integrated systems, games-based technologies and video-based technologies. TR services can be implemented through integrated systems that either combine multiple sensory and technical interfaces on both sides, or integrate one technology on the remote side with a different technology on the clinician/near side [7].

Teleplay refers to the use of interactive, exploratory “games” with built-in therapeutic or monitoring capabilities, and with interface parameters that can be adjusted depending on the patient’s progress. Home-based therapy can be conducted through the development of systems or serious games providing multiple and simultaneous users with opportunities for rehabilitation [8,9].

Teleconferencing is the process of two or more users interacting across a distance, supported by telecommunications. Videoconferencing, i.e., teleconferencing that also supports video among its communication channels, can be conducted in a wide variety of ways: webcams (when the visual images can be of coarse resolution and quality in general); specific videoconferencing equipment (when facial features or other high quality visual images or movement are important); complex videoconferencing systems (when a broad field of vision or multiple sites are involved, such as for client staffing or professional training); via Internet using existing phone lines at no greater cost than computer Internet use; via installed and dedicated lines which offer much greater security and dependability, but requires high initial installation costs; via cell phone with videoconferencing capabilities to transmit audio and video over a Wi-Fi or cellular network [7].

Nevertheless, to the best of our knowledge, TR has not yet been applied to genetic syndromes. Considering that traditional treatments for genetic syndromes, such as Rett Syndrome, Down’s syndrome, or Tourette’s syndrome, focus on specific cognitive and communication training, or physical and occupational therapy, considering the use of new rehabilitative approaches for genetic disorders (eye-tracker technologies and virtual reality), and finally, considering the effectiveness of TR interventions in subjects with MDs, the need for adopting specific techniques to study the applicability of TR in genetic syndrome becomes crucial [10,11,12,13,14,15].

This paper reports reflections and early findings on the ongoing project Telerehabilitation, counseling and training in Rett Syndrome, funded by Rotary International and carried out by the Airett Center for Innovation and Research in Verona and the University of Messina, Italy.

2. Telerehabilitation, Counseling and Training in Rett Syndrome (TCTRS) Project

2.1. Background

Rett syndrome (RTT) is a complex genetic disorder, caused by mutations in the X-linked gene encoding for a regulator of epigenetic gene expression, methyl CpG binding protein (MeCP2) [16]. Although mutations in MECP2 gene are the main cause of RTT disease, other mutations are associated with RTT, such as: FOXG-1 and CDLK-5 [17,18].

Since it is a rare disease, affecting 1:10,000 people, the distribution of patients within the territory is a big challenge for patients and their families, as there are few specialized centers and specialists (e.g., physicians or therapists). In this context, in additional to institutional actors, a crucial role is played by patient organizations, which actively involves all the potential caregivers around the patient, i.e., families, educators, rehabilitation centers etc.

Airett (Associazione Italiana Rett, www.airett.it) is a patient-family organization that promotes research projects, rehabilitation practices for RTT people and specific education programs for caregivers. In the last few years, Airett has carried out several projects aiming at studying and applying new technologies and rehabilitation techniques to the physical and cognitive enhancement of patients affected by Rett Syndrome.

The TCTRS project, realized in conjunction with the University of Messina, is a step forward in this direction. The main goal is to bring closer what is far, providing patients with access to skilled specialists by exploiting the possibilities given by new technologies. Consequently, the massive use of technology-enabled interventions produces a large amount of acquired data, which can subsequently be analyzed to better address therapies and provide better decision-making support. Moreover, TCTRS explores the possibilities provided by modern computer vision technologies in combination with teleconferencing, paving the way for a wider application of multimodal interaction for collaborative environments [19].

2.2. Technological Architecture

The TCTRS system (Figure 1) is composed of two types of workstations, namely a local workstation and a remote workstation. The local workstation is intended to be used patient-side, while the remote one is located at a specialized care center.

Through the local workstation the patient and caregiver connect with specialists for a cognitive or physical rehabilitation session. It is worth noting that, for Rett Syndrome subjects, the patient is never alone, but always accompanied by a caregiver (with different skill levels, ranging from parents to therapists).

The software platform is a Web Application, that leverages on the CISCO Webex API for network connection and video streaming.

The following sections discuss the main components of the platform.

2.3. The telerehabilitation software

Existing teleconferencing platforms may act as a technological foundation but cannot provide the whole set of services needed for TR. In particular, the special requirements of patients affected by Rett Syndrome, and other MDs with similar characteristics, demand very specific approaches and tools. As a consequence, we designed and implemented an ad-hoc architecture to manage the TR sessions and to store and analyze the acquired data that is tailored to the needs of Rett subjects, exploiting all the knowledge acquired with Airett after years of experience in using new technologies for both cognitive and physical rehabilitation. The main characteristic of the software is that it is transparent to the patient, since it does not interfere with the ordinary tools that patients use in their everyday rehabilitation activities, but gives in addition the possibility for a caregiver to be supervised and trained by a remote specialist during the rehabilitation sessions.

The software is a web application with the following main features:

• User authentication
• Videoconferencing
• Acquisition and storage of rehabilitation data from multiple input sources (eye-tracker, webcam, 3D camera)
• Analysis of rehabilitation data
• Recording of data on the general condition of the patient

For both user authentication and video-conferencing, the software interacts with Cisco Application Program Interfaces (APIs), so that the user can authenticate our software with Cisco credentials. The software then allows to make video calls directly to Cisco videoconferencing devices, such as Cisco Webex Board. Nevertheless, we customized the video acquired from the local workstation, since we needed additional information from the screen and other peripherals, such as 3D camera and eye-tracker. Details about the composition of the streamed video will be discussed in the following section.

The software also gives the possibility to analyze the acquired data on rehabilitation sessions after the sessions end. This is a very important feature, since it makes it straightforward to record and follow the physical and cognitive improvements of patients. Moreover, the software allows the specialist to remotely record and analyze several parameters of the patients.

The software also differentiates cognitive and physical rehabilitation sessions, since they have different and specific needs and settings, that are discussed below.

2.4. The Local Workstation

As shown in Figure 1, the hardware of the local workstation is composed of a laptop equipped with an eye-tracker, a webcam, a 3D camera and a headset. Note that Rett subjects can only control eye movements, so the eye tracker is a crucial component, being the only input peripheral that Rett subjects can use to interact with a computer.

2.4.1. Cognitive Telerehabilitation Sessions

During a TR session the software allows the local caregiver to video call a remote specialist. As shown in Figure 2, once the video call starts, the software acquires video streams from the laptop screen, the laptop webcam, an external webcam, and data from the eye-tracker. The local caregiver starts a cognitive rehabilitation session with rehabilitation software specifically tailored for Rett patients [20]. During the session, the patient is always in audio contact with the remote specialist, who follows what is going on through the shared screen and the videos. The specialist can also see patient interactions, i.e., the patient’s gaze on the screen. Hence, thanks to this setting and to the acquired data, the specialist can monitor how the patient and the caregiver interact (from front and side views), and how the patient interacts with the rehabilitation software (through the eye-tracker pointer on screen).

After the session ends, the video call is terminated and the acquired data are stored in the remote database. Importantly, the remote specialist can record pieces of video and text notes on its own, without distracting the local caregiver and/or patient.

2.4.2. Physical Telerehabilitation Sessions

A physical TR session starts by video calling a remote specialist, as for cognitive rehabilitation. In this case, however, there is no interaction between the patient and the local workstation (Figure 3). The local workstation is only used to acquire data and to let the caregiver and the specialist communicate.

In the context of physical rehabilitation, the novel contribution of our approach is that during the sessions the movements of the patient are not only video recorded, but also reconstructed and 3D-mapped in real time.

By applying computer vision and artificial intelligence (AI) techniques, the artificial skeleton of the patient is mapped in a 3D space in real-time. The remote specialist can supervise the acquired data, since the skeleton is superimposed to the patient image in the video stream, and hence can annotate if the data acquired during the session can be considered valid. The skeleton data are then used to calculate the overall pose and the angles between bones near to joints, giving an objective measure of improvements in the patient’s movements.

2.5. The Remote Workstation

As shown in Figure 1, the remote workstation is composed of two devices, namely a Cisco Webex Board and a workstation.

During the TR sessions, the remote specialist answers the video calls from the caregiver and follows the session interacting with the Cisco Webex Board. As already mentioned, during the sessions, the specialist can remotely activate video sequences recording and add text notes to it without disturbing the local caregiver. These annotations (video and text) are triggered by specific text typed in the video call chat, e.g., writing “Start recording” initiates the recording event. All these annotations are recorded in the database on the web application server and are grouped by session and available both to local caregiver and remote specialist.

The workstation lets the specialist analyze data of the TR sessions. This is a very important feature, as it enables the specialist to make ex-post analyses for both cognitive and physical sessions. Specifically, in the case of physical rehabilitation, the specialist has a synoptic view of the acquired video and the 3D skeleton. The skeleton is also clickable, giving information about the angles between the bones, so that the specialist can take note of any improvements in the movements of the patient. This is a crucial tool since it adds objectivity and corroborates the observation of therapists.

Moreover, we plan to apply machine learning and advanced analysis techniques to the acquired data to provide decision-making support to the specialist, when the amount of data is adequate for this purpose [21].

Furthermore, the system that we have presented above, is also suited for counselling and training, since it provides professional videoconferencing tools both for one-to-one (TR, counseling or training) and group (training) video calls.

2.6. System Specificity

As we have already mentioned in the above sections, several telerehabilitation systems have been proposed. Nevertheless, our system introduces some specificity that derives from the Rett syndrome. In [22], the authors present a telerehabilitation system for physical therapy sessions. In contrast to that method, our approach is able to detect and recognize two persons, i.e., it can distinguish the patient from the operator. Since Rett patients cannot make physical therapy on their own, this is a crucial requirement. Moreover, several sophisticated commercial products exist (for example: khymeia.com), but they lack some important feature for Rett Syndrome, e.g., the eye-tracker.

Our system is then specifically suited for Rett Syndrome, even if we do not exclude to study its applicability to other multidisability contexts.

3. Impact of Project Adoption by Families and Care Centers

We designed our system to be as acceptable as possible for families above all. We know how hard it is for families to manage Rett patients in everyday life, so we intentionally avoided using wearable sensors or other invasive components. We strongly believe that the success of such a project cannot ignore usability for caregivers, in terms of simplicity, affordability and familiarity with the devices.

The local workstation setting makes use of a laptop equipped with an eye-tracker, webcam, 3D camera, and Bluetooth headset. These devices are general purpose and their use does not require specific skills. Eye-trackers and 3D cameras might appear as ‘specialistic’ devices, but caregivers of Rett subjects are very familiar with eye-tracking and a 3D camera looks and works like a conventional webcam. Such devices are not cost-prohibitive. Moreover, what is spent on enabling a TR workstation is saved avoiding time and cost-consuming journeys to reach specialized centers.

Additionally, equipment for the remote rehabilitation center does not imply excessive investment and long training. Despite the complexity of the project, the users will mostly use ‘conventional’ tools, i.e., video calls or web application interfaces, which are nowadays commonly used among non-specialistic users. In our preliminary tests, a training of just 2 h was enough to start using the system.

Nevertheless, the broad adoption of the proposed platform implies spending time in one-to-one training, not only for acquiring technical skills, but also to effectively communicate the benefits of TR.

The TCTRS platform can be customized for different MD users and can provide similar advantages. Moreover, in the case of milder cognitive impairment, the local caregiver may be substituted by an AI-based virtual agent for cognitive rehabilitation. Some experimentation has already been carried out [20] but more investigation will be needed to assess the applicability of “general purpose” customizable platforms, such as the Red platform [23], to the specific case of cognitive rehabilitation.

The proposed system, hence, represents a very important opportunity for care centers, families and schools dealing with Rett patients. The TCTRS platform, in fact, does not affect significantly the rehabilitation tools that caregivers normally adopt for rehabilitation, but gives them the opportunity for continuous monitoring by an expert, online and offline. In everyday life, for example, families use cognitive rehabilitation software or make physical rehabilitation sessions but are subjected to travels for reaching specialized centers for training (to learn what to do) or monitoring (to verify periodically how they do).

Moreover, our system can be also used to propose entertainment and educational activities to groups of Rett patients. At the time we are writing this paper, Italy is facing the Covid-19 emergency, so therapists and patients cannot move from their homes. Schools and centers are closed, so the families have no possibility to involve their children in any activity. We are currently using our system to propose educational and entertainment material through the telerehabilitation system.

4. Conclusions

This paper presents the Telerehabilitation, Counseling and Training in Rett Syndrome (TCTRS) Project, discussing some key points that are addressed within the project. To the best of our knowledge, it is the first attempt to realize a telerehabilitation (TR) system for Rett Syndrome with a holistic approach. In the design process, all the decisions focused on creating a comprehensive, affordable, easy-to-use system without burdening families, caregivers, or rehabilitation centers with time and cost-prohibitive approaches. Our preliminary tests are encouraging, and confirmed the design and realization choices. We are now planning extensive tests, to collect data and feedback on system usage from a wide number of families and caregivers.

Part of our future work will concentrate on applying machine learning approaches to add decision-making support to our system, and study how an interactive avatar can improve motivation and performance in Rett patients as other contexts [20].