Digital Alternative Communication for Individuals with Amyotrophic Lateral Sclerosis: What We Have

Amyotrophic Lateral Sclerosis is a disease that compromises the motor system and the functional abilities of the person in an irreversible way, causing the progressive loss of the ability to communicate. Tools based on Augmentative and Alternative Communication are essential for promoting autonomy and improving communication, life quality, and survival. This Systematic Literature Review aimed to provide evidence on eye-image-based Human–Computer Interaction approaches for the Augmentative and Alternative Communication of people with Amyotrophic Lateral Sclerosis. The Systematic Literature Review was conducted and guided following a protocol consisting of search questions, inclusion and exclusion criteria, and quality assessment, to select primary studies published between 2010 and 2021 in six repositories: Science Direct, Web of Science, Springer, IEEE Xplore, ACM Digital Library, and PubMed. After the screening, 25 primary studies were evaluated. These studies showcased four low-cost, non-invasive Human–Computer Interaction strategies employed for Augmentative and Alternative Communication in people with Amyotrophic Lateral Sclerosis. The strategies included Eye-Gaze, which featured in 36% of the studies; Eye-Blink and Eye-Tracking, each accounting for 28% of the approaches; and the Hybrid strategy, employed in 8% of the studies. For these approaches, several computational techniques were identified. For a better understanding, a workflow containing the development phases and the respective methods used by each strategy was generated. The results indicate the possibility and feasibility of developing Human–Computer Interaction resources based on eye images for Augmentative and Alternative Communication in a control group. The absence of experimental testing in people with Amyotrophic Lateral Sclerosis reiterates the challenges related to the scalability, efficiency, and usability of these technologies for people with the disease. Although challenges still exist, the findings represent important advances in the fields of health sciences and technology, promoting a promising future with possibilities for better life quality.


Introduction
Amyotrophic Lateral Sclerosis (ALS) is a progressive and irreversible neurodegenerative disease that affects an individual's motor neurons.As a result, there is a gradual loss of functionality in voluntary movements, respiratory function, and communication [1][2][3][4][5].For people with ALS, having access to an ecosystem that integrates multi-professional assistance and assistive technologies, particularly Augmentative and Alternative Communication resources, has been shown to be essential to preserving communication and interaction skills and enhancing quality of life and survival as the disease advances [6][7][8][9][10][11].
As ALS progresses, functional losses intensify.Thus, the communicative process, autonomy, social interaction, and participation are partially or entirely affected.To compensate for these losses, which are related to functional and motor abilities caused by this disease, several kinds of research have been developed, with a common objective: to promote improvement in the quality of life of patients with ALS.Some of this research has been directed toward alternative communication, as this is one of the main issues for ALS patients.This is because many of these patients lose the ability to communicate, which can cause social isolation and loss of autonomy [12,13].Therefore, research in this field is often based on Human-Computer Interaction, to promote Augmentative and Alternative Communication methods, using devices or information systems and applications within the scope of assistive technologies [14][15][16][17][18][19][20][21].
In Augmentative and Alternative Communication, there are different mechanisms and paradigms for controlling interfaces based on Human-Computer Interaction.Bioelectric signals, for instance, are widely used and investigated mechanisms in neuroscience, rehabilitation, and Brain-Computer Interface (BCI).In BCI, brain signals and electroencephalography devices are widely used for Human-Computer Interaction, especially when the person with ALS is in a locked-in state and cannot voluntarily move their eyes [22][23][24][25][26][27][28].In a systematic review, Jaramillo-Yánez et al. [29] showed studies investigating electromyography signals for Human-Computer Interaction through muscle contractions and gesture recognition.Other studies have also suggested that, by applying electro-oculogram-based features, it is possible to control interfaces by capturing predefined eye movements (up, down, left, and right) or blinking [30][31][32][33][34][35].
Despite scientific and technological advances in the field of Augmentative and Alternative Communication using bioelectric signals, it is still a challenge to introduce devices in this category to the home environment, for use by individuals with severe motor disabilities, such as people with ALS.These limitations are related to home usability (or the ability to handle an instrument), the time necessary to select characters or items in an Augmentative and Alternative Communication interface, and the necessity of electrodes that must be attached to the patient, which usually causes fatigue and discomfort and discourages from adopting the resource [12,[36][37][38].
Other approaches to Human-Computer Interaction, highlighted in significant areas of Computer Vision and Machine Learning, use computational methods based on image processing.For example, some studies have used the eyes of individuals with a severe motor disability, along with one or more cameras, to capture image data for processing and defining patterns, such as blinking [13,39,40] or pupil movement [41][42][43][44][45][46].Thus, the result of the images' digital processing coming from the eyes of the user acts as input for a Human-Computer Interaction system.
Fathi et al. [40] presented two categories of image-based techniques: with and without infrared.Methods for eye tracking with infrared are more effective; however, prolonged exposure to infrared may cause health damage or discomfort to the eye.Also, it may require specific hardware for the attachment to the individual's head.The methods that use cameras without infrared lights are simpler; however, the detection aspects of eye movements or eye blinking are more complex, which may compromise the system's accuracy, relative to the desired selection target on the interface [42,47].
In the context of Augmentative and Alternative Communication and image-based mechanisms for people with ALS or other diseases belonging to the so-called locked-in syndrome, it is challenging to craft solutions based on Human-Computer Interaction in a home environment, which requires low effort for typing and low cost as well, that are efficient and accessible.The devices for Augmentative and Alternative Communication that are considered to have high performance are usually commercial and are provided with additional elements, such as infrared and sensors for image processing, which represents high cost and robustness.
In this context, the following question arises: is it possible to build a low-cost resource, using an eye-image-based method that relies on Computer Vision and Machine Learning techniques for Augmentative and Alternative Communication, interaction, and inclusion of people with ALS?To answer this research question, this paper presents, based on the methodological features of a Systematic Literature Review [48][49][50][51][52], an investigation of primary studies that explore eye-based Human-Computer Interaction systems and technologies for Augmentative and Alternative Communication for people with ALS.

Materials and Methods
This research was developed based on the systematic review guidelines proposed by Kitchenham [48] and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist [53], and it was registered with PROSPERO (registration no.CRD42021230721) [54].Initially, as a fundamental part of the protocol, five Research Questions (RQ) were developed (see Table 1  The process of identifying primary studies related to the investigation object of this Systematic Literature Review consisted of searches in six repositories: Science Direct, Web of Science, Springer, IEEE Xplore, ACM Digital Library, and PubMed.Searches in all databases were performed on 18 November 2021.Except for PubMed, two search strings (SS01 and SS02) were used in the searches.Specifically for PubMed, a third search string (SS03) was considered, which was defined from the Medical Subject Headings (MeSH) thesaurus.The search strings are presented below: After identifying and defining the initial set of records, screening was performed, to select a subset of eligible primary studies.This process was organized and executed by applying three elementary procedures: (i) Inclusion Criteria-IC; (ii) Exclusion Criteria-EC; and (iii) Quality Assessment Criteria-QA.

below).
In procedure (i), a subset of primary studies was defined, based on the Inclusion Criteria (Table 2) applied through the filters made available in the repositories.In procedure (ii), a screening guided by the Exclusion Criteria (Table 2) based on the title reading, abstract, and keywords was performed on the subset of primary studies.Rayyan [55], a web application for systematic reviews, assisted in this step (ii).To determine the final set of eligible articles, and to seek answers to the Research Questions (see Table 1), a screening, guided by the Quality Assessment Criteria (see Table 3), was performed from the entire reading of the primary studies.An elimination condition (QA01) and an evaluation metric, called score (see Equation ( 1)), were used for the qualification and ranking of the studies.The score was the arithmetic mean of the weights (w) assigned for each Quality Assessment Criteria.The weight (w), which could vary between 0, 0.5, and 1.0, measured how satisfactory the response of that article was for a given Quality Assessment Criteria, as shown in Equation ( 2).Primary articles that scored 0.5 or higher (i.e., 0.5 ≤ score ≤ 1.0) were considered eligible for this Systematic Literature Review.Two reviewers assigned scores, and elementary data from the final set of eligible studies, extracted based on the research questions, were summarized in Table 4. where: n QA : variable used to represent the total of Quality Assessment Criteria; -w QA : variable used to determine the value referring to the weight w assigned to the Quality Assessment Criteria under analysis (see the possible values in the Equation ( 2)).

Results
The detailed quantitative results of the protocol execution of this Systematic Literature Review have been summarized in Figure 1.After identifying 9084 records and performing screening, consisting of the application of Inclusion Criteria (8586 studies excluded), Exclusion Criteria (449 studies excluded), and Quality Assessment Criteria (24 studies excluded), a set of 25 primary studies were considered eligible and were included in this Systematic Literature Review, to answer the Research Questions (Table 1).The most comprehensive research results were organized and presented in the same sequence as the Research Questions.The analysis is based on the data extracted from the 25 eligible articles, briefly described in Table 4.This table was organized in groups of Human-Computer Interaction, Score, and year of publication of the articles, arranged in descending order.

Research Question 01
Based on the primary studies, and as shown in Figure 2, the strategies evidenced for Human-Computer Interaction through the eyes are divided into four categories: Eye-Gaze; Eye-Blink; Eye-Tracking; and hybrid strategies, which combine some of the previous categories.The Human-Computer Interaction approach based on Eye-Gaze (36% of the studies) was the highlighted technique found in the research, a technique that generally seeks to estimate gaze direction based on pupil movement on the horizontal (left and right) and vertical (up and down) axes, to select the target object on the interface [56][57][58][59][60][61][62][63][64].The Eye-Blink strategy was evidenced in 28% of the studies [65][66][67][68][69][70][71].In this category, the approach to target selection varies and can be based on the detection/identification of voluntary eye blinking (long eye-blinks) [65], the simulation of analogous mouse clicks (right or left eye-blink) [66-68,70,71] and the sequential combination of blinks in a temporal space [69].With the same percentage, 28% of the primary studies provided Human-Computer Interaction approaches to Eye-Tracking, i.e., identifying and classifying the effective pupil direction [72][73][74][75][76][77][78].Despite its similarity to Eye-Gaze, this strategy seeks to estimate gaze direction in relation to the image pixels, which is more accurate and goes beyond horizontal and vertical lines.The studies by Zhao et al. [79] and Xu and Lin [80], which belong to the category of hybrid strategies, combine Eye-Gaze and Eye-Blink strategies.

Research Question 02
The algorithmic techniques explored in the primary studies varied, due to the Human-Computer Interaction techniques presented in section 3.1 (Research Question 01) and the environment configuration in which the camera was allocated for acquiring images/video from the user, as shown in Figure 3, in the step called Video Acquisition.Figure 3 also shows a generic workflow of the procedures (tasks) and the respective Computer Vision or Machine Learning computational techniques often used to solve the challenges comprising Human-Computer Interaction through pupil tracking or blink detection.In the set of primary studies, mainly for the face detection/localization and eye detection/localization steps, the use of Computer Vision algorithmic resources from the Open Source Computer Vision Library (OpenCV) [81] and Dlib [82] was observed.For the face detection/localization task, for example, the authors mentioned the use of techniques based on the Viola-Jones algorithm or the Haar cascade classifier, available in the OpenCV library, and on Dlib's built-in facial landmark detector [56,58,60,65,70,74,80].Other sources of computational tools for image processing and Computer Vision were also explored in the face detection/localization step: for example, Singh and Singh [66] and Singh and Singh [67] used the Viola-Jones algorithm from the MathWorks company.Authors Zhang et al. [57] explored face detection with the Machine Learning Kit on iOS and landmark detection with Dlib.
The eye detection/localization task challenges were mainly related to eye clipping and pupil or iris enhancement.The computational techniques for eye clipping are intrinsically linked to the video acquisition approach, which does not have a Head-Mounted Camera and which aims to delimit or extract the region of interest (the eye).The commonly explored computational techniques were the Viola-Jones algorithm [66,67,74], algorithmic models based on Facial Landmark Points [57,70], and Geometrical Dependencies coupled with Binarization/grayscale/OpenCV strategies [56,63,65,75,78,79].
For enhancing the pupil (or iris) in an eye image, the robust and versatile technique for detecting circles called the Circular Hough Transform [58,62,63,73,74] is highlighted.Other techniques covering the eye detection/localization task can also be mentioned, such as the Limbus tracking method [59], gradient-based algorithms [60,80], Machine Learning models based on Hierarchical Temporal Memory [61,64], Erosion with a cross-shaped structure element [68], segmentation based on thresholding [69], and the Clustering Method of Unbroken Pixel Lines [76].
From the perspective of detecting eye blinking, the authors explored computational models based on Template Matching [65,68], optical flow technique and pixels' motion analysis [66,67], mathematical formulas to calculate Eye Aspect Ratio (EAR) [70,80] or iris height and width [79], and the 2 Pixel Verification Methodology (black and white: open eye; black and black: closed eye) [69].In addition to using the Template Matching technique, Missimer and Betke [68] incorporated the Lucas-Kanade optical flow algorithm and finite state machines into the proposed model.Krapic et al. [71] used software called eViacam with integrated implementations of Motion Analysis from the authors [83,84] and computational techniques (not specified) from the OpenCV Library.
In the set of studies where the purpose was to develop a Human-Computer Interaction strategy based on gaze direction, it is evident that the authors followed the workflow, to identify the pupil/iris in the images in advance, using varied methods such as Circular Hough Transform [58,62,63,73,74] or gradient-based algorithm [60,80], and that they extracted values related to the coordinates that served as input to mathematical models (called Geometrical Dependencies in this work) that calculated and identified the gaze direction [58,60,62,63,73,74,[78][79][80].The authors Eom et al. [56] and Yildiz et al. [62] used Geometrical Dependencies to train and create Machine Learning models, using neural networks and the K-Nearest Neighbor algorithm, respectively.
Other approaches to classify gaze direction can also be mentioned.Zhang et al. [57] used Template Matching.Abe et al. [59] explored the vertical eye-gaze detection method, which is also based on the Limbus tracking technique.Rozado et al. [61] and Rozado et al. [64] combined Machine Learning models based on Hierarchical Temporal Memory with ITU Gaze Tracker (open source library software) and the Needleman-Wunsch algorithm, respectively.In a study by Oyabu et al. [76], the pupil position was defined using the Clustering Method of Unbroken Pixel Lines.Also performing mathematical operations, Aharonson et al. [75] calculated the pupil position using two different algorithms: a parametrical interpolationbased algorithm (called a polynomial) and a model-based algorithm (called a projection).Park and Park [72] built an expert embedded system, Pupil Center Corneal Reflection, to track pupils through hardware with attached adaptive lights and a mathematical model-based program.Kaushik et al. [77] used the EyeScan software.

Research Question 03
The performance-related evaluation of the computational techniques explored in the primary studies showed promising results in control group testing.The analysis of the 13 studies that reported the performance of the techniques in percentage terms shows that the average accuracy (Acc) reached the value of 94.12% (std = 4.14; median = 95%) [57,58,61,[63][64][65][66][67][68]74,77,79,80].The approach proposed by authors Park and Park [72] obtained accuracy of 1-2 • .In addition to the accuracy of 95.17%, Królak and Strumiłło [65] measured Recall and Precision, which obtained values of 96.91% and 98.13%, respectively.Singh and Singh [67] also measured performance on more than one metric, showing 91.2% Acc and 94.11% Precision.Abe et al. [59] presented the average error in two perspectives of eye-gaze detection: vertical detection (0.56 • ) and horizontal detection (1.09 • ).The authors Rahnama-ye-Moqaddam and Vahdat-Nejad [60] reported an average error rate of 5.68% and, now looking at the best error rate obtained by the system, Yildiz et al. [62] presented 0.98% as a result.All the computational techniques explored are identified in Table 4.
Other approaches to evaluating technology performance for Human-Computer Interaction have been used.Eom et al. [56] conducted computer experiments with a control group and summed the individual participants' gaze movement error (vertical and horizontal).Zhang et al. [57] evaluated, in addition to the Eye-Gaze system, the usability of Augmentative and Alternative Communication software, through a questionnaire, with questions based on the Likert scale [85].Similarly, Krapic et al. [71] evaluated usability tests.Rupanagudi et al. [69] evaluated and compared the speed of the proposed algorithm to another approach in the literature.With an evaluation system based on pattern recognition, Rakshita [70] reported the efficiency of the approach (without quantifying).Saleh and Tarek [73] evaluated the proposal based on an interface with six targets representing user needs.Aharonson et al. [75] constructed a table containing each user's mean deviation in degrees.The experimental results in Oyabu et al. [76] were presented by calculating the time, using a "click experiment screenshot" system.Kavale et al. [78] showed the performance of the techniques used through images.
In the Eye-Blink category, all seven studies explored image capture using cameras not mounted on the user's head [65][66][67][68][69][70][71].Three studies equipped the cameras with some type of light source projected onto the user's eye or face, with one being infrared lights [69] and two being lamps [66,67].Video acquisition in studies belonging to the Eye-Tracking category varied between approaches with Head-Mounted Cameras [73,75,77], two of them with infrared [73,77], and with non-head-mounted cameras equipped with [72,76,78] and without [74] infrared.The authors Zhao et al. [79] and Xu and Lin [80], from the hybrid Human-Computer Interaction strategies category, investigated Augmentative and Alternative Communication techniques from images collected from cameras with infrared.

Research Question 05
The data extracted from the primary studies to answer these Research Questions have been summarized in Figure 4. Figure 4 clearly shows that only one study-by authors Rahnama-ye-Moqaddam and Vahdat-Nejad [60], for Eye-Gaze-performed experimental tests on a person with ALS.A second study considered participants with other (unspecified) disabilities.Królak and Strumiłło [65] included 12 people in experimental testing to evaluate the Human-Computer Interaction approach through Eye-Blink.All primary studies performed tests with healthy controls, with an average of 10.76 participants per study (std = 12.3; median = 5).

Discussion
This Systematic Literature Review investigated 25 primary studies on image-based Human-Computer Interaction approaches using simple, low-cost cameras for Augmentative and Alternative Communication for people with ALS.Initially, as an answer to the problem question, the results point to the possibility and feasibility of developing low-cost technologies for Human-Computer Interaction through eye imagery.However, there are still challenges to be explored in the broad areas of Computer Vision, Machine Learning, and Augmentative and Alternative Communication, related not only to the cost but also to the efficiency and usability of Human-Computer Interaction technologies that are used through the eyes, particularly in the context of people with ALS.From this perspective, this Systematic Literature Review organized and discussed, in sequence, the main findings.
The surveys highlighted in this study could be relevant if they aligned with sharing knowledge and the Augmentative and Alternative Communication resource (the final product).Therefore, the goal must be to improve the functional capacities of people with motor disabilities-that is, the autonomy of those individuals.In this way, it may be possible to mitigate the effects of social isolation.Moreover, it also promotes the exercise of rights, citizenship, fundamental freedoms, and health care for people with ALS.This aspect is very significant, for it acts directly on health promotion, well-being, and the reduction of inequalities, which may reflect in the promotion of equity.These factors are even foreseen in the Sustainable Development Goals (SDGs) that are part of the United Nations (UN) 2030 Agenda, particularly SDGs 3 (3.8)and 10 [86].Therefore, this issue is not only about developing new technologies or simply providing low-cost solutions but is also about acting through technological mediation as research that induces social inclusion, reduces inequalities, and promotes equity-values often not measurable in scientific research of a more technological nature.
Although most of the results occurred in healthy controls, they suggest the viability of investment in research in this field.However, public health policymakers must prioritize research in this area, to ensure that the poorest people diagnosed with ALS have access to assistive technologies that can improve their quality of life.It is not enough to develop new technologies: it is necessary to ensure that people with ALS have access to them, regardless of their social conditions.Securing investment in research in this field is essential, not only for providing access to people with this disease -which is fundamental -but also for ensuring the sustainability and advancement of studies in this area, which is often neglected by the industry, as the market is very limited.
ALS is considered rare, and, despite efforts to seek digital health solutions, there are still significant challenges to be tackled: these include the need for more data, studies, and evidence on disease incidence and prevalence, which are essential but scarce pieces of information in the context of global health [12,.There are few records or epidemiological studies in Brazil, and only two studies at national level have been mentioned in the scientific literature.In 1998, Dietrich-Neto et al. [108] conducted a national survey and reported incidence and prevalence rates of 0.4 and 1.2 per 100,000 in-habitants, respectively.More recently, when analyzing the period from 2004 to 2013, the researchers Moura et al. [109] estimated the average incidence of ALS to be 0.461 cases per 100,000 inhabitants (with a trend of increasing incidence over the years), a rate similar to that of Dietrich-Neto et al. [108].It is worth mentioning that, in Brazil, until 2019, there was no compulsory notification system or national registry of Amyotrophic Lateral Sclerosis, which may have led to under-reporting [107,110].
To address this problem of under-reporting in Brazil, Barbalho et al. [111] emphasize the National ALS Registry, an applied research project supported by the Brazilian Ministry of Health.According to Barbalho et al. [111], this National Registry is a project still in progress and under implementation throughout Brazil, the goal of which is to continuously map all people with ALS in the country online.Through the National Registry, it will be possible to develop epidemiological studies and analyses that can support the decision making of public authorities in the design of health policies in the context of ALS in Brazil, for example.In this sense, Law Project No. 4691 of 2019 [112] aimed to make the notification of rare diseases mandatory in Brazil, and so the National ALS Registry is a structuring part of this Law Project.Noteworthy in Brazil is the state of Rio Grande do Norte, which is in the Northeast Region of the country, as it was the first Brazilian state to publish Law No. 10,924 of 10 June 2021 [113], which made the notification of ALS compulsory.
There are many challenges in the ALS context, and it is clear that they go beyond the areas of health sciences and technology.However, understanding transdisciplinarity and the appropriate use of these technologies or digital health solutions could significantly improve access to quality health care, reduce inequalities, and improve life quality, especially for people with ALS.Therefore, it is also necessary to consider technologies as tools for society's social and sustainable development.

Conclusions
This paper, through the execution of a Systematic Literature Review protocol, investigated primary studies in the literature and highlighted five relevant points that could directly contribute to development and technological effectiveness in providing eye-imagebased Human-Computer Interaction strategies regarding Augmentative and Alternative Communication for people with ALS.The first point showed the Human-Computer Interaction approaches based on eye images: Eye-Gaze (36%) [56][57][58][59][60][61][62][63][64]; Eye-Blink (28%) [65][66][67][68][69][70][71]; Eye-Tracking (28%) [72][73][74][75][76][77][78]; and hybrid strategies (8%) [79,80].These Human-Computer Interaction approaches are the results of efforts by the scientific community to develop low-cost solutions and to indicate the feasibility of their use as assistive technologies for Augmentative and Alternative Communication for people with ALS or other diseases that compromise functional abilities.The computational resources related to Computer Vision/Machine Learning techniques and to hardware support for image acquisition and enhancement were also examined and described in Table 4, which summarizes the answers to these and the other investigated points.
The computational models identified showed potential for face and eye detection and for eye movement tracking tasks or eye state classification (open or closed).However, there were limitations regarding experiments on people with ALS and, in some studies, the methodological density of the model structure and application.In addition to these limitations, it is important to highlight that computational techniques have reached an efficiency threshold (regarding performance), i.e., they are well-consolidated for Human-Computer Interaction through the eyes.However, it is worth noting that controlled computational experiments with a low and undiversified number of users may mask the actual results, presenting good results in the tests but without the ability to generalize the model.These aspects could be explored in further research and related to approaches without using a Head-Mounted Camera or infrared, which could direct other tests, considering people with ALS without causing discomfort.
The purpose of this Systematic Literature Review was to gather findings on eye-imagebased Human-Computer Interaction approaches for the Augmentative and Alternative

Figure 1 .
Figure 1.Result of the search and screening process of primary studies for this systematic review.

Figure 2 .
Figure 2. Strategies for Human-Computer Interaction (HCI) based on eye images.

Figure 4 .
Figure 4.The number of subjects used in the primary studies [56-80].

Table 2 .
Inclusion and Exclusion Criteria.

Table 4 .
Summary of the main characteristics of the articles included in the systematic review.