Augmented Reality, Serious Games and Picture Exchange Communication System for People with ASD: Systematic Literature Review and Future Directions

For people with Autism Spectrum Disorder (ASD), using technological tools, such as augmented reality (AR) and serious games remain a new and unexplored option. To attract people with ASD who have communicative, social, emotional and attention deficit disorders to behavioral treatments, an attractive environment is needed that ensures continuity during treatment. The aim of the current work is to efficiently examine systematic reviews and relevant primary studies on ASD solutions from 2015 to 2020, particularly those using the traditional Picture Exchange Communication System (PECS), the application of augmented reality and those that propose serious games, thereby providing an overview of existing evidence and to identify strategies for future research. Five databases were searched for keywords that may be included within the broad Autism Spectrum Disorder ‘ASD’ umbrella term, alongside ‘augmented reality’, ‘serious games’ and ‘PECS’. We screened 1799 titles and abstracts, read, and retained 12 reviews and 43 studies. The studies scrutinized in our systematic review were examined to answer four primary and four sub-research questions, which we formulated to better understand general trends in the use of approaches for attracting people with ASD to behavioral therapies. Additionally, our systematic review also presents ongoing issues in this area of research and suggests promising future research directions. Our review is useful to researchers in this field as it facilitates the comparison of existing studies with work currently being conducted, based on the availability of a wide range of studies in three different areas (AR, SG and PECS).


Introduction
Autism Spectrum Disorder (ASD) is a neurodevelopment condition that affects many abilities including social, verbal and physical skills. It typically arises in early childhood, before the age of three years [1]. The World Health Organization stated that, based on studies conducted in the past 50 years, it is expected that there will be an increase in the prevalence of ASD globally, with estimation that one in every 160 children will have an ASD. Researchers believe that there are three possible means for investigation the studies. This study focused on reviewing many diverse technologies such as smart glasses, mobile devices and video display systems, and focused on abilities such as social skills, the ability to communicate, attention and cognitive skills. In [17], about augmented reality in autism treatment there is two aspects were identified that were found to be common to almost all current studies on ASD: (1) the high communication and cognitive performance of people with ASD; (2) experimental designs aimed at improving social skills, language and relationships. Adnan et al. [18] focused on the satisfaction and confidence aspect of using AR applications among children with ASD.
The study found that using a visual approach in AR applications for training social interaction skills increased the enthusiasm and passion of ASD children. In [19], the purposes, limitations and effectiveness of AR and the quality and results of evaluation methods were analyzed among 10 recently published studies. These studies were divided into many categories based on content, social skills, emotion recognition and other factors. A central issue in the survey presented in [20] was determining which activities would be possible and most effective for applying the use of AR to develop techniques for ASD children and how to go about doing so. The authors note that few of studies recently published using different strategies (such as 3D cards) were directly related to approving treatments for ASD children.
On the other hand, for using the concept of serious games with ASD, three surveys were included in this area. In [21], the following meta-analyses revealed that limitations in applying the principles of serious game design may be computer-based socio-emotional interventions. Furthermore, a serious game assessment tool was developed to implement serious game principles when designing game-based interventions targeting ASD individuals. Studies such as [22] on the effectiveness of serious games for people with ASD deal with social skills, rather than targeting other adaptive behavior skills, since the field of serious game design has been successful in addressing many adaptive behavior skills and intellectual performance. Finally, through the survey presented in [23] measuring the effectiveness of information and communication technology and the use of serious games to treat people with ASD, authors Of the current study concluded that serious games incorporating ASD that are currently available have some limitations in terms of evidence of their clinical benefit.
In relation to PECS's applications with ASD, as well as surveys that integrate the fields of AR, serious games and PECS's technology with ASD, we found no surveys during the research process. Addressing these contexts represents one of the main contributions of the current survey. To identify all contributions, Table 1 provides a summary of previously published work and lists their limitations to identify the main contributions of the current survey.

Main Contribution of the Presented Work
Based on the previous review of the existing surveys, the current research presents the following primary contributions.
(1) Conducted a systematic and accurate survey of studies applying the PECS approach.
Our goal is to understand the way of using PECS with ASD children. (2) In contrast to existing surveys that focused on the number of targeted skills, as all previous surveys have demonstrated in related studies, and which primarily addressed social and communication skills, the current survey focused on improving and developing existing solutions. So we will try to review existing solutions using AR, SG and PECS with ASD children. (3) Aimed to find common factors in the field of AR environments, serious games and PECS's applications, and to illustrate how combining these fields can contribute to the existing body of research. (4) Aimed to establish technical solutions for addressing different personality traits by defining specific criteria for people with ASD, as existing surveys did not consider solutions in terms of these differences and instead combined individuals with ASD with other groups (e.g., those with intellectual disabilities).   There are many definitions of ASD, all of which canter on the notion that ASD can be described in the simplest terms as impairment in social communication that develops early during childhood and is caused by a heterogeneous set of neurodevelopmental conditions, such as repetitive sensory and motor behaviours that are associated with a strong genetic component [25][26][27][28]. Autism can be defined in terms of the pathological anatomy of the cerebral cortex. It involves only a slight disturbance in the basic and diamond radial organization of neurons and glia in this particular area of the brain [29]. Studies have expressed it as a slight disorder that is extremely variable in dendritic orientation and in the decrease in size and spacing between the small radial columns of neurons in different cortical regions, including the frontal lobe [30,31]. As shown in Figure 1, small for some mandibular tracts (indicated by the red lines) and anterior temporal joints (indicated by the yellow lines) and are responsible for stabilizing human contact. However, the severity and size of the disorder may differ between people; accordingly, different types of autism present with different characteristics as shown in Table 2 [18]. This survey focuses on all autism types except Asperger's syndrome and pervasive developmental disorder not otherwise because these represent severe disorders that require complex clinical procedures.
Although there is no drug treatment for autism spectrum disorder, early treatment can make a significant difference in the lives of children with autism [32]. Following the diagnostic process by a specialist, a treatment method will depend on determining the severity of the disorder and the child's age. Typically, the earlier treatment can be started, the better the results will be. There are many therapeutic approaches that ASD children can practice at home, in school and in specialized clinics. Specific types of treatment methods can be presented as follows: behavior and communication treatments, applied behavior analysis, educational therapies, family therapies, speech therapy, occupational therapy, social skills classes, hippo therapy and using the PECS [33]. Table 2. Describe the types of autism [18].

Type of Autism Describe
Autistic disorder

The Picture Exchange Communication System
As previously noted, one of the methods of treating ASD is the PECS. The PECS system was developed by Andy Bundy and Laurie Frost in 1994 [34,35]. It is a communication system that mainly depends on exchanging images to express basic needs and communicate with others [36,37]. The PECS system targets children with a nonverbal autism spectrum disorder. It works using the exchange of well-known signals, images and symbols to enable effective communication as one of the Augmentative and Alternative Communication (AAC) strategies [38]. The PECS approach has the advantage of being a system supported with outdoor equipment at low technical costs because it relies only on painted drawings and recording the images in these drawings [39] as shown in Figure 2. Furthermore, PECS provides more tangible reinforcement, as children receive tangible objects in its first stage, which teaches how to order first, which in turn enhances dealing with the components of the surrounding environment. Furthermore, PECS is distinguished by the fact that it does not require a child to have existing communication skills to learn about communication. Research by Bundy and Frost demonstrated that 59% of ASD children who were trained using PECS spontaneously developed independent speech [40].
The PECS approach comprises six stages. In the first stage, a picture is exchanged to discover how to deal and obtain the required item through pictures and this process develops until it reaches the sixth stage, in which simple questions and mutual comments are answered [41]. Table 3 briefly explains each of these stages. The PECS approach functions as per guidelines established by [34]. Table 3. Stages of PECS.

Stage Name
Description Outcome

1.
How to Communicate Children learn to exchange pictures of objects or activities they desire.
Training to take the picture from the table and put the image in the hands of the communication partner.

Distance and Persistence
Children learn to generalize this new skill if they continue to use individual images in different places and with different individuals. They are also taught to be more persistent and persistent in communication.
He goes to the profile, grabs the photo, goes to the communication partner, gets attention, and leaves the picture in hand.

Picture Discrimination
Children learn to choose between two or more pictures to order the things they want.
To ask for the things he wants by moving to the communication book and choosing the appropriate image from among several images Table 3. Cont.

4.
Sentence Structure Children learn to attach a simple sentence to tape using the picture "I want" followed by a picture of the required thing.
To request existing and non-existent things using multi-word phrases by navigating to the communication file and taking a picture (I want) + (picture of the desired thing) and placing it on the sentence bar and taking the sentence bar from the communication file and directing it to the communication partner and giving it to him 5.
Answering Questions Children learn to use PECS to answer the question, "What do you want?" To automatically ask for many things and to answer the question (What do you want?)

Commenting
Children are now learning how to comment in response to questions such as: "What do you see?", "What do you hear?" And "What is this?" They also learn how to construct sentences starting with "I see", "hear", "feel", "it", and so on.
To answer (What do you want?) (What do you see?) (What do you hear?) (What is this?) And automatically ask and comment on events he sees.

Serious Games
Other concepts that can be incorporated into ASD treatment is that of serious games. Although several different definitions can be applied, "serious games" involve many different types including training simulations, computer games and sports; serious games can be defined as digital games with several purposes, such as providing an experience of multimedia, as opposed to simply serving as entertainment [42]. They can also be defined as games that are designed with a primary purpose other than entertainment. Moreover, some [43] posit that the term "serious" in serious games refers to messages or input that is part of in-game content aimed at providing knowledge or training skills, i.e., providing the player with knowledge or experience management in a suitable environment [43]. To identify that serious games involve applications comprising experience, entertainment and multimedia, through the previous definition, it is clear that serious games are the result of the merging of three concepts, which are training simulation, computer games, and sports. As shown in Figure 3. Serious games can be combined with the latest technologies to contribute to increasing their benefits and purpose. The rapid development of serious games is expected to increase in the future [44,45]. To ensure the success of serious games and to apply them to their full extent, several criteria that contribute to their benefits must be considered [43]. Five of these criteria include activity, modality, interaction style, and environment and application area. 'Activity' refers to determining the type of activity for which a game was designed. Many types of activity, including physical activity, target health and movement activities such as recreational games [46,47] and games aimed at obese children [48]. Another type of activity is physiological, which can be used in rehabilitation games [49] and to diagnose health conditions [50]. The activity can also be mental, i.e., one that is used to learn [51,52] and provide training in different fields. The 'modality' criterion refers to the senses that are used to transfer information from a device to the game participants. The most common senses in this regard are sight, hearing and touch, with recent attempts to include the olfactory sense [53]. To use serious games to their full extent, attempts have been made to integrate a wide range of senses in their mechanics, e.g., enabling participants to mix music or engage in tactile experiences [54]. The 'interaction' criterion refers to the specific input used within the game environment to enable participant interaction; these include traditional inputs, such as a keyboard, mouse and joystick, as well as more modern and smart inputs, such as enabling tactile experiences (touch) and tracking eye gaze and movement using sensors. Correctly selecting entries together with game requirements increases the success and effectiveness of serious games. One example that provides a good description of the input used for interaction was explored in [55] (Whitehead et al. 2010), i.e., the Sensor Network for Active Play system. This example involved a game aimed at effecting physical activity by connecting various sensors to the feet and arms of a game user.
The 'environment' criterion refers to the type of digital environment used in a game. Different types of serious game environments can be combined. The environment may be a two-dimensional (2D) or 3D game, or a combination of both. Alternatively, the game environment can employ an augmented, virtual or mixed reality. These types will be defined in more detail in the next section. The AR or virtual reality is widely used in serious games such as those defined in [54]. Mixed reality is also discussed in [48]. This criterion addresses whether the game requires permissions to locate the player. It also determines whether the game is portable. In addition, the criterion also determines whether play is affected online, in addition to the number of players in the game and whether it is individual or collective as discussed in [47]. The fifth and final criterion that must be determined in the design of serious games is the 'application area', which refers to serious games' target areas for application. These include education and advertising, which dominate serious games by up to 57% according to [56], while the remaining percentage relates to less frequently addressed areas such as healthcare, rehabilitation, communication between people, luxury, tourism, antiquities and others. Figure 4 briefly illustrates the primary criteria for serious games.

Augmented Reality
One of the most important environments that can be used in serious games is known as AR. In the 1960s, AR prototypes were invented by Evan Sutherland and his students, in which code was used to display 3D graphics. Augmented reality was first recognized as an emerging technology in 2007 [57]. The technology involves 3D interactive recorded data that combines real and virtual objects [58]. Figure 5 (as presented by [59]) illustrates that AR is part of a connected chain that extends between real and virtual environments and includes AR and augmented virtual reality (AVR), where AR is closer to the real world and AVR is closer to the purely virtual environment. Two main types of AR can be defined as follows: • Location-based AR, which depends on devices or mobile phones that have a Global Positioning System (GPS) to display digital media as discussed in [60]; • Visual-based AR, which depends on the user's phone camera orientation to the object in the digital environment [61,62]. These two types of AR must, however, be able to preserve the shape of virtually created elements that merge with elements in reality, thereby rendering them compatible with the real environment. Furthermore, the performance of the virtual environment should be effective in real-time [63,64].
Augmented and AVR are often confused because while they are almost the same, there are subtle differences between them. In an AR, the user reference diagram is linked to reality but additional technical elements are added, making it appear as if these items are default aspects of reality. In an AVR environment, the user reference diagram is linked to a completely virtual world, and the player uses a virtual environment to engage in a particular experience [63,65,66]. Table 4 shows the most important differences between AR and AVR [14,67,68]. Augmented reality operates within a framework that includes three main phases. In the first stage, tracking, recording, display technologies and processing processes take place in real-time. This stage is important for creating the virtual object. The second stage includes interaction devices and procedures, in addition to presentation and configuration processes. The third stage is the application stage and includes the user interface. Figure 6 illustrates the different components of AR [69,70].

Review Methodology
This survey follows the methodology of a systematic review to examine current research work. A comprehensive view of recent proposals for rehabilitating children with ASD is provided and serves as a means for examining and evaluating current studies relevant to the topic to provide comprehensive information to the research community in several areas. In addition, this survey delivers a new proposition by developing the limitations of the current studies. This survey was conducted by following the recommended guidelines accordingly [71]. The survey process comprised three phases, i.e., planning, conducting and reporting. Figure 7 summarizes the steps required for a systematic review [72]. Details of these steps are explained in the following sections.  Although many applications have been proposed in the field of rehabilitation for children with ASD, many technical interventions that contribute to increasing rehabilitation effectiveness are still required. This study reviewed several ways in which to develop rehabilitation for children with ASD by integrating approaches into three domains, i.e., augmented reality, serious games and the PECS approach. These areas were previously individually suggested in the reviewed literature; the current survey broadens the discussion and investigates integrating the suggested areas according to the following research questions [72]. Figure 8 shows the current relationship between these fields, considering the lack of research that combines them.

Defining the Research Questions
To achieve the main objectives of this study, the PICOC method [72] was used to define the primary and sub-research questions. The main research questions are as follows. • SRQ1: Can developing AR serious games attract people with ASD? • SRQ2: Can all the phases of PECS be implemented in a mobile application? • SRQ3: How long can people with ASD continue being attracted to the application of PECS? • SRQ4: How much can we benefit from previously reviewed papers that target people with ASD?

Review Protocol Development
The protocol development process contributes to the process of identifying the methods used in the systematic review components (review background, MRQ and SRQ development, search strategy, inclusion and exclusion criteria, data extraction and data synthesis).
The result of the review protocol development process aimed to reduce study bias and distinguish between SLR and traditional methods of reviewing the literature [73][74][75].

Conducting the Review
Search strategy: As shown in Figure 9, the search strategy aimed to extract data from the relevant research papers. Accordingly, a search strategy must be developed to ensure that scholars obtain the largest possible number of relevant studies. To ensure that additional academic publications are included, manual and automatic research methods were used, in addition to examining the content included in the review. An automatic search was used to find preliminary studies related to people undergoing ASD rehabilitation. The automatic search of library databases via the Internet was conducted based on search keywords. The following online databases (with their accompanying website link) were included in the search strategy:  The proposed steps in [13] were followed to derive the search terms as follows.
Seeking keywords using the main and sub-research questions, four main keywords/ phrases were identified (autism spectrum disorder, augmented Reality, PECS, serious games).
(1) Alternate spellings, synonyms and abbreviations for each keyword were defined.
(2) The keywords used in the relevant searches were defined.
(3) Similar keywords were defined and divided into four categories for each of the relevant proposed domains so that the common category for all domains was 'autism spectrum disorder'. (4) Used the OR logical operator to combine keywords in each category. (5) Using the AND logic operator by combining keywords across categories. (6) Each of the search rules differed in terms of filtering searches; Table 5 shows the specifically applied filtering and the particular query formed from the keywords. Table 5. Database query and filter actions.

Criteria for Including and Excluding Articles
From a search of five databases, 1799 related studies were found. Next, incorrect quotes screening reduced studies related by title, keywords, abstract and conclusion to 172 studies. Next, 31 duplicates were removed to reduce the related studies to 141. Using the forward and backward technique, the number of studies was further reduced to 84. First, the title of each study was considered. Next, the content of each primary study was reviewed starting with the abstract. The primary studies incapable of addressing one or more of the RQs related to this SLR were excluded from the list of related studies. Next, an analysis of the remaining studies was carried out based on the selection criteria. Then, inclusion and exclusion criteria (see Table 6) were applied as part of the selection criteria to reduce the number of included studies. The collective selection process reduced the related studies to 43. The references section of each remaining study was then manually checked to identify additional related studies but did not yield additional new and related studies. Finally, a qualitative assessment of the primary studies was performed; no additional new and related studies were found. The qualitative assessment of related studies is described below. Table 6. Criteria for inclusion and exclusion of the articles.

Inclusion Criteria Exclusion Criteria
Studies are written in English only Studies for which the full text is not available Studies have been published from 2015 to 2020 Duplicate studies Studies type is Journal articles and conference papers.
Studies that do not directly aim to help people with autism spectrum disorder only (without other disabilities).
All methodology except (Review, Survey) Studies involving VR and MR in studies related to augmented reality with ASD. Studies that provide answers to the main research question or the sub-research questions Studies that include all types of game-based learning and do not specialize in serious game. Studies not related to the applications of PECS

Manual Search
The citations to related studies were traced using the forward and backward technique in [76,77], and the included databases were searched to find the studies cited in the selected studies. Among others, the Mendeley tool was used to sort and manage all studies and to remove duplicates. Using the manual search step, a systematic review of the research was confirmed as complete and relatively comprehensive.
Selection of Studies As shown, using the steps for effecting automatic searches, 141 related studies were found. In the manual search, all articles were checked using the Mendeley tool. Duplicate articles were removed by checking the titles of each article. As a result, a total of 121 articles were obtained. The focus then moved to reviewing the summary of each article, which subsequently included 98 articles. In the final step, these articles were examined by focusing on their content, reducing the overall number to 43. Figure 11 shows the stages for selecting articles, based on checking the inclusion and exclusion criteria.

Quality Assessment of Primary Studies
Quality assessment (QA) was performed for each study to measure the quality of the content presented. A set of quality assessment questions was created to determine the suitability of the related studies.
(1) QA1: Are the research goals/objectives clearly defined? (2) QA2: Is the solution based on AR/serious games/PECS clearly defined? (3) QA3: Is the research methodology specified in the article? (4) QA4: Are the search results reported?
The above four QA criteria were tested for the 43 included research papers to determine their reliability. The QA comprised three quality schema stages, i.e., high, medium and low, in which the quality of each paper relied on its loading score. For example, papers that satisfied the criteria were awarded a score of 2, papers that partially satisfied the criteria were awarded a score of 1 and papers that did not provide any information regarding the question and did not satisfy the criteria were awarded a score of 0. Consequently, based on the four QA defined criteria, studies with a score of 6 or above were considered to be of high quality, studies with a score of 5 or above were considered to be of medium quality, and studies with a score below 4 were considered to be of low quality. Table 7 presents the QA list for each study.  Data Extraction and Synthesis A set of guidelines described in [121] and related to the data extraction process was followed to determine the relevant information from the related studies The data extraction method was designed to accurately record all information for each study in a summary. Features were determined considering all the research questions related to the systematic review.

Combining an Augmented Reality Environment with Autism Spectrum Syndrome
A summary of related studies in the AR field, based on publication year, study type, country, participants, evaluation time, intervention, research strategy, methodology, device type, skills targeted and limitations are presented in Table 8. Figure 12 shows the number of studies published between 2015 and 2020. As is evident, a number of studies were recently published due to the growing interest in AR technology. This interest was reflected in the announcements of large companies supporting AR technology in their products (e.g., Apple). Among the related studies, 36% were published in journals, while the largest percentage was published as part of conference proceedings (64%). We note that the percentage of studies published as part of conferences was greater than the percentage of studies published in journals. We anticipated this to have been due to limitations that will be discussed in detail in the section on ongoing issues, which could be bypassed and published in conferences.
Through deeper analysis of the relevant studies, the following results emerged.
• Figure 13 shows the number of included studies per country. The bulk of these works were from Taiwan (six published studies), followed by Malaysia, Brazil, Qatar, Portugal and the United States (one study per country); 11 studies did not clarify their country of publication. This indicates that researchers in Taiwan are interested in integrating AR in the rehabilitation of people with ASD and may also be attributed to the increased prevalence of autism. Based on a World Health Organization (https:// www.who.int/news-room/fact-sheets/detail/autism-spectrum-disorders, accessed on 13 December 2021) report, the prevalence of autism worldwide is between 1 and 6. This prevalence is rated between 2.8% and 4.1% in Taiwan [122].    The face2face unit is not a precisely described unit that contains a series of levels and difficulty settings The system was built without the involvement of the autism specialist or caregivers

•
The devices used in AR systems for the rehabilitation of ASD patients were limited. Two studies used smart glasses; six studies used computers (the most common device used) in rehabilitation centres and 12 studies used mobile devices (smartphones, tablets and laptops). Three studies that used smartphones were obtained and another three used a tablet device. We found only one study that used a head-mounted device; one study did not mention the type of device used. • Figure 14 shows the various skills targeted in the related studies; these can be classified as follows. The largest number of studies (eight) targeted social communication skills, e.g., the effectiveness of asking for help and expressing abilities. Five studies involved facial expression skills and focused on six expressions (anger, fear, disgust, happiness, sadness and surprise). Six studies included cognitive skills, e.g., academic skills (arithmetic, geography, biology, numbers and letters), as well as attention skills, mental representations, focus skills and the recognition of objects. Daily living skills were represented in only one study. One study did not reveal specifically addressed skills.  Table 9 presents a summary of studies related to the field of serious games. It is presented in terms of year of publication, study type, country, participants, evaluation time, intervention, research strategy, methodology, device type, targeted skills, activity, modality, interaction style, environment and limitations. Figure 15 shows the number of studies published between 2015 and 2020. Only a small number was published during the past five years; the highest number was in 2016 (at a rate of three studies) and in 2019, this decreased to one study. We expect that this had been due to a lack of game designers interested in the field of ASD rehabilitation. The percentage of conference publications (67%) was larger compared with journal articles (33%). Regarding the number of studies per country, six countries (Canada, the United Kingdom, Egypt, the Philippines and Serbia) each published one study. There were six studies for which the country of publication was not noted.

Serious Games with Autism Spectrum Disorder
Devices used in serious games for ASD rehabilitation were small in number; five studies included computer use and six included mobile devices (smartphones, tablets and laptops). One study using a tablet device and one employing a laptop were obtained. Figure 16, shows the variety of skills targeted in related serious games studies. The largest number of studies targeted cognitive skills (six), which included skills such as understanding, academic skills, financial concept skills and computational thinking skills. This was followed by two studies that involved facial expression skills; two additional studies included social and communication skills. Only one study included emotional skills, motor and cognitive skills.
We previously noted that the activity criterion included three types, i.e., mental, physical and physiological activity. However, physiological activity was not included in the current paper because it targets and treats autistic individuals with impaired movement, as well as visual and auditory sensitivity. All 12 related studies targeted mental activities, in contrast to physical activities, which were implicit in two studies with mental activities; no study focused only on physical activities.  Studies related to serious games included three styles: tangible interfaces, keyboard and/or mouse and the Microsoft Kinect system. Tangible interfaces were used at a rate of 58% and included physical or digital interfaces. Representing a traditional interaction style, a keyboard and/or mouse was used in up to 25% of studies. A relatively new style involved using the Kinect (17%). Figure 17 shows an unexpected consequence of the environments used in serious game studies. Despite the variety of environments in serious games, they were limited to 2D aspects in eight studies. Conversely, 3D, 2D/3D, AR and multi-player games were each represented by one study (excluding studies that included VR).  Table 10 summarizes the results of the study on the application of PECS in terms of publication year, type of study, country, participants, and the number of phases applied, evaluation time, device type, intervention, search strategy, methodology and limitations. Only a few studies on the applications of PECS were published (nine in total); the highest value was found in 2018 (four) and in 2019 this decreased to two studies.

Applying the Picture Exchange Communication System with Autism Spectrum Disorder
Unlike the fields of AR and serious games, 67% of related studies were published in journals, while conference papers represented 33%. We note that the percentage of studies published in journals was almost twice that of conferences papers.
The PECS studies included three countries, i.e., Taiwan (one study), Indonesia (two) and South America (one). Five studies did not denote a country of origin.
When considering studies related to the number of applied PECS phases, we found that they were limited to the first four phases (One study exception). Figure 18 shows that four studies applied four phases and three studies used three phases. Only one study used all six phases and another only applied the concept of PECS.   The reason for the most studies being limited to the first four phases was because phases five and six of PECS require people with ASD to answer a set of questions and talk about their feelings; this is not provided for in PECS's applications.
We found PECS applications to be limited to mobile devices (100%). One study focused on the use of smartphones and one other on using a tablet; six studies all included mobile devices. One study made no mention of the device type.
Among the important results in the techniques and approaches for delivering intervention through the applications of PECS, we found that AR had been used in three studies. The remaining studies were more diversified and included many different technologies and approaches, such as child-computer interaction, the Treatment and Education of Autistic and Related Communication Handicapped Children methodology (TEACCH ® ) and Autisdata software, the goal-directed design method, aided learning, human-computer interaction and mobile instant messaging.

Discussion
The main findings related to each RQ and SRQ of the current study is briefly described below. . All results indicated that the use of AR technology had a positive effect on the progress of speech and language therapy in particular, as well as the social skills and communication in general for people with ASD. Moreover, the experience of AR technology was not limited to its impact on social skills and communication; other positive effects such as increased interest in the education process through participation, enjoyment and motivation were also indicated. The educational processes for people with ASD require long-term continuity; therefore, AR technology provides a very enjoyable and stimulating learning environment with a wide variety of sources for learning activities, which can help to avoid students becoming bored quickly. In addition, AR technology was able to deepen an understanding of the relationships between social situations and the different behaviours of individuals with ASD; in this context, it was observed that AR enhanced their social cues. Moreover, through exercises, role-playing games and interactive games using augmented reality, social skills were significantly affected.
Despite its positive effects, however, there are limitations to the use of AR in the rehabilitation of people with ASD. Scanning signs or objects using camera capturing are not designed in a manner that considers the capabilities of people with ASD. These signs and objects must be in a specific position, have good lighting and must be at a reasonable distance. For AR characters, these were represented by three types in the reviewed studies, i.e., cartoons, avatars and emojis. However, in most studies reviewed in this current research, the gestures and facial expressions of the characters were not considered. Only one study created an avatar; in this case, the faces of the participants were captured by taking several photos at different angles ([A_9]); however, this will be a complicated process if the number of participants is large.
• MRQ2: What is the effect of applying the standards of the serious game developed for children of ASD?
Through their findings, studies relating to the efficient use of serious gaming standards in many learning practices enable their implicit understanding. These findings indicate a positive effect in terms of enhancing the motor skills and interactive functions of people with ASD, particularly children. The results also showed the effectiveness of serious games in terms of improving the language skills of children before and after using the game. Moreover, more interactive games motivated children to work towards common goals. Importantly, serious gaming standards were able to enhance the visual support factor, which must be taken into consideration when working on the rehabilitation of people with ASD. However, studies related to this context were limited to implementing traditional standards and these were not properly applied. Only one study among all those described, employed the AR environment ([S_12]), despite its positive impact on educational activities. Due to the low level of diverse environments in serious games studies, a lack of standard interaction patterns emerged because the environments were limited to the traditional mouse and keyboard interaction methods, as well as tangible interfaces.
• MRQ3: How effective are the developed PECS applications for ASD children?
The effect of PECS applications was effective and positive in all respects. Studies showed that using the benefits of PECS applications effectively improved communication among ASD individuals with communication problems (verbal/social communication). The reasons for improvement included reducing the burden of carrying many traditional cards to make it easier to manage them; PECS applications apply a large range of symbols and images alongside the feature of adding sound effects to teach pronunciation and motivate people with ASD to learn. Moreover, PECS applications alleviated the difficulties faced by caregivers, such as reducing verbal repetition and accessing cards faster, thereby overcoming spatial and time constraints and enabling ASD progress monitoring without indirectly interfering at any time and any place after the completion of the session. Despite the many advantages we have mentioned, the related studies were small in number and several presented limitations. The applications lacked interaction and verbal communication (applied in phases five and six of PECS), providing one study ([P_3]), It is suggested to implement phases 5 and 6 in a cooperative manner between two players, through simple comments, asking for help and giving opinions while playing. Moreover, studies lacked participation and interaction with the caregivers, parents or friends of those with ASD; involving these groups could potentially increase the effectiveness of treatment using PECS's applications, particularly in advanced stages of training. Most of the studies that addressed the need for the mouth movement feature which focus on clarifying how to pronounce the names of things and the mouth movement while the word is pronounced.

•
MRQ4: Can the application of PECS be developed with AR technology?
In the [P_2], [P_3] and [P_4] studies, it emerged that it is possible to design PECS applications using AR technology. The results also indicated an improvement in the ability of verbal communication, but more studies are needed over longer periods. Another major limitation is that the PECS approach has not been effectively integrated with AR technology, as AR was included only in some stages.
As we previously noted, one study included in this review was related to serious games that used an AR environment ([S_12]). This study aimed to motivate children with ASD to acquire emotional recognition skills by attracting their attention and stimulating them with AR technology. What distinguishes this study is that its approach represented fun and interesting ways for people with ASD, where the GameBook was used to represent and interact the events of the story using AR technology. In contrast, also the study is a prototype that has not been evaluated for the reliability of serious games using AR technology.
(2) SRQ2: Can all phases of PECS be applied in an AR application Based on studies [P_3] and [S_8], it seems possible to create a PECS application that supports all six phases of this approach. Studies demonstrated positive results for developing communication and interaction skills through the application of PECS. However, These two studies is insufficient for demonstrating the effectiveness of supporting all phases of the PECS approach in AR application, as further testing involving a larger group of participants is required. Moreover, this study employed AR technology and, as such, are prone to the same limitations that we mentioned.
(3) SRQ3: How long can ASD children continue to be attracted to PECS's applications?
Maintaining the attention of individuals with ASD during treatment using the PECS approach is a primary obstacle, as treatment sessions for as long as five years may be needed to gain and retain minimum verbal skills. Therefore, attracting the attention of people with ASD for the longest possible period is required. None of the studies included herein could conduct PECS evaluation sessions for long periods; the longest time reached by related studies was four weeks ([P_1]). Accordingly, many more studies that evaluate the attractiveness of using PECS applications and that measure the maximum possible period for maintaining the attention of ASD individuals when using PECS applications are needed. The attraction factor mainly depends on the technologies that are incorporated in PECS applications.
(4) SRQ4: How much can we benefit from previously reviewed papers that target ASD children?
Through an in-depth analysis of all related studies, many aspects emerged that present several areas of potential development for future studies. These aspects are described in the following points.

•
Participants: Based on the inclusion criteria for the current survey, all studies involving individuals with ASD that present with one specific symptom (mild, moderate, or severe) and no movement or cognitive impairments, special needs, or visual and auditory sensitivity were reviewed. Participants in the studies related to this survey included children and adolescents with ASD. This a group of ASD did not show symptoms of autism in strangers; but, the situation becomes embarrassing in dealing with them despite their high ability to learn, so they are not treated equally with other groups in which the symptoms of autism are a phenomenon. The different personality traits of people with ASD called for different techniques and interventions to be implemented in their rehabilitation, as most techniques that rely on sound or visual effects may negatively affect people with ASD who have auditory/visual sensitivity. Two included studies addressed the difference in personality traits of people with ASD during the development of the proposed learning systems ([P_3] and [S_4]). By allowing the systems to have flexible configurations to suit the needs of all those with ASD, such as choosing the gender, application color, and volume of sounds in the application etc. In terms of the number of participants, this ranged in most studies from one-to-four participants. These small samples may have been due to the difficulty of dealing with large numbers of individuals with ASD, based on the emotions they may experience during an intervention. Furthermore, each patient needs to be treated independently by a person they are familiar with. ) the use of traditional devices (computers) was beneficial in terms of the variety of devices that could be linked to it, e.g., adding a motion sensor or large display screens; however, it was restricted in that they could only be used in rehabilitation centres. Additionally, many of these devices were expensive and it may also be confusing for people with ASD to interact with more than one device at the same time. Furthermore, advanced technology devices require an expert to operate and manage them. The use of smart glasses presented potential benefits in two studies ([A_13] and [A_16]). The use of smart glasses was found to be less distracting and less demanding in terms of cognitive workload. This was because smart glasses do not require the use of hands, which is beneficial in terms of not distracting the user and allowing them to only focus their visual attention during the learning process. However, the side-effects of using smart glasses for a long time for people with ASD remain unknown. Moreover, smart glasses are not able to address the visual problems of ASD individuals. ). These devices present multiple advantages, such as their portability, the ability to use them at anytime and anywhere, as well as their low cost compared with more traditional devices, which are limited in certain places and times. Moreover, people with ASD exhibited positive reactions when using smartphones and tablets. Despite promising results, however, some studies also expressed concerns about the use of smartphones and tablets, as prolonged and unregulated use can lead to isolation among ASD individuals and reduce their awareness of the surrounding (social and physical) environment; this is particularly the case for ASD individuals who primarily experience a lack of social communication skills. Selected studies showed that smartphones increased stress and pain due to remaining in the same position for an extended time.
Head-mounted devices were used in one study ([A_8]). These represent close-up displays that provide more control for those with ASD in terms of what they can observe, unlike the screens of mobile devices, which have a limited field of view. In addition, head-mounted devices enhance a sense of fun and can present a realistic AR experience. The disadvantage of head-mounted devices appears to be their need for operating in a low-light environment.
• Culture: East Asian culture featured predominantly in the reviewed studies. Interestingly, no established studies were found in an Arabic culture setting. One study derived from Qatar ([A_5]) and another from Egypt ([S_5]); however, these studies depended on having knowledge of the English language or translating it only into Arabic.

•
Interventions: By combining the advantages of AR studies, serious games studies and PECS application studies, regardless of the characteristics of the participants, a diversified series of research can be collected that addresses current research problems in correlated studies.
We found one serious games study that employed an AR environment ([S_12]). The study aimed to create a game that would attract children's attention to develop social and communication skills. This approach used the software Gamebook, which depends on a graphic interface, character interactions, voice and text narration and AR technology by using a children's book. The AR in Gamebook provides an experience within a safe and attractive environment and can enhance a child's attention and imagination without diminishing it. Moreover, through the game's website, the system provides a feature that is able to monitor the results of children without direct intervention, thus supporting the child's independence. All of these features were included in this work without financial costs or the use of traditional peripheral devices such as a keyboard and mouse.
Conversely, one serious games study also targeted social and communication skills but did so using the PECS method ([S_8]). The game in this study included 15 levels, which were divided as follows. (1) Levels that achieved differentiation and recognition skills through the use of the first, second and third stages of the PECS approach. (2) Levels that achieved the skills of building sentences and communicating with others through the use of the fourth stage of the PECS approach. (3) Levels (14 and 15), in which progress in communication skills was achieved by asking for help and making comments while playing with other players, and which followed a cooperative approach that achieved the values of participation and cooperation between players.
We also found three studies that combined AR with a PECS ([P_2], [P_3] and [P_4]) approach. In [P_2], the first three phases of the PECS approach were implemented alongside an AR technology intervention. Augmented reality technology displayed an animal class as 3D objects in a scene, thereby enhancing the attraction factor for ASD individuals. In the stage that was implemented, the goal was to achieve object recognition skills by clicking on the icon for each animal. The animal that was clicked display for viewing in a 3D image. In [P_3], the 3D objects were expanded because the application was designed with seven menus, each representing a stage of the PECS system; a final list was also presented to exit the game. The first stage comprised five categories, i.e., the classes, a dining room menu, a bathroom menu, a bedroom menu and a playground menu. In each category, all the components of the selected category were displayed. At the end of each stage, a short test was presented to assess ASD children. In [P_4], the first four phases of the PECS approach were used. Here, a conventional PECS card was scanned by smartphone camera and each card was represented by 3D objects. In the related studies, AR was used with PECS applications in a traditional manner, i.e., the representation of objects in 3D form only, without interaction or having a logical sequence of events. We believe that by leveraging the standards of serious gaming and the benefits of an AR environment to develop PECS applications, the problems that emerged in the reviewed studies could be avoided. None of the included studies adopted such an approach.
• Independence: A relationship was observed between the type of devices used in studies and support for autonomy in working on the system without the presence of the device operator for the duration of the treatment session. In studies that relied on using a computer, an expert or care provider was required to operate them or to assist in using peripheral devices. Therefore, mobile devices support independence via their ease of use. ) indicated that games that specifically reinforce cooperative behaviours can help to develop the social behaviours of ASD individuals because they enhance an understanding of others, use simple interactive dialogue (such as asking for help) and make use of eye contact. However, these studies explained that some ASD individuals may more frequently engage in collaborative activities only to affect blame and to complain. In some instances, reactions made by ASD individuals may be unexpected in the event of mistakes on behalf of one of the parties involved.

Ongoing Issues and Future Research Directions
In this section, we provide a summary of ongoing issues and possible future directions for research based on the reviewed studies.

Ongoing Issues
Two types of ongoing issues (general and private) emerged from the included studies. The general issues refer to common issues in all of the reviewed studies, regardless of their particular field. Private issues are those that are specific to one area among those discussed herein.

General Ongoing Issues
• Several participants The number of children with ASD who participated in all of the reviewed studies was small, ranging in most (20) studies from one-to-four participants. Interestingly, there are few studies that recruited a large number of participants, as the number of study participants was [A_19] reach to 200 participants over a period time of up to 3 years in AR studies. Hence, the included studies do not represent a strong body of evidence as the findings have not been validated by more systematic evaluations.

•
Duration of the experiment Behavioural therapies for people with ASD require long-term skills training. These issues are among the most prominent. In most of the reviewed studies that lasted up to four weeks, it is important to note that the number of sessions had not been counted because there was no clarification on the required sufficient period for acquiring skills. As already stated, one study ([A_19]) spanned a period of three years. We believe that the reason for the short duration of the experiment in this particular study can be ascribed to the fact that many of the studies had been conducted for specific conferences; accordingly, the duration of the experiment was adapted based on the timing of these conferences.

•
Completing the experiment Completing an experiment can be a difficult task for people with ASD. We observed in most of the studies the withdrawal of one or two participants from the experiment for several reasons. One of these reasons related to experimental difficulties, that is, either because of the complexity of the system or because of the selection of participants whose characteristics did not match the experience. Other causes were related to the mood and physical state of people with ASD (aspects such as tension, fear and hunger must be considered). Furthermore, in some studies, people with ASD did not interact with researchers because they did not feel comfortable interacting with strangers.

•
Duration of use of mobile devices Despite the reported positive results and stimulus interaction resulting from ASD individuals using mobile devices, there is a worrying trend related to the use of these devices. Evidence in studies indicates that the use of mobile devices can cause feelings of isolation for people with ASD, which may reduce their awareness of social and physical environments, particularly for ASD individuals with reduced communication and social skills. As such, prolonged use of these devices during the treatment period may contribute to a deterioration of these skills.
• Culture As previously noted, most of the reviewed studies were based in East Asia, indicating a lack of other additional cultural approaches in related studies. The bulk of studies were written in the English language or propose a studies based to the translation of the English language without carrying the original culture of the country of origin for the study. The images used in these studies represent selected cultures and may thus reduce the effectiveness of the social communication process. None of the included studies focused on studying the effect of culture and AR applications on the rehabilitation of people with ASD.

•
Augmented Reality The main problem in studies related to AR environments is that these environments involve only a display of objects with/without cards. As such, they lack a connection to any logical sequence of events. Moreover, although applications that rely on AR environments are being developed, few of them consider the needs of people with ASD in their communication with caregivers or with other ASD individuals. Moreover, a technical issue related to displaying AR objects currently appears unclear; the cause may be due to shaking of the camera device, particularly shaking caused by ASD individuals. Another reason may be attributed to lighting. In addition, in the studies included herein, these applications lacked response speed or a sense of real interaction with ASD sufferers.

• Serious Games
One of the most prominent issues regarding serious games was the lack of diversity in the applied standards; furthermore, these standards were not properly employed. Due to the lack of diverse environments in the studies focused on serious games, an absence of standards for the interaction type was evident because this was limited to traditional methods. Furthermore, as was the case for the development of AR applications, which are limited to being created by software developers, most studies related to serious games often developed applications that lacked a sense of fun. A reason for this may be because the games are designed for sponsor providers or researchers who did not consider using creative game designers who are better equipped to create applications that ensure a focus on fun and engaging users.

•
Picture Exchange Communication System (PECS) The lack of applications using PECS have makes it a poorly researched topic with several obstacles. Most PECS applications were applied in a traditional manner that simulated the use of a manual card. These were primarily 2D digital displays with added sounds; as such, they lacked visual and sensory interaction. Moreover, based on this design, the application made the process of moving between images difficult, either because of the size of the image, which was often small or, alternatively, because of the process of dragging images to form sentences. As we previously noted, most of the PECS applications were limited to the first four stages, creating issues linked to reduced social interaction, which the final two stages of PECS can provide.

Future Research Directions
The information presented in the 'Discussion' and 'Open Issues' sections can assist in predicting future trends that may contribute to additional studies linked to developing the field of rehabilitation for people with ASD.
Participants By reviewing the number of participants and the personal characteristics that must be considered in studies related to the rehabilitation of people with ASD, future studies may opt to include their number of participants. Autism centres can be contacted to carry out a systematic evaluation of up to 12 participants, both male and female, with no other diagnoses except for ASD. The age group ranging from three to 12 years of age can also be included. This age group is able to rapidly acquire communication and social skills but is the least focused on by ASD research.
Device type We support the use of mobile devices for several reasons including their attractiveness to people with ASD. Moreover, they can be used almost anywhere without significant financial costs and supports the pattern of repetition required by behavioural therapies for people with ASD. It is possible to avoid the isolation that accompanies frequent and prolonged use of mobile phones in future studies by imposing specific periods of use, e.g., during treatment sessions in clinical centres. This can be applied by adding a feature for caregivers through which to set the time for using the application, from one hour to three as a maximum.
Culture Studies can be enriched in the future by integrating local culture in the rehabilitation of people with ASD. We aim to create an application that represents Arabic culture through the creation of characters depicted in local dress or engaging in social customs to help ASD individuals acquire and reinforce Arab social behaviours.

Interventions
The incorporation of PECS applications in serious AR environment-based games can provide solutions that ensure that the limitations present in each of the studied areas are improved in future research. An AR environment can provide 3D characters in a simulated real environment to help ASD individuals gain social skills in an attractive and comfortable environment for people with ASD. Moreover, serious games provide a logical sequence of events by following a storytelling approach, which can guarantee aspects such as fun and suspense and implicitly enhance social skills. By carefully diversifying the standards of serious games, educational environments can be created. In future studies, we aim to apply all of the standards shown in Figure 19. In this regard, for the PECS methodology, all six stages can be applied, including a treatment sequence that will provide an implicit aspect in serious games.

Independence
Using mobile devices can enable people with ASD to be autonomous because they can use the application without assistance. In future studies, this can provide independence through the creation of simple application interfaces, alongside tips that can help guide ASD individuals during the application experience.
Monitoring progress Relevant studies on monitoring progress can be useful for future research. A progress monitoring feature can be included in the application to allow caregivers to accurately monitor the progress of ASD individuals. A breakdown of each stage completed and the time it took the participant to complete it can be included. Charts can also be added to facilitate the monitoring of participant progress.
Cooperation Consideration should be given to the problem of how cooperation skills can be enhanced within the target skill framework, particularly when more than one player is involved and the game is conducted over the Internet. Accordingly, in future studies, an AR avatar will be added to represent a parent or friend of the ASD individual. This will be done by adding an image of the individual and having the application create an avatar using said image. People with ASD will then be able to engage in a range of simple conversations and collaborate with the avatar to finish the skill-set target frameworks alongside a familiar character.

Conclusions
Augmented reality environments have recently gained significant attention in a range of different contexts. This environment can be used efficiently if it is combined with the rehabilitation of people with ASD, e.g., using serious games and PECS applications. In this paper, a rigorous SLR was conducted about these approaches using 172 research papers in three domains published from 2015 to 2020. In the final stage, a total of 43 papers were examined. Augmented reality was represented in 22 studies, serious games studies represented 12 studies and a total of nine PECS application studies were included. We began by detailing the background and relationship between these three areas. We then reviewed existing surveys related to these three areas and found no other surveys related to PECS applications. This is one of the main contributions of this survey. We highlighted the studies related to each field in detail and noted their results. We also discussed the benefits of these studies for the research community and highlighted how future studies can be informed by making use of our review findings. Finally, we provided insights about the remaining issues and future directions for research to improve the application of PECS alongside AR-based serious games.