Next Article in Journal
Simulation of Aerosol Evolution within Background Pollution for Nucleated Vehicle Exhaust via TEMOM
Previous Article in Journal
Deep Learning Based Real Age and Gender Estimation from Unconstrained Face Image towards Smart Store Customer Relationship Management
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Applied Sciences
Volume 11
Issue 10
10.3390/app11104550
applsci-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Mobile Augmented Reality Technologies for Autism Spectrum Disorder Interventions: A Systematic Literature Review

by
Xiaojie Lian
1,2 and
Mohd Shahrizal Sunar
1,2,*
1
School of Computing, Faculty of Engineering, Universiti Teknologi Malaysia, Skudai Johor 81310, Malaysia
2
UTM-IRDA Media and Game Innovation Centre of Excellence, Institute of Human Centered Engineering, Universiti Teknologi Malaysia, Skudai Johor 81310, Malaysia
*
Author to whom correspondence should be addressed.
Appl. Sci. 2021, 11(10), 4550; https://doi.org/10.3390/app11104550
Submission received: 7 April 2021 / Revised: 5 May 2021 / Accepted: 10 May 2021 / Published: 17 May 2021
(This article belongs to the Section Computing and Artificial Intelligence)
Browse Figures
Versions Notes

Abstract

:
Over the past decade, enhanced computing capabilities and mobile technologies have begotten the upsurge of innovative mobile health (mHealth) solutions, and many research efforts have occurred recently in the area of technology-based interventions (TBI) for autism spectrum disorders (ASD). Mobile augmented reality (MAR) refers to AR systems that use the handheld mobile device medium (mobile phones, tablets or smart glasses). This article reports the results of a systematic review undertaken on the use of MAR for ASD-related skills learning from the year 2010. It aims to provide an insight into the current state of research on MAR interventions and to provide guidance to relevant designers and researchers. We searched seven databases and retrieved 625 articles initially. After exclusion and screening, 36 articles were reviewed reporting on using MAR to improve various skills of children and adolescents with ASD, and 10 research questions related to PICO (P: Population, I: Intervention, C: Comparison, O: Outcomes) were addressed. This study identifies challenges that still exist in the research efforts towards the development of applications exploiting the MAR for ASD interventions: technology issues, research design consideration, subjective assessment etc. The studies examined suggest researchers should focus on users and improve the quality of the MAR app. In addition, more effective research methods and evaluation methods could be involved in future studies to facilitate the development of MAR intervention applications.

1. Introduction

Autism spectrum disorder (ASD) is a neurodevelopmental disability varying in severity across the spectrum with two main characterized symptoms: (1) impaired communication and social interaction and (2) repeated activities and interest [1]. The World Health Organization (WHO) reports that approximately 1 in 160 children has been identified as having ASD, which affects the entire range of daily living activities from a very young age, restricting social participation of individuals and even need lifelong care [2]. Due to increasing attention to autism and increased awareness of clinical screening, the relevant data may increase significantly. According to the fifth edition of the Diagnostic and statistical manual of mental disorders (DSM-5), all different subtypes, including autism, disintegrating diseases in children, Asperger syndrome, Rett Diseases and generalized developmental disorders are classified as one diagnosis: autism spectrum disorder (ASD) [3].
Early Intensive Behavioral Intervention can improve the defect of autism to a certain extent [4]. However, the materials of some traditional media intervention methods, such as Video Modeling (VM), are usually too long and have no interaction mechanism [5,6,7]. It is difficult to dynamically adjust their attention focus and switch their attention position. Over the last years, information and communication technologies (ICTs) have been widely utilized to help the healthcare industry [8]. In the field of autism rehabilitation, technology-based interventions (TBI) include social robots [9], computer-based interventions [8,10,11,12], VR [13,14], tablet computers [15], serious games [16,17] and other TBI interventions [18,19].
As a new type of human-computer interaction (HCI) technology, augmented reality (AR) has evolved drastically over the years, which can combine real scene information and virtual information to provide rich visual information and diverse interactive experiences [20,21,22,23]. Evidence supports that people with autism are keen to process visual information and use digital devices, and AR provides new ideas for improving their learning experience [24,25,26]. Their parents also report that technology, including smartphones and tablets, plays a key role in addressing these behavioral challenges [27,28]. MAR applications make the treating process more interesting [29,30,31,32,33] and affect the academic achievement of children with ASD in a positive way [34,35,36,37,38,39,40].
Several literature reviews have been published about the use of augmented reality technologies in interventions with children and adolescents with ASD. Marto et al. have conducted a systematic literature review (SLR) of 16 primary studies on the use of AR for the rehabilitation of people diagnosed with ASD [41]. However, only eight studies on MAR were included. The authors in reference [42] reviewed 10 preliminary studies published from 2012 to 2016 and put forward some suggestions for future development and evaluation. Similarly, there are only six studies of MAR in the collection. In 2018, Adnan et al. [43] listed the development and research routes of AR in the field of autism intervention, while the search for literature was inadequate. Khowaja et al. [44] have conducted an SLR of primary studies published in 2005–2018 on the use of AR to learn different skills for children and adolescents with ASD and proposed the research classification of ASD; however, their study did not address MAR, and the research timeframe needs to be updated. Berenguer et al. [45] carried out an evaluation of the effectiveness of AR technologies on ASD. In this paper, we conduct an SLR of the use of MAR for ASD-related interventions from 2010 to 2020 and address ten research questions (RQ) related to PICO in order to identify current trends, future prospects and possible gaps related to the mobile AR technologies in the field of autism spectrum disorder.

2. Methods

According to Kitchenham [46], a systematic literature review (SLR) is a method of identifying, evaluating and interpreting all available research relevant to a topic area, or particular research question. We conduct this research in accordance with the PRISMA guidelines [47], and we follow recommendations outlined by Kitchenham.

2.1. Research Questions

A total of ten research questions (RQ) were formulated following the PICO framework to implement a comprehensive review of the topic [48]. In this study, the Population (P) is composed of autistic individuals enrolled in the intervention process; The Intervention (I) considered is the use of AR applications for ASD; The Comparison (C) is not applicable since the aim of this study is to describe the application of MAR; Finally, the Outcomes (O) refers to the main outcomes obtained. As a result, we elaborated ten RQs that guide the purpose of the study, as shown in Table 1.
In RQ1, the channel refers to the demographic information of the primary studies, such as the published years, article types, and quality we assessed. In RQ8, the setting refers to the environment in which the intervention is implemented. In RQ9, sustainable means keeping the skills learned over time, and in RQ10, generalization refers to the transfer of skills learned from one situation to a new situation.

2.2. Search Strategy

A search of peer-reviewed published literature was implemented in February 2020 for articles related to the use of MAR technologies in ASD interventions. The search process was conducted using keywords that link to our topic, and the keywords combined by the Boolean operators are determined as follows: (“Autism*” OR “Autism Spectrum Disorder” OR “ASD”) AND (“Mobile augmented reality” OR “AR” OR “Augmented reality”) AND (“Mobile” OR “Tablet” OR “Smartphone” OR “Phone” OR “Smartglass”).
The following inclusion criteria were used:
  • Articles published after 2010;
  • Studies published in English;
  • Research from peer-reviewed journals or conferences, books and lecture notes;
  • Articles focused on mobile AR applications for ASD people;
  • Articles must be a full or short version (not an abstract).
Seven online databases linked to the scientific fields relevant to both technologies and ASD interventions were selected to search primary studies for this review. These databases include Web of Science, Scopus, ACM Digital Library, ScienceDirect, IEEE Xplore, SpringerLink, PubMed and Google Scholar. The search and selection process is described in Figure 1, with the following steps:
  • A separate search was carried out in the identified online databases, and search results were manually imported to EndNote for further screening;
  • Individually check and remove the incorrect references (such as abstracts or contents, etc.) from the EndNote;
  • To remove all the duplicates studies;
  • To check the research title, abstract, introduction and conclusion, and exclude documents that do not meet the inclusion criteria;
  • Add the research we missed from other review paper;
  • Perform a full-text review to delete non-mobile AR studies (such as device used computer or Kinect, etc.).
Applsci 11 04550 g001 550
Figure 1. Literature search and selection process.
Figure 1. Literature search and selection process.
Applsci 11 04550 g001

2.3. Quality Assessment

The importance of the selected articles was weighted for quality assessment, and the eight quality assessment questions were used together with their weight to evaluate the completeness and content relevance of the primary studies. CORE Conference Ranking 2020 [49] and Journal Citation Reports (JCR) 2019 [50] were used accordingly for conferences/workshops and the journal index papers. The criteria used, along with their weight, are presented in Table 2.

2.4. Data Extraction

The data extraction process aims to identify relevant information from the primary studies in depth and explain all research questions related to SLR. The process was stood by the RQs and Data extraction process defined in Table 1, and the review results are shown in Table 3 and Table 4.

3. Results

Among the 625 articles extracted from online database search results, we identified 491 incorrect and duplicates references. The 134 remaining papers were then checked for inclusion on the basis of the abstract and keywords, and there were 47 potential papers for inclusion. A further 20 references were eliminated after a full-text review, and 36 articles were included in the end. According to the results of the quality assessment, three papers got the maximum score of 8, three scored 7.5, and one paper scored 6 (see Table 3). Most of the reviewed papers (65%) are above half of the total score. This fact confirms that the reviewed papers are appropriate for the research conducted.

3.1. RQ1: What Channels Are Used to Publish Research Articles in ASD Interventions Exploiting AR Technology?

According to the review results, studies are published in different channels. Figure 2 shows the number and type of papers per year. The reviewed articles were published from 2011 onwards, with 61.1% (n = 22) of them being published after 2018. More than one-half (58.3%) were reported in journals, while 38.8% is published in conference proceedings, only 2.7% is published in books. Almost half (47.2%) of the sources are recorded in technology-related journals or conference proceedings. The remaining papers were published in educational, medical conferences and journals. Journal of Research on Technology in Education appeared two times [59,60], while the rest of the articles are distributed in 19 journals, 14 conferences and 1 book. 37.5% (n = 9) of publications presented in the JCR ranking, and 21.4% (n = 3) of conferences listed in CORE rankings. As for the active researchers, it was found that the author Keshav and Sahin both have three studies as the first author and McMahon, Chung, Lee, Vahabzadeh and Escobedo have two studies as the first author each.

3.2. RQ2: What Mobile Device Was Used?

Researchers have used different devices in the intervention, and the technologies and devices used in each project include mobile smartphones, laptops, iPads, smart glasses and iPod touch was listed in Table 4. More than one-half (55.6%) of research intervention projects use mobile smartphones (n = 10) and smart glasses (n = 10). The proportion of iPad and tablet used is 19.4% (n = 7) and 13.9% (n = 5) each. It was reported that they could use any mobile device (tablet, smartphone or laptop) in three studies, and iPod appeared one time.

3.3. RQ3: What Are the Targeted Population?

Figure 3 presents their distribution. As we can see, less than 5 people participated in most of the studies (55.6%, n = 20), while 13.9% had 6–9 subjects and 25% (n = 9) recruited 10–20 participants. The number of participants in the research conducted by Keshav et al. [51] and Antao et al. [53] is more than 20. Furthermore, the brief overview of the basic demographic characteristics of participants of each study was demonstrated in Table 4. In total, 36.1% of the studies (n = 13) did not report the demographic information of participants in detail or even not recruit participants in the least, while four papers briefly present the number of participants. For instance, Lee et al. [58] used intelligence quotient (IQ) scores to test participants to determine whether they are suitable for participating in project research. The last studies (n = 19) described participants’ inclusion criteria and demographic characteristics in detail. In 14 studies included a total of 77 participants stated the proportion of male and female participants, male accounted for 81%. Only one study reported parental involvement.

3.4. RQ4: For What Purposes Have Been Applied the Proposed Technology Solution?

Figure 4 shows the classification of the target skills in the primary studies, and 16 main categories were identified, including social communication, behavior, learning, social reciprocity, facial expressions and emotion, attention, navigation, literacy, language skill, social skills practice, pretend play, motor skill, job interview, daily prayers, brush teeth and bakery. The majority of research conducted was related to improving social communication skills (31%), followed by behavior (11.1%) and learning (11.1%), while the remaining skills are used in two or one study each. There are four studies in which authors have used two or more skills [63,72,80,81]. The studies related to social communication and behavior were briefly described below.
A series of research has been conducted on the smart glasses systems Brain Power Autism System (Empowered Brain system), which uses AR and emotional artificial intelligence to assist the social communication of autistic children and adults. Keshav et al. [51] evaluated the tolerance and usability of the Empowered Brain system with ASD. Sahin et al. [52] investigated the safety and negative effect of EBS along with its effectiveness and usability in a classroom environment.
Lorenzo et al. [68] have conducted a study to assess the effectiveness of AR training programs named Quicker Vision app to improve social skills with autism. Taryadi et al. [73] have combined the use of AR with the PECS method (Picture Exchange Communication System) in training the communication ability of children with autism. The prototype developed by the authors called Multimedia AR app uses images, video and sounds that allow an individual to work with PECS by just scanning a QR code instead of selecting an image in traditional PECS. Lee [60] presented a report about an augmented reality coloring book (ARCB) to teach ASD children how to recognize and understand some specific social signals. This is the only study that has been found about augmented reality coloring books. ARCB can keep the participants’ attention and maintain the positive intervention results to the maintenance stage. Nubia et al. [74] have reported the use of an AR app to improve the communication field in children with autism. The goal is to investigate whether AR intervention will increase the attention process and speech expression.
Wang et al. [70] have conducted a study to enhance the effectiveness of autistic children’s request for help and expression functions. The authors developed the Auto Organizational Menu (AOM), which combined augmented reality (AR) with video of key parts with action (KPV) and evaluated it through the observation method. Abou et al. [72] developed a framework to support children with ASD and ID study in an enhanced education environment to improve social communication and behavior, while the users of the designed system are parents and teachers, which may reduce the engagement of autistic children. Regarding the skill of behavior, Vahabzadeh et al. [57] have investigated the changes in ADHD-related symptoms in children, adolescents, and young adults with ASD immediately after the use of the Empowered Brain system. The authors used the ABC-H scale to test the participants and grouped them according to their scores. Likewise, Vahabzadeh et al. [63] presented the behavioral and social-emotional effects of using the Empowered Brain on students with ASD to assess the feasibility and efficacy. They conducted a study to investigate if EBS reduces the irritability and hyperactivity of the users. Liu et al. [55] performed one session intervention in school and reported a lower ABC score in the post-test, which improved nonverbal communication, eye contact and social participation.

3.5. RQ5: What Research Designs Are Used in the Studies?

According to the data extraction results (see Table 4), a quarter (25%) of the published articles are no-experimental design, which is characterized by no random assignment design, no control group and no multiple measures. Two-thirds of the research was conducted with a single-case experimental design, and 8.3% (n = 3) reported a quasi-experimental design with a control group. Lorenzo et al. [68] conducted a pre-posttest control group design, which grouped 11 individuals (10 male and 1 female) into control (n = 5) and experimental group (n = 6). It is not a random assignment experiment, for they used a non-probabilistic intentional sampling method. Nubia et al. [74] and Antao et al. [53] presented a posttest control group design, which compared the evaluation of different groups of participants at the end of the evaluation.
In addition to the simplest pretest-posttest single-case experimental design, there also includes a multiple baseline single-case design that increases the internal validity by testing multiple baselines, alternating treatment single-subject design and withdrawing single-case designs.

3.6. RQ6: What Methods Are Used to Evaluate the Outcomes?

The outcome measurement method is used to record the results or outcomes in the mobile AR intervention. According to William M.K. Trochim, there are four main types of measurement: survey research, scaling, qualitative research and unobtrusive measures [84]. Among the primary studies, four main outcome measurement methods have been used, and these methods include an interview, questionnaire, observation and scaling. In total, 36.1% of the papers utilized a combination of different methods for measurement, while 16.7% of the studies did not report it, and the others used a single measurement method. More specifically, 17 studies used survey research (8 for questionnaires and 9 for interviews) along or with other measurement methods, 11 studies applied qualitative observation research, and 14 applied scaling. Survey research includes the design and implementation of interviews and questionnaires, and scaling involves consideration of the main methods of formulating and implementing scales. Qualitative research outlines a wide range of non-numeric measurement methods.

3.7. RQ7: What Are the Outcomes Obtained by the Application of the Proposed Solutions?

Table 4 depicts the major outcomes of the primary studies. In total, 22.2% of them reported results of the development of an app, prototype framework and concept, while did not conduct user studies [72,76,78,79,80,83]. The majority of research (42%, n = 15) shown an improvement of participant’s ability and performance. In the study conducted in [51,56,73], the qualitative analysis shows that participants enhanced their social interaction and communication skills. Of the studies, 16.7% found that they have improved continuous attention and engagement [26,54,55,70,73,74,82]. In the studies conducted by Cihak et al. [59] and McMahon et al. [62], the authors found that all participants were more independent after the AR training. The statistical analysis in [57] shows a reduction in irritability of ASD children. The results of [68] revealed no statistically significant differences between control and experimental groups with or without using AR-based intervention; however, the qualitative feedback provided by the researchers revealed an improvement in the focus of attention and motivation among children through AR, which can provide fruitful results in the development of skills for children with ASD. The quantitative analysis using tests in [58,60,62,70] shows that the test score of all the participants was improved, while Vahabzadeh et al. [57] revealed that the outcome is the decreased ABC-H score. Furthermore, in the study conducted in [52,69,71,85], the authors found that the usability of the applied mobile AR application. Keshav et al. [65] explored the relationship between student performance and validated clinical measures and then demonstrated game performance is related to the severity of ADHD symptoms.

3.8. RQ8: Which Settings Are Used in the Primary Studies?

From the primary studies, there are three types of environment for primary research, including school environment, community environment and laboratory environment. The majority of research was conducted in a school environment (50%, n = 18), followed by the lab (22.2%). In the study conducted in [62,68], the setting used was community environment, and Nubia et al. [74] conducted their research in a clinic, while seven studies did not mention the setting environment.

3.9. RQ9: How Sustainable the Outcome of the Intervention Is?

The durability effects of MAR have rarely been investigated, and only six studies assessing near/far effects. In [58,60,67], participants performed the same tasks as in the baseline phase procedure 6 weeks after the intervention, and the results were similar in immediate and delayed post-tests, indicating that the enhanced correct rate was maintained after the intervention. Cihak et al. [59] conducted the maintenance phase 9 weeks after the intervention, and Escobedo et al. [26,56] began the maintenance phase after 1 week. Taryadi and Kurniawan [73] mentioned the maintenance phase but did not describe much.

3.10. RQ10: How Generalized the Intervention Result Is?

From the primary studies, only one study by Lee et al. [58] conducted generalization probes at various instances (baseline, intervention and maintenance) of their research, but they did not present the generalization results.

4. Discussion

Authors should discuss the results and how they can be interpreted from the perspective of previous studies and of the working hypotheses. The findings and their implications should be discussed in the broadest context possible. Future research directions may also be highlighted.
The diversity of the publication sources could be explained due to the high adaptability of the emerging mobile augmented reality technology, which allows them to be applied in different fields. The research team includes Keshav, Sahin, Vahabzadeh and Liu, and it is more active than the others, which have published a quarter of the primary studies (n = 9) on using the Empowered Brain on students with ASD. As for the researchers, McMahon and Cihak are also slightly more active than others, followed by Lee (n = 3) and Escobedo (n = 2). Initially, researchers mainly published their results through conferences, while after the year 2013, journals are preferred by researchers as being the most “prestigious” and impactful source to publish their studies. Increasingly advanced mobile devices provide fertile soil for applying mobile AR solutions to solve real-life problems of the ASD population. In view of the results, research related to the use of mobile AR technology for developing interventions in autism is expected to undergo an important increase in the coming years.
As far as the population is concerned, MAR studies were widely conducted with children rather than adolescents. This agrees with Marto et al. [41], who conducted the research on applying AR for autist patients with 16 studies. Their parents tend to provide intensive treatment for them at younger ages, which may be related to the golden period of the intervention; however, it has no evidence. Participants also have large deviations between male and female, almost for 4:1, that may be caused by the different prevalence rates of male and female with autism and the difficulty of recruiting participants. It is a common problem of the small sample size, which possibly due to the difficulty of recruiting participants. Further studies should address teen years for covering their needs, recruit more participants of different ages and reduce participants’ gender bias. Furthermore, the limited participation of parents in the examined studies could also be related to privacy and security issues.
As for the selection of devices in intervention, the cost of technology is one of the essential factors to consider [86]. Handheld devices, such as smartphones (11 studies) and tablets, have become an obvious choice for use because they are getting cheaper and cheaper and widely available. The use of smart glasses (9 studies covering the Empowered Brain system) has also increased these years due to their size, portability and flexibility.
Regarding the target skill, most parts of these studies (n = 11) presented social communication as the variable to improve with the AR technology, which may be because social disorders are one of the most significant characteristics that affect the lives of people with autism. The last target skills are related to defect repair and skills learning as well, which is different from the educational assistance of typically developed students. In the future, we can design products that meet their need to improve the user’s satisfaction.
With regard to the research design, these studies are devoted to examining whether the intervention method or project causes some outcomes or results. Research design can combine research projects (participants, measurements, interventions, etc.) and can be used to organize research to show how the research projects solve the research problem together. Among the primary studies, two research design methods have been used, including the quasi-experimental design and non-experimental design, and the quasi-experimental design divided into single case design and quasi-experimental design with a control group. The single-case design, also known as single-subject design, refers to a research design that can only use one subject in the research process. Although there is only one subject, it can also clearly determine whether there is a causal relationship between variables. However, all the studies did not use the randomized experiment design. This is because individual differences are normally large along the autism disorder spectrum and have different performances in each; therefore, treatment usually focuses only on a single individual’s development rather than a group. Multiple baselines across subject design are regarded as a standard and evidence-based method in many computer-based treatments used in special education. It is a fundamental experimental method for research in the field, and, in actual practice, it does not require control groups or many participants. Based on this, maybe it is better to design a multiple baseline across subject design than a posttest or pretest-posttest single-case design. In the study conducted by Lorenzo et al. [68], the author used different evaluators before and after the test, and the sample size is too small. The number of participants and the factors that may affect reliability during the research process should be considered. In addition, studies without research design are also a major issue for the research effectiveness in this area.
Measurement methods are also the main concerns in research projects. In the methods used in these studies, both survey research and qualitative research require human participation, and it will increase the error rate and reduce the effectiveness and reliability of the measurement if observers are not adequately trained. Comprehensive training of observers before the intervention, calculating the data multiple times, or using a computer-assisted data collection method can reduce the inadvertent introduction of personnel errors. Furthermore, eye tracking is a technique used to measure a person’s eye movements when interacting with a stimulus [87]. Greene et al. [88] examined the utility of eye-tracking as an outcome measure and stated that it has the potential to function as a valid treatment outcome measure for use with individuals with ASD of varying ages and levels of impairment. Eraslan et al. [89] assessed the ability to search for web information in individuals with high-functioning autism. Rezae et al. [90] assessed visual preferences for mobile device interfaces in individuals with autism. Yaneva et al. [91] applied eye-movement tracking to the diagnosis of high-functioning autism in adults using web stimuli. However, there is still a gap that directly uses eye-tracking technology to measure the AR intervention outcomes. Based on the current findings, future studies could investigate eye-tracking technology as a measure in pre-and post-intervention to track responses to intervention in individuals with ASD.
The outcomes reveal that the application of mobile AR in the field of autism intervention is successful to a certain extent. All the studies report a positive result related to target skills. From the perspective of target skills, the achievements are all related to the education of defect repair and improvement of basic life skills of the autistic population. It is to be noted that the research on defect repairment and is much larger than the research on education. This is consistent with the core goal of autistic population interventions, which is to reduce core symptoms or related defects, achieve self-care in life, improve quality of life and reduce family stress, rather than pursuing academic success [92]. From the perspective of outcome measurement methods, studies using survey and qualitative research measurement methods both report better various performances and the improvement of participants’ capabilities, and studies using scaling give the scores improvement of participants. The studies without user research only mention the usability of related apps.
Regarding the research setting, most of the research is conducted in a school and laboratory, and two studies are in the community environment, while some studies do not mention the setting. To a certain extent, the social skills of autistic children and adolescents may be improved with training in a laboratory setting, but their core issue (social) still needs to be practiced in a real social environment. The ultimate goal of the intervention is to integrate them into society and family, while there is no family environment mentioned. It should be emphasized that school and institutional research settings are very important because there has peer participation. However, special education teachers can only accompany autistic children for a few years, and they will eventually return to their families and enter society. Future research can involve more family interventions, enable parents to participate more and guide their children to live an active life. More family environments can involve and enable parents to participate more and be their mentors.
The analysis results of the MAR durability effects showed that most of the studies conducted their research without the far effects of AR, which may be due to the limitation of research time. Part of the articles studied the durability effects of MAR in a short period (up to 9 weeks), while other articles focusing on long-term effect research have reported half a year to one year. It is almost a blank area. Research on generalization and maintenance is still in the infant stage. The long-term effect and generalization of mobile augmented reality can push the evidence-based practical research on mobile augmented reality further. The application of learned skills to other environments, such as expressing greetings learned in the lab to strangers in society which is also a major issue for persons with ASD.
However, like every other review, this paper has some limitations such as the keywords, inclusion and exclusion criteria we chose contain subjective elements, which might affect the number of selected primary studies. Furthermore, regarding the studies we reviewed, we can summarize the major problems and limitations faced by the studies.
  • Small sample;
  • Research design issues;
  • Not generalizable results;
  • Short intervention time;
  • Participants are not differentiated by age;
  • No use of control groups;
  • Difficult in recruiting participants;
  • Lack of gender parity of the sample;
  • App design issues;
  • Internet connection issues.
Many of the existing mobile AR applications for people with ASD are not considered very practical due to insufficient functionality. Designers must focus on users and design products that have high interactivity, quality and meet user needs to improve user satisfaction and make mobile AR become widely accepted. As for the issue of internet connection, which is a considerable indicator in most mobile AR-related user experience studies, the solutions can be focused on the selection of mobile devices and intervention locations. Furthermore, the studies on internet stability and speed, especially in a rural area, are still in the infant stage, and more effort should be put into this particular area.
The main issue related to the evaluation of mobile AR in people with ASD may point to the research design method, sample and the intervention period. Currently, there is no research report on randomized experiments in this field; thus, the effectiveness of randomized experiments research design still needs to be investigated. The most mentioned in the studies are the small sample size and short intervention time. The determination of the participants’ number is related to the research design method, region, and many other factors. Recruiting more participants, establishing inclusion criteria and conducting strict screening can improve the accuracy of the research. The distribution of participants between females and males reveals an extremely limited gender bias, and more female autistic participants can be recruited to solve this issue.
Intervention time also had been added as one of the future directions while solutions can be focused on extending the research period. The long-term impact research is helpful to verify the stability of mobile AR in the intervention with autism and is more conducive to the promotion of this technology.
Technology assessment is very challenging and time-consuming but a necessary task in any research effort. Focusing on mobile AR applications, the evaluation is even more challenging since there is no gold standard. Researchers can use the combination of quantitative and qualitative measurement methods to further optimize technology usability and user studies to improve the stability and effectiveness of research. On another hand, due to the study on generalization, there is one more issue to be focused on that can help the research of evidence-based practices; researchers can improve their design by taking account of durability and generalization of mobile AR effects. The most notable feature of mobile AR is that it is portable. For related research, it should be verified in a wider variety of participants and diverse environments instead of only in the experimental setting.
Although mobile AR technologies have not been a novel issue, they still keep a broader development space for autism treating. The use of mobile AR applications in an educational environment is a relatively recent development, and there is a need for further studies designing scientific intervention process in combination with the existing effective evidence-based teaching methods to maximize the impact of technology.

5. Conclusions

Early findings indicate that mobile augmented reality has a positive effect on ASD and has great potential to help teachers and parents intervene more effectively early on. Despite the promising results reported by almost all well-conducted studies, we must be mindful of the existing issues. As an emerging interdisciplinary area of research, future studies should follow systematic experimental research methods and rigorous evaluation processes if possible. For instance, physiological assessment methods, such as eye-tracking, can be used to improve the validity of the measurements. In addition, gender bias needs to be minimized for the selection of participants. We should also be aware that when it comes to intervention studies that focus on the development of a single individual, the larger the number of participants is not better. Furthermore, it is important to design MAR systems with a user-centered approach, involving more family members and de-signing products that meet their needs in order to improve the learning outcomes obtained during the intervention and to increase user satisfaction. Finally, research on the long-term effects and diffusion of MAR should be conducted to further promote evidence-based practice research on mobile augmented reality.
Overall, this systematic review presents the use of mobile AR technology in autism spectrum disorder interventions and answers 10 research questions to provide MAR-based solutions for skill learning for individuals with ASD and gives directions for research in this area to make MAR better for individuals with autism. It is hoped that this review could provide useful insight and guidance for researchers, educators and those who are concerned with ASD.

Author Contributions

Conceptualization, X.L. and M.S.S.; writing—original draft preparation, X.L.; writing—review and editing, X.L. and M.S.S.; supervision and review, M.S.S. Both authors have read and agreed to the published version of the manuscript.

Funding

This research received funding partially from the Universiti Teknologi Malaysia through Prototype Research Grant Q.J130000.2809.00L44.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Acknowledgments

Special thank to Malaysia Ministry of Higher Education and Universiti Teknologi Malaysia, Skudai, Johor, Malaysia for research grant under Fundamental Research Grant Scheme FRGS/1/2018/ICT01/UTM/02/8 and Prototype Research Grant Q.J130000.2809.00L44. We also thank the XRlab team members at Media and Game Innovation Centre of Excellence, Institute of Human Centered Engineering, Universiti Teknologi Malaysia for your kind support.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. A.P.A. Diagnostic and Statistical Manual of Mental Disorders [DSM-5]; A.P.A.: Washington, DC, USA, 2013. [Google Scholar]
  2. WHO. Autism Spectrum Disorders; WHO: Geneva, Switzerland, 2019. [Google Scholar]
  3. Grzadzinski, R.; Huerta, M.; Lord, C. DSM-5 and autism spectrum disorders (ASDs): An opportunity for identifying ASD subtypes. Mol. Autism 2013, 4, 12. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Eldevik, S.; Hastings, R.P.; Hughes, J.C.; Jahr, E.; Eikeseth, S.; Cross, S. Meta-Analysis of Early Intensive Behavioral Intervention for Children With Autism. J. Clin. Child Adolesc. Psychol. 2009, 38, 439–450. [Google Scholar] [CrossRef] [PubMed]
  5. Mcconnell, S.R. Interventions to facilitate social interaction for young children with autism: Review of available research and recommendations for educational intervention and future research. J. Autism Dev. Disord. 2002, 32, 351–372. [Google Scholar] [CrossRef] [PubMed]
  6. Axe, J.B.; Evans, C.J. Using video modeling to teach children with PDD-NOS to respond to facial expressions. Res. Autism Spectr. Disord. 2012, 6, 1176–1185. [Google Scholar] [CrossRef]
  7. Bellini, S.; Akullian, J. A Meta-Analysis of Video Modeling and Video Self-Modeling Interventions for Children and Adolescents with Autism Spectrum Disorders. Except. Child. 2007, 73, 264–287. [Google Scholar] [CrossRef]
  8. Khowaja, K.; Salim, S.S. A systematic review of strategies and computer-based intervention (CBI) for reading comprehension of children with autism. Res. Autism Spectr. Disord. 2013, 7, 1111–1121. [Google Scholar] [CrossRef]
  9. Pennisi, P.; Tonacci, A.; Tartarisco, G.; Billeci, L.; Ruta, L.; Gangemi, S.; Pioggia, G. Autism and social robotics: A systematic review. Autism Res. 2016, 9, 165–183. [Google Scholar] [CrossRef]
  10. Ramdoss, S.; Mulloy, A.; Lang, R.; Oreilly, M.F.; Sigafoos, J.; Lancioni, G.E.; Didden, R.; Zein, F.E. Use of computer-based interventions to improve literacy skills in students with autism spectrum disorders: A systematic review. Res. Autism Spectr. Disord. 2011, 5, 1306–1318. [Google Scholar] [CrossRef]
  11. Ramdoss, S.; Lang, R.; Mulloy, A.; Franco, J.; O’Reilly, M.; Didden, R.; Lancioni, G. Use of computer-based interventions to teach communication skills to children with autism spectrum disorders: A systematic review. J. Behav. Educ. 2011, 20, 55–76. [Google Scholar] [CrossRef] [Green Version]
  12. Fletcher-Watson, S. A targeted review of computer-assisted learning for people with autism spectrum disorder: Towards a consistent methodology. Rev. J. Autism Dev. Disord. 2014, 1, 87–100. [Google Scholar] [CrossRef] [Green Version]
  13. Lorenzo, G.; Lledó, A.; Arráez-Vera, G.; Lorenzo-Lledó, A. The application of immersive virtual reality for students with ASD: A review between 1990–2017. Educ. Inf. Technol. 2019, 24, 127–151. [Google Scholar] [CrossRef]
  14. Mesa-Gresa, P.; Gil-Gómez, H.; Lozano-Quilis, J.-A.; Gil-Gómez, J.-A. Effectiveness of virtual reality for children and adolescents with autism spectrum disorder: An evidence-based systematic review. Sensors 2018, 18, 2486. [Google Scholar] [CrossRef] [Green Version]
  15. Kagohara, D.M.; van der Meer, L.; Ramdoss, S.; O’Reilly, M.F.; Lancioni, G.E.; Davis, T.N.; Rispoli, M.; Lang, R.; Marschik, P.B.; Sutherland, D. Using iPods® and iPads® in teaching programs for individuals with developmental disabilities: A systematic review. Res. Dev. Disabil. 2013, 34, 147–156. [Google Scholar] [CrossRef]
  16. Noor, H.A.M.; Shahbodin, F.; Pee, N.C. Serious Game for Autism Children: Review of Literature. Int. J. Psychol. Behav. Sci. 2012, 6, 554–559. [Google Scholar]
  17. Zakari, H.M.; Ma, M.; Simmons, D. A review of serious games for children with autism spectrum disorders (asd). In Proceedings of the International Conference on Serious Games Development and Applications, Berlin, Germany, 9–10 October 2014. [Google Scholar]
  18. Aresti-Bartolome, N.; Garcia-Zapirain, B. Technologies as support tools for persons with autistic spectrum disorder: A systematic review. Int. J. Environ. Res. Public Health 2014, 11, 7767–7802. [Google Scholar] [CrossRef]
  19. Ibrahim, Z.; Alias, M. A review on using assistive technology to enhance social skills competence among children with autism spectrum disorder (ASD). Adv. Sci. Lett. 2018, 24, 4250–4254. [Google Scholar] [CrossRef]
  20. Van Krevelen, D.; Poelman, R. A survey of augmented reality technologies, applications and limitations. Int. J. Virtual Real. 2010, 9, 1–20. [Google Scholar] [CrossRef] [Green Version]
  21. Goh, E.S.; Sunar, M.S.; Ismail, A.W.; Andias, R. An Inertial Device-based User Interaction with Occlusion-free Object Handling in a Handheld Augmented Reality. Int. J. Integr. Eng. 2018, 10, 159–168. [Google Scholar]
  22. Su, G.E.; Sunar, M.S.; Ismail, A.W. Device-based manipulation technique with separated control structures for 3D object translation and rotation in handheld mobile AR. Int. J. Hum. Comput. Stud. 2020, 141, 102433. [Google Scholar] [CrossRef]
  23. Bai, Z.; Blackwell, A.F.; Coulouris, G. Using Augmented Reality to Elicit Pretend Play for Children with Autism. IEEE Trans. Vis. Comput. Graph. 2015, 21, 598–610. [Google Scholar] [CrossRef] [Green Version]
  24. Uzuegbunam, N.; Wong, W.H.; Cheung, S.-C.S.; Ruble, L. MEBook: Multimedia Social Greetings Intervention for Children with Autism Spectrum Disorders. IEEE Trans. Learn. Technol. 2018, 11, 520–535. [Google Scholar] [CrossRef]
  25. Goh, E.S.; Sunar, M.S.; Ismail, A.W. 3D Object Manipulation Techniques in Handheld Mobile Augmented Reality Interface: A Review. IEEE Access 2019, 7, 40581–40601. [Google Scholar] [CrossRef]
  26. Escobedo, L.; Tentori, M.; Quintana, E.; Favela, J.; Garciarosas, D. Using Augmented Reality to Help Children with Autism Stay Focused. IEEE Pervasive Comput. 2014, 13, 38–46. [Google Scholar] [CrossRef]
  27. O’Nions, E.; Happé, F.; Evers, K.; Boonen, H.; Noens, I. How do parents manage irritability, challenging behaviour, non-compliance and anxiety in children with autism spectrum disorders? A meta-synthesis. J. Autism Dev. Disord. 2018, 48, 1272–1286. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  28. Bagatell, N. The routines and occupations of families with adolescents with autism spectrum disorders. Focus Autism Other Dev. Disabil. 2016, 31, 49–59. [Google Scholar] [CrossRef]
  29. Lopezherrejon, R.E.; Poddar, O.; Herrera, G.; Sevilla, J. Customization Support in Computer-Based Technologies for Autism: A Systematic Mapping Study. Int. J. Hum. Comput. Interact. 2020, 36, 1–18. [Google Scholar]
  30. Bhatt, S.K.; De Leon, N.I.; Aljumaily, A. Augmented reality game therapy for children with autism spectrum disorder. Int. J. Smart Sens. Intell. Syst. 2014, 7, 519–536. [Google Scholar] [CrossRef] [Green Version]
  31. Casas, X.; Herrera, G.; Coma, I.; Fernández, M. A Kinect-based Augmented Reality System for Individuals with Autism Spectrum Disorders. In Proceedings of the International Conference on Computer Graphics Theory and Applications, Rome, Italy, 24–26 February 2012. [Google Scholar]
  32. Chen, C.H.; Lee, I.J.; Lin, L.Y. Augmented reality-based self-facial modeling to promote the emotional expression and social skills of adolescents with autism spectrum disorders. Res. Dev. Disabil. 2015, 36, 396–403. [Google Scholar] [CrossRef]
  33. Magrini, M.; Curzio, O.; Carboni, A.; Moroni, D.; Salvetti, O.; Melani, A. Augmented Interaction Systems for Supporting Autistic Children. Evolution of a Multichannel Expressive Tool: The SEMI Project Feasibility Study. Appl. Sci. 2019, 9, 3081. [Google Scholar] [CrossRef] [Green Version]
  34. Lee, I.-J. Kinect-for-windows with augmented reality in an interactive roleplay system for children with an autism spectrum disorder. Interact. Learn. Environ. 2020, 1–17. [Google Scholar] [CrossRef]
  35. Da Silva, C.A.; Fernandes, A.R.; Grohmann, A.P. STAR: Speech therapy with augmented reality for children with autism spectrum disorders. In Proceedings of the International Conference on Enterprise Information Systems, Lisbon, Portugal, 27–30 April 2014. [Google Scholar]
  36. Chung, P.J.; Vanderbilt, D.L.; Soares, N.S. Social Behaviors and Active Videogame Play in Children with Autism Spectrum Disorder. Games Health J. 2015, 4, 225–234. [Google Scholar] [CrossRef]
  37. Chen, C.H.; Lee, I.J.; Lin, L.Y. Augmented reality-based video-modeling storybook of nonverbal facial cues for children with autism spectrum disorder to improve their perceptions and judgments of facial expressions and emotions. Comput. Hum. Behav. 2016, 55, 477–485. [Google Scholar] [CrossRef]
  38. McMahon, D.D. Augmented Reality on Mobile Devices to Improve the Academic Achievement and Independence of Students with Disabilities. Ph.D. Thesis, University of Tennessee, Knoxville, TN, USA, 2014. [Google Scholar]
  39. Syahputra, M.F.; Arisandi, D.; Lumbanbatu, A.F.; Kemit, L.F.; Nababan, E.B.; Sheta, O. Augmented reality social story for autism spectrum disorder. J. Phys. Conf. Ser. 2018, 978, 012040. [Google Scholar] [CrossRef]
  40. Hossein, S.M.; Mirinezhad, S.Y.; Ilanloo, A.; Aghsaghloo, N. Utilizing SURF Features and KLT Tracking Algorithm in Augmented Reality (AR), Using Kinect V. 2 with the Aim of Autism Therapy. In Proceedings of the National Conference on Computer, Information Technology and Applications of Artificial Intelligence, Ahvaz, Iran, 5 February 2020. [Google Scholar]
  41. Marto, A.; Almeida, H.A.; Gonçalves, A. Using Augmented Reality in Patients with Autism: A Systematic Review. In Proceedings of the ECCOMAS Thematic Conference on Computational Vision and Medical Image Processing, Porto, Portugal, 16–18 October 2019. [Google Scholar]
  42. Karamanoli, P.; Tsinakos, A.; Karagiannidis, C. The Application of Augmented Reality for Intervention to People with Autism Spectrum Disorders. J. Mob. Comput. Appl. 2017, 4, 42–51. [Google Scholar] [CrossRef]
  43. Adnan, N.H.; Tunggal, D.; Abdullasim, N. Systematic Review on Augmented Reality Application for Autism Children. J. Adv. Res. Dyn. Control Syst. 2018, 10, 26–32. [Google Scholar]
  44. Khowaja, K.; Banire, B.; Al-Thani, D.; Sqalli, M.T.; Aqle, A.; Shah, A.; Salim, S.S. Augmented reality for learning of children and adolescents with autism spectrum disorder (ASD): A systematic review. IEEE Access 2020, 8, 78779–78807. [Google Scholar] [CrossRef]
  45. Berenguer, C.; Baixauli, I.; Gómez, S.; Andrés, M.; Stasio, S.D. Exploring the Impact of Augmented Reality in Children and Adolescents with Autism Spectrum Disorder: A Systematic Review. Int. J. Environ. Res. Public Health 2020, 17, 6143. [Google Scholar] [CrossRef]
  46. Kitchenham, B. Procedures for performing systematic reviews. Keele UK Keele Univ. 2004, 33, 1–26. [Google Scholar]
  47. Moher, D.; Altman, D.G.; Liberati, A.; Tetzlaff, J. PRISMA statement. Epidemiology 2011, 22, 128. [Google Scholar] [CrossRef] [Green Version]
  48. Higgins, J.P.; Thomas, J.; Chandler, J.; Cumpston, M.; Li, T.; Page, M.J.; Welch, V.A. Cochrane Handbook for Systematic Reviews of Interventions, 2nd ed.; The Cochrane Collab: Hoboken, NJ, USA, 2019; pp. 13–29. [Google Scholar]
  49. Education, C.R. CORE Rankings Portal. Available online: https://www.core.edu.au/conference-portal (accessed on 27 September 2020).
  50. InCites Journal Citation Reports. Available online: https://jcr.clarivate.com (accessed on 10 October 2020).
  51. Keshav, N.U.; Salisbury, J.P.; Vahabzadeh, A.; Sahin, N.T. Social Communication Coaching Smartglasses: Well Tolerated in a Diverse Sample of Children and Adults with Autism. JMIR mHealth uHealth 2017, 5, e140. [Google Scholar] [CrossRef]
  52. Sahin, N.T.; Keshav, N.U.; Salisbury, J.P.; Vahabzadeh, A. Safety and Lack of Negative Effects of Wearable Augmented-Reality Social Communication Aid for Children and Adults with Autism. J. Clin. Med. 2018, 7, 188. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  53. Antão, J.Y.F.D.L.; Abreu, L.C.D.; Barbosa, R.T.D.A.; Crocetta, T.B.; Guarnieri, R.; Massetti, T.; Antunes, T.P.C.; Tonks, J.; Monteiro, C.B.D.M. Use of Augmented Reality with a Motion-Controlled Game Utilizing Alphabet Letters and Numbers to Improve Performance and Reaction Time Skills for People with Autism Spectrum Disorder. Cyberpsychol. Behav. Soc. Netw. 2020, 23, 16–22. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  54. Singh, K.; Shrivastava, A.; Achary, K.; Dey, A.; Sharma, O. Augmented Reality-Based Procedural Task Training Application for Less Privileged Children and Autistic Individuals. In Proceedings of the International Conference on Virtual-Reality Continuum and Its Applications in Industry, Brisbane, QLD, Australia, 14–16 November 2019. [Google Scholar]
  55. Liu, R.; Salisbury, J.P.; Vahabzadeh, A.; Sahin, N.T. Feasibility of an Autism-Focused Augmented Reality Smartglasses System for Social communication and Behavioral coaching. Front. Pediatr. 2017, 5, 145. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  56. Escobedo, L.; Nguyen, D.H.; Boyd, L.E.; Hirano, S.H.; Rangel, A.; Garciarosas, D.; Tentori, M.; Hayes, G.R. MOSOCO: A mobile assistive tool to support children with autism practicing social skills in real-life situations. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Austin, TX, USA, 5–10 May 2012. [Google Scholar]
  57. Vahabzadeh, A.; Keshav, N.U.; Salisbury, J.P.; Sahin, N.T. Improvement of Attention-Deficit/Hyperactivity Disorder Symptoms in School-Aged Children, Adolescents, and Young Adults With Autism via a Digital Smartglasses-Based Socioemotional Coaching Aid: Short-Term, Uncontrolled Pilot Study. JMIR Ment. Health 2018, 5, e25. [Google Scholar] [CrossRef]
  58. Lee, I.J.; Chen, C.-H.; Wang, C.-P.; Chung, C.-H. Augmented Reality Plus Concept Map Technique to Teach Children with ASD to Use Social Cues When Meeting and Greeting. Asia Pac. Educ. Res. 2018, 27, 227–243. [Google Scholar] [CrossRef]
  59. Cihak, D.F.; Moore, E.J.; Wright, R.E.; McMahon, D.D.; Gibbons, M.M.; Smith, C. Evaluating Augmented Reality to Complete a Chain Task for Elementary Students With Autism. J. Spec. Educ. Technol. 2016, 31, 99–108. [Google Scholar] [CrossRef]
  60. Lee, I.J. Augmented reality coloring book: An interactive strategy for teaching children with autism to focus on specific nonverbal social cues to promote their social skills. Interact. Stud. 2019, 20, 256–274. [Google Scholar] [CrossRef]
  61. McMahon, D.D.; Cihak, D.F.; Wright, R.E.; Bell, S.M. Augmented Reality for Teaching Science Vocabulary to Postsecondary Education Students With Intellectual Disabilities and Autism. J. Res. Technol. Educ. 2015, 48, 38–56. [Google Scholar] [CrossRef]
  62. McMahon, D.; Cihak, D.F.; Wright, R. Augmented Reality as a Navigation Tool to Employment Opportunities for Postsecondary Education Students With Intellectual Disabilities and Autism. J. Res. Technol. Educ. 2015, 47, 157–172. [Google Scholar] [CrossRef]
  63. Vahabzadeh, A.; Keshav, N.U.; Abdus-Sabur, R.; Huey, K.; Liu, R.; Sahin, N.T. Improved Socio-Emotional and Behavioral Functioning in Students with Autism Following School-Based Smartglasses Intervention: Multi-Stage Feasibility and Controlled Efficacy Study. Behav. Sci. 2018, 8, 85. [Google Scholar] [CrossRef] [Green Version]
  64. Sahin, N.T.; Keshav, N.U.; Salisbury, J.P.; Vahabzadeh, A. Second Version of Google Glass as a Wearable Socio-Affective Aid: Positive School Desirability, High Usability, and Theoretical Framework in a Sample of Children with Autism. JMIR Hum. Factors 2018, 5, e1. [Google Scholar] [CrossRef]
  65. Keshav, N.U.; Vogt-Lowell, K.; Vahabzadeh, A.; Sahin, N.T. Digital Attention-Related Augmented-Reality Game: Significant Correlation between Student Game Performance and Validated Clinical Measures of Attention-Deficit/Hyperactivity Disorder (ADHD). Children 2019, 6, 72. [Google Scholar] [CrossRef] [Green Version]
  66. Dragomir, M.; Manches, A.; Fletcher-Watson, S.; Pain, H. Facilitating pretend play in autistic children: Results from an augmented reality app evaluation. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility, Galway, Ireland, 22–24 October 2018. [Google Scholar]
  67. Lee, I.J.; Lin, L.-Y.; Chen, C.-H.; Chung, C.-H. How to Create Suitable Augmented Reality Application to Teach Social Skills for Children with ASD. State Art Virtual Real. Augment. Real. Knowhow 2018. [Google Scholar] [CrossRef] [Green Version]
  68. Lorenzo, G.; Gómez-Puerta, M.; Arráez-Vera, G.; Lorenzo-Lledó, A. Preliminary study of augmented reality as an instrument for improvement of social skills in children with autism spectrum disorder. Educ. Inf. Technol. 2019, 24, 181–204. [Google Scholar] [CrossRef]
  69. Sahin, N.T.; Abdus-Sabur, R.; Keshav, N.U.; Liu, R.; Salisbury, J.P.; Vahabzadeh, A. Case Study of a Digital Augmented Reality Intervention for Autism in School Classrooms: Associated with Improved Social Communication, Cognition, and Motivation via Educator and Parent Assessment. Front. Educ. 2018, 3. [Google Scholar] [CrossRef]
  70. Wang, C.-P.; Tsai, C.-H. Requesting Help Module Interface Design on Speech-Generating Device and Augmented Reality for Children with Autism Spectrum Disorder. In Proceedings of the 2nd Eurasian Conference on Educational Innovation, Singapore, 27–29 January 2019. [Google Scholar]
  71. Nuraini, C.K.; Mohd, F.S.; Suparjoh, S.; Khidir, N.A.S.M. Application of Augmented Reality in Learning Bakery for Autism Spectrum Disorder. Int.J. Eng. Adv. Technol. 2019, 9, 2616–2620. [Google Scholar]
  72. Abou El-Seoud, S.; Halabi, O.; Geroimenko, V. Assisting Individuals with Autism and Cognitive Disorders: An Augmented Reality-Based Framework. Int. J. Online Biomed. Eng. 2019, 15, 28. [Google Scholar] [CrossRef]
  73. Taryadi; Kurniawan, I. The improvement of autism spectrum disorders on children communication ability with PECS method Multimedia Augmented Reality-Based. J. Phys. Conf. Ser. 2018, 947, 012009. [Google Scholar] [CrossRef]
  74. Nubia, R.M.; Fabian, G.R.; Wilson, R.A.; Wilmer, P.B. Development of a mobile application in augmented reality to improve the communication field of autistic children at a Neurorehabilitar Clinic. In Proceedings of the 2015 Workshop on Engineering Applications-International Congress on Engineering (WEA), Bogotá, Colombia, 28–30 October 2015. [Google Scholar]
  75. Quintana, E.; Ibarra, C.; Escobedo, L.; Tentori, M.; Favela, J. Object and Gesture Recognition to Assist Children with Autism during the Discrimination Training. In Proceedings of the Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, Buenos Aires, Argentina, 3–6 September 2012. [Google Scholar]
  76. Vullamparthi, A.J.; Nelaturu, S.; Mallaya, D.D.; Chandrasekhar, S. Assistive Learning for Children with Autism Using Augmented Reality. In Proceedings of the IEEE Fifth International Conference on Technology for Education, Kharagpur, India, 18–20 December 2013. [Google Scholar]
  77. Xu, Q.; Cheung, S.S.; Soares, N. LittleHelper: An augmented reality glass application to assist individuals with autism in job interview. In Proceedings of the Asia Pacific Signal and Information Processing Association Annual Summit and Conference, Hong Kong, China, 16–19 December 2015. [Google Scholar]
  78. Brandão, J.; Cunha, P.; Vasconcelos, J.; Carvalho, V.; Soares, F. An augmented reality gamebook for children with autism spectrum disorders. In Proceedings of the International Conference on E-Learning in the Workplace, New York, NY, USA, 10–12 June 2015. [Google Scholar]
  79. Chung, C.-H.; Chen, C.-H. Augmented Reality Based Social Stories Training System for Promoting the Social Skills of Children with Autism. In Advances in Ergonomics Modeling, Usability and Special Populations; Springer: Cham, Switzerland, 2017; Volume 486, pp. 495–505. [Google Scholar] [CrossRef]
  80. Daniel, V.L.T.; Ting, H.; Photchara, R.; Haruo, T. An AR Puzzle Application for Improving Emotion Recognition for AS Children. In Proceedings of the 2019 The 3rd International Conference on Digital Technology in Education, Yamanashi, Japan, 25–27 October 2019. [Google Scholar]
  81. Juhlin, D.; Morris, C.; Schmaltz, P.; Shane, H.; Schlosser, R.; O’Brien, A.; Yu, C.; Mancini, D.; Allen, A.; Abramson, J. The PTC and Boston Children’s Hospital Collaborative AR Experience for Children with Autism Spectrum Disorder. In Proceedings of the International Conference on Human-Computer Interaction, Orlando, FL, USA, 26–31 July 2019; Springer: Cham, Switzerland, 2019; Volume 11573, pp. 116–122. [Google Scholar] [CrossRef]
  82. Tang, T.Y.; Xu, J.; Winoto, P. Automatic Object Recognition in a Light-Weight Augmented Reality-based Vocabulary Learning Application for Children with Autism. In Proceedings of the 2019 3rd International Conference on Innovation in Artificial Intelligence, Suzhou, China, 15–18 March 2019. [Google Scholar]
  83. Pradibta, H.; Wijaya, I.D. Designing AR daily prayers for children with ASD. In Proceedings of the 2017 International Seminar on Application for Technology of Information and Communication (iSemantic), Semarang, Indonesia, 7–8 October 2017. [Google Scholar]
  84. Trochim, W.; Donnelly, J.P. The Research Methods Knowledge Base, 3rd ed.; Atomic Dog and Cengage Learning: Mason, OH, USA, 2008. [Google Scholar]
  85. Keshav, N.U.; Salisbury, J.P.; Vahabzadeh, A.; Sahin, N.T. But will they even wear it? Exploring the tolerability of social communication coaching smartglasses in children and adults with autism. bioRxiv 2017, 164376. [Google Scholar] [CrossRef] [Green Version]
  86. Blattgerste, J.; Renner, P.; Pfeiffer, T. Augmented reality action assistance and learning for cognitively impaired people: A systematic literature review. In Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments, Rhodes, Greece, 5–7 June 2019. [Google Scholar]
  87. Sun, Q.C.; Xia, J.C.; Nadarajah, N.; Falkmer, T.; Lee, H. Assessing drivers’ visual-motor coordination using eye tracking, GNSS and GIS: A spatial turn in driving psychology. Spat. Sci. 2016, 61, 1–18. [Google Scholar] [CrossRef]
  88. Greene, R.K.; Parish-Morris, J.; Sullivan, M.; Kinard, J.L.; Dichter, G.S. Dynamic Eye Tracking as a Predictor and Outcome Measure of Social Skills Intervention in Adolescents and Adults with Autism Spectrum Disorder. J. Autism Dev. Disord. 2021, 51, 1173–1187. [Google Scholar] [CrossRef]
  89. Eraslan, S.; Yaneva, V.; Yesilada, Y.; Harper, S. Web users with autism: Eye tracking evidence for differences. Behav. Inf. Technol. 2019, 38, 678–700. [Google Scholar] [CrossRef]
  90. Rezae, M.; Chen, N.; McMeekin, D.; Tan, T.; Krishna, A.; Lee, H. The evaluation of a mobile user interface for people on the autism spectrum: An eye movement study. Int. J. Hum. Comput. Stud. 2020, 142, 102462. [Google Scholar] [CrossRef]
  91. Yaneva, V.; Le, A.H.; Eraslan, S.; Yesilada, Y.; Mitkov, R. Detecting High-Functioning Autism in Adults Using Eye Tracking and Machine Learning. IEEE Trans. Neural Syst. Rehabil. Eng. 2020, 28, 1254–1261. [Google Scholar] [CrossRef]
  92. World Health Organization. International Classification of Diseases; 9th Revision; WHO: Geneva, Switzerland, 1990. [Google Scholar]
Applsci 11 04550 g002 550
Figure 2. Publications per year and type.
Figure 2. Publications per year and type.
Applsci 11 04550 g002
Applsci 11 04550 g003 550
Figure 3. Distribution of participants’ number in studies.
Figure 3. Distribution of participants’ number in studies.
Applsci 11 04550 g003
Applsci 11 04550 g004 550
Figure 4. Publications per year and type.
Figure 4. Publications per year and type.
Applsci 11 04550 g004
Table
Table 1. Research questions.
Table 1. Research questions.
No.Research QuestionsData Extraction Process
RQ1What channels are used to publish research articles in ASD interventions exploiting AR technology?Venue and year of publication is required.
RQ2What device was used?The used technological devices are required.
RQ3What are the targeted population?Characteristics of participants should be defined.
RQ4For what purposes have been applied the proposed solution?The targeted skills of selected studies should be extracted.
RQ5What research designs are used in the studies?The research design of selected studies should be defined.
RQ6What methods are used to evaluate the performance of participants with the provided AR technology intervention?The assessment methodology should be presented.
RQ7What are the outcomes obtained by the application of the proposed solutions?The major outcomes of the study should be presented.
RQ8Which settings are used in the primary studies?Details on the setting (classroom, home, controlled research environment, etc.) are required.
RQ9How sustainable the outcome of the intervention is?Detailed information about maintenance.
RQ10How generalized the intervention result is?Detailed information about generalization.
Table
Table 2. Quality assessment criteria.
Table 2. Quality assessment criteria.
No.Quality Assessment QuestionWeight
QA1Are the demographic information of the participants clearly stated?Yes (+1), Partially (+0.5), No (+0)
QA2Are the data collection methods adequately described?Yes (+1), Partially (+0.5), No (+0)
QA3Is the paper published in a recognized source?Conferences: CORE A * or A (+1.5), CORE B (+1), CORE C (+0.5), not included in CORE ranking (+0).
Journals: ranked Q1 (+2), ranked Q2 (+1.5), ranked Q3 or Q4 (+1), no JCR ranking (+0)
Other sources: (+0)
QA4What is the number of participants in the evaluation phase?≥20(+3), 10–20 (+2), 6–9 (+1.5), 3–5 (+1), 1–2 (+0.5), 0 or not specified (+0)
QA5Is AR-based solution clearly defined?Yes (+1), Partially (+0.5), No (+0)
QA6Are future works mentioned?Yes (+1)/No (+0)
Table
Table 3. This is the QA score and paper types and venues of the selected studies.
Table 3. This is the QA score and paper types and venues of the selected studies.
Ref.Authors (Year)VenuesTypeQAS
[51]Keshav et al. (2017)JMIR MHEALTH AND UHEALTHJournal8
[52]Sahin et al. (2018)Journal of Clinical MedicineJournal8
[53]Antao et al. (2020)Cyberpsychology Behavior and Social NetworkingJournal8
[26]Escobedo et al. (2014)IEEE Pervasive ComputingJournal7.5
[54]Singh et al. (2019)ACM SIG International Conference on Computer Graphics and Interactive TechniquesConf7.5
[55]Liu et al. (2017)Frontiers in PediatricsJournal6
[56]Escobedo et al. (2012)ACM Annual Conference on Human Factors in Computing SystemsConf5.5
[57]Vahabzadeh et al. (2018)JMIR MENTAL HEALTHJournal5.5
[58]Lee et al. (2018)Asia-Pacific Education ResearcherJournal5.5
[59]Cihak et al. (2016)Journal of Special Education TechnologyJournal5
[60]Lee (2019)Interaction StudiesJournal5
[61]McMahon et al. (2015)Journal of Research on Technology in EducationJournal5
[62]McMahon et al. (2015)Journal of Research on Technology in EducationJournal5
[63]Vahabzadeh et al. (2018)Behavioral SciencesJournal5
[64]Sahin et al. (2018)JMIR HUMAN FACTORSJournal5
[65]Keshav et al. (2019)ChildrenJournal5
[66]Dragomir et al. (2018)ACM SIGACCESS Conference on Computers and AccessibilityConf5
[67]Lee, et al. (2018)State of the Art Virtual Reality and Augmented Reality KnowhowChapter5
[68]Lorenzo et al. (2018)Education and Information TechnologiesJournal5
[69]Sahin et al. (2018)Frontiers in EducationJournal4.5
[70]Wang et al. (2019)Eurasian Conference on Educational InnovationConf4.5
[51] Keshav et al. (2018)JMIR HUMAN FACTORSJournal4
[71]Nuraini et al. (2019)International Journal of Engineering and Advanced TechnologyJournal3.5
[72]Abou et al. (2019)International journal of online and biomedical engineeringJournal3.5
[73]Taryadi et al. (2018)Journal of Physics: Conference SeriesConf3.5
[70]Wang et al. (2019)Eurasian Conference on Educational InnovationConf3.5
[74]Nubia et al. (2015)Workshop on Engineering Applications—International Congress on EngineeringConf3.5
[75]Quintana et al. (2012)Iberoamerican Congress on Pattern RecognitionConf3
[76]Vullamparthi et al. (2013)IEEE Fifth International Conference on Technology for EducationConf3
[77]Xu et al. (2015)Asia-pacific Signal & Information ProcessingConf2.5
[78]Brandão et al. (2015)International Conference on E-Learning in the WorkplaceConf2
[79]Chung & Chen (2016)Advances in Intelligent Systems and ComputingJournal2
[80]Daniel et al. (2019)International Conference on Digital Technology in EducationConf2
[81]Juhlin et al. (2019)International Conference on Human-Computer InteractionConf1
[82]Tang et al. (2019)International Conference on Innovation in Artificial Intelligence Conf1
[83]Pradibta et al. (2017)International Seminar on Application for Technology of Information & CommunicationConf1
Conf: Conference; QAS: Quality Assessment Scores; Ref.: References.
Table
Table 4. Summary of selected studies (n = 36).
Table 4. Summary of selected studies (n = 36).
Authors (Year)Target SkillParticipantsIntervention SettingResearch DesignOutcome MeasurementOutcomes
Brandão et al. (2015)Facial Expressions and Emotion-D: Mobile device (tablet, smartphone or laptop)--An innovative AR Game Book has been developed
Escobedo et al. (2012)Social skill practice3 ASD child + 9 NT child
8–11 y.o		D: Smartphone
P: School
T:7 weeks		Multiple condition designBehavioral observations: video
transcript coding analysis
Interviews		Increased social interaction and reduced social and behavioral errors
Escobedo et al. (2014)Attention12 ASD child 3–8 y.o, 7 TeachersD: Smartphone
P: School
T: 8 weeks		Single subject with multiple condition designBehavioral observations: video
sequential analysis		Improved continuous attention and engagement
Cihak et al. (2015)Brush teeth3 (M) ASD child 6–7 y.oD: iPod touch
P: School
T: Single session		Multiple probe across participants designPerformance: percentage of
independent implementation steps Questionnaire on social effectiveness
Follow-up surveys		Increased independent performance
McMahon et al. (2015)Literacy1 ASD + 3 ID adult 19–25 y.o.D: iPad (3rd generation)
P: Lab		Multiple probe across behaviors/skills designVocabulary tests
Likert-type survey		Increased number of correct responses
McMahon et al. (2015)Navigation1 ASD adult 21 y.o + 3 ID adultD: iPhone 4s
P: Community		Adapted alternating treatment single- subject design (ATD)Percentage of independent navigational checks
Correct responses rate		More independent navigation decisions when using the AR navigation tool
Chung & Chen, (2017)Social reciprocate-D:Laptop--A prototype for the social skill “greeting”
Keshav et al. (2017)Social Communication21 ASD child + adult 11.9 y.o 4.4–21.5D: Google Smartglasses
P:Lab		Single-case experimental designObservations: caregiver report
Likert scale		Well tolerated and usable of the provided BPAS system
Sahin et al. (2017)Social Communication18 ASD child + adult 12.2 y.o 4.4–21.5D: Google Smartglasses
P: Lab		Single-case experimental designStructured interviewsNo significant negative effects
Sahin et al. (2018)Social Communication1 (M) ASD adult (fully-verbal) 13.92 y.oD: Google Smartglasses
P: School
T: 3 weeks		Case study incorporating elements of a single-case DesignSocial Responsiveness Scale 2 (SRS-2)No result in adverse behaviors
Vahabzadeh et al. (2018)Behavior8 ASD child + adult 15 y.o 11.7–20.5D: Google Smartglasses
P: Lab
T: 2 days		Single-case experimental designAberrant Behavioral Checklist (ABC-H) scoresDecreased ABC-H score
Vahabzadeh et al. (2018)Social Emotional; Behavior4 (M) ASD 7.5 y.o 6.7–8.8,
3 Educators		D: Google Smartglasses
P: School
T: 6 weeks		Single-case experimental designAberrant Behavioral Checklist (ABC-I; ABC-H; ABC-L/SW)
Likert scale		Reduced irritability, hyperactivity, and social withdrawal
Sahin et al. (2018)Social Communication8 (7M; 1F) ASD 11.7 y.o 6.7–17.2, CaregiversD: Google Smartglasses
P: Lab		Single-case experimental designSemi-structured interviewThe children successfully used smartglass and did not feel the pressure
Keshav et al. (2018)Attention1 (M) ASD 13 y.oD: Google Smartglasses
P: School
T: 2 weeks		Single-case experimental designDigital logs and In-person interviewsPractical and feasible of the system, improved social interaction
Keshav et al. (2019)Social Communication7 (6M; 1F) ASD 15.6 y.o 14–18D: Google Smartglasses
P: School
T: 1 weeks		Single-case experimental designAberrant Behavior Checklist (ABC-H)
the combined ADHD subscale of the TRF		Game performance is related to the severity of ADHD symptoms
Liu et al. (2017)Behavior treatment/usability2 ASD (M) child 8–9 y.oD: Google Smartglasses
P: School
T: 1 session		Single-case experimental designAberrant Behavior Checklist (ABC)
Post-intervention interview		improved nonverbal communication, eye contact, and social engagement
Abou et al. (2019)Social Communication; Behavior3 ASDD: Mobile device (tablet, smartphone or laptop)Single-case experimental design post-testQuestionnaire
Likert scale		A framework was developed
Daniel et al. (2019)Emotion Recognition; Attention-D: Tablet--Developed an AR app
Dragomir et al. (2018)Pretend play7 (6M; 1F) ASD 8–14 y.oD: Tablet
P: School
T: 5 weeks		Single-case experimental design
pre-post test		Video analysis
Feedback		Significant increased mean frequency of pretend play acts
Juhlin et al. (2019)Language skill; Navigation D: Mobile device -Three high level concepts were developed
Lee et al. (2018)Social reciprocate3 (2M; 1F) ASD 8.8 y.o 8–9D: Tablet
P: School		Single-subject research with a multiple-baselines across-subject designSST test, Correct rate, Likert scale
Questionnaires and interviews
Parental reports		Increased target responses of the SST test
Lee et al. (2018)Social reciprocate3 (2M; 1F) ASD 7.73 y.o 7–9D: Laptop
P: School
T: 6 weeks		Single subject in a multiple baseline across subject design pre-post-testSocial Story trial (SSTs) tests
5-point Likert scale
Social behavior experts’ score
Parents questionnaires and interview
feedback reports		Decreased error rate. Increased overall role-playing score
Lorenzo et al. (2018)Social communication11 (10M; 1F) ASD 3.99 y.o 2–6D: Smartphones
P: Community		Pre-Post test design with control group AR vs. non-ARAutistic Spectrum Inventory Whitney U test
Wilcoxon Signed-Rank Test Mann-Whitney U test		No statistically significant differences found, improved attention and motivation
Singh et al. (2019)Learning; Usability12 (6M; 6F) ASD 10.3 y.o 9–12D: Laptop
P: School
T: 1 session		Single-case experimental designObservations
Subjective questionnaire		Longest time in understanding instructions when using the AR mode, while the in-person mode was the fastest.
Tang et al. (2019)LiteracyTD child + adultD: Smartphone
P: University campus		-ObservationsAttracted attention of children
Taryadi & Kurniawan, (2018)Social communication12 ASDD: Smartphone
P: Lab
T: 15 sessions		Single subject experimental designBehavioral observations, interviewsignificant improvement in the autistic children’s communication ability
Anta et al. (2020)Motor skill48 ASD child + 48 TD childD: Laptop
P: Institute		Control experiment ASD vs. TDMann–Whitney U test, Wilcoxon signed-rank testimproved in reaction times, Improved ability to identify numbers
Wang et al. (2019)Social communication3 (2M; 1F) ASDD: Laptop/smartphone
P: School		Multiple case studies and withdrawal designsObservationsimproved correct judgment rate and situational understanding rate
Quintana et al. (2012)Learning-D: Tablet/smartphone--An AR system-MOBIS has been developed
Nubia et al. (2015)Social communication6 (5M; 1F) ASD 3–9 y.oD: Smartphone
P: Clinic
T: 1 session		Control experiment AR vs. non-ARQuestionnaire
Observations		increase in the attention process and appearance of verbal language.
Pradibta et al. (2017)Daily Prayers-D: Smartphone--Augmented Reality Flash Cards (ARDOA) app has been developed
Vullamparthi et al. (2013)LearningParentsD: Smartphone
P: School		-InterviewAn AR system has been developed and has appreciated by parents
Xu et al. (2015)Job interviewResearcherD: Google Glass Smartglasses
P: Lab		-Observationsdemonstrated an acceptable level of performance
Lee et al. (2019)Social cues3 ASDD: Tablet
P: School		Multiple-baselines across-subject designQuestionnaire
Observations		increased answer correct rate and higher rate in the maintenance phase
Wong et al. (2019)Learning4 ASD + 1 SpeEdu teacherD: Tablet/smartphone
P: School		Pre-post experimental research designPerformance (success)
Observation		higher achievement scores than BLS (Static graphic)
Nuraini et al. (2019)Bakery10 (9M; 1F) ASDD: Smartphone
P: School		Single subject experimental designQuestionnaireexcellent responses
NT: Neurotypical children; TD: Typically Developing; y.o: Years old; M: Male; F: Female; D: Devices used; P: Intervention conducted place; T: Intervention time.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Lian, X.; Sunar, M.S. Mobile Augmented Reality Technologies for Autism Spectrum Disorder Interventions: A Systematic Literature Review. Appl. Sci. 2021, 11, 4550. https://doi.org/10.3390/app11104550

AMA Style

Lian X, Sunar MS. Mobile Augmented Reality Technologies for Autism Spectrum Disorder Interventions: A Systematic Literature Review. Applied Sciences. 2021; 11(10):4550. https://doi.org/10.3390/app11104550

Chicago/Turabian Style

Lian, Xiaojie, and Mohd Shahrizal Sunar. 2021. "Mobile Augmented Reality Technologies for Autism Spectrum Disorder Interventions: A Systematic Literature Review" Applied Sciences 11, no. 10: 4550. https://doi.org/10.3390/app11104550

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop