Next Article in Journal
Long-Term Treatment Strategies of Pediatric Multiple Sclerosis, Including the use of Disease Modifying Therapies
Previous Article in Journal
Correction: Anup C. Katheria. Neonatal Resuscitation with an Intact Cord: Current and Ongoing Trials. Children 2019, 6, 60
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Children
Volume 6
Issue 6
10.3390/children6060072
children-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Digital Attention-Related Augmented-Reality Game: Significant Correlation between Student Game Performance and Validated Clinical Measures of Attention-Deficit/Hyperactivity Disorder (ADHD)

by
Neha U. Keshav
1,
Kevin Vogt-Lowell
1,
Arshya Vahabzadeh
1,2 and
Ned T. Sahin
1,3,*
1
Brain Power, 1 Broadway, Cambridge, MA 02142, USA
2
Department of Psychiatry, Massachusetts General Hospital, Boston, MA 02114, USA
3
Department of Psychology, Harvard University, Cambridge, MA 02138, USA
*
Author to whom correspondence should be addressed.
Children 2019, 6(6), 72; https://doi.org/10.3390/children6060072
Submission received: 18 February 2019 / Revised: 1 May 2019 / Accepted: 14 May 2019 / Published: 28 May 2019
Browse Figures
Versions Notes

Abstract

:
As many as half of school children with autism spectrum disorder (ASD) exhibit symptoms of attention-deficit/hyperactivity disorder (ADHD), resulting in marked negative academic, social, and behavioral outcomes. The focus of the US Food and Drug Administration (FDA) on real-world data from novel digital sources, and the emergence of Current Procedural Terminology (CPT) codes to reimburse for digital monitoring and neurobehavioral testing suggest an increasing acceptance of the role of technology in augmenting clinical care and research. Empowered Brain is an augmented reality and artificial intelligence-based social-emotional communication aid for students with ASD. In this study, student performance on Empowered Brain is correlated to validated clinical measures of ADHD. Seven high school students with a diagnosis of ASD were recruited from a public high school. All students were assessed for severity of ADHD-related symptoms via three clinical gold-standard assessments, namely the Aberrant Behavioral Checklist (ABC), Social Responsiveness Scale 2 (SRS-2), and Teacher Report Form (TRF). Students used Empowered Brain over a one-week period. We measured the correlation of student in-game performance (as measured by point- and star-based rewards) relative to the hyperactivity subscale of the ABC (ABC-H), and the ADHD-subscale of the TRF. All seven students completed the study and managed to successfully use Empowered Brain. Students received a culminative total of 32 sessions, an average of 4.6 sessions per student (range 2–8). Student in-game performance demonstrated highly significant correlation relative to ABC-H (points: p = 0.0013; stars: p = 0.0013), and significant correlation to TRF ADHD scores (points: p = 0.012; stars: p = 0.012). No adverse effects were noted among students who used Empowered Brain. New technologies may herald novel ways of identifying and characterizing symptoms of ADHD in student populations. This study provides evidence that Empowered Brain in-game performance correlates with ADHD symptom severity in students with ASD. Larger samples are required to validate these findings, with more diverse participants that can also widen the generalizability of these findings to a broader range of brain conditions that manifest with inattention, impulsivity, and hyperactivity. Through further research, we may find that such technologies can help us to identify and longitudinally monitor such symptoms, and potentially aid in severity stratification and digital phenotyping.

1. Introduction

Autism spectrum disorder (ASD) is a life-long, neurodevelopmental condition predominantly characterized by challenges in social communication [1]. People with ASD have been shown to have worse educational [2,3], occupational [2,3], health [4], and social outcomes [5] than their neurotypical peers. Transition between adolescence and adulthood is an especially difficult developmental period for these individuals [6].
Up to half of individuals with ASD demonstrate symptoms associated with attention-deficit/hyperactivity disorder (ADHD) [7,8], manifestations that include symptoms from three key domains: motor hyperactivity, inattention, and impulsivity [1]. The combination of ADHD and ASD have been shown to be associated with greater negative outcomes that those seen in ASD populations alone, including worse academic performance and school-based behavioral problems [9]. The presence of ADHD-related symptomatology, even without a formal ADHD diagnosis, is associated with inferior academic function [10]. Research has highlighted that individuals with both ASD and ADHD have a greater number and severity of co-occurring mental health conditions than individuals with ASD alone [11]. Individuals with ASD and ADHD are also less likely to have their ADHD recognized and treated [12]. Additionally, these individuals respond worse to stimulant medication, the first-line medication for ADHD [13,14]. Monitoring of ADHD symptoms in this dual-diagnosed group is particularly challenging.
At present, there are a variety of ways that ADHD can be diagnosed. Most typically, ADHD is a clinical diagnosis that is given by a behavioral specialist following the gathering of a clinical/educational history, observations of the individual, and collateral history from family members, educators, and other professionals. For almost 50 years, ADHD rating scales have supplemented this process, with current use favoring scales such as the Swanson, Nolan and Pelham Questionnaire, Fourth Edition (SNAP IV), Conners 3rd edition, and the Vanderbilt ADHD diagnostic rating scale (VADRS) (see reviews [15,16]). The diagnostic process focuses on identifying impairment in functioning in one of the three core symptom domains seen in ADHD, namely inattention, hyperactivity, and impulsivity. Unfortunately, there is little standardization of ADHD diagnosis, as there are different assessment processes between specialists, conflicting recommendations from professional medical societies, and subjective evaluative components that undermine diagnostic accuracy [17,18]. These factors are exacerbated by the highly heterogeneous nature of ADHD and, relevant to this report, are further complicated by the presence of co-occurring ASD.
While a range of effective pharmaceutical treatments has been developed for ADHD, there is a lack of robust, ecologically valid measures to quantitatively assess and monitor the response to medical treatment. The importance of ecological validity should not be underestimated when dealing with individuals who have conditions that are diagnosed based on behavioral criteria. Obtaining patient-generated data from outside of clinical settings using new digital devices has been emphasized by FDA interest in the use of real world data (RWD) and real world evidence (RWE), to understand the potential benefits and adverse effects related to a medical intervention [19]. Digital tools can become mainstream ways to monitor and assess patients, including the use of technology to perform neuropsychological testing, as demonstrated by new Current Procedural Terminology codes for 2019. Current Procedural Terminology codes are used to identify specific billable services to health insurers and Medicare.
Attempts to develop quantitative assessment tools for ADHD has led to the development of several continuous performance tests (CPTs). These tests aim to objectively measure ADHD symptoms, especially in research settings. Examples of CPTs include Test of Variables of Attention (TOVA) and Connors CPT [18]. Typically, as part of a CPT assessment, participants are asked to differentially respond to a series of rapidly presented visual and/or auditory cues. Participants are asked to respond positively to some cues, but to withhold respond to others. These tests are often administered under strictly-controlled settings, with limited external distractions. CPTs may be particularly useful in assessing attention and impulsivity, as they assess an individual’s selective (correctly responding) and sustained attention (performance over time). However, these tests do not typically monitor the motor hyperactivity component of ADHD [18]. While inattention/impulsivity may present in a range of behavioral conditions, hyperactivity appears to be more specific to ADHD [18]. CPTs are not designed as stand-alone diagnostic instruments but may be used as part of a more comprehensive evaluation. In addition, concerns exist regarding the clinical utility of CPTs, as well as their limited specificity and sensitivity [20,21]. The strict environmental conditions in which CPTs are administered may have both poor ecological validity and limited generalizability compared to home and school environments [18]. Even within these real-world settings, attentional and behavioral demands vary based on context. meaning specific impairments can become more observable during certain situations.
Therefore, there is a need to develop novel technologies to assess ADHD-symptomatology. These technologies should objectively assess symptoms across all ADHD-symptom clusters (attention/impulsivity/hyperactivity) and should also demonstrate high ecological validity/ability to gather real-world data.
In this paper we report on the use of Empowered Brain during typical public high school classroom settings. Empowered Brain is a novel technology that combines computerized smartglasses, like Google Glass, and a series of game-like software modules. It is designed to be used by students as a wearable socio-affective aid as part of a daily school or home intervention program. Empowered Brain uses artificial intelligence to analyze video, audio, affective, and behavioral data and provides users with in-game rewards, such as points and stars, based on their performance. In this study of students with ASD, we report on the correlation between student ADHD symptoms and their in-game performance measurements. Student ADHD symptoms were determined through the use of validated scales, namely the hyperactivity subscales of the aberrant behavioral checklist (ABC) and the ADHD component of the Teacher Report Form (TRF) [22,23,24]. Both of these scales include items that assess all of the main symptom domains of ADHD. No additional staff, time, or equipment resources were provided during the time period that the study was conducted.
Emerging research has demonstrated that Google Glass can be used as an assistive communicative and therapeutic technology for people with ASD, although both encouraging results and controversy have arisen. The Kushki lab at the University of Toronto has demonstrated the feasibility and usability of their unique system using Google Glass as a communication aid in ASD [25]. The Wall Lab at Stanford University (Autism Glass) has also shown that Glass may be a feasible social communication tool; however, they have been the subject of media controversy given concerns about lack of transparency, scientific rigor, and ethical practice [26]. Given the largely nascent nature of this technological field, and the vulnerable population that is being studied, it is important to ensure that robust scientific validation and ethical standards are maintained.

2. Methodology

2.1. Technological Background

Empowered Brain is a wearable technology based on computerized glasses loaded with assistive software [27]. Empowered Brain provides users with real-time social communication and attention coaching [28,29,30,31]. The computerized glasses are most commonly Google Glass Explorer Edition (Figure 1).
Empowered Brain is worn on the face in a similar manner to glasses. Users of the device see visual guidance through a translucent display over their right eye and hear sounds through a right-sided bone conduction device. Empowered Brain consists of a variety of sensors, including a camera, microphone, accelerometer, gyroscope, and an inward-facing infrared sensor. Data from the sensors are gathered and are processed through AI-powered software. In-game rewards of points and stars are awarded based on performance and displayed in real-time to the user. These in-game rewards are awarded based on the same sensor data, and a number of points will result in a star reward that unlocks an “achievement” in the game and results in a temporary cartoon mask or embellishment of the partner’s face visible through the screen.
Empowered Brain has previously been found to be feasible to use across multiple studies [27,32]. The use of Empowered Brain has been found to be associated with improvements in social communication, irritability, and symptoms of ADHD [27,31,32]. The technology has also been found to be free of any major adverse effects [33] and capable of accurate real-time measurement of human behavior in brain-related conditions [34]. Empowered Brain has also been rated as desirable for school use [35], and has been studied in multiple longitudinal school-based intervention programs [28,29,32]. Empowered Brain was invented and engineered by Brain Power (Cambridge, MA, USA) and is based on novel software and hardware development partnerships afforded in part through relationships with X (formerly Google X, Mountain View, CA, USA), Affectiva (Boston, MA, USA), and Amazon (Seattle, WA, USA).

2.2. Study Design

A convenience sample of high school students were selected by educators to take part in the study. Criterion for inclusion was a current diagnosis of ASD according to school Individualized Education Program (IEP) records. An IEP is a collaboratively created document for students with special educational needs. An IEP outlines their educational needs and goals, while also setting out the interventions and resources that will be provided to the student. Educators were asked to assess students via an aberrant behavioral checklist (ABC) and the Social Responsiveness Scale (SRS-2). The ABC and SRS-2 are validated behavioral and social communication measures commonly used in ASD research, respectively [22]. Additionally, educators completed a TRF, a validated measure of behavioral and emotional problems in children [23,24]. All students who were referred, consented, and had educators complete their intake measures, were included in this study.
Over a one-week period of time, students were able to use Empowered Brain dependent on the availability of an educator to provide the intervention (Figure 2). The technology is designed to be studied and used in ecologically valid environments. Educators were therefore asked to incorporate the use of the technology in their classroom routine without the provision of additional support, such as staff, time, or other resources. Each session was approximately 10 min in duration and could occur up to a maximum of twice-daily during school days. Student performance was monitored by Empowered Brain device and software, and real-time feedback was given to students regarding points and stars obtained. The data collected from the session, including student performance, is stored and analyzed in a web-accessed portal. This allows teachers to generate summary reports of student performance in a format compatible to IEP reports.
The intervention involved a dyadic interaction between the student and the educator. The student wore Empowered Brain in a similar manner to glasses, and the educator sat in front of the student. At the start of the intervention, the educator powered on the system and loaded the Empowered Brain module. The educator then provided the system to the student and initiated an unstructured academic discussion with him/her regarding a variety of recommended topics, such as homework assignments, classroom projects, or behaviors in relation to home and/or peers. During this time, Empowered Brain monitored the student-educator interaction and provided the student with appropriate visual and auditory guidance to help direct his/her attention to emotional and social communication cues.
As the participant followed the provided digital guidance, for instance by increasing and sustaining attention to the central facial area of the educator, he/she received in-game rewards.
During the interaction, improvement in student social communication skills is met with increased game difficulty, whereas lapses in attention or reduced markers of social engagement are met with increased guidance/decreased game difficulty.

2.3. Outcome Measures

The main assessed measures used in this study were those from the hyperactivity subscale of the Aberrant Behavior Checklist (ABC–H) [36] and the combined ADHD subscale of the TRF [37]. The ABC consists of five subscales, namely: (1) Irritability, Agitation, Crying; (2) Lethargy/Social Withdrawal; (3) Stereotypic Behavior; (4) Hyperactivity/Noncompliance; and Inappropriate Speech. The ABC-H is a particularly useful measure of ADHD symptoms in ASD populations [38]. The ABC-H has been used extensively as a screening and outcome measure in ADHD-related studies in ASD populations [13,27,31,38,39,40,41,42]. The ABC-H contains items that assess ADHD-related symptoms, including hyperactivity, impulsivity, inattention, and defiance [36]. The TRF is an empirically-validated syndrome scale capable of identifying emotional and behavioral problems in school-age children [43]. It has six-DSM orientated scales that have been identified by experts as being consistent with DSM-5 diagnostic categories. The TRF refers to “problem” symptoms, namely depression, anxiety, somatic, thought, and attention-deficit/hyperactivity. The TRF has been validated across many societies and countries [44], and has been used in research studies into ADHD [45].

2.4. Participants and Setting

There were seven participants in this research study (M = 6: F = 1) (Table 1). Mean age of the participants was 15.6 years (range 14–18 years). Participants identified as being white (n = 5), African-American (n = 1), and Hispanic/Latino (n = 1). Participant language ability ranged from non-verbal to fully verbal. Participants received the intervention concurrently with their usual lessons, in their usual classroom setting, as facilitated by their usual educators. Educators received training on how to use Empowered Brain.

2.5. Consent and Institutional Review Board Status

Research of Empowered Brain running on multiple head-worn computing devices and its use with children and adults with ASD was approved by Asentral, Inc., IRB, an affiliate of the Commonwealth of Massachusetts Department of Public Health (protocol no. BPSCOM2, date of approval 17 November 2017). The study was performed in accordance with relevant guidelines and regulations, and in accordance with the Helsinki Declaration. Written informed consent was obtained from all parents/legal guardians of all minors involved in this study and consent was provided by all capable participants. Consent to conduct this research was also obtained from all educators involved in the study. Written informed consent was obtained for the publication of photographs and other identifiable information of participants.

3. Results

Participants (n = 7) had a mean SRS-2 T-score of 78.3 (range 73 to >90), and a mean ABC-H of 7.9 (range 0–20). Participants’ mean TRF ADHD raw score was 7.7 (range 0–15) with a mean TRF ADHD T-score of 56.7 (50–65). Participant ABC-H score and TRF ADHD raw scores were significantly correlated (p = 0.01).
Participant in-game performance was successfully gathered from all participants. Participants had a mean of 4.6 Empowered Brain sessions (total number 32, range 2–8). All participants were able to demonstrate an ability to obtain rewards.
Participant in-game performance, as assessed by stars and points rewards, was correlated to ABC-H and TRF ADHD raw scores through the calculation of a Spearman Rank correlation coefficient. A p-value was determined from this score. Participant in-game performance demonstrated highly significant correlation to baseline ABC-H for both game points (p = 0.0013) and game stars (p = 0.0013) (Table 2). TRF ADHD raw scores were also significantly correlated to student in-game performance in regard to points (p = 0.012) and stars (p = 0.012) (Table 3). Mean participant performance is displayed in Figure 3 and Figure 4 (ABC-H) and Figure 5 and Figure 6 (TRF ADHD). Given that the game points and star measures were not independently-related game components, and were directly linked to one another, statistical correction was not deemed necessary. No adverse effects were noted among students who used Empowered Brain.

4. Discussion

New technologies may herald novel ways of identifying and characterizing symptoms of ADHD in student populations. Over 6.4 million children and adolescents in the US have been diagnosed with ADHD [46], and the condition currently impacts over 10 million US adults [47]. Despite this, research suggests that ADHD symptoms continue to be under-recognized despite leading to adverse educational and health outcomes. Technology may allow for a highly scalable and objective assessment of ADHD symptoms. While technology could augment traditional subjective approaches, its utility and ecological validity must be assessed.
This study provides supportive evidence that Empowered Brain is a technology that can be used by students and teachers within a regular classroom setting. The usability of this system was demonstrated despite methodologically-imposed limitations designed to ensure high ecologically validity. These limitations include student use of the technology, as facilitated by their own regular teachers, during their regular lessons/school day, and within their own classrooms. No additional time, staff, or fiscal resources were provided except for initial teacher training. The study demonstrated that the in-game performance of the students was significantly correlated to two clinically validated measures of ADHD symptom severity. The findings were highly significant in the case of the ABC-H measure.
While this study has a sample size of only seven participants, to the authors knowledge, this is the first report of the use of assessment of ADHD symptom severity through the use of socio-emotional game performance on smartglasses. The data were collected over a one-week period and involved a total of 32 game sessions, each lasting 10 min in duration, providing for 320 min of use. Given these findings, it would be important to conduct further studies that address questions raised by this study. Firstly, further research should attempt to reproduce the results with a larger student sample size. Secondly, use of a broader age range of students across different school types could aid the generalizability of the findings across a broader range of students. Thirdly, a more detailed assessment of the presence of confounding disorders such as anxiety and depression should be undertaken, especially as they influence social interaction.
Future work will focus on gathering data on the use of Empowered Brain in non-ASD populations who may experience challenges in inattention, hyperactivity, and impulsivity. As symptoms of inattention and impulsivity occur across a broad transdiagnostic range of conditions, the possibility of using this technology across a much broader student and clinical population can also be considered.
Despite these limitations, Empowered Brain has already been shown to improve social communication and behavior in people with ASD when used as a daily socio-affective coaching tool. While providing this intervention through game-like experiences, Empowered Brain collects and analyzes important information about the user through data-gathering sensors and artificial intelligence-based analysis. While further research is warranted, the potential for Empowered Brain to be able assess ADHD symptom load would be an interesting possibility. A digital tool that is practical to use while providing quantitative stratification of ADHD symptom severity could help identify the needs of students while not unduly impacting school resources. This technological approach may have utility to further aid digital phenotyping efforts seen in mental health conditions [48,49].
Additional research is warranted to determine whether the system could function in a longitudinal monitoring role. Empowered Brain has already demonstrated its ability to generate real-world data and real-world evidence in a series of research studies. Gathering more robust evidence to validate the ability of Empowered Brain to objectively and accurately measure ADHD symptoms is important. This validation could allow for the use of such technology as an outcome measure tool in ADHD clinical research. The ability to do this in a quantitative and standardized manner is highly desirable. We should also consider that there is evidence Empowered Brain can improve ADHD symptoms. Therefore, there is a possibility that participant ADHD symptom load may have changed during their use of Empowered Brain in this study (mean number of sessions: 4.6). We should also note that prior attempts to use CPT as an objective measure of ADHD symptoms have often focused on the measurement of attention and impulsivity, with limited assessment of hyperactivity/motor behavior. This is important given inattention and impulsivity are commonly encountered transdiagnostic symptoms across many behavioral conditions, whereas hyperactivity may be much more specific to ADHD [18]. The Empowered Brain captures user performance and interaction across all three ADHD symptom domains, including hyperactivity, allowing the system to provide a more comprehensive approach to ADHD assessment as compared to most CPTs.
The research and development of technologies that can quantify behavior represents a broader trend to use technology to obtain patient-generated data from non-clinical settings. This data can help to understand the risks and benefits associated with new health interventions [RWE). Additionally, digital patient monitoring and neuropsychological testing have been introduced as new codes for the Current Procedural Terminology code set for 2019, allowing them to be billable services for health insurers and Medicare.

5. Conclusion

This study demonstrates that Empowered Brain in-game performance correlates with, and may be predictive of, ADHD-related symptoms in students with ASD. These preliminary findings were obtained in real world educational settings, without the provision of any additional school resources. Further research with larger participant numbers is required to replicate these findings. There is potential for this type of technology not only to aid identification and longitudinal monitoring of such symptoms, but also to provide for an assessment of severity level and subtype. Larger and more diverse samples will be required to improve the generalizability of the present findings to a broader range of brain conditions that manifest with inattention, impulsivity, and hyperactivity.

Author Contributions

Conceptualization, N.U.K., A.V., N.T.S.; methodology K.V.-L., N.U.K., A.V., N.T.S.; software N.T.S.; formal analysis, K.V.-L., N.U.K., A.V., N.T.S.; investigation, K.V.-L., N.U.K., A.V., N.T.S.; data curation, A.V., N.U.K., N.T.S., writing—original draft preparation, K.V.-L., N.U.K., A.V., N.T.S. writing—review and editing, K.V.-L., N.U.K., A.V., N.T.S.; visualization, A.V. & N.T.S.

Funding

This work has been supported by the Office of the Assistant Secretary of Defense for Health Affairs, through the Autism Research Program under Award No. W81XWH 17-1-0449. Early work to transform smartglasses into biomedical sensors was supported in part by the United States Army Medical Research and Materiel Command under Contract No. W81XWH-14-C-0007 (awarded to TIAX, LLC). Opinions, interpretations, conclusions, and recommendations are those of the authors and are not necessarily endorsed by the Department of Defense.

Acknowledgments

The authors acknowledge Google, Inc., (Mountain View, CA, USA) for a generous grant and also the Glass team at X (formerly Google X) for technical guidance on Glass software development. The research is also supported through in-kind artificial intelligence and technological resources from Amazon and Amazon Web Services, and Affectiva. The authors also thank the families, children, educators, and administrators who provided their time and effort in participating in this research study.

Conflicts of Interest

The authors are doctoral level scientists (N.U.K., A.V., N.T.S.), and lab managers (K.V.-L.) who hold affiliations at Brain Power. Brain Power is a congressionally-funded for-profit organization that conducts research and development of neurotechnologies that can improve the lives of people with brain differences and brain injuries. The grant and in-kind funders of the study had no role in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

References

  1. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (DSM-5®); APA Publishing: Washington, DC, USA, 2013. [Google Scholar]
  2. Shattuck, P.T.; Narendorf, S.C.; Cooper, B.; Sterzing, P.R.; Wagner, M.; Taylor, J.L. Postsecondary education and employment among youth with an autism spectrum disorder. Pediatrics 2012, 129, 1042–1049. [Google Scholar] [CrossRef] [PubMed]
  3. Taylor, J.L.; Mailick, M.R. A longitudinal examination of 10-year change in vocational and educational activities for adults with autism spectrum disorders. Dev. Psychol. 2014, 50, 699–708. [Google Scholar] [CrossRef] [PubMed]
  4. Kohane, I.S.; McMurry, A.; Weber, G.; MacFadden, D.; Rappaport, L.; Kunkel, L.; Bickel, J.; Wattanasin, N.; Spence, S.; Murphy, S.; et al. The co-morbidity burden of children and young adults with autism spectrum disorders. PLoS ONE 2012, 7, e33224. [Google Scholar] [CrossRef] [PubMed]
  5. Eaves, L.C.; Ho, H.H. Young adult outcome of autism spectrum disorders. J. Autism Dev. Disord. 2008, 38, 739–747. [Google Scholar] [CrossRef] [PubMed]
  6. Roux, A.M.; Shattuck, P.T.; Cooper, B.P.; Anderson, K.A.; Wagner, M.; Narendorf, S.C. Postsecondary employment experiences among young adults with an autism spectrum disorder. J. Am. Acad. Child Adolesc. Psychiatry 2013, 52, 931–939. [Google Scholar] [CrossRef] [PubMed]
  7. Leyfer, O.T.; Folstein, S.E.; Bacalman, S.; Davis, N.O.; Dinh, E.; Morgan, J.; Tager-Flusberg, H.; Lainhart, J.E. Comorbid psychiatric disorders in children with autism: Interview development and rates of disorders. J. Autism Dev. Disord. 2006, 36, 849–861. [Google Scholar] [CrossRef] [PubMed]
  8. Simonoff, E.; Pickles, A.; Charman, T.; Chandler, S.; Loucas, T.; Baird, G. Psychiatric disorders in children with autism spectrum disorders: Prevalence, comorbidity, and associated factors in a population-derived sample. J. Am. Acad. Child Adolesc. Psychiatry 2008, 47, 921–929. [Google Scholar] [CrossRef] [PubMed]
  9. Chiang, H.L.; Kao, W.C.; Chou, M.C.; Chou, W.J.; Chiu, Y.N.; Wu, Y.Y.; Gau, S.S. School dysfunction in youth with autistic spectrum disorder in Taiwan: The effect of subtype and ADHD. Autism Res. 2018, 11, 857–869. [Google Scholar] [CrossRef] [PubMed]
  10. Loe, I.M.; Feldman, H.M. Academic and educational outcomes of children with ADHD. J. Pediatr. Psychol. 2007, 32, 643–654. [Google Scholar] [CrossRef] [PubMed]
  11. Murray, M.J. Attention-deficit/hyperactivity disorder in the context of autism spectrum disorders. Curr. Psychiatry Rep. 2010, 12, 382–388. [Google Scholar] [CrossRef]
  12. Joshi, G.; Faraone, S.V.; Wozniak, J.; Tarko, L.; Fried, R.; Galdo, M.; Furtak, S.L.; Biederman, J. Symptom profile of ADHD in Youth with high-functioning autism spectrum disorder: A comparative study in psychiatrically referred populations. J. Atten. Disord. 2017, 21, 846–855. [Google Scholar] [CrossRef] [PubMed]
  13. Research Units on Pediatric Psychopharmacology Autism Network. Randomized, controlled, crossover trial of methylphenidate in pervasive developmental disorders with hyperactivity. Arch. Gen. Psychiatry 2005, 62, 1266–1274. [Google Scholar] [CrossRef] [PubMed]
  14. Erickson, C.A.; Posey, D.J.; Stigler, K.A.; Mullett, J.; Katschke, A.R.; McDougle, C.J. A retrospective study of memantine in children and adolescents with pervasive developmental disorders. Psychopharmacology 2007, 191, 141–147. [Google Scholar] [CrossRef] [PubMed]
  15. Collett, B.R.; Ohan, J.L.; Myers, K.M. Ten-year review of rating scales. V: Scales assessing attention-deficit/hyperactivity disorder. J. Am. Acad. Child Adolesc. Psychiatry 2003, 42, 1015–1037. [Google Scholar] [CrossRef] [PubMed]
  16. Bied, A.; Biederman, J.; Faraone, S. Parent-based diagnosis of ADHD is as accurate as a teacher-based diagnosis of ADHD. Postgrad. Med. 2017, 129, 375–381. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  17. Ogundele, M.O.; Ayyash, H.F.; Banerjee, S. Role of computerised continuous performance task tests in ADHD. Prog. Neurol. Psychiatry 2011, 15, 8–13. [Google Scholar] [CrossRef]
  18. Hall, C.L.; Valentine, A.Z.; Groom, M.J.; Walker, G.M.; Sayal, K.; Daley, D.; Hollis, C. The clinical utility of the continuous performance test and objective measures of activity for diagnosing and monitoring ADHD in children: A systematic review. Eur. Child Adolesc. Psychiatr. 2016, 25, 677–699. [Google Scholar] [CrossRef] [PubMed]
  19. Jarow, J.P.; LaVange, L.; Woodcock, J. Multidimensional evidence generation and FDA regulatory decision making: Defining and using “real-world” data. JAMA 2017, 318, 703–704. [Google Scholar] [CrossRef]
  20. Lovejoy, M.C.; Rasmussen, N.H. The validity of vigilance tasks in differential diagnosis of children referred for attention and learning problems. J. Abnorm. Child Psychol. 1990, 18, 671–681. [Google Scholar] [CrossRef]
  21. Trommer, B.L.; Hoeppner, J.-A.B.; Lorber, R.; Armstrong, K. Pitfalls in the use of a continuous performance test as a diagnostic tool in attention deficit disorder. J. Dev. Behav. Pediatr. 1988, 9, 339–345. [Google Scholar] [CrossRef]
  22. Constantino, J.N.; Gruber, C.P. Social Responsiveness Scale (SRS); Western Psychological Services: Torrance, CA, USA, 2012. [Google Scholar]
  23. Achenbach, T.M. Child Behavior Checklist and Related Instruments. In The Use of Psychological Testing for Treatment Planning and Outcome Assessment; Maruish, M.E., Ed.; Lawrence Erlbaum Associates, Inc.: Hillsdale, NJ, USA, 1994; pp. 517–549. [Google Scholar]
  24. Young, T.; Palta, M.; Dempsey, J.; Skatrud, J.; Weber, S.; Badr, S. The occurrence of sleep-disordered breathing among middle-aged adults. N. Engl. J. Med. 1993, 328, 1230–1235. [Google Scholar] [CrossRef] [PubMed]
  25. Kinsella, B.G.; Chow, S.; Kushki, A. Evaluating the usability of a wearable social skills training technology for children with autism spectrum disorder. Front. Robot. AI 2017, 4, 31. [Google Scholar] [CrossRef]
  26. Furfaro, H. Meet the “Bad Boy” of Autism Research. Scientific American. 2018. Available online: www.scientificamerican.com/article/meet-the-bad-boy-of-autism-research/ (accessed on 15 January 2019).
  27. Liu, R.; Salisbury, J.P.; Vahabzadeh, A.; Sahin, N.T. Feasibility of an autism-focused augmented reality smartglasses system for social communication and behavioral coaching. Front. Pediatr. 2017, 5, 145. [Google Scholar] [CrossRef] [PubMed]
  28. Sahin, N.T.; Abdus-Sabur, R.; Keshav, N.U.; Liu, R.; Salisbury, J.P.; Vahabzadeh, A. (Eds.) Case study of a digital augmented reality intervention for autism in school classrooms: Associated with improved social communication, cognition, and motivation as rated by educators and parents. Front. Educat. 2018, 3, 57. [Google Scholar] [CrossRef]
  29. Keshav, N.; Vahabzadeh, A.; Abdus-Sabur, R.; Huey, K.; Salisbury, J.; Liu, R.; Sahin, N. Longitudinal socio-emotional learning intervention for autism via smartglasses: Qualitative school teacher descriptions of practicality, usability, and efficacy in general and special education classroom settings. Educ. Sci. 2018, 8, 107. [Google Scholar] [CrossRef]
  30. Keshav, N.U.; Salisbury, J.P.; Vahabzadeh, A.; Sahin, N.T. Social communication coaching smartglasses: Well tolerated in a diverse sample of children and adults with autism. JMIR Mhealth Uhealth 2017, 5, e140. [Google Scholar] [CrossRef] [PubMed]
  31. Vahabzadeh, A.; Keshav, N.U.; Salisbury, J.P.; Sahin, N.T. Improvement of attention-deficit/hyperactivity disorder symptoms in school-aged children, adolescents, and young adults with autism via a digital smartglasses-based socioemotional coaching aid: Short-term, uncontrolled pilot study. JMIR Ment. Health 2018, 5, e25. [Google Scholar] [CrossRef]
  32. Vahabzadeh, A.; Keshav, N.U.; Abdus-Sabur, R.; Huey, K.; Liu, R.; Sahin, N.T. Improved socio-emotional and behavioral functioning in students with autism following school-based smartglasses intervention: Multi-stage feasibility and controlled efficacy study. Behav. Sci. 2018, 8, E85. [Google Scholar] [CrossRef]
  33. Sahin, N.T.; Keshav, N.U.; Salisbury, J.P.; Vahabzadeh, A. Safety and lack of negative effects of wearable augmented-reality social communication aid for children and adults with autism. J. Clin. Med. 2018, 7, E188. [Google Scholar] [CrossRef]
  34. Salisbury, J.P.; Keshav, N.U.; Sossong, A.D.; Sahin, N.T. Concussion assessment with smartglasses: Validation study of balance measurement toward a lightweight, multimodal, field-ready platform. JMIR Mhealth Uhealth 2018, 6, e15. [Google Scholar] [CrossRef]
  35. Sahin, N.T.; Keshav, N.U.; Salisbury, J.P.; Vahabzadeh, A. second version of google glass as a wearable socio-affective aid: Positive school desirability, high usability, and theoretical framework in a sample of children with autism. JMIR Hum. Factors 2018, 5, e1. [Google Scholar] [CrossRef] [PubMed]
  36. Aman, M.G.; Singh, N.N.; Stewart, A.W.; Field, C.J. The aberrant behavior checklist: a behavior rating scale for the assessment of treatment effects. Am. J. Ment. Defic. 1985, 89, 485–491. [Google Scholar]
  37. Rescorla, L.A.; Achenbach, T.M. The Achenbach System of Empirically Based Assessment (ASEBA) for Ages 18 to 90 Years. In The Use of Psychological Testing for Treatment Planning and Outcomes Assessment: Instruments for Adults; Maruish, M.E., Ed.; Lawrence Erlbaum Associates Publishers: Mahwah, NJ, USA, 2004; pp. 115–152. [Google Scholar]
  38. Lyall, K.; Schweitzer, J.B.; Schmidt, R.J.; Hertz-Picciotto, I.; Solomon, M. Inattention and hyperactivity in association with autism spectrum disorders in the CHARGE study. Res. Autism Spectr. Disord. 2017, 35, 1–12. [Google Scholar] [CrossRef]
  39. King, B.H.; Wright, D.M.; Handen, B.L.; Sikich, L.; Zimmerman, A.W.; McMahon, W.; Cantwell, E.; Davanzo, P.A.; Dourish, C.T.; Dykens, E.M.; et al. Double-blind, placebo-controlled study of amantadine hydrochloride in the treatment of children with autistic disorder. J. Am. Acad. Child Adolesc. Psychiatry 2001, 40, 658–665. [Google Scholar] [CrossRef]
  40. Arnold, L.E.; Aman, M.G.; Cook, A.M.; Witwer, A.N.; Hall, K.L.; Thompson, S.; Ramadan, Y. Atomoxetine for hyperactivity in autism spectrum disorders: placebo-controlled crossover pilot trial. J. Am. Acad. Child Adolesc. Psychiatry 2006, 45, 1196–1205. [Google Scholar] [CrossRef] [PubMed]
  41. Bent, S.; Hendren, R.L.; Zandi, T.; Law, K.; Choi, J.E.; Widjaja, F.; Kalb, L.; Nestle, J.; Law, P. Internet-based, randomized, controlled trial of omega-3 fatty acids for hyperactivity in autism. J. Am. Acad. Child Adolesc. Psychiatry 2014, 53, 658–666. [Google Scholar] [CrossRef] [PubMed]
  42. McCracken, J.T.; Badashova, K.K.; Posey, D.J.; Aman, M.G.; Scahill, L.; Tierney, E.; Arnold, L.E.; Vitiello, B.; Whelan, F.; Chuang, S.Z.; et al. Positive effects of methylphenidate on hyperactivity are moderated by monoaminergic gene variants in children with autism spectrum disorders. Pharmacogenomics J. 2014, 14, 295–302. [Google Scholar] [CrossRef] [PubMed]
  43. Achenbach, T.M.; Rescorla, L. ASEBA School-Age Forms & Profiles; ASEBA: Burlington, VT, USA, 2001. [Google Scholar]
  44. Ivanova, M.Y.; Achenbach, T.M.; Rescorla, L.A.; Dumenci, L.; Almqvist, F.; Bathiche, M.; Bilenberg, N.; Bird, H.; Domuta, A.; Erol, N. Testing the teacher’s report form syndromes in 20 societies. Sch. Psychol. Rev. 2007, 36, 468–483. [Google Scholar]
  45. Sasser, T.R.; Kalvin, C.B.; Bierman, K.L. Developmental trajectories of clinically significant attention-deficit/hyperactivity disorder (ADHD) symptoms from grade 3 through 12 in a high-risk sample: Predictors and outcomes. J. Abnorm. Psychol. 2016, 125, 207–219. [Google Scholar] [CrossRef]
  46. Visser, S.N.; Danielson, M.L.; Bitsko, R.H.; Holbrook, J.R.; Kogan, M.D.; Ghandour, R.M.; Perou, R.; Blumberg, S.J. Trends in the parent-report of health care provider-diagnosed and medicated attention-deficit/hyperactivity disorder: United States, 2003–2011. J. Am. Acad. Child Adolesc. Psychiatry 2014, 53, 34–46.e2. [Google Scholar] [CrossRef]
  47. Kessler, R.C.; Berglund, P.; Chiu, W.T.; Demler, O.; Heeringa, S.; Hiripi, E.; Jin, R.; Pennell, B.E.; Walters, E.E.; Zaslavsky, A.; et al. The US national comorbidity survey replication (NCS-R): Design and field procedures. Int. J. Methods Psychiatr. Res. 2004, 13, 69–92. [Google Scholar] [CrossRef] [PubMed]
  48. Onnela, J.P.; Rauch, S.L. Harnessing smartphone-based digital phenotyping to enhance behavioral and mental health. Neuropsychopharmacology 2016, 41, 1691–1696. [Google Scholar] [CrossRef] [PubMed]
  49. Vahabzadeh, A.; Sahin, N.; Kalali, A. Digital suicide prevention: Can technology become a game-changer? Innov. Clin. Neurosci. 2016, 13, 16–20. [Google Scholar] [PubMed]
Children 06 00072 g001 550
Figure 1. (A) Google Glass versions 1–3 (Explorer Edition, Enterprise Edition, Enterprise Edition 2) on an adult human face. Corresponding weight and facial occlusion characteristics are outlined. (B) Images of other augmented reality platforms (Magic Leap One, Microsoft Hololens, Meta AR) demonstrating increased weight and facial occlusion relative to Google Glass platforms. (C) Google Glass Enterprise Edition 2 with optional safety frames, and in side profile view without safety frames. Low occlusive characteristics are still maintained with addition of frames.
Figure 1. (A) Google Glass versions 1–3 (Explorer Edition, Enterprise Edition, Enterprise Edition 2) on an adult human face. Corresponding weight and facial occlusion characteristics are outlined. (B) Images of other augmented reality platforms (Magic Leap One, Microsoft Hololens, Meta AR) demonstrating increased weight and facial occlusion relative to Google Glass platforms. (C) Google Glass Enterprise Edition 2 with optional safety frames, and in side profile view without safety frames. Low occlusive characteristics are still maintained with addition of frames.
Children 06 00072 g001
Children 06 00072 g002 550
Figure 2. Overview of Empowered Brain as related to attention-deficit/hyperactivity disorder (ADHD) symptom stratification. (A–C) Non-study student wears Empowered Brain smartglasses and proceeds to engage with educator in live classroom setting. (D,E) Attention and game performance charts from Empowered Brain web account, a secure cloud-accessible data portal that reports student learning and performance. (F–G) Visual snapshots of what the student sees through the optical display during moments of achievement while using Empowered Brain. These snapshots are automatically recorded and available for teacher and student review through the student’s Empowered Brain account. Consent was obtained from all identifiable individuals to publish their photographs in publicly accessible research journals.
Figure 2. Overview of Empowered Brain as related to attention-deficit/hyperactivity disorder (ADHD) symptom stratification. (A–C) Non-study student wears Empowered Brain smartglasses and proceeds to engage with educator in live classroom setting. (D,E) Attention and game performance charts from Empowered Brain web account, a secure cloud-accessible data portal that reports student learning and performance. (F–G) Visual snapshots of what the student sees through the optical display during moments of achievement while using Empowered Brain. These snapshots are automatically recorded and available for teacher and student review through the student’s Empowered Brain account. Consent was obtained from all identifiable individuals to publish their photographs in publicly accessible research journals.
Children 06 00072 g002
Children 06 00072 g003 550
Figure 3. ABC-H Score vs. Mean Point Rewards Per Session.
Figure 3. ABC-H Score vs. Mean Point Rewards Per Session.
Children 06 00072 g003
Children 06 00072 g004 550
Figure 4. ABC-H score vs. Mean Star Rewards per Session.
Figure 4. ABC-H score vs. Mean Star Rewards per Session.
Children 06 00072 g004
Children 06 00072 g005 550
Figure 5. TRF ADHD score vs. Mean Star Rewards per Session.
Figure 5. TRF ADHD score vs. Mean Star Rewards per Session.
Children 06 00072 g005
Children 06 00072 g006 550
Figure 6. TRF ADHD score vs. Mean Star Rewards per Session.
Figure 6. TRF ADHD score vs. Mean Star Rewards per Session.
Children 06 00072 g006
Table
Table 1. Participant Demographics.
Table 1. Participant Demographics.
ParticipantAgeGenderEthnicityGradeEducatior-Rated Verbal Ability
115MWhite9thFull
215FBlack/African American9thMinimal
314MHispanic/Latino9thNon-verbal
415MWhite9thFull
516MWhite10thFull
618MWhite10thFull
716MWhite10thFull
Table
Table 2. Mean Participant ABC-H Score vs In-game Performance (Points & Stars Rewards).
Table 2. Mean Participant ABC-H Score vs In-game Performance (Points & Stars Rewards).
ParticipantABC-H ScoreMean PointsMean Stars
120143.40.125
204089.32.83
3112400.5
4420672.2
54797.61
61633.50
7042223.75
Spearman Rho−0.95−0.95
P-Value0.0013 **0.0013 **
** highly significant < 0.01).
Table
Table 3. Mean Participant Teacher Report Form (TRF) attention-deficit/hyperactivity disorder (ADHD) Raw Score vs In-game Performance (Points & Stars Rewards).
Table 3. Mean Participant Teacher Report Form (TRF) attention-deficit/hyperactivity disorder (ADHD) Raw Score vs In-game Performance (Points & Stars Rewards).
ParticipantTRF ADHD Raw ScoreMean PointsMean Stars
112143.40.125
254089.32.83
3102400.5
4820672.2
54797.61
61533.50
7042223.75
Spearman Rho−0.89−0.89
P-Value0.012 *0.012 *
* significant < 0.05.

Share and Cite

MDPI and ACS Style

Keshav, N.U.; Vogt-Lowell, K.; Vahabzadeh, A.; Sahin, N.T. Digital Attention-Related Augmented-Reality Game: Significant Correlation between Student Game Performance and Validated Clinical Measures of Attention-Deficit/Hyperactivity Disorder (ADHD). Children 2019, 6, 72. https://doi.org/10.3390/children6060072

AMA Style

Keshav NU, Vogt-Lowell K, Vahabzadeh A, Sahin NT. Digital Attention-Related Augmented-Reality Game: Significant Correlation between Student Game Performance and Validated Clinical Measures of Attention-Deficit/Hyperactivity Disorder (ADHD). Children. 2019; 6(6):72. https://doi.org/10.3390/children6060072

Chicago/Turabian Style

Keshav, Neha U., Kevin Vogt-Lowell, Arshya Vahabzadeh, and Ned T. Sahin. 2019. "Digital Attention-Related Augmented-Reality Game: Significant Correlation between Student Game Performance and Validated Clinical Measures of Attention-Deficit/Hyperactivity Disorder (ADHD)" Children 6, no. 6: 72. https://doi.org/10.3390/children6060072

APA Style

Keshav, N. U., Vogt-Lowell, K., Vahabzadeh, A., & Sahin, N. T. (2019). Digital Attention-Related Augmented-Reality Game: Significant Correlation between Student Game Performance and Validated Clinical Measures of Attention-Deficit/Hyperactivity Disorder (ADHD). Children, 6(6), 72. https://doi.org/10.3390/children6060072

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop