Effect of Sensory Feedback on Turn-Taking Using Paired Devices for Children with ASD
Abstract
:1. Introduction
1.1. Robotic Toys to Engage Children in Interaction
1.2. Robotic Toys to Describe Children’s Behaviors
1.3. Proposal and Contribution
- The introduction of different interaction rules using paired devices for turn-taking.
- Investigation of the effect of different interaction rules using paired devices on the turn-taking behavior of children with ASD.
- Investigation on the capacity and viability of using COLOLO for automated quantitative measurement of children’s and therapist behavior during therapy sessions.
2. Methodology
2.1. COLOLO: Paired Robotic Devices
2.2. COLOLO in Turn-Taking Interventions
2.3. Interaction Rules and Hypotheses
2.4. Turn-Taking Behavior Analysis Using Data from COLOLO
- Time: the time a message from any of the toys is received by the server.
- Label: “RECEIVE” indicates that the tilt sensor in a device detected a manipulation and then the server received a notification. If the manipulation was made during its turn, a new line will be added in the log with the label of “CHANGE”, and the server then forwards the message to the paired device.
- ID: the identification number of the sender device. There is a number for the therapist’s device and for the participant’s device.
- Color: the information related to the light color of the sender.
3. Evaluation
3.1. Experimental Setup
3.2. Pilot Study
3.3. Experimental Study: Comparison of Two Interaction Rules
3.3.1. Effect on Children’s Behavior during Turn-Taking Interventions
- Condition: During the therapist’s turn and when a message is being exchanged. Indicators: Each time the child directs his gaze from the therapist’s facial region or the therapist’s device to his device.
- Condition: During the child’s turn and when a message is being exchanged. Indicators: Each time the child directs his gaze from his device to the therapist’s facial region or the therapist’s device.
- Successful attempt of turn-taking: described by clear attempts to manipulate the device that result in a device changing color (the child manipulates his toy during his turn)
- Failed attempt of turn-taking: described by clear attempts to manipulate the toy that did not result in a change of color (the child manipulates his toy during the therapist’s turn)
3.3.2. Turn-Taking Analysis by COLOLO
- Each time the child took his turn: described by clear attempts to send a message by manipulating COLOLO, with the gaze on one of the devices or the therapist’s face (Pattern A).
- Each time the child waited without manipulating COLOLO during the therapist’s turn: the child is looking at the therapist or paired device, showing no intention of manipulating the device (Pattern B).
4. Results
4.1. Effect on Manipulation
4.2. Effect on Gaze Shifting
4.3. Effect on Failed Attempts of Turn-Taking
4.4. Turn-Taking Analysis by COLOLO
5. Discussions
5.1. Effects of Sensory Feedback on Turn-Taking Behaviors
5.2. COLOLO as a Descriptor of Turn-Taking Behaviors
6. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Boucher, J. Interventions with Children with Autism—Methods Based on Play; Sage: Thousand Oaks, CA, USA, 1999. [Google Scholar]
- Boutot, E.A.; Guenther, T.; Crozier, S. Let’s play: Teaching play skills to young children with autism. Educ. Train. Dev. Disabil. 2005, 40, 285–292. [Google Scholar]
- Baron-Cohen, S. Autism and symbolic play. Br. J. Dev. Psychol. 1987, 5, 139–148. [Google Scholar] [CrossRef]
- Jarrold, C.; Boucher, J.; Smith, P. Symbolic play in autism: A review. J. Autism Dev. Disord. 1993, 23, 281–307. [Google Scholar] [CrossRef] [PubMed]
- Williams, E.; Reddy, V.; Costall, A. Taking a Closer Look at Functional Play in Children with Autism. J. Autism Dev. Disord. 2001, 31, 67–77. [Google Scholar] [CrossRef] [PubMed]
- Boucher, J. Editorial: Interventions with children with autism—Methods based on play. Child Lang. Teach. Ther. 1999, 15, 1–5. [Google Scholar] [CrossRef]
- Wulff, S.B. The symbolic and object play of children with autism: A review. J. Autism Dev. Disord. 1985, 15, 139–148. [Google Scholar] [CrossRef] [PubMed]
- Scassellati, B.; Admoni, H.; MatariÄ, M. Robots for Use in Autism Research. Annu. Rev. Biomed. Eng. 2012, 14, 275–294. [Google Scholar] [CrossRef] [PubMed]
- Robins, B.; Otero, N.; Ferrari, E.; Dautenhahn, K. Eliciting Requirements for a Robotic Toy for Children with Autism—Results from User Panels. In Proceedings of the 16th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2007), Jeju, Korea, 26–29 August 2007; pp. 101–106. [Google Scholar]
- Ingersoll, B.; Schreibman, L.; Tran, Q.H. Effect of Sensory Feedback on Immediate Object Imitation in Children with Autism. J. Autism Dev. Disord. 2003, 33, 673–683. [Google Scholar] [CrossRef] [PubMed]
- Piaget, J. Play, Dreams, and Imitation in Childhood; Norton Library: Norton, MA, USA, 1962; p. 579. [Google Scholar]
- Harper, C.B.; Symon, J.B.G.; Frea, W.D. Recess is Time-in: Using Peers to Improve Social Skills of Children with Autism. J. Autism Dev. Disord. 2008, 38, 815–826. [Google Scholar] [CrossRef] [PubMed]
- Rieth, S.R.; Stahmer, A.C.; Suhrheinrich, J.; Schreibman, L.; Kenned, J.; Ross, B. Identifying Critical Elements of Treatment: Examining the Use of Turn Taking in Autism Intervention. Focus Autism Dev. Disabil. 2014, 29, 168–179. [Google Scholar] [CrossRef]
- Yoder, P.; Stone, W.L. Randomized comparison of two communication interventions for preschoolers with autism spectrum disorders. J. Consult. Clin. Psychol. 2006, 74, 426. [Google Scholar] [CrossRef] [PubMed]
- Aldred, C.; Green, J.; Adams, C. A new social communication intervention for children with autism: Pilot randomised controlled treatment study suggesting effectiveness. J. Child Psychol. Psychiatry 2004, 45, 1420–1430. [Google Scholar] [CrossRef] [PubMed]
- Mundy, P.; Crowson, M. Joint attention and early social communication: Implications for research on intervention with autism. J. Autism Dev. Disord. 1997, 27, 653–676. [Google Scholar] [CrossRef] [PubMed]
- Billard, A.; Robins, B.; Nadel, J.; Dautenhahn, K. Building robota, a mini-humanoid robot for the rehabilitation of children with autism. Assist. Technol. 2007, 19, 37–49. [Google Scholar] [CrossRef] [PubMed]
- Wainer, J.; Robins, B.; Amirabdollahian, F.; Dautenhahn, K. Using the humanoid robot KASPAR to autonomously play triadic games and facilitate collaborative play among children with autism. IEEE Trans. Auton. Mental Dev. 2014, 6, 183–199. [Google Scholar] [CrossRef]
- Dautenhahn, K. Socially intelligent robots: Dimensions of human–robot interaction. Philos. Trans. R. Soc. Lond. B Biol. Sci. 2007, 362, 679–704. [Google Scholar] [CrossRef] [PubMed]
- Soleiman, P.; Moradi, H.; Mahmoudi, M.; Teymouri, M.; Pouretemad, H.R. The Use of RoboParrot in the Therapy of Children with Autism Children: In Case of Teaching the Turn-Taking Skills; Springer: Berlin/Heidelberg, Germany, 2011. [Google Scholar]
- Barakova, E.; Gillessen, J.; Feijs, L. Social Training of Autistic Children with Interactive Intelligent Agents. J. Integr. Neurosci. 2009, 8, 23–34. [Google Scholar] [CrossRef] [PubMed]
- Brok, J.C.J.; Barakova, E.I. Engaging Autistic Children in Imitation and Turn-taking Games with Multiagent System of Interactive Lighting Blocks. In Proceedings of the 9th International Conference on Entertainment Computing, Seoul, Korea, 8–11 September 2010; pp. 115–126. [Google Scholar]
- Klin, A. Attributing social meaning to ambiguous visual stimuli in higher-functioning autism and Asperger syndrome: The social attribution task. J. Child Psychol. Psychiatry Allied Discip. 2000, 41, 831–846. [Google Scholar] [CrossRef]
- Happé, F.G. Communicative competence and theory of mind in autism: A test of relevance theory. Cognition 1993, 48, 101–119. [Google Scholar] [CrossRef]
- Scassellati, B. How social robots will help us to diagnose, treat, and understand autism. In Robotics Research; Springer: Berlin/Heidelberg, Germany, 2007; pp. 552–563. [Google Scholar]
- Rehg, J.M. Behavior Imaging: Using Computer Vision to Study Autism. MVA 2011, 11, 14–21. [Google Scholar]
- Anzalone, S.M.; Boucenna, S.; Ivaldi, S.; Chetouani, M. Evaluating the engagement with social robots. Int. J. Soc. Robot. 2015, 7, 465–478. [Google Scholar] [CrossRef]
- Albinali, F.; Goodwin, M.S.; Intille, S.S. Recognizing stereotypical motor movements in the laboratory and classroom: A case study with children on the autism spectrum. In Proceedings of the 11th International Conference on Ubiquitous Computing, Orlando, FL, USA, 30 September–3 October 2009; pp. 71–80. [Google Scholar]
- Funahashi, A.; Gruebler, A.; Aoki, T.; Kadone, H.; Suzuki, K. Brief Report: The Smiles of a Child with Autism Spectrum Disorder during an Animal-Assisted Activity May Facilitate Social Positive Behaviors? Quantitative Analysis with Smile-Detecting Interface. J. Autism Dev. Disord. 2014, 44, 685–693. [Google Scholar] [CrossRef] [PubMed]
- Pan, Y.; Hirokawa, M.; Suzuki, K. Measuring K-degree facial interaction between robot and children with autism spectrum disorders. In Proceedings of the 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2015), Kobe, Japan, 31 August–4 September 2015; pp. 48–53. [Google Scholar]
- Rudovic, O.; Lee, J.; Dai, M.; Schuller, B.; Picard, R. Personalized machine learning for robot perception of affect and engagement in autism therapy. arXiv, 2018; arXiv:1802.01186. [Google Scholar]
- Taffoni, F.; Focaroli, V.; Formica, D.; Gugliemelli, E.; Keller, F.; Iverson, J. Sensor-based technology in the study of motor skills in infants at risk for ASD. In Proceedings of the 4th IEEE RAS EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob), Rome, Italy, 24–27 June 2012; pp. 1879–1883. [Google Scholar]
- Rivera, D.; García, A.; Alarcos, B.; Velasco, J.R.; Ortega, J.E.; Martínez-Yelmo, I. Smart toys designed for detecting developmental delays. Sensors 2016, 16, 1953. [Google Scholar] [CrossRef] [PubMed]
- Vega-Barbas, M.; Pau, I.; Ferreira, J.; Lebis, E.; Seoane, F. Utilizing smart textiles-enabled sensorized toy and playful interactions for assessment of psychomotor development on children. J. Sens. 2015, 2015. [Google Scholar] [CrossRef]
- Nunez, E.; Matsuda, S.; Hirokawa, M.; Yamamoto, J.; Suzuki, K. Paired robotic devices to mediate and represent social behaviors. In Proceedings of the 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2015), Kobe, Japan, 31 August–4 September 2015; pp. 722–727. [Google Scholar]
- Nuñez, E.; Matsuda, S.; Hirokawa, M.; Yamamoto, J.; Suzuki, K. An approach to facilitate turn-taking behavior with paired devices for children with Autism Spectrum Disorder. In Proceedings of the 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2016), New York, NY, USA, 26–31 August 2016; pp. 837–842. [Google Scholar]
- Matsuda, S.; Nunez, E.; Hirokawa, M.; Yamamoto, J.; Suzuki, K. Facilitating Social Play for Children with PDDs: Effects of Paired Robotic Devices. Front. Psychol. 2017, 8, 1029. [Google Scholar] [CrossRef] [PubMed]
- Suzuki, K.; Hashimoto, S. 1 bit Communication Network for Remote Interaction. In Proceedings of the HCI International 2005, Las Vegas, NV, USA, 22–27 July 2005. [Google Scholar]
- Suomela, J.; Ylikorpi, T. Ball-shaped robots: An historical overview and recent developments at TKK. In Field and Service Robotics; Springer: Berlin/Heidelberg, Germany, 2006; pp. 343–354. [Google Scholar]
- Salter, T.; Davey, N.; Michaud, F. Designing & developing QueBall, a robotic device for autism therapy. In Proceedings of the 23rd IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2014), Edinburgh, UK, 25–29 August 2014; pp. 574–579. [Google Scholar]
- Pierce, K.; Schreibman, L. Increasing complex social behaviors in children with autism: Effects of peer-implemented pivotal response training. J. Appl. Behav. Anal. 1995, 28, 285–295. [Google Scholar] [CrossRef] [PubMed]
- Pierce, K.; Schreibman, L. Multiple peer use of pivotal response training to increase social behaviors of classmates with autism: Results from trained and untrained peers. J. Appl. Behav. Anal. 1997, 30, 157–160. [Google Scholar] [CrossRef] [PubMed]
Participant | Chronological Age (Year; Month) | Developmental Quotient | PARS | CARS |
---|---|---|---|---|
P1 | 4; 7 | 43 | 46 | 44 |
P2 | 4; 5 | 39 | 21 | 36 |
P3 | 5; 8 | 70 | 26 | 32.5 |
P4 | 3; 8 | 44 | 21 | N/A |
© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Nunez, E.; Matsuda, S.; Hirokawa, M.; Yamamoto, J.; Suzuki, K. Effect of Sensory Feedback on Turn-Taking Using Paired Devices for Children with ASD. Multimodal Technol. Interact. 2018, 2, 61. https://doi.org/10.3390/mti2040061
Nunez E, Matsuda S, Hirokawa M, Yamamoto J, Suzuki K. Effect of Sensory Feedback on Turn-Taking Using Paired Devices for Children with ASD. Multimodal Technologies and Interaction. 2018; 2(4):61. https://doi.org/10.3390/mti2040061
Chicago/Turabian StyleNunez, Eleuda, Soichiro Matsuda, Masakazu Hirokawa, Junichi Yamamoto, and Kenji Suzuki. 2018. "Effect of Sensory Feedback on Turn-Taking Using Paired Devices for Children with ASD" Multimodal Technologies and Interaction 2, no. 4: 61. https://doi.org/10.3390/mti2040061