The Use of Social Robots in the Diagnosis of Autism in Preschool Children

Abstract

The present study contributes to the research problem of applying social robots in autism diagnosis. There is a common belief that existing diagnostic methods for autistic spectrum disorder are not effective. Advances in Human–Robot Interactions (HRI) provide potential new diagnostic methods based on interactive robots. We investigated deficits in turn-taking in preschool children by observing their interactions with the NAO robot during two games: (Dance with me vs. Touch me). We compared children’s interaction profiles with the robot (five autistic vs. five typically developing young children). Then, to investigate turn-taking deficits, we adopted a rating procedure to indicate differences between both groups of children based on an observational scale. A statistical analysis based on ratings of the children’s interactions with the NAO robot indicated that autistic children presented a deficient level of turn-taking behaviors. Our study provides evidence for the potential of designing and implementing an interactive dyadic game between a child and a social robot that can be used to detect turn-taking deficits based on objective measures. We also discuss our results in the context of existing studies and propose guidelines for a robotic-enabled autism diagnosis system.

1. Introduction

An increasing number of children are diagnosed with autistic spectrum disorders (ASD—the term used interchangeably with autism) globally [1]. Symptoms include deficits in the social behaviors that are needed to establish adequate relations and bonds with other people [1,2]. There is a large body of research on autism which demonstrates that integral to establishing social relations is the concept of turn-taking [1,3,4]. Typical forms of turn-taking behavior include an individual’s reactions to the actions of other people (e.g., response to a greeting, response to others’ questions, introducing oneself), imitating the actions of another individual and/or the ability to engage in the pretend play [1,2,3]. Researchers from various scientific disciplines have attempted to characterize the etiology, pathogenesis and symptoms of ASD, but these results are considered ambiguous [1]. It appears that the most common approach that explains the nuclear symptoms of autistic spectrum disorders emerges from the cognitive psychology domain. For instance, cognitive research [3] indicates that deficits in turn-taking may be indicative of lack of ability to build a proper “theory of the mind” of other people. Such a theory serves as a basis for conducting proper social interactions (in a typically developed population). Therefore, the observation of abnormal turn-taking patterns during ongoing interactions between a child and their parents, caregivers and other objects in the child’s social environment becomes crucial in assessing the presence of ASD.

The use of turn-taking deficits to indicate the presence of ASD is justified in a range of classical studies in the fields of psychology and pedagogy [2] and in the latest studies on the neurological basis of autism [5]. However, despite a large body of research on ASD diagnosis [6,7,8], relatively little effort has been made to standardize diagnostic instruments to differentiate autism (especially its mild form) from typical development [1]. Existing tools are based mainly on information from the parents (or caregivers) of autistic children. See, for instance, the most popular ASD diagnosis methods: ADI-R (Autism Diagnostic Interview–Revised) [9], M-CHAT (The Modified Checklist for Autism in Toddlers) [10], ADEC (Autism Detection in Early Childhood) [8], CARS-2 (The Childhood Autism Rating Scale) [11] and ASRS (Autism Spectrum Rating Scales) [12]. Since these tools are based on information from the parents and caregivers of autistic children [8,9,10,11,12], their reliability is limited. The exception is an approach such as ADOS (Autism Diagnostic Observation Schedule) [13], which is enriched with interactive protocols. Moreover, the effectiveness of ASD diagnosis is compromised by difficulties in differentiating autism from other developmental disorders (for instance, sensory integration disorder, dysphasia, minimal brain dysfunction, e.g., ADHD) [1]. In summary, the diagnosis of autism often employs a range of clinical tools complemented with long-term observation of the autistic child, often leading to ambiguous diagnostic outcomes.

Further development of ASD diagnostic tools can benefit from achievements in related research fields such as social robotics or Human–Robot Interactions (HRI) [14,15,16,17,18,19,20,21]. It is worth noting that in the area of robotics there are many initiatives concerning autism therapy. The majority of these are focused on the use of robots as a therapeutic intervention, but few address assessment. Representative examples include projects such as AuRoRa (http://aurora.herts.ac.uk/, accessed on 1 May 2022), IROMEC, DREAM (https://dream2020.github.io/DREAM/, accessed on 1 May 2022), DE-ENIGMA (https://de-enigma.eu/, accessed on 1 May 2022) and SARACEN (https://twitter.com/saracenrobot, accessed on 1 May 2022), as well as other works conducted by Dautenhahn and Werry [22], Robins et al. [23], Scassellati et al. [16], Diehl et al. [24] and Esteban et al. [25] and commercial initiatives such as AskNAO [26]. Importantly, the main implication of these works is that autistic children have been shown to be more involved in interactions with robots than with other people [16,22,27]. This in turn suggests that the interaction between an autistic child and a robot can serve as a basis for the development of an observational tool for the diagnosis of ASD. It can also act as therapeutic stimulation, where it has been highlighted that children playing with robots can have a developmental impact [28,29,30]. Other researchers have emphasized that the robotic environment can provide predictable and reliable situations in which the complexity of interactions can be controlled and gradually increased [29]. In addition, engagement of the child playing with a robot enables more complex behavior by engaging another child or adult in a triad. In this way, autistic children can gradually practice turn-taking behaviors and tackle their social deficits [27,31,32]. Scassellati et al. [16] clearly summarize how autistic children can benefit from interaction with social robots: (i) a robot can serve as a child’s instructor that demonstrates proper social behavior, which is guided via interaction with the child; (ii) a robot can respond in the manner of a toy that mediates social behavior between the child and other people; and (iii) a robot can act as an agent, thus allowing him/her to express emotions and/or desires.

The idea of using robots or other modern technology in the field of autism diagnosis first appeared in the literature more than a decade ago [14,15,16,17,19,24]. Scassellati [14] first outlined the idea of using social robots to address critical issues in autism diagnosis. This research is complemented by later work [15] on the field of social robotics, relevant technologies and the study of autism. These studies rely on passive sensing (i.e., without directly engaging in interactions, passive sensors record information on human user response such as gaze direction, focus of attention, position, vocal prosody). Scassellati et al. [16] comprehensively outlined several important aspects of applying robotics in autism research. The main conclusion of this work is that social robots could be used to diagnose autism due to their use in a range of diagnostic settings, constant behavioral evaluations and standardized stimuli. Petric [17] proposed a robot-assisted protocol based on the Autism Diagnostic Observation Schedule (ADOS) and adapted for use with the NAO robot; however, this researcher did not develop a reliable experimental environment and provided no further experimental evaluations of the proposed protocol. However, Ramirez-Duque et al. [19] made some progress and posited that early detection of ASD symptoms through behavioral observation in the natural environment may be improved by a protocol for Child–Robot Interaction (CRI) and technology for automated assessment of behavior. The proposed CRI protocol developed by Ramirez-Duque et al. [19] was inspired by a well-established joint attention protocol designed for computer vision, with a network of RGB-D (Red, Green, Blue, Depth) sensors for measuring children’s movements. Together, these studies suggest that the challenge of HRI-enabled autism diagnosis is ongoing, and no final technological solution is yet available. It is also worth noting that the availability of a stable technology, i.e., the construction of a social robot for diagnosing autism, is hampered by several technological constraints, in particular those related to the difficulty in designing specific sensors. In conclusion, the literature clearly shows that the challenges associated with the design of an HRI-enabled ASD diagnosis tool is of interest to several international groups of researchers and is dependent on the technology readiness level of social robots.

A ‘top-down’ approach to the robotic system design process was also applied by [14,15,17]. The approach adopted by these authors is as follows: First, a set of features to be detected by the robotic system for the purposes of autism diagnosis is identified. Next, user requirements are defined, along with the system specification. Alpha-version components of the system are then selected. Finally, a small-scale study is carried out to evaluate selected aspects of the final diagnostic system. However, none of these studies have ended up with a fully functioning diagnostic system for autism. Consequently, no satisfactory solutions exist due to the discrepancy between user requirements and currently available technology. Hence, in our research we apply the ‘bottom-up’ approach instead. In particular, we focus on primarily one symptom of autism, i.e., turn-taking deficits. On the basis of currently available technology, we have designed a dyadic game between an interactive robot and preschool children which aims to elicit a variety of turn-taking behaviors that are subsequently assessed by a specialist observer.

It is important to note that turn-taking has attracted considerable attention in recent research on robotics and autism therapy. For instance, Scassellati et al. [16] argue that the phenomenon of turn-taking is a central design issue in applying socially assistive robots (SAR) in autism research. This phenomenon has also been the focus of several other studies by Kose-Bagci [33], Brok and Barakova [32], Iacono et al. [34] and Kimand Clarke [35]. The latter article does not focus on the use of robots but suggests that tablet devices can be themselves effective tools for supporting turn-taking behaviors in children with autism. Kim and Clarke [35] conducted studies based on a virtual character implemented in the iTake Turns$\mathrm{\copyright }$ app, but this game has some potential to be ported to a robot such as NAO. In terms of using robots to diagnose turn-taking deficits, there is an important issue concerning the availability of existing robots that can diagnose interactive behaviors in conditions similar to those in the everyday social environment of a child. A variety of robots have been used in autism research. Scassellati et al. [16], Esteban et al. [25] and Wood et al. [36] provided a representative overview of well-known robotic technology and platforms for these purposes. There are also a number of initiatives which were not taken into account in the aforementioned works, such as those of [37,38]. There are only two humanoid robots in this group that are able to move around and are also commercially available: NAO (https://www.softbankrobotics.com/emea/en/nao, accessed on 1 May 2022) and ZENO (https://www.hansonrobotics.com/zeno/, accessed on 1 May 2022). NAO is an older design, but to date, it is the most widely used robot (see, e.g., Esteban et al. [25], Brok and Barakova [32], Tapus et al. [27], Bernardo [31]). For this reason, the NAO robot was chosen for use in this study.

From a robotic perspective, the research problem outlined in this study can be formulated as follows. It is possible to design and implement software applications that enable preschool children to play with a robot that elicits turn-taking in its various forms and allows a specialist observer to assess the developmental stage of the child’s competence on the basis of an objective measure (collected ratings). Thus, the underlying idea of the present research is that HRI offers the potential to differentiate interaction patterns of children with ASD vs. typically developing children when interacting and playing with a robot, which can be used diagnostically.

Our specific research questions can be summarized as follows:

Q1:

Are there significant differences between turn-taking behaviors in neurotypical children compared with children with autism when interacting with a social robot?

Q2:

Can human–robot social interactions differentiate patterns of interest and motoric behaviors in children with a diagnosis of autism?

2. Study Design

2.1. The NAO Robot and the Interactive Games Designed for the Study

NAO is a humanoid robot that is the size of a large doll; it has extensive capabilities in terms of perception, mobility, behavioral expression and human-oriented communication. The software interface of the NAO platform enables the implementation of complex human–robot interaction (HRI) scenarios that include verbal and non-verbal communication, human recognition and tracking, visual contact and choreography of movements [39].

It is important to note that when investigating the potential of applying social robots in ASD diagnosis (described in the second last paragraph of the Introduction), the specific features of the robot that will be used are of importance. These important features include the way the robot behaves, communicates, perceives objects and expresses its states. These features should match the child’s capabilities, sensitivities and individual preferences. To the best of our knowledge, despite several studies on social robots and autism therapy, there is no off-the-shelf robotic technology that matches the present research problem well.

However, a good starting point may be the ASK NAO software [26]. It is important to note that the ASK NAO software was developed for specific user requirements that do not consider autism diagnosis. Thus, for the purpose of the study, we have developed dedicated software applications for NAO. Here, we discuss users’ requirements for implementing interactive scenarios for both games on the NAO platform (see below).

Taking into account the requirements of the diagnosis of autism, as well as the implementation requirements for the perceptual and expressive capabilities of NAO, we have proposed detailed scenarios of an interactive game for preschool children that facilitate the differentiation of turn-taking behaviors between autistic children and typically developing children. The scenario was developed over two iterations, each containing three phases: design, implementation and evaluation [40]. An additional phase was used to adjust the scenario and software parameters to the specific experimental conditions associated with children experiencing symptoms of ASD; this phase was previously reported in the pilot study [41].

In terms of the finite-state machine (FSM) shown in Figure 1, the interaction scenario included three main states: conversation, playing Touch me and playing Dance with me. In the state called conversation, the robot sits on the floor, initiates verbal communication with a child (in particular, it introduces itself), invites the child to play and asks him/her to choose one of the two available games. In the state called Dance with Me, the robot performs a choreographed dance to a piece of music entitled Rainy Day Games by the Green Orbs band. The melody is rhythmic, but not very fast. The choreography involves all the parts of the robot’s body, where the complexity of its movements increases gradually, and individual parts of the body are activated alternately. First, the arms of the robot are raised to the side alternately, then the movement is formed of a combination of squats and synchronous sideways movement of the arms; this is followed by alternating movements of the arms up and down, alternating with forward and backward movements; sideways movements of the hips follow, then back to the squats and synchronous sideways movements of the arms. Then there is alternating bending of the elbow joints with the arms extending outwards, rocking of the hips and finally swinging of upright arms. Before returning to the conversation state, NAO bows and thanks the child for dancing with it. In the state called Touch me, NAO first sits on the floor and asks the child to touch one of its body parts: the head, the palm or the leg. If the child reacts correctly, the robot rewards the child with a verbal applause (by saying bravo). The correctness of a child’s response is confirmed by raising a hand. In the case of an incorrect reaction, the robot informs the child of this error and encourages him/her to try again. After a correct response or after three attempts, the robot asks the child about his/her willingness to continue playing. The robot maintains visual contact with the child throughout the course of the game. Note that the scenarios specified above can be used for applications other than eliciting turn-taking from a child, e.g., developing the psychomotor coordination of a child or teaching them about the parts of the body.

The proposed interaction scenarios were designed to activate interest of the child with ASD into the moving object and tailor needs of the diagnostician and technological capabilities of NAO. To do so, we adopted the theoretical perspective of designing HRI with a social robot outlined by Scasselati in [16] indicating three recommendations how children with autism can strongly benefit from such interactions (see Section 1). We assumed that the Dance with me scenario fits the first recommendation (a robot can serve as a child’s instructor that demonstrates proper social behaviour) while the Touch me scenario follows the second recommendation (a robot can respond in the manner of a toy that mediates social behaviour between the child and other people. It is worth noting that if a child touches a robot during the Dance with me scenario, this behavior cannot be classified as a response to the robot’s voice command. If at all, this is a phase of some kind of imitation process. On the other hand, the Touch me scenario precludes the emergence of spontaneous child reactions because the NAO robot in this case was not moving.

2.2. Participants

Ten children aged $5.5$ to $6.5$ years ($M=6.08$, $SD=0.4$) took part in the study. Five children (four boys and one girl) from the control group exhibited typical developmental patterns. The other five children were diagnosed with autism (also four boys and one girl), as indicated by kindergarten records. The diagnoses were delivered by a team of specialists consisting of a psychiatrist, a psychologist and a pedagogue. The study was carried out in the kindergartens attended by the participating children and was approved by the local committee for research ethics at the University of Lower Silesia. Before undertaking the study, the legal guardians of the children were acquainted with the research aims and procedures used in the study, and they signed consent forms for the participation of the children in the experiment and the filming of the study.

2.3. Procedure and Materials

The experimental procedure consisted of two sessions: familiarization with NAO and the main experiment. In the familiarization session, NAO introduces itself to the child and performs two short, rhythmic dances; this takes about 5 minutes. The main experiment took place after a few minutes’ break; it began with the Touch me game, followed by the Dance With Me game. Both games lasted approximately 5 minutes for each child. After each experiment with a child, there was a break that lasted several minutes, then the experiment was repeated with another child. During each visit to a given kindergarten, a maximum of three children participated in the experiment. A photo of an ongoing game from the experiment is shown in Figure 2.

The main experiment was carried out in a kindergarten room. All potentially distracting elements were removed. Apart from the child–NAO pair, a teacher was present who assisted the child and took care of his/her safety during the experimental session. The teacher occasionally intervened when needed. An experimenter was also present who supervised the course of the experiment remotely using the standard NAO GUI. The interactions between the robot and the children were recorded using a video camera; these recordings were then subjected to detailed analysis by a team of competent raters working in the special education field. These professionals evaluated the video-recorded material by searching for turn-taking profiles (autistic vs. neurologically typical ones) based on the predefined, observable indicators included in the observational scale (see

Table A1. The Observation Scale of Child’s behaviour in Interaction with a Robot.

The Child’s ID		Competent Rater’s ID
	Instruction: In the part The interaction with the robot Please focus on the child’s activity while playing with the robot. Please provide “a precise number of all observed movement activities of the child”: (i) imitating the gestures and movements of the robot; (ii) touching the robot on command. The quality of the imitation or the quality of the instructions performed does not matter. Reactions of a child when supported by a teacher should also be approved if the child initiated them. In the Interest in the robot and Additional Movements sections, the child’s different reactions while playing with the robot should be assessed globally and the intensity/frequency of these reactions should be indicated on a scale.
I Interaction with the robot
1	Movement activity of the child during the Touch me game
	Count each movement activity initiated by a child at the robot’s command and ending with a touch of the robot.
	Give the number of all movement activities initiated by the child at the robot’s command, ending with a touching the robot.
	$N_{ch}^{\mathrm{touch}\mathrm{me}}=$
2	Imitation reactions
	Count each child’s response that imitates the robot’s movements during its dance
	Give the total number of the child’s responses that imitate the robot’s movements during the dance.
	$N_{ch}^{\mathrm{dance}\mathrm{with}\mathrm{me}}=$
II Interest in the robot
(0-none, 1-sporadic, 2-average, 3-frequent, 4-very intense, 5-constant)
3	Visual contact	0	1	2	3	4	5
	The child looks towards the robot or into the eyes of the robot
4	Motor activity	0	1	2	3	4	5
	The child shows movement activity in the presence of the robot: e.g., approaches the robot, reduces the distance to the robot, approaches the robot with the teacher.
5	Expression of gestures	0	1	2	3	4	5
	The child uses gestures to express emotions while playing with the robot: e.g., points to the robot, picks up the robot.
6	Spontaneous expression	0	1	2	3	4	5
	The child speaks to the robot (e.g., ‘Hello, robot.’). The child speaks about the robot (e.g., ‘It looks at me.’)
7	Emotional reactions	0	1	2	3	4	5
	The child smiles at the robot, can laugh spontaneously in the presence of the robot.
III Additional movements
(0-none, 1-sporadic, 2-average, 3-frequent, 4-very intense, 5-constant)
8	Stereotypical movements	0	1	2	3	4	5
	The child jumps and/or spins in circles.
9	Compulsions	0	1	2	3	4	5
	The child grabs his clothes and twists them around. The child grabs his or her hair and twists it.
10	Movement mannerisms	0	1	2	3	4	5
	The child waves or flaps his arms; clapping.

in the Appendix A). Subsequently, statistical analyses of the collected ratings data began in order to examine whether the turn-taking profiles of the autistic children that had been presented during interaction with the robot differed from the profiles of typically developing children.

2.4. Psychological Assessment of Children’s Behavior from Video Material: The Procedure Used by the Competent Raters

As discussed, this research involved a rating procedure. The age of the competent judges (six women) ranged from 25 to 64 years ($M=41$, $SD=16.4$). They all came from the education field and worked with children with developmental dysfunctions on a daily basis. None of the competent raters worked in the kindergartens where the experiment took place; moreover, they had no information about the diagnoses of the children who participated in this study.

The rating procedure used The Observation Scale of Child’s behaviour in Interaction with a Robot, which is shown in Table A1. The scale measured turn-taking behavior indicators in both games with the robot as well as the level of the children’s interest in the game with the NAO robot (for instance, see Goldstein and Ozonoff [42], Frith [1], Flether [3]) and the presence of motor behavior specific to the ASD population, e.g., movement mannerisms (see DSM-5 [43], Goldstein and Ozonoff [42], Schopler and Mesibov [2], Eigsti et al. [5], Frith [1]). We developed this scale for the purposes of this project since no standardized questionnaire was available for such an experimental setup. Development of this scale was based on the literature: mainly on the ASD criteria specified in a book based on the DSM V by Goldstein and Ozonoff [42] and other works on autism such as Wing et al. [44], Marvin et al. [45], Watson et al. [46], Fletcher et al. [3], Schopler and Mesibov [2], Egisti et al. [5] and Frith [1]. In the previous research of Arent et al. [41], as in here, the scale delivered results that are replicable and justified according to current knowledge in the autism field. The long-term goal of the project is to establish an interactive method of HRI-enabled diagnosis of autism. Hence, in the future, the scale might be further developed as the project methodology itself continues to advance.

The rating procedure began with the instruction of the observational raters. They were given The Observation Scale of Child’s behavior in Interaction with a Robot, and the experimenter verbally explained the scale content and the method of completing it. The experimenter introduced the scale to the raters and emphasized the most important points. The raters watched films of the interactions of the children and the robot. While watching the movie, the competent raters evaluated the children’s behavior with The Observation Scale of Child’s behaviour in Interaction with a Robot. For each film, the competent raters used a separate scale for each child to evaluate turn-taking behavior in the course of the interaction with the robot.

With regard to both measurable indicators of turn-taking (see the point 1 of the Observation Scale in Table A1), while watching the films, the raters counted the presence of turn-taking behaviors presented by children in a given moment. In particular, in the Touch Me game, it was assumed that turn-taking was manifested when the child properly followed the robot’s instructions to touch its “body”; turn-taking when dancing with the robot was revealed by a child’s behavioral imitation of the robot’s movements. Thus, the numerical indicator, $t{r}_{\mathrm{touch}\mathrm{me}}$, of turn-taking in the Touch me game was the ratio parameter of touches of a robot’s body performed by a child and counted by the raters, denoted by $N_{ch}^{\mathrm{touch}\mathrm{me}}$, divided by the total number of Touch Me commands given by the robot to the child, denoted by $N_r^{\mathrm{touch}\mathrm{me}}$ for each child:

$$t{r}_{\mathrm{touch}\mathrm{me}}=\frac{N_{ch}^{\mathrm{touch}\mathrm{me}}}{N_r^{\mathrm{touch}\mathrm{me}}}.$$

(1)

When the child’s attempts to touch the robot were confirmed by the robot’s voice, this was counted as a manifestation of turn-taking behavior. For some children, $t{r}_{\mathrm{touch}\mathrm{me}}>1$, because the child could follow a single command from the robot more than once. It is a consequence of the robot’s voice command–child’s reaction–robot’s confirmation procedure discussed in Section 2.1. Due to imperfect sensors or inappropriate touch, the child sometimes has to make several attempts to receive confirmation from the robot. In other words, repeated touching was due to a lack of immediate response from the robot and was intended to elicit such a response (a turn-taking). This particular interaction is characterized by an effort to maintain contact with the robot for a longer period of time, which is cyclic in nature. The turn-taking lasts for an extended period of time, and therefore it cannot be associated with a single measurement. The child touched the robot until the robot’s visible reaction confirmed the execution of the command. Thus, when a child sought contact with the robot, this demonstrated the phenomenon of turn-taking (i.e., the robot gives a command, the child executes it and the robot confirms the execution of the command). Given the advances in robotic technology, the problem of limited tactile sensors in NAO will be eliminated in newer versions of NAO or its successors. We believe that behavioral outcomes of such technological constraint as it stands can be neglected because the child–robot interaction itself was not affected and distorted while the Touch me scenario was in play. The indicator of turn-taking in the Dance with me game, which is denoted by the ratio parameter $t{r}_{\mathrm{dance}\mathrm{with}\mathrm{me}}$, is equal to the ratio of the number of imitational behaviors of the robot’s movements (performed by the child and counted by the rater), denoted by $N_{ch}^{\mathrm{dance}\mathrm{with}\mathrm{me}}$ (dance with me), and the total number of the robot’s movements during the dance that could potentially be imitated by the child, denoted by $N_r^{\mathrm{dance}\mathrm{with}\mathrm{me}}$ (dance with me). Thus, the ratio was the following:

$$t{r}_{\mathrm{dance}\mathrm{with}\mathrm{me}}=\frac{N_{ch}^{\mathrm{dance}\mathrm{with}\mathrm{me}}}{N_r^{\mathrm{dance}\mathrm{with}\mathrm{me}}}.$$

(2)

We assumed that both types of interactions that were presented in games between the robot and children would evoke turn-taking behaviors. As indicated in the literature, typical forms of turn-taking behavior are an individual’s reactions to the actions of other people (e.g., response to a greeting, response to others’ questions, introducing oneself), imitation of the actions of another individual and/or the ability to engage in a pretend game [1,2,3]. Thus, one should expect that while interacting with NAO, children with ASD will show deficits of social reciprocity in a situation in which they play a game with the robot. Since the interaction between the child and the robot unfolds over time, one would expect that deficits in contingent reactions in the game with NAO are diagnostic for children from the ASD population. Thus, abnormal turn-taking patterns that emerged during an ongoing interaction between a child and the social robot were operationalized as significant deviations of the value of $t{r}_{\mathrm{dance}\mathrm{with}\mathrm{me}}$ from 1.

It should be noted that deviations of $t{r}_{\mathrm{dance}\mathrm{with}\mathrm{me}}$ from 1 can be influenced by the degree of familiarity of a child with the robot. In the case of the Touch me game, children always know what the robot expects them to do, because the robot instructs them verbally and the expected behaviors are simple. In the case of the Dance with me game, children first need to become used to the choreography (see Section 2.1) and then to learn the suitable behaviors allowing them to imitate parts of choreography in response to the robot’s dance. Consequently, the values of $t{r}_{\mathrm{dance}\mathrm{with}\mathrm{me}}$ are lower than $t{r}_{\mathrm{touch}\mathrm{me}}$ relative to expected for both typical and autistic children during a learning phase.

Since there was a possibility that the initial ASD diagnoses might be misleading, we decided to check the diagnosis of autism by measuring the child’s interest in the robot and additional movements. We expected autistic children to be more interested than typical children in the robot due to the increased focus of autistic individuals on inanimate objects as compared to the attention they pay to human objects [1,2]. Regarding additional movements, they are common in autism [1,2,5].

The raters evaluated the children’s interest in the robot using a Likert-type scale (see point 2 of the Observation Scale included in Table A1). The Interest in the robot subscale consisted of the following items: Visual contact, Motor activity, Expression of gestures, Spontaneous expression and Emotional reactions. Each item is defined in detail in the scale shown in Appendix A. The raters referred to each of these items by indicating a suitable value from 0 to 5 on the scale associated with the considered item. The Interest in the robot subscale showed satisfactory reliability, $\alpha =0.66$. The answers in this subscale were summed to produce the overall result of the subscale.

With reference to additional movements, the competent raters also used an appropriate subscale in the Observation Scale, shown in Table A1 ( point 3). The Additional movements subscale consisted of Stereotypical movements, Compulsions and Movement mannerisms. Each of these positions is defined in detail in the form shown in the Appendix. For each of these positions, the competent raters chose an appropriate number from 0 to 5 to assess the severity of symptoms. The Additional movements subscale had satisfactory reliability, $\alpha =0.77$.

The inter-rater reliability of the ratings for the whole scale, as analyzed using Kendall’s W coefficient, was at the satisfactory level, $W=0.65$, $p<0.001$.

3. Results

Our analysis compared the results for autistic and typically developing children based on the collected ratings provided by raters. For comparison, the grouping variable was the diagnosis of the child (typically developing vs. autistic). First, we started with the effects of the presence of ASD symptoms during the interaction with the robot by analyzing the measures of the child’s Interest in the robot, and Additional movements. Then, we tested the turn-taking profiles in both populations by inspecting numerical indicators of turn-taking, $t{r}_{\mathrm{dance}\mathrm{with}\mathrm{me}}$ and $t{r}_{\mathrm{touch}\mathrm{me}}$, which are based on non-parametrical statistics (The Mann–Whitney, Kruskal–Wallis and ${\chi }^2$ test statistics).

3.1. Non-Parametric Analysis of Turn-Taking in the Touch Me and Dance-with-Me Games

We performed a non-parametric test (Kruskal–Wallis) to analyze each competent rater across the Touch-me and Dance-with-me conditions. We found significant differences for all competent raters (six individuals), yielding a higher turn-taking ratio for the Touch me condition than for the Dance with me condition. The analysis resulted in the six significant differences that are presented in

Table 1. Non-parametric (Kruskal–Wallis) test for turn-taking ratios by rater, regardless of diagnosis.

Rater No	Me (Touch me)	Me (Dance with me)	H(2)	p
1	1.056	0.005	13.377	0.0003
2	1.000	0.066	14.925	0.0001
3	0.955	0.041	14.004	0.0002
4	0.955	0.036	14.560	0.0001
5	1.100	0.020	13.418	0.0002
6	0.838	0.071	13.866	0.0002

. Note that the values included in Table 1 are consistent with the expectations concerning $t{r}_{\mathrm{touch}\mathrm{me}}$ and $t{r}_{\mathrm{dance}\mathrm{with}\mathrm{me}}$ discussed in Section 2.4.

The results presented to this point (in particular Table 1) relate to the raters’ consistency. The next results (

Table 2. The amount of autistic and typically developing children who did, or did not, perform the Touch me commands that were given by the robot.

	Child’s Diagnosis
	Typical Development	Autism
commands executed	29	7
commands not executed	1	23

,

Table 3. The amount of autistic and typically developing children who did, or did not, imitate the robot’s movements in the Dance with me game.

	Child’s Diagnosis
	Typically Developing	Autism
commands executed	23	7
commands not executed	7	23

and

Table 4. Results of non-parametric (Mann–Whitney) comparison of autistic vs. typically developing children in the additional measures, i.e., Interest in the robot and Additional movements.

Measure	Diagnosis	Me	Comparison Results
Interest in the robot	Autism	13	Mann–Whitney $U=220.0$, $p<0.001$
	Typically developing	7
Additional movements	Autism	4	Mann–Whitney $U=167.5$, $p<0.001$
	Typical development	0

) directly address the research questions [Q1] and [Q2] from Section 1.

Next, we ran ${\chi }^2$ test statistics in order to compare turn-taking behavior between both populations in the Touch me condition.

For this test, the results of $t{r}_{\mathrm{touch}\mathrm{me}}$ were divided into two categories based on the median value ($Me=1$). This division indicated that among 30 observations of typically developing children (i.e., 6 raters observing 5 children), there were 29 observations of children who performed the robot’s commands and 1 observation of a child who did not. Regarding autistic children, this division indicated that among 30 observations of these children (i.e., 6 raters observing 5 toddlers), there were 7 observations of them performing the robot’s commands and 23 observations of children who did not. The results of this division are presented in Table 2. The ${\chi }^2$ test revealed that the results of typically developing children were significantly higher than those of autistic children, ${\chi }^2(\mathrm{df}=1)=33.61$, $p<0.001$.

Subsequently, the ${\chi }^2$ test was performed for the Dance with me game. The results of $t{r}_{\mathrm{dance}\mathrm{with}\mathrm{me}}$ were divided into two categories based on the median value ($Me=0.036$). This division indicated that in 30 observations of typically developing children (i.e., observations of 5 children, carried out by 6 raters), there were 23 observations of children who imitated the robot’s movements and 7 observations of children who did not. With regard to children with ASD, this indicated that in 30 observations (i.e., 5 children and 6 raters) there were 7 observations of children who imitated the robot’s movements and 23 observations of children who did not. The effects of this division are shown in Table 3. The ${\chi }^2$ test also showed that the results from typically developing children were significantly higher than those of autistic children, ${\chi }^2(\mathrm{df}=1)=17.07$, $p<0.001$.

3.2. Observational Measures of ASD Symptoms, i.e., Children’s Interest in the Robot and Additional Movements

• Interest in the robot subscale. The Mann–Whitney test for non-parametric analysis of the collected ratings showed that interest in the robot for autistic children ($Me=13$) was higher than presented by typically developing children ($Me=7$). The distributions in the two groups differed significantly (Mann–Whitney $U=220.0$, $n1=n2=30$ (5 children times 6 competent raters), $p<0.001$).
• Additional movements subscale. Further analysis of the presence of ASD symptoms based on the same test indicated that autistic children expressed more additional movements ($Me=4$) than typically developing children ($Me=0$) during interaction with NAO. The distributions in the two groups differed significantly (Mann–Whitney $U=167.5$, $n1=n2=30$, $p<0.001$).

The above results regarding Interest in the robot and Additional movements are also included in Table 4.

These results support the initial ASD diagnoses by showing that autistic symptoms are present in children diagnosed with ASD.

Selected detailed statistics, complementing the results from this section can be found in Appendix B.

4. Discussion

The present study was designed to address whether using interaction between autistic children and the NAO robot could be used to develop a diagnosis tool to screen ASD symptoms in preschool children, including behaviors such as turn-taking. Under the conditions of two programmed scenarios for two game contexts (the Touch me and Dance with me games) which induce turn-taking behaviors, it was found that children with ASD produced significantly less turn-taking behaviors than children with a typical developmental pattern [Q1]. This supports the suggestion that assessing turn-taking plays a valuable role in diagnosing autism and that the interaction between an autistic child and a robot can provide predictable and reliable situations in which the complexity of interactions can be an efficiently observed.

Addressing the second research question [Q2], it was found that autistic children were rated as expressing more interest in the robot and more additional movements than typically developing children during interactions with the robot. These results have face validity when considering the characteristics of autism that have been observed in other studies. In the Touch me game, children were asked to interact closely with the robot (e.g., touch the robot’s body parts) in a completely new situation. As opposed to the general population, individuals with autism are usually severely aroused in new situations [47,48]. This state of arousal can also intensify when children find themselves in an unexpected situation. As arousal emerges, additional movements (e.g., mannerisms or stereotypical movements) are produced by individuals with autism; these movements are characteristic of children with ASD and are far less frequently displayed by typically developing children. If they do occur among typically developing children, they are less intense (DSM V, 2013). In addition, children with ASD may experience ambivalent tendencies [47] (together with anxiety), which may result in paradoxically high interest in the robot. High interest in inanimate objects also exemplifies well-known observations related to the preference of children with ASD for objects over people [1]. The aforementioned anxiety and ambivalent tendencies are also plausible causes of the reduced turn-taking that was observed among the autistic population in the Touch me and Dance with me games.

The robot environment, in particular, the Dance with me game, allows the diagnostic evaluation not only of imitation but also of other ASD symptoms, such as visual motor coordination, which is dependent on multisensory integration [48]. Properly functioning sensory integration forms the basis of visual motor coordination, motor planning and praxis, i.e., phenomena that are disturbed in autistic children. In turn, this imitation, which is weakened in ASD individuals, is often related to their difficulties in dividing, sharing and focusing attention. Additionally, results from the additional subscales confirmed the presence of other ASD symptoms. In particular, there was greater interest in the robot among the autistic children compared to the typically developing children. This is additional evidence for the strong focus on inanimate objects [1] (e.g., on the robot) that is characteristic of autism. Additionally, analysis of the movements presented by autistic children in the games indicated a substantial increase in additional movements, such as mannerisms or stereotypical movements. Additional movements are also symptoms of autism [1,2,5]. Thus, the behavioral patterns observed in the collected video material revealed symptoms characteristic of autism. The observation of increased interest in the robot and a high degree of additional movements can further feed the development of a HRI-enabled diagnostic system that supports the differentiation of additional autism symptoms (apart from a basic focus on turn-taking in HRI). In summary, these results support the proposition that social robots can be used in the diagnosis of autism by eliciting and subsequent analysis of a variety of stereotypical behaviors.

5. Limitations and Further Directions of Study

The findings of the present study support the potential of using social robots in the diagnosis of autism and justifies the continuation of activities in this area as the relevant technologies progress. Currently, NAO with the proposed software applications can be treated as a supplementary tool, thanks to which specialists can acquire additional diagnostic intuition to be used with standard tools.

The experience gained in this experimental study led us to the conclusion that the games we used for interactive diagnosis of turn-taking deficits need to be improved and further developed. This should be achieved by having close and extensive cooperation with the target users, including children, psychiatrists/psychologists and therapists. The ultimate goal is to identify a set of game scenarios that would be attractive for children, useful for specialists and technically feasible for engineers.

First, automation of the robot-enabled diagnostic system discussed in this work is necessary to deliver an efficient tool for specialists that guarantees high objectivity of the diagnosis process. Recent progress in RGB-D vision and studies on deep learning suggest that it should be possible to develop a system that is capable of detecting and recording a child’s spontaneous turn-taking behavior with a level of quality that is acceptable for diagnosis. The flexibility of the NAO SDK, in conjunction with a wide variety of tactile sensors, provides the possibility of developing a reliable sensing system for commanded turn-taking behavior. Development of eye-tracking systems presents a good basis for the detection of children’s attention. Finally, it should be kept in mind that we are developing a component of a potential system that supports autism diagnosis but for which for no protocol has yet been defined. This implies the need for further research from the fields of robotics and psychology to implement our proposal (a component of the target diagnostic system).

The work should culminate in a system based on artificial intelligence methods, which, on the basis of the knowledge acquired from specialists, competent judges and current observations, will enable automatic diagnosis with high reliability.

6. Conclusions

Our results allow us to conclude that the robot NAO with the proposed software applications (or some updated and extended version of this robot with newer applications) can be used to detect turn-taking deficits in children when observing them playing with NAO. If a specialist has a sense of how a typical child plays with NAO, then they will be able to see possible deficits in the turn-taking of a child with autism by observing him/her play with NAO (through a simple differentiation).