Important Preliminary Insights for Designing Successful Communication between a Robotic Learning Assistant and Children with Autism Spectrum Disorder in Germany

Abstract

Early therapeutic intervention programs help children diagnosed with Autism Spectrum Disorder (ASD) to improve their socio-emotional and functional skills. To relieve the children’s caregivers while ensuring that the children are adequately supported in their training exercises, new technologies may offer suitable solutions. This study investigates the potential of a robotic learning assistant which is planned to monitor the children’s state of engagement and to intervene with appropriate motivational nudges when necessary. To analyze stakeholder requirements, interviews with parents as well as therapists of children with ASD were conducted. Besides a general positive attitude towards the usage of new technologies, we received some important insights for the design of the robot and its interaction with the children. One strongly accentuated aspect was the robot’s adequate and context-specific communication behavior, which we plan to address via an AI-based engagement detection system. Further aspects comprise for instance customizability, adaptability, and variability of the robot’s behavior, which should further be not too distracting while still being highly predictable.

1. Introduction

Children diagnosed with Autism Spectrum Disorder (ASD) experience a variety of behavioral as well as social challenges in their daily lives [1,2,3,4]. This is particularly true for children with no or poor verbal communication skills. Frequently, they are struggling with the processing of their own as well as others’ emotions and consequently show dysfunctional or unexpected behavior [5]. These children often lack the ability to act appropriately in everyday social situations [5], which results in difficulties in social relationships and vocational or academic environments [6].

However, by means of planned therapeutic intervention programs, the children’s empathic capacities as well as their (socially) appropriate behavior can be facilitated [7,8]. There is a widespread consensus that behavior-based autism-specific interventions, using the principles of human learning, are highly effective [9,10,11]. A behavior-based intervention aims to improve behavioral and/or functional skills as well as child development using strategies of behavior therapy or social skill training [12]. This, for instance, can consist of play-based strategies to increase joint attention, engagement, and reciprocal communication [9]. Although autistic traits last a lifetime, the children can be empowered to improve their socio-emotional and functional skills substantially [6,13,14]. Particularly when the intervention occurs early, negative effects of the disorder can be reduced tremendously and the quality of the affected persons’ lives can be improved [15,16,17,18]. In sum, the benefits of therapeutic interventions are clearly visible. This benefits not only the children themselves but also their families and caregivers: effective interventions provide parents with more self-efficacy, reduce stress levels, and lead to an upward cycle of the children’s behavioral skills and inner-family relations [19].

The therapeutic exercises need to be repeated frequently, which demands much time and effort from the children’s therapists, parents, and other caregivers. Since children with autism are easily distracted and have troubles focusing on the exercises by themselves [20], they need to be accompanied by someone with full attention during those training sessions. Caregivers need to monitor disruptive emotions or behaviors and intervene when necessary. Excellent skills of observation and analysis of behavior are crucial. Altogether, this results in a great need and therefore high demand for well-founded support [21]. Due to, inter alia, the highly time-consuming and simultaneously highly specific nature of the interventions, there are not enough therapists available to satisfy this demand [22,23,24]. This was observed to be particularly the case in Germany, with a heightened intensity in rural areas with a lower density of specialists [25].

By improving and expanding therapeutic options, this deficient situation can be alleviated. With the aim to lift some weight of the therapists’ and affected parents’ shoulders, we investigate the potential of a robotic learning assistant to accompany and support the children while they execute various therapeutic exercises to improve their socio-emotional and functional skills (see Figure 1 for the goals, propositions, and benefits of the research approach). The robot is planned to be equipped with an AI-based state detection system to monitor whether the child is still focused on the exercise and to intervene in case the focus drifts. This way, the state detection system enhances the robot’s communication behavior and enables more effective, motivating interventions. To promote the success of employing a robotic learning assistant as described in German households, an extensive requirement analysis with relevant stakeholders is needed. Against this background, interviews with parents and therapists of children with ASD were conducted in Germany. While the parents can give in-depth insights regarding their children’s specific behaviors and needs, the therapists offer broader assessments of the affected children’s general behaviors and needs when training their socio-emotional and functional skills. The goal was to consider the stakeholders’ expectations, needs, concerns, and suggestions regarding the usage of this new technology in the context of psychotherapeutic ASD treatments. Ultimately, we aimed to receive valuable insights with regard to feasibility, usefulness, and effectiveness for a successful technological development of a robotic learning assistant and its interaction behavior. Although there is already extensive international research on the deployment of robots in the ASD field (as presented in the following), the question remains as to how these insights apply to persons affected in Germany. For instance, there is pronounced skepticism towards new technologies within Germany in particular [26]. Therefore, the results will be considered in the context of previous research concerning the employment of robots in ASD treatment from other countries (e.g., UK, USA, Italy, Japan) in order to investigate similarities and differences.

1.1. Challenges for Children with Autism and Their Caregivers

Autism is identified as ‘spectrum disorder’ since the symptoms can occur in a wide variety of combinations with different levels of severity resulting in a wide range of abilities and patterns of deficits [27]. ASD comprises a broad set of symptoms, of which some occur more frequently than others. In general, ASD is commonly associated with impairments in social communication, interaction, as well as imagination, and is often accompanied by repetitive behavior patterns [28,29,30]. The Fifth Edition of the American Psychiatric Association’s Diagnostic and Statistical Manual (DSM-5) provides standardized criteria to diagnose ASD [28]. According to the manual, people with ASD typically display communication deficits which can manifest in inappropriate responses during conversations, misunderstandings of nonverbal interactions, and/or difficulties in establishing friendships in their own age groups [28]. These socio-cognitive impairments hinder children with ASD from appropriately communicating with their social environment and from actively taking part in society [23]. Moreover, people with ASD are observed to cling heavily to routines, to react particularly sensitively to changes in their environment, and/or to focus intensely on inappropriate items [28]. ASD not only affects social behavior and communication skills, but also other functional aspects such as sensory responsiveness, play, and motor activity [23,27]. In many cases, children with ASD further suffer from learning disabilities which is connected to their general developmental delay [23,31,32]. Researchers argue that children with ASD typically engage less with their environment, particularly their social environment, which is a substantial component for learning, practicing new skills, and acquiring knowledge [31]. Missing these important opportunities for learning has detrimental consequences for both, their cognitive and social development.

Adding to these challenges, ASD often occurs together with attention deficit/hyperactivity disorders [20]. More specifically, hyperactivity, instability, and low attention spans are observed in children with ASD, particularly in younger children [33]. As a consequence, this contributes to their social deficiencies and causes comprehension problems with training exercises [33]. Therefore, in most cases a trained person with full attention needs to be present during ASD training sessions to monitor the children’s attention and to intervene whenever additional motivation or a reminder to finish a task are needed [21]. A great amount of time but also cognitive and mental resources are needed to adequately support a child with ASD during those learning sessions. Due to a lack of specialized services, this often needs to be done by the parents which adds to the families’ burdens [34]. Moreover, children with ASD also struggle with generalization, i.e., transferring acquired capabilities to other situations or contexts [23]. Exercises therefore need to be repeated frequently in various everyday situations which cannot all be covered in the therapy sessions. The training of social-interactive, emotional, and practical life skills therefore often takes place outside the therapy context in everyday real-life situations [23]. This demands a high amount of time as well as constant focus (i.e., cognitive, and mental resources) by the children’s caregivers which monitor the children’s attention and intervene whenever additional motivation or reminders are needed. Summing up, different learning methods need to be applied to children with ASD to reduce the deficits they are experiencing. Foremost, a high amount of support and focus as well as a strategic therapeutic approach are needed [23].

However, this high need for adequate therapeutic support is often not satisfied, at least not in a timely fashion [22,23,24]. Therefore, another challenge which caregivers of children with ASD face is a general lack of available child and youth therapists and low numbers of institutions and practices specialized in ASD [23,35,36]. This deficiency was observed in Germany with particular severity in rural areas with a lower density of specialists [25]. Even where autism care centers are available, there is a lack of common standards for expert qualification as well as for the methods delivered [23]. Additionally, access to behavior-based autism-specific interventions is often hampered and delayed by a lack of consistent jurisdictional regulation among funding agencies [37]. From an international perspective, Europe is still far behind the service quality and availability of the United States [38]. The waiting time can be several years, resulting in various negative consequences for the children’s development since early interventions are of the essence [15,16,36]. Consequently, technological solutions are highly needed to reduce the described gaps and deficits in the therapeutic care of children with ASD and to relieve the children’s caregivers [39].

1.2. Using (New) Technologies to Support Children with Autism

To address the described care gap problem and to support children with ASD and their caregivers, new technologies can offer manifold solutions as a variety of research projects show. Children affected by ASD appear to be more comfortable and relaxed when interacting with digital learning tools, if they offer an appropriate and appealing user interface design [40]. For instance, a portable learning platform for early interventions was found to be an effective tool to enhance certain skills of children with ASD such as receptive and expressive linguistic abilities [24]. Likewise, the use of a virtual agent guiding children on the spectrum through learning programs has been tested with success [41] and a robot-assisted intervention was found to be beneficial for improving children’s psychosocial skills [42]. There are many projects examining the use of social robots in the context of ASD, particularly for psychotherapeutic purposes (e.g., [8,43,44,45,46,47]). An overview by Huijnen et al. [48] shows that social robots are used to enhance communication, social interaction and relations, independent living, emotional wellbeing, and preschool skills in children with ASD. In a systematic review, Papakostas et al. [49] come to the conclusion that “the specific field of application of social robots is at the first development step; however, it is expected to be of great concern to the research community in the coming years” (p. 1).

A big benefit of robots is that they can assume the role of a social interaction partner [50] without inducing the same level of social pressure known to occur when a human is present. Unlike robots, humans have been found to send and interpret social cues in a higher quantity and quality [51,52]. Robots can be used to encourage children during ASD training sessions, for instance by giving feedback to a child that is practicing social skills such as self-initiating interactions or imitating observed actions [46]. The interaction with a social robot was shown to foster social interaction skills, empathy, and adequate responses of children with ASD [43]. In contrast to a human therapist, a social robot is able to isolate particular (emotional) responses without sending mixed signals, which enables a unique form of incremental therapy [52]. Altogether, a robot is highly beneficial for children with ASD in educational, therapeutic, and assistive contexts due to their simple, predictable, and non-intimidating nature in comparison to humans [47,52].

1.3. Requirements for Social Robots to Be Used for Children with Autism

Some initial analyses regarding requirements for child- and ASD-appropriate robot design show that the robot should be customizable and easily reprogrammable, not too human-like (appearance clearly distinguishable from a human), below child height, robust, and congruent, for instance regarding appearance and intonation [47,53,54,55,56]. Diverse international studies (e.g., from UK, USA, Italy, and Japan) delivered interesting insights in the design and functioning of the robots to achieve the aforementioned positive effects [39,45,57].

With the development of the Keepon robot, a Japanese team promoted a minimal, creature-like design including only simple features, namely attention expression (via gaze direction) and emotion expression (e.g., pleasure, fear, excitement; through body movements; [45]). Kozima et al. [45] argued that such a minimal and comprehensive design particularly allows a greater understanding of socially meaningful information and promotes exchange, interaction, physical contact, and a shared mental state.

A research team in the UK decided for a simplistic human-like design and robust hardware but integrated more advanced features in their Kaspar robot [39]. After years of revising and adopting their prototype, the researchers highlighted the usefulness of an integrated speaker, the inclusion of tactile sensors and magnets (for Kaspar to hold objects), as well as the possibility for torso movements. Further, for a greater ease of use and robustness they invested in wireless connectivity (via WIFI) and better concealment (no hanging wires) of Kaspar. At last, they specifically focused on the development of a semi-autonomous system through a sense-think-act architecture with the goal to reduce the cognitive load of the human operator while keeping some levels of external control [39].

In contrast to these rather simplistic designs, researchers in Italy and the US investigated the use of more realistic-looking robots, suspecting greater effectiveness regarding the transfer of the learnings from the robot-human interaction to human-human interaction [57]. In their cross-cultural study, the research teams from Italy (using Alice robot) and the US (using Zeno RoboKind) laid the foundation for the use of realistic-looking robots, showing that children and adults with ASD did not show signs of fear, were highly interested in interacting with the robots and showed high levels of engagement [57]. According to the authors, the benefit of realistic robots is that they can be used to simulate human-to-human social interactions with controlled repeatability, tireless repetition, and absence of frustration or other uncontrolled negative affect of the therapist towards the child [57]. This could lead to more effective social training and transfer to real human-human interaction. Their results show that realistic robots are non-frightening, appealing and engaging to children with ASD and further can hold attention throughout a session and foster the desire for more interactions with the robot [57]. Following interactions with realistic robots, children with ASD displayed increased social engagement (e.g., increased affect, eye contact, verbalization, and instances of theory of mind) [57].

With regard to perceived benefits, scholars as well as affected caretakers often report workload reduction [58,59]. Since there is a high demand for trying new therapies and ASD therapy is costly and time consuming, robots may serve as valuable assistants [58]. Robots could automate repetitive interventions and engage the child while caretakers focus on other responsibilities such as prompting and praising the children [59]. Another benefit would be that robots may foster children’s independence from therapists’ prompts or assistance [59] and increase their readiness to learn [60]. For instance, the robot could be used in place of another adult when a child needs to demonstrate the generalization of a skill or in place of a peer at the same level of learning (particularly useful under circumstances such as during the COVID-19 pandemic) [59,60]. In that case, the robot can be adapted to match the skill level of the respective child to simulate a peer, while the robots’ predictability would pose another benefit for children with ASD [59,60]. Another mentioned benefit is that children with ASD enjoy interacting with computers (iPads, PCs, video games [61]) and robotic assistants encourage children to be mobile which may counteracts the physical effects other computer-based programs may have on them [62]. Furthermore, therapists explained that by introducing new technologies, children with ASD would be exposed to novel things in a positive way which helps to decrease problematic behavior and increase interest [30]. Ultimately, this can help children with ASD to learn and to adapt to other unfamiliar situations in the future [59].

On the other side, there are also several concerns mentioned by caretakers. A lot of them surround the topic of variability and adaptability since children with autism pose a user group with immense heterogeneity [63]. It is possible that robots will be more useful for some individuals than for others and that different scripts may be needed for different children [57,60]. Children with ASD show a wide range of behavioral, social, learning, affective, and cognitive challenges as well as needs and their actions cannot always be predicted [59,61,63]. Consequently, the robot’s ability to adapt to the children’s needs and their changing behavior (e.g., problem behavior, personalization, and motivation) poses a great challenge [59]. Other concerns circle around the topic of costs and benefits [64]. Parents as well as therapists emphasize that robotic systems need to be offered at an affordable price [39,62,64]. Therapists also expressed concerns that using robots could increase workload due to heightened staff training and preparation times [59,61]. This is underlined by therapists’ preference to supervise the interactions between robot and child and to tele-operate the robot rather than it being fully automated [58]. In line with that, many therapists are also concerned with the usability and ease of use for integrating the robot within their workflows [60,62]. Other concerns comprise the children’s safety (pinch points, chances of electrical shock, sharp edges [39]), children’s fear of the robot [45], time delays between the child’s response and the robot’s feedback (delays in prompting and reinforcement can lead to learning errors [59]), physical limitations (e.g., prompting or fine motor skills [59]), responses to unpredictable conversations and behaviors [59], and the overuse of technology [59,60], which could prevent children from engaging fully with other people or activities [60].

1.4. AI-Equipped Robotic Learning Assistant for Children with Autism Spectrum Disorder

In order to support both therapists and parents in the treatment of children with ASD, a social robot may accompany and support children at different levels of the disorder and different stages of development with the completion of various therapeutic exercises. Compared to a real person, a robot is assumed to reduce the pressure of a challenging social situation and to create a unique learning environment for children diagnosed with ASD [29,54]. Robots are already prevalent in the field of ASD treatment; however, this robotic system is planned to expand existing technologies with a targeted initiation of learning processes, intelligent steering of attention, and mitigating of distracting emotions.

In stationary learning situations (e.g., while interacting with digital or analog learning materials) the robot is planned to offer support via child-oriented interventions. For this purpose, the robot will be equipped with an AI-based state detection system. Children with ASD are observed to be alexithymic [65] which comprises troubles with recognizing and expressing own emotions [5]. Particularly during therapeutic exercises, monitoring the children’s emotions is of central importance in order for the therapists or caregivers to react adequately or intervene when necessary [21]. When disturbing emotions are recognized early, measures to reduce them or to avoid an escalation can be taken, which promote the success of the exercise session. Here, affective computing may provide remedy by offering a way to automatically detect the emotional state of children with ASD. Emotion recognition predominantly focuses on facial expressions and paralinguistic features [66,67,68] which constitute core disciplines in machine learning and pattern recognition [69]. For instance, facial expression markers for emotions, such as sadness, depression, fear, and anger, are derived from facial landmark detection algorithms which analyze changes of numerous muscular action units [66,67]. Patterns can be classified categorially (e.g., emotional vs. non-emotional) or by assigning a specific label (e.g., joy, anger, sadness; [69]). In the current context of ASD training sessions, emotions could potentially be classified as disruptive, fostering, and neither disruptive nor fostering. An additional assignment of intensity is also possible.

The automatic detection of emotional and behavioral changes can potentially be implemented in or made accessible for a robotic learning assistant. Since robot technology is already advancing in automatically detecting and responding to users’ actions, the prospect of robots that can detect the moods and preferences of children with autism and adapt their behavior accordingly in real time is investigated [54]. For instance, a current project in Greece (Social Robots as Tools in Special Education [70]) investigates how Lattice Computing (LC) models together with machine learning techniques can be used to construct a representation of a child’s behavioral state [71,72]. With this kind of ability, the robot could take over the task of initiating, monitoring, and effectively motivating the child while executing autism-specific training tasks. Whenever disruptive behavior is detected, the robot could intervene.

As reaction to the identified distractions and dysfunctional learning states, the robot is planned to attempt to restore the learning motivation via (a) appropriate emotion coping nudges (e.g., “Hey, enough dreaming for now. Let’s finish the task!”; “Good job focusing on your tasks like this!”) as well as (b) prompting interaction and games (e.g., “Hey, let’s take a quick break and see who wins this game!”). The empathetic communication behavior of the robot (e.g., “You seem sad, I am sorry about that”) is supposed to promote the robotic learning assistant to an actual companion who supports the child in a loyal, patient, and friendly way. Furthermore, the robot may relieve therapists, parents, and other caregivers during the children’s training sessions by assisting them with monitoring and motivating the children [21]. Here, valuable insights of the persons which are closest to the children are needed.

1.5. Research Questions

Altogether, we need an extensive requirement analysis based on valuable insights from caregivers’ personal experiences for a successful design of the robotic learning assistant and its interaction with the children. The results and design approaches from the diverse international research projects we described before will be considered and reflected on in the light of the outcomes of the current user requirements obtained in Germany. For a successful application of the proposed robotic learning assistant in Germany, additional and more specific requirements are needed from the affected caregivers (parents and therapists). Against this background, the following research questions are used as guideline for the requirement interviews:

RQ1: What are the general conditions of ASD therapy? What are (a) therapeutic learning areas, (b) specific challenges and deficits, and (c) disruptive and beneficial learning factors? Are (d) new technologies currently used and/or desired in (German) ASD therapy settings?

RQ2: How is the prospect of using an AI-equipped robotic learning assistant in ASD therapy settings evaluated? What are (a) perceived benefits and concerns, (b) desired or required features, appearance characteristics, and skills/functions, and (c) the perceived benefits and concerns regarding two specific robot types being used as robotic learning assistant?

2. Materials and Methods

In total, eleven explorative in-depth interviews were executed with parents of children diagnosed with ASD (n = 5) as well as therapists specialized in the treatment of ASD-related deficiencies (n = 6). A semi-structured interview guideline was developed by human–robot interaction psychologists and experts from the field of autism-specific behavior therapy (see Appendix A and Appendix B) and the interviews lasted between 58 and 107 min (M = 87.35, SD = 17.97). The parents answered the interview questions focusing on in-depth insights regarding their children’s specific behaviors and needs and the therapists extended this by offering insights they gained based on their training and experiences with regard to general behaviors and needs of children with ASD when training their socio-emotional and functional skills. This multi-perspective approach profits from the parents’ high personal relevance and the therapist’s extensive expertise, making the gained insights particularly valuable for the planned robotic assistant for ASD treatment in Germany. The interviewees (parents and therapists) were recruited via a therapy program for children with ASD which is provided by the FH Münster University of Applied Sciences and other associated ASD treatment programs. Later on, the robot is planned to be also tested and evaluated by parents and therapists associated with the same ASD treatment programs, which is why it was of particular relevance to include them early on in the development of the robotic learning assistant.

2.1. Sample

2.1.1. Parent Interviews

Three interviews were conducted with one parent and two interviews were conducted with two parents. Accordingly, there was a total of seven interviewed parents, of which three were male and four were female with an age range from 23 to 42 (M = 33.86, SD = 7.11) years. The educational level ranged from university entrance qualification to university degree. All participants were either married or in a committed relationship. In three of the cases, the child with ASD was an only child, otherwise there was either one or two siblings. The biological age of the children ranged between 3 and 10 years (M = 6, SD = 3.67) with a developmental age ranging from 2 to 5 years (M = 3, SD = 1).

2.1.2. Therapist Interviews

A total of six therapists were interviewed of which five reported to be female and one to be male with ages ranging between 22 and 38 (M = 29.67, SD = 5.75) years. Regarding the educational level, all therapists hold a university degree either in psychology, pedagogy, social work, or ergotherapy. One therapist had less than one year of experience, two over ten years, the rest between five and seven years. All interviewed therapists have worked or currently work with children with ASD using Applied Behavior Analysis (ABA) and Autism-Specific Behavioral Therapy (ABT; [12,73,74,75]).

2.2. Procedure

After written informed consent was obtained, the research project was introduced briefly along with the object and purpose of the interviews. Following that, demographic data was collected. The parents were asked about their children’s current therapy and treatment situation such as individual learning goals and deficits, typical procedures, and methods of a therapy session, as well as therapeutic exercises they do at home. Therapists answered questions about their professional pathway, their daily work routine including the procedure of a typical ASD treatment session, problems and challenges they often encounter, and their patients (e.g., age and developmental stage). Following that, the parents’ and therapists’ current technological background was assessed by asking about experiences with and attitude towards the usage of new technologies in the ASD therapy context.

Next, the basic idea of a platform for ASD-related information, exercise material and exchange opportunities, which is another subproject of the research project, was presented to and evaluated by the parents and therapists, which is not further elaborated in this paper (since the focus lies on the robotic learning assistant). Subsequently, parents and therapists were asked about attitudes and expectations as well as wishes concerning the use of robots in psychotherapy (see Appendix A and Appendix B for the complete interview guide). Specific desired characteristics, for instance regarding functions and appearance, were assessed as well as whether children with ASD would benefit from and be interested in this kind of technology. In the next section of the interviews, two use case scenarios with two different socially interactive robots were presented to the participants including exemplary videos of the robots. The presented robots included the Cozmo robot by Anki© (San Francisco, CA, USA) (see Figure 2a) and the QTrobot by LuxAI© (Luxembourg, Luxemburg) (see Figure 2b). These two robot types were presented to show two different approaches to socially interactive robots and offer vivid images for a more consensual understanding and substantial discussion. The QTrobot was chosen since it is particularly designed for the use in behavioral therapy for children with ASD. The robot can show facial expressions via an LCD display and has a variety of advantageous pre-installed software such as speech recognition and hardware (e.g., cameras and microphones; [76]). The Cozmo robot was chosen as additional, contrasting example since it differs optically and functionally to a large extent from the QTrobot: it is much smaller, non-anthropomorphic, more limited in its verbal interaction abilities, and costs only a fraction. The Cozmo robot is predominantly used as a toy, but also to examine HRI phenomena (e.g., [77,78]). For each robot type, the interviewees answered questions regarding their first impression and opinion of the robot and whether they would welcome the use of this robot within the therapeutic context of children with ASD. The last question assessed participants’ preference for either the QTrobot or Cozmo robot by asking them to name positive and negative aspects of each robot type from their point of view.

3. Results

Interviews were transcribed and then categorized with a mixed deductive-inductive coding scheme approach. The coding software MAXQDA was used for the transcription as well as the content analysis. Intercoder reliability was calculated with about 25% of the data (two out of eleven interviews) and two judges using the coefficient “Kappa” [79,80]. The results indicate a substantial agreement (κ = 0.71 and κ = 0.79; [79,80]). The interviews, analyses, and scientific interpretations were performed in German. Translations of quotes from the interviews are provided for consistency and convenience.

3.1. Current Therapeutic Situation

The interviewed parents as well as the therapists reported to work on the basis of ABA/ABT. With regard to the place of treatment, parents stated that their children received therapy in a domestic (4/5) or in an outpatient setting (1/5), while the therapists also mentioned daycare centers (1/6) as a therapy setting in addition to domestic treatment (5/6) and treatment in outpatient facilities (e.g., “in the autism therapy center”; 4/6). The therapists reported to treat the children either with two or less sessions per week (2/6) or with more than two sessions per week (2/6). The parents stated that their children receive several therapy sessions per week (e.g., “three to four times a week in any case”; 5/5) and additionally once a month (1/5) in another therapy facility. The duration of the individual therapy sessions varied between 45 and 150 min, reported by the parents (3/5) as well as therapists (6/6).

3.2. Therapeutic Learning Areas

The interviewed therapists most frequently mentioned the therapeutic learning areas communication and social interaction (e.g., “promoting both passive and active vocabulary”; 4/6). This goes in line with the parents’ statement, who named verbal communication (5/5), non-verbal communication (e.g., “the pointing gesture, we actually practiced that a lot”; 2/5), and social interaction (e.g., “especially this change of perspective”; 3/5). In addition to the promotion of gross and fine motor skills, which were reported by therapists (3/6) as well as parents (3/5), the therapists stated to support the children in training independence and practical life skills (e.g., “how do I make a phone call”; 2/6). Likewise, the parents list changing clothes (3/5), eating independently (3/5), hygiene (e.g., “washing hands”; 2/5), and further aspects (2/5) such as traffic education (1/5) as learning fields for practical life skills. Additionally, emotion recognition and regulation (e.g., “recognizing your own needs […] and then to verbalize it”; 2/6) as well as perception (e.g., “something like eye-hand coordination”; 2/6) were named as learning areas by the therapists. One therapist (1/6) further stated that the learning areas are basically unlimited and depend on the individual needs of each child.

3.3. Challenges and Deficits

The interviewed parents and therapists of children with ASD named a wide range of challenges they face as well as deficits they experience regarding the children’s therapeutic care. On the one hand, the parents criticized the deficient professional consultation (e.g., “they didn’t take us seriously, they blamed it on the fact that he must be too spoilt”; 2/5), the lack of assistant and treatment programs (e.g., “we would have wished for more support”; 3/5), as well as the Internet as often major but unreliable source for information regarding ASD (2/5). They further explain that they are often under high time pressure due to extensive therapy-related preparation and follow-up processes (3/5) and the pronounced supervisory demand of their children (3/5). The therapists often have to balance parents’ needs and their aspiration for an effective therapy (3/6) while considering comorbidities (e.g., “me and my colleagues are not trained as psychotherapists and therefore hardly able to treat compulsions, anxieties, and depression symptoms”; 2/6) and flexibly adjusting a session’s structure if necessary (1/6).

3.4. Disruptive vs. Beneficial Learning Factors

During training sessions, parents (2/5) and therapists (4/6) report to often deal with refusal behavior or insufficient compliance by the children (e.g., “he dislikes everything that is new in the beginning”). Parents further name the children’s lack of motivation and/or concentration (5/5) as well as disruptive emotions, such as anger, negative mood, and frustration (e.g., “if a piece is missing, then there is anger”; 3/5) as detrimental factors during ASD therapy sessions. Helpful are according to the parents precise instructions (4/5), repetitions (e.g., “it is very important for him to make the exercises very often, so that he can do it himself automatically”; 4/5), clear structures, framing, and orientation (e.g., “you have to set boundaries”; 4/5), pauses (3/5), fun elements such as games (e.g., “they do the learning sessions in a playful way”; 3/5), as well as adjustments of the exercises to the child’s developmental level (3/5). As positive reinforcement, parents name food (4/5), praise/applause (e.g., “when he did something well, then he wants applause and then he starts clapping hands himself and is very happy about it”; 3/5), visual stimuli (e.g., “he goes out onto the balcony for a few minutes and watches the passing cars”; 3/5), a game (e.g., “on the phone or tablet”; 2/5), and soap bubbles (1/5). Therapists also name pauses (3/6) and games (3/6) as successful reinforcers, while additionally emphasizing the importance of a low-stimulus environment (e.g., “only have the material there that is absolutely required”; 6/6), establishing a personal relationship (4/6), individual support (e.g., “a need that can also be extraordinary, then you don’t stick to the standard procedure”; 5/6), a learning curriculum with a stable structure (5/6), and generalizing the learning content by changing contextual factors (e.g., “that’s usually most effective when it is transferred to the natural situation”; 4/6). The therapists stated that in general it is important that the children comprehend their disorder (3/6) and the content of the therapy sessions to foster their intrinsic motivation and compliance (2/6).

3.5. Usage of (New) Technologies

The parents reported that they (5/5) and their children (4/5) have a high technical affinity and that they use media-supported exercises to train their children’s abilities (e.g., “videos from YouTube to learn sentence structuring”; 5/5). According to the therapists, the children’s technological skills depend on their cognitive level (2/6). Additionally, they evaluate the children’s general technical affinity as rather high (e.g., “when the same tasks are provided via tablet they are done very quickly”; 6/6) and mention that they are able to execute digital learning material autonomously (4/6). The interviewed therapists themselves state to be open towards new technologies (e.g., “it can be very supportive for the therapies”; 6/6) and that they already use technology- and/or media-based exercises in their treatments (e.g., “we do social role plays and watch them together on video afterwards to show which behavior is appropriate and which is not”; 6/6).

3.6. Usage of a Robotic Learning Assistant in ASD Therapy

3.6.1. Perceived Benefits and Concerns

The interviewed parents are generally very open towards the idea of using a social robot with an AI-based state detection system to support their children (5/5; see Figure 3 for the parents’ and therapists’ perceived benefits) and think that their children would accept the robot as stress-resistant (e.g., “it does not allow itself to be provoked”; 2/5) companion (4/5) to exercise tasks together (4/5). Similarly, the therapists also described the robot as stress-resistant (e.g., “without getting tired or frustrated”; 2/6), accepted companion for the children (e.g., “such a technological device is rather interesting, because it makes them special”; 5/6). They added the following benefits: the interaction with a robot is simpler and more predictable than with a person (2/6) and the robot can take over some therapeutic activities (e.g., “the robot has not taken over the function of the therapist completely, but a little bit and that […] is great […] due to the lack of therapists”; 4/6).

Despite this general positive attitude, the interviewed parents also noted that the children’s level of development may not be sufficient to use the robot (e.g., “it would be difficult for his age to really work with it or interact with it”; 4/5; see Figure 3 for the parents’ and therapists’ perceived concerns), the children’s interest may decrease rapidly (3/5), and an unsupervised usage of the robot would not be possible (2/5). Furthermore, the robot may not be able to react sufficiently (e.g., “you still have to respond to him in order to keep him in line”; 3/5) and only offer a limited amount of exercise options (2/5). Additionally, data protection (e.g., “the videos could fall into the wrong hands”; 4/5) as well as financial (3/5) concerns were expressed and a negative image based on science fiction portrayals was mentioned (e.g., “threatening at first sight if you have these images on your mind”; 1/5). The interviewed therapists on the one hand expressed concerns regarding the children being too distracted by the robot (e.g., “too much excitement”; 3/6) and losing interest in it over time (3/6). On the other hand, they worried about a displacement of social interaction (e.g., “despite the use of media and robots, the focus should still be on interpersonal relationships and direct human contact”; 5/6), exaggerated trust in the robot including its AI-based decisions (1/6), and insufficient operating skills of the parents (1/6).

3.6.2. Desired Features, Appearance, and Functions

Regarding the robot’s desired features, parents name a connection to a computer (4/5), great mobility (e.g., “that it can also run after the child, something we would do otherwise”; 3/5), a human-like voice (2/5), a touchscreen (1/5), and a built-in (security) camera (e.g., “when you’re […] in the garden, you could have another little look to see if he’s actually still sitting at the table”; 1/5). A child- or generally human-like (5/5) or animal-like (2/5) appearance with a robust exterior (4/5) and a height approximately as tall as a child (e.g., “at eye level; 5/5) is preferred by parents. The robot’s skills are suggested to comprise acoustic feedback (4/5), communication skills (3/5), and general caring skills (e.g., “giving him something to eat, something to drink”; 1/5).

In addition to mobility (5/6) and a human-like voice (2/6), the interviewed therapists name adjustable speech modes (e.g., “varying the instructions a little […] from very structured to a little more open”; 2/6), an integrated orientation help (e.g., “a progress bar so that the child sees or the robot says ‘Come on! Two more times”; 2/6), general customizability (e.g., “it would be nice if the robot could say the name of the child”; 2/6), and simple operability (2/6) as desired features. The robot’s appearance is preferred by the therapists to be low-stimulus (1/6), human-like (2/6), but without too many human-like characteristics (e.g., “one would need to see that it is a robot and no human being”; 5/6), with a robust exterior (e.g., “we often observe that children throw things from the table, when they are frustrated”; 4/6) and a small size (2/6). Regarding skills and functions, the therapists wish for communication skills (3/6), acoustic feedback/noises (e.g., “verbal praise is accentuated with clapping and an unsuccessful attempt with […] a typical melody going down”; 2/6), detection of and feedback to emotions (e.g., “You look tired. Do you need a break?”; 3/6), as well as teaching and supporting practical life skills (e.g., “that brushing teeth for example is accompanied”; 2/6). The robot further is desired to have functions to support role plays (2/6) and to give an overview regarding the child’s therapeutic process (e.g., “within the robot it is saved what the child already did”; 2/6). Please see Figure 4 for an overview of the parents’ and therapists’ desired features, appearance, and functions of a robotic learning assistant.

3.6.3. Perceived Benefits and Concerns Regarding Cozmo and QTrobot as Robotic Learning Assistants

The Cozmo robot evoked a general positive impression among the parents with a high perceived fun factor (4/5), whereby its great mobility (1/5) and low price (1/5) were particularly highlighted. However, the robot was also criticized for being a toy rather than a learning help (3/5), its limited communication skills (1/5), and for being too cognitively demanding (1/5). The therapists can imagine using the Cozmo robot as therapy assistant (6/6), reinforcement (5/6), for support or role model (e.g., “for children that are a little shy, scared to do something, that you can send it forward so that they can watch”; 3/6). They appreciate its mobility (3/6) as well as that it is not too human-like (2/6). It was criticized by the therapists that the robot appears to be a toy rather than a learning assistant (e.g., “does not provide a calm learning atmosphere, but rather a bit of restlessness”; 4/6), with restricted communication skills (2/6) and not clearly recognizable expressions of emotions (1/6).

The QTrobot was evaluated by the interviewed parents as realistic learning assistant (4/5), being able to display emotions (4/5), and having a fitting size (2/5) as well as an interesting appearance (1/5). As negative aspects, the parents questioned the robot’s usefulness (e.g., “Does it actually need a robot that is just additionally praising the child?”; 3/5) and its restricted mobility (1/5). The interviewed therapists pictured the QTrobot as therapy assistant (6/6), for general reinforcement (2/6) and prompting (3/6), in the context of emotion expression and detection (e.g., “show the diversity of human expressions in a more systematic way “; 4/6) and autonomy training (3/6). They praised its adequate way of speaking (e.g., “the speaking rate was great because all of us talk too fast for humans with autism”; 3/6) and its usefulness for younger children (2/6). However, the therapists also questioned the necessity or usefulness of the robot (1/6) and criticized that switching between the QTrobot and the tablet could be too distracting (2/6) as well as its disruptive arm noises (1/6). They further expressed concerns regarding the robot’s facial expressions (e.g., “I don’t know how much the autistic child benefits from the face, whether it really looks there”; 2/6) and its way of speaking, particularly the insufficient clarity of expression (2/6), delayed speech and reaction (2/6), and the robot-likeness/artificiality (e.g., “there is the risk that the child starts talking after and develops a robot voice”; 2/6).

4. Discussion

To receive valuable insights for the successful employment of a robotic learning assistant for children with ASD in German households we interviewed parents and therapists of children with ASD. While the parents gave in-depth insights regarding their children’s specific behaviors and needs, the therapists offered broader assessments of general behaviors and needs when training the children’s socio-emotional and functional skills. As the interview results show, children with ASD as well as their parents and therapists face a variety of challenges which are rooted in the disorder itself but also stem from deficits in the German therapeutic care system. The interviewed caretakers are generally very open towards the idea of using technologies to support children with ASD, evaluate the children’s technical affinity as rather high, and already use technology in some respects. However, there appears to be no consistent, well-established use of technology-based solutions within the German ASD therapy and learning context yet. Against this background, a technology-based approach is proposed to support children with ASD in improving their socio-emotional and functional skills with a robotic learning assistant and an implemented AI-based state detection system. Several design implications and aspects to consider for the development of such a robotic learning assistant were retrieved from the interviews.

Foremost, there needs to be a focus on sophisticated speech and interaction skills as well as customizability, adaptability, and variability of the robot’s behavior. As it was made clear to us, every child has different needs and therefore therapeutic elements (which would include the robot) need to be constantly adapted to the current demands. This is underlined by Robins et al. [81] who assessed user requirements concerning a robotic toy for children with ASD: “for some children something very simple with a very limited level of technology might feel quite good; while others would need more complexity to sustain their interest” (p. 105). In line with that, the majority of previous research also underlines the extraordinary heterogeneity among children with ASD [57,59,60,61,63]. Attention needs further be paid to designing the robot’s behavior and appearance in a way that is not too distracting while preserving its benefit of being more predictable and thus comfortable for children with ASD (compare with [59,60]). This goes in line with the more simplistic design approaches by Kozima et al. [45] and Wood et al. [39]. Therefore, this approach appears to be recommendable in the German context as well, in contrast to more realistic approaches, for instance by Hanson et al. [57]. This is further emphasized by the concerns mentioned by the interviewed therapists that the robot should not replace social interactions entirely (compare with [58,60]) and further not look too human-like. Previous research also mentions the importance of promoting user focus, usability, and reliability [39,60,62], which goes in line with designing the robot behavior to be interesting but not distracting, easy to use, and predictable. The predictability while keeping the child interested is an important balance that should be focused on in the further development of the robot’s behavior. In conclusion, the robot needs to show structured behavior so the children know what to expect, at the same time the behavior needs to evolve to sustain their attention after the novelty effect declines (compare with [81]). Wood et al. [39] further mentioned affordability as important aspect, which reflects the parents’ financial concerns mentioned in our interviews. Affordability was also mentioned in a number of other studies as important point to consider [62,64]. Extending previous research, we need to consider the mentioned data protection concerns when employing the robot, which appears to reflect Germans’ heightened sensitivity towards this topic (e.g., in comparison to the USA [82]).

Another important aspect that needs to be focused on is the robot’s ability to detect and react to the child’s current state, which will address the desired emotion detection and response skills mentioned by the therapists as well as the sophisticated communication skills which were considered important by parents as well as therapists. This highlights the need to carefully develop a platform to provide a reliable detection and reaction system, comparable to the sense-think-act architecture of the Kaspar robot [39]. As argued by Wood et al. [39], the development of such a detection-reaction system might profit from a semi-autonomous structure to keep some level of external control, flexibility, and personalization in the interaction, while reducing the cognitive load of the external person controlling the robot. Since both, therapists as well as parents, favored the QTrobot–for instance due to its ability to display emotions and to communicate adequately–this platform will be used in the course of this research project.

4.1. Limitations and Future Research

One limitation is the rather small sample size and that all participants were recruited via the same ASD treatment program and interviewed using Internet-based technologies which could affect the generalizability of the results. Furthermore, due to the qualitative nature of this study, no assumptions about the prevalence of the identified requirements can be made. However, in the ASD field, generalizability is generally a challenge since the disorder manifests in various forms and no child experiences the same situation as another. Therefore, the technology needs to be adapted to the respective child and situation in any case. Our insights still help us to consider a great variety of aspects such as the relevance of a low-stimulus environment that are helpful to most children while keeping in mind the importance of adaptability and personalization of the robot and its behavior. For more diverse and at the same time more generalizable insights, future studies should include a larger sample of parents and therapists with different backgrounds. This could be done for example in form of a large-scale online study based on the results of our explorative interviews. Additionally, it would be interesting to find out more about how the requirements observed in this study change or evolve after interacting with the proposed robotic learning assistant, which would be interesting to investigate as cross-sectional and longitudinal studies.

4.2. Outlook: Detecting Engagement in Children with Autism

Social robots have already been successfully employed to assist children on the spectrum to practice important behaviors of social interactions [83] such as joint attention, turn-taking, or imitation behavior [54]. The success of this robotic support increases when the robot is able to generate coordinated and timely behavior considering situational factors and social surroundings [50,84,85]. Adapting to the current context, for example by considering the child’s prevailing emotions, will enable the robot to offer customized and thus more effective support in the situation where it is needed (as shown by previous research; e.g., [86,87,88]). For instance, children react more expressively and positively to a robot which changes its voice, body posture, eye-color, and gestures in order to adapt to the children’s emotions [89].

The potential of using audio-, video-, and physiology-based state recognition in various forms to enhance technology-assisted interventions of children with ASD is already explored in the literature (e.g., emotion recognition [21]; measuring social engagement [90]; assessment of engagement [85,91]; visual attention estimation: [92]). Usually, emotion recognition is based on speech or facial expression analysis [66,67,68,69]. However, based on the literature [5,93] and a video screening of ASD therapy sessions, it can be concluded that an automated detection of emotions of children with ASD via speech or facial expressions will pose great challenges, making it almost technically unfeasible. Symptoms of ASD often include impoverished facial display of affect [94] and high social anxiety [95]. Consequently, changes in facial expression are rather unlikely [5]. Furthermore, children with ASD often display wide-ranging speech and language deficits [96], which was also indicated by the interviewed therapists. Thus, obtaining sufficient facial expression or speech data to develop an adequate detection system will be difficult to achieve and may still not successfully function during interactions with children on the spectrum.

Therefore, since the application of classic facial expression and speech analysis holds various problems when working with ASD children [93], a new technological approach is proposed focusing on automated engagement detection (instead of emotion detection). The approach concentrates on the concept of engagement since various studies already proofed the feasibility of the detection in children with ASD. For example, Lemaignan et al. [97] measured to what extend children engaged with a robot during an interactive task based on a concept from the field of computer-supported collaborative learning. Kim et al. [98] concentrated on behavioral engagement which is indicated by the response time to tasks and prompts of a therapist. The level of engagement in typically developing children can be indicated by cues like eye-gaze and head pose (see [99] for a review of methods to measure the engagement levels of children during child-robot interactions in an educational settings). However, those cues seem to not directly appear in children with ASD [100]. As a result, engagement is easier to detect in typically developing children than children with ASD [101]. Our proposed approach therefore investigates the role of body movement, particularly the posture of the upper body, using 3D skeleton data as well as gaze direction.

Several studies showed promising results in combining these two concepts [102,103,104]. For instance, Anagnostopoulou et al. [105] focused on engagement estimation based on pose key points for children with ASD during interaction with robots. Marinoiu et al. [106] investigated the feasibility of body movement and posture for emotion recognition using 3D skeleton data. Mainly due to difficulties in capturing small changes in engagement, most of the work reports binary engagement (engaged vs. disengaged, [85]). Despite the difficulties, we do not think that a binary classification is suitable in the context of children with ASD because it cannot reflect the complexity and individual differences of the disorder. Therefore, a three-level continuum scale is planned which is divided into low, medium, and high engagement.

This new concept is specifically suited for children with ASD since it does not depend on verbal utterances and facial expressions and therefore bypasses the described problems regarding speech and facial expressivity while still delivering valuable information for the robotic learning assistant. Furthermore, body language, expressed via posture, provides an alternative way in understanding the children’s emotions [107]. With this approach, the robot can intervene whenever the child displays severe signs of disengagement. The robotic learning assistant equipped with the proposed engagement detection system is currently designed with careful attention to the requirements obtained from this study and will be evaluated in future studies.

5. Conclusions

Based on the results of the interviews and the presented insights from previous research [5,93,94,95], we conclude that in Germany new technological solutions need to be explored and tested to adequately support children with ASD and their caregivers in training the children’s socio-emotional and functional skills. Here, robots pose a promising approach since they can take over the role of a social interaction partner [50] but with reduced social pressure [52] due to being less complex and intimidating than a human interaction partner [51]. Previous research already provides evidence for the benefits of using a robot in the context of ASD therapy (e.g., [43,46,47,51,52,54]), while this study offers specific insights for therapeutic settings in Germany which are compared to requirement obtained in studies from other parts of the world (e.g., [39,45,57,81]).

According to the interviewed parents and therapists, one of the most relevant aspects for a successful application in ASD therapy settings is the robot’s adequate and context-specific communication behavior. To enhance the quality of the robot’s communication skills, an AI-based detection system will be developed with the goal to enable the robot to autonomously monitor the children’s engagement during the ASD training exercises. Since parents as well as therapist are very favorable towards the usage of new technologies in this context, we are confident that our approach is promising for the German ASD therapy setting. This should particularly be the case when we consider potential obstacles and relevant design specifications that were identified in the interviews such as that the robot’s behavior should be customizable and predictable without being too distracting from the relevant exercise.