Design for Children’s Playful Learning with Robots

Abstract

This article presents an investigation of the implications of designing for children’s playful learning with robots. This study was carried out by adopting a Research through Design approach that resulted in the development of a novel low-anthropomorphic robot called Shybo. The article reports the main phases of the project: the preliminary and exploratory research that was carried out to define a list of design requirements; the design of the robot and its supplementary materials for carrying out playful learning experiences; and the evaluation of the project that involved both parents and children. The robot, in fact, was finally tested as part of a two-hour experience that engaged children in activities related to the associations between sounds and colours. The article presents and discusses the results of this evaluation to point out positive aspects of the experience, emerging issues and hints for future works. These are documented to share lessons learned that might be supportive of the general development of children’s playful learning and cognitive experiences with robots.

1. Introduction

Nowadays, the ability of robots to support children’s learning is quite renowned and their application in educational contexts in constantly increasing. They are used to support both technical, such as programming, and non-technical subjects [1], such as STEM (science, technology, engineering, and math) disciplines [2]. They are even employed to support storytelling [3] and creative activities.

This great success that robots are experiencing in education is the result of a broad body of research that in the last decades focused on the role of experience as inseparable aspect of learning [4]. Piaget [5] explained in his theory how “knowledge is not a commodity to be transmitted”, and situated cognition scholars expressed the idea that “to know is to relate” [4]. Knowledge is rather constructed through the experience and the interaction with materials, and children are “active builders of their own mental structure” [6]. Consequently, the act of programming introduced a transformation in the process of learning. In this activity, children play an active and self-direct role, and knowledge is acquired for a personal purpose [6] rather than an external imposition.

Within the framework of these studies, play was also brought at the centre of the debate about education as a crucial aspect of human learning together with the importance of artefacts [7]. By playing, children have the opportunity to unconsciously learn those habits necessary for their intellectual growth [8]. Play also has the particular function of letting children deal with new objects and situations by recalling existing schemas, blocks of intelligent behaviours, in a process that Piaget calls assimilation [5]. Thus, toys, daily life objects and the whole surrounding environment represent an expansion of the individual abilities for building knowledge [4]. Through play, an artefact has the chance to become an “object-to-think-with” [6], whether they are computers [6], robots or non-technological objects, such as a pendulum [9].

This practice-based, explorative, and intrinsically motivating [10] approach to education takes the name of playful learning. According to Fisher et al. [11], in playful learning, children that approach academic contents through free and guided play acquire greater cognitive and social skills than via traditional and direct instruction practices. However, it has to be noted that playful learning is not about adding entertainment to education, to make it less unpleasant [12]. It rather challenges educators, teachers and designers to carry out projects and activities able to support playful experiences by enriching the environment with artefacts that provide experiential learning opportunities, and by supporting and guiding children in their exploration [11].

These theories reveal a large body of design opportunities, in which innovative educational activities, as well as artefacts, can be explored. Thus, the implications of designing for children’s playful learning were investigated through the development of a novel low-anthropomorphic robot called Shybo.

This robot is able to perceive sounds and to react through minimal behaviours, namely lighting up in different colours and moving the hat. Given the aim of letting children construct their knowledge, Shybo is designed to be used as character for stories and experiences that can be related to academic contents or to abstract concepts such as identity and emotional intelligence. To do so it is characterized by a key feature: the ability to be trained. To play with Shybo, in fact, children have to train it by playing sounds and associating colours to them.

This project was carried out as part of a joint research that involved the UXD Polito research group, from Politecnico di Torino (Italy), and X-Studio, from Tsinghua University (China). Referring to literature (Section 2), the project was developed through a methodology (Section 3) that combines participatory actions to the physical development of a prototype and accessories (Section 5). The participatory actions consisted of a preliminary investigation of the scenario, for defining the design requirements (Section 4), and of activities to evaluate the project (Section 6). In particular, the evaluation stage of the project was carried out as a two-hour experience in a primary school. Feedback and insights, regarding both the robot and the playful learning experience, were collected with the intent of contributing to the knowledge about how to design acceptable child-robot experiences.

Accordingly, the results (Section 7) are reported and discussed (Section 8) to share a lesson learned and contribute to the understanding of the opportunities and challenges posed by designing playful learning experiences for children.

2. Related Works

The design of playful learning experiences for children is a complex challenge. “Designers” (intended as every professional involved in such project) are asked not only to define which artefact to use, they are also requested to investigate the ways the artefacts may enable children to relate them to their experience of the word, with the final intent of constructing new knowledge.

In this regard, a decisive contribution was given by the research, and the activities carried out at the Lifelong Kindergarten of MIT (Massachusetts Institute of Technology), over the last twenty-five years. Here, researchers investigated ways to expand the range of concepts that children can learn through the direct manipulation of physical objects [13]. To do so, they developed a series of what they call “manipulative materials” which embed computational capabilities inside traditional toys, such as blocks, beads and balls. As reported by Resnick [13], their work was guided by three underlying principles: encourage design projects, leverage new media, and facilitate personal connections. Resnick, however, points out the fact that the introduction of new media is aimed at fostering creative thinking, not at developing creative technologies, and that the media per se cannot ensure a playful-learning experience [14]. Decisive aspects appeared to be the capacity of meeting children’s interests, the ability to support different play styles, and the use of familiar objects in unfamiliar ways [14].

The most popular example of robotics applied for playful learning, developed at MIT, is LEGO Mindstorm [15]. Launched by LEGO in 1998 [15], the Mindstorm Robotic Invention System is a product line that combines programmable bricks with sensors, actuators and LEGO technics elements, for engaging children in playful learning activities. This product resulted from a series of initiatives and research that were carried out at MIT Media Lab in the nineties, such as an annual challenge called LEGO Robot Design Competition, and, especially, the Programmable Brick project [16].

LEGO Mindstorm is nowadays quite diffused and used in several contexts, and its use for playful learning activities is getting popular. Furthermore, several robots for teaching computational thinking are nowadays spreading, such as Thymio [17] and Cubetto [18].

Less systematic and long term studies, instead, were focused on the investigation of playful learning with robotic characters designed to show social behaviours, rather than construction kits. Most of the projects that employ social robots in educational contexts tend to develop them to play the role of a teacher or a caregiver [19]. However, some studies demonstrated how a playful interaction with a robot as children’s peers could be beneficial regarding both engagement and learning. Tanaka and Matsuzoe [19], for instance, by introducing the concept of the care-receiving robot, showed how children increase their performances in terms of learning by teaching to the robot. The idea of children as teachers is also presented by Gordon et al. [20]. The authors present a tangible toolkit for programming social robots, a DragonBot in their case, through a playful and fun interaction. In other studies, instead, the robot assumed different role: a mediator. Marti and Iacono [21] designed a robot companion for children with disabilities, for empowering them to explore different play modalities. Kronreif et al. [22], instead, designed a Cartesian coordinate robot that enables children with severe physical disabilities to play with existing toys, such as bricks.

Although both aspects of play and learning are addressed in the mentioned examples, these do not converge fully in playful learning experiences. Nevertheless, these kinds of examples provide useful methodological references for design processes that often address particular user requirements and take into account context specificities. These methods for designing social robots for play or education, in fact, often present some common traits. They usually include explorative stages in which both existing knowledge as well as specific findings, emerging from user studies, are used to define the requirements that subsequently guide the design and development of the robot and the interactions. In addition, the testing phases are peculiar: these are usually carried out in real contexts, with representative samples of participants, and there is keen interest in qualitative data, rather than quantitative.

Regarding the actual design of robots for children’s playful learning, Edith K. Ackermann [23] provide handy hints. In particular, in her work about AniMates, she points out some crucial aspects of animated toys. AniMates are animated in the sense that they exhibit specific behavioural attributes, attitudes, or social skills [23]. She does not refer to robots or educational content explicitly. However, she explains how animated toys represents self-reflective tools that can help children in their process of identity formation and social-emotional development. According to the author, these animated toys can also be described as toys that feel or sense. These, in fact, are sensitive to particular features of their environment and children can affect their behaviour by manipulating these features.

This idea of animated toys that feel may be combined with the idea of manipulative materials that can support children’s self-exploration and knowledge building, with the intent of developing not only an empowering artefact but also an intrinsically motivating experience around it.

3. Methodology

This project was carried out applying a Research through Design approach [24]. The process resulted from the combination of the design and development of a robotic artefact and a series of supplementary materials with participatory design actions, referring to existing literature about child-robot interaction studies, that combined.

The purpose and the requirements of the playful learning activity were identified in a preliminary research phase, consisting of literature review, scenario analysis, and an exploratory study [25] that engaged parents through a questionnaire and children in a hands-on workshop. The exploratory study was particularly meant for investigating the current scenario of children’s play, in urban areas of China.

The requirements, then, guided the design of Shybo and its supplementary materials for carrying out playful learning experiences in educational context. Together with the physical prototyping of Shybo, the making of the game elements and the design of a related activity, this phase has also involved a preliminary playtest of the experience.

Finally, the activity with the robot Shybo and its game was tested in a primary school with twelve children. This test was meant for evaluating the solution’s soundness, engaging potential, as well as its ability to foster critical thinking and to generate arousal. The data collection consisted of children’s feedbacks through an after experience questionnaire and two evaluating tools, and observations based on the video recordings of the experience. In parallel with the activity with children, parents were invited to fill a questionnaire about Shybo, the activity and some general aspects regarding them and their children.

4. Preliminary Research and Requirements

The first phase of the project involved an exploratory study [25] aimed at investigating the habits of Chinese children and observing how they approach certain activities, which were structured referring to real activities for active learning, aimed at understanding how children perceive concepts such as colour, sounds, emotions and their associations. To this end, parents were asked to fill a questionnaire, while children were involved in a hands-on workshop. The workshop consisted of three different activities, supported by a set of cultural probes [26]: acting and guessing emotions; drawing soundscapes; and associating sounds, objects and colours.

This exploratory study was crucial for defining a series of requirements that concern, mainly, the purpose of the project proposal. Through the questionnaire, in fact, it was possible to notice how children’s daily life is tightly scheduled, resulting in a dominance of educational activities and limited time dedicated to playing. The unbalance towards education rather than play was also noticed in the parent’s opinions about the characteristics that a good toy should have. In most of the cases, it has to have a clear educational purpose. Another peculiar aspect identified through the questionnaire is that children, especially during the week, have small chance to play socially, with adults or peers. According to these findings, three requirements were identified: fitting into existing habits, making clear the learning potential of the proposed solution, and promoting social engagement.

The observation of the hands-on workshop, instead, allowed observing how children spontaneously engage these kinds of activities creatively and critically. Regarding the soundtracks, for instance, they automatically tended to make complex interpretations of each track creating stories and adding interesting details. Relating to the activity about sound-colour associations, it was possible to notice how they were spontaneously creating their own rules of approaching the activity and explaining their decisions. Thus, these observations allowed defining three other requirements also related to the project’s purpose. Rather than designing the most efficient social robot, the challenge became designing for giving control to children, by considering the robot as part of a broader storytelling that can be developed though open and customizable experiences.

These six requirements were then supplemented with other concerning the specific aspects of robot’s design, emerging mostly from the literature review. The design of robots usually addresses three main aspects, namely morphology, nonverbal behaviours and interaction schemes [27]. Accordingly, the robot’s design requirements were defined referring to these three aspects. Regarding nonverbal behaviours, the robot has to communicate different statuses through movement, and it has to show explicit input-output relations. The first is because the sense of animacy and causality spontaneously emerge with the visual processing of movement [28]. The second, instead, refers to the work of Ackermann [23], who explains how toys that are sensitive to some features of the environment can be controlled and affected by children’s actions.

About morphology, the robot has to show an iconic appearance and provide some physical affordances. The need for an iconic appearance is motivated by the willingness of providing lifelike features that can be attractive and raise a sense of familiarity [29] while avoiding the risks of surface mimicry [30] that can lead to uncanny feelings [31]. Physical affordances [32], instead, are needed for inviting and facilitating user’s interaction.

Finally, also regarding the interaction schemes, two requirements were identified. On the one hand, the use of the robot as a mediator of the interaction engages and motivates children to interact with the physical environment. On the other hand, the interaction has to give control to children, at some levels. Although it is related to the requirement emerged from the exploratory study, this somehow differs since it refers to the idea, by Ackermann [23], of toys that feel and that can be controlled by children. In this can, thus, the act of giving control determines a feature of the robot.

5. Robot Design

In the attempt of meeting the requirements emerged from the preliminary research, a trainable sound-reactive robot, called Shybo, was developed. The design process was characterised by sketching, 3D modelling, prototyping and documentation. In particular, a series of five low and high fidelity prototypes was developed for investigating different aspects of the robot, such as nonverbal behaviours, interaction with users and morphology.

5.1. Shybo

The proposed solution is Shybo: a small robot that perceives sounds and reacts by lighting up in different colours and through a minimal nonverbal behaviour, namely the movement of the hat.

Shybo, in Figure 1, is designed to be used as a character for stories, aimed at letting children construct knowledge that can be related to academic contents or more abstract concepts such as identity and emotional intelligence. It is intended to be part of broader learning experiences, which can be carried out in class with groups, fitting into existing habits and answering to the requirement of promoting social engagement. To do so, it is designed to be accompanied by a set of elements that can change and be defined according to the context and the educational interest of the situation, also answering to the need for open and customizable experiences.

Given the interest in giving control to children, for expressing their visions and interpretations of experiences, the robot has no pre-set colour–sound combinations and children have to train it to play. The training consists of simple actions: switching modality, selecting a colour, and recording sounds. At the bottom surface of the robot is located a switch that changes the status of the robot from play to train. In this state, the child can select a colour by turning the nose of the robot, as shown in Figure 1a. Once a colour is chosen, the child can associate a sound to that by pushing the robot’s mouth (a button). As shown in Figure 1b, when the mouth is pushed a small red light indicates that the robot is recording. Once it is released, Shybo automatically saves the association of that sound to the selected colour category. In this way, children can potentially choose to make sound with any kind of object and they can record multiple sounds on the same colour category, paying attention to the similarities of the various sounds.

Shybo is designed as a low-anthropomorphic robot, consisting of a round-shaped head and a hat. Addressing the requirement of an iconic appearance, its face is obtained using electronic components, such as a potentiometer (nose), a button (mouth) and a LED ring (eye). The components also represent an intuitive training interface of the robot, answering to the need for giving control to children. The shape and the face-interface answer to the need for providing physical affordances to children. The rounded body, in fact, invites to the physical contact and let children hold the robot in their hands. The “nude” electronics components, instead, provide hints of the use.

By perceiving sounds and reacting through light and colour, the robot presents an explicit input-output relation that easily allows children to see the effects of their actions. In this way, the robot can be used as a mediator if the interaction that can take place between children and elements of the physical environment, such as musical instruments or daily life objects used to make sounds. The robot’s reactivity and status are also manifested through the movement of the hat, which can be at an intermediate position when Shybo is switched off, totally open when it is active, and closed and shaky when it is scared (Figure 1c).

5.2. Supplementary Materials

The first experimental experience with Shybo was meant to explore with children some primary concepts of colour theory and discuss together the qualities of sounds, and how sounds and colours can be associated. These aspects were, then, intended to be used through a game with Shybo, that would require children to remember colour–sound combinations and coordinate in groups for playing. Accordingly, a set of game elements was designed and developed.

The set includes twelve musical instruments, five coloured paper cards, four boards of colours, a board game, a set of game cards, four pawns, and an hourglass.

The musical instruments, consisting of both existing and custom made instruments, were firstly used to let every child play, find similarities among sounds, train Shybo and, then, to play the game for obtaining certain colours on Shybo. For better understanding the role of each game element, the list of game rules is listed below:

• Divide the players into groups of 3, so to have 4 groups.
• Each group plays in turns. One turn last 1 min, showed by the hourglass.
• During every turn, each member of the group has to do a different thing. One child reads the card, one takes the object, and the last child moves the pawn on the board. Every turn the children can exchange the role.
• Every card has a coloured circle or a description of colour. The teams have to obtain those colours on the robot by making sounds. The sounds were previously associated with colours by children.
• In one turn each group can do as many cards as it can. Every card done allows moving one step forward on the board.
• If the robot gets scared by the sounds, the team lose the turn, unless the card requires the red colour.
• The team that arrives first at the end of the board wins.

6. Evaluation

The evaluation of the project has encompassed two main actions: a questionnaire for parents and a playtest with children. During the playtest with the kids, different methods were used to collect data. On the one hand, the whole experience was video recorded for subsequent observation and transcription of the verbal interaction. On the other hand, children were invited to self-report data about the experience through three tools: an after experience questionnaire, inspired by the Godspeed questionnaires [33], an Again and Again table [34] and a Difficultometer [34].

The playtest was firstly aimed at observing both usability and user experience aspects. Play related experiences, especially pure entertainment applications, in fact, require not only to evaluate the key factors that affect the playability of the proposed solution but also to understand the emotional experience of users [35].

The questionnaire, instead, was meant to investigate factors of social acceptance and societal impact that, in human-robot interaction studies, assume a crucial role in acceptance. Robots, in fact, represent a more complex challenge regarding evaluation compared to computer based applications. For instance, expectations and resistances toward robots are significantly affected by the fact that people’s mental model of these artefacts is more anthropomorphic than the one of other systems [36]. Furthermore, various sociocultural and perceptual factors, such as education, life and work conditions, attitude toward technology, and forms of attachments can determine the way people approach and behave with robots [37].

Accordingly, the evaluation of the project presented in this article was carried out with the attempt of providing useful insight regarding both the general purpose and the specific aspects of the project. With regards to the purpose, on the one hand, the study was aimed at getting insights about the perceived usefulness, which is crucial for product acceptance [38] of the solution. On the other hand, the goal was understanding the compatibility with the context and the existing practices, which determine the way potential users give meanings to the proposed solution in relation to their values and beliefs [39].

Regarding the robot, three main factors were observed, namely likeability, learnability, and perceived animacy. Robot’s likeability is addressed since it has been reported that this factor can significantly affect the users’ way and willingness to interact with a robot, for instance reducing or increasing the distance during the interaction [40]. Learnability, namely how easy is the system to be learned for novice users [37] is particularly important for this project, and it is mainly related to the training mode of the robot. This aspect, in fact, may invalidate the overall experience, by moving the focus from sound-colour associations to how to train the robot. Perceived animacy, instead, is crucial for understanding the efficacy of the interaction that relies on the user’s ability to attribute individual mental states based on certain forms and movements [41]. The solution presented in this project, in fact, relies on the children’s ability to perceive the robot’s status, active or scared, according to its light and hat movement.

Finally, the overall experience is evaluated referring to two most important aspects: enjoyment and engagement. According to Xie et al. [42], in fact, these two are integral, and prerequisite aspect’s of children’s playful learning experiences. Still referring to the authors, this work address enjoyment as children’s intrinsic motivation, in which people engage the learning activity for their own sake, rather than for receiving some external reward or avoid some external punishment [10]. Furthermore, Xie et al. refer to Salomon and Globerson [43] who provide a useful conceptualisation of engagement within the context of learning experiences, as a mindful activity that requires cognitive effort and deep processing of new information.

6.1. Participants

Engaging a representative sample of the population in the study is of primary importance to get ecological validity and make better generalisations of the real world for which the solution is intended [44]. Moreover, in the case of this project, this good practice acquires even more importance. Children respond more readily and strongly than adults to social robots [45] and even a minimal robot movement, such as gaze movement [46], can affect the perception of animacy and likeability. Referring to these considerations, the evaluation stage was carried out involving twelve Chinese children (N = 12) aged 6–9 years old (age: M = 7.08, SD = 0.95). They were half females and half males (F = 6; M = 6). They all attend extracurricular courses, on average, twice a week (M = 1.9, SD = 1.3). Parallel to playtesting with children, one parent for each child was asked to answer a questionnaire. The parents involved (N = 12), aged between 28 and 41 years old (M = 33.25, SD = 3.86), were eleven females and 1 male (F = 11; M = 1). Regarding education, six of them have a university degree, four a high school diploma, and two attended only the middle school. In addition, their current employment is diverse: seven are teachers, two are farmers, one is a legal assistant, and one is an accountant.

6.2. Questionnaire

The questionnaire was composed by 27 questions, organized in two sections: general information about both the parent and him/her child, and feedbacks on Shybo. Parents were asked to rate (on five values Likert scale) their agreement to some aspects of the project. Firstly, they had to give feedback on the relevance of the purpose addressed (Q1), namely promoting children’s reasoning and the appropriateness of the solution developed for that purpose (Q2). Then, they had to evaluate how much interesting is the robot’s feature of being trainable (Q3), and if children would like to play with a robot like this (Q4). Finally, parents were asked to say if they think that children can learn from the observation of sound-colour associations showed by the robot (Q5), and if they would allow their children to attend a class that employs the use of this robot in a game with rules (Q6).

Parents were also asked to rate the Shybo’s appearance from a picture provided in the questionnaire. This question consisted of a semantic differential scale, in which eight couples of opposite adjectives were presented. The couples, extracted from the Godspeed questionnaires set [33], are listed in

Table 1. The table presents the list of attributes used to evaluate the robot’s appearance.

Negative Descriptors	Positive Descriptors	Descriptor’s Category	Attribute
Dislike	Like	Appeal	Likeability
Awful	Nice	Beauty	Likeability
Unfriendly	Friendly	Friendliness	Likeability
Complex	Simple	Simplicity	Suitability
Inappropriate for children	Appropriate for children	Appropriateness	Suitability
Unsafe	Safe	Safety	Suitability

.

This part of the questionnaire is meant to address two main aspects of the robot’s appearance: likeability and suitability for children. To do so, three groups of descriptors were identified for each category. Regarding likeability, the scales asked to evaluate the appeal, beauty and friendliness. While for assessing the suitability parents were invited to give feedbacks on the robot’s simplicity, safety and appropriateness for children.

It has to be noted that, due to time issues, parents that were asked to fill a questionnaire had no chance to see Shybo before and their knowledge about the experience, that their children were attending, was based on descriptions included in the questionnaire.

6.3. Playtest

The playtest, which is shown in Figure 2, was organized as a two-hour activity, subdivided into four main phases. The detailed procedure is reported in

Table 2. The table shows the sequence of activities that characterised the playtest with children. In the video sequence column, the time of start and end are reported, together with the number of the video recording. This information is provided to help understand how the video coding process was carried out.

Step	Video Sequence	Description
1	00:00/00:01 (video 1)	Children are welcomed in the conference room of the school
2	00:01/00:35 (video 1)	Each child is invited to take one musical instrument from the front table
3	00:35/01:55 (video 1)	One child at the time is invited to play his/her musical instrument while the other listen
4	01:55/06:01 (video 1)	Children are invited to find sounds similar to the one of their instrument and to make groups of two or three children
5	06:01/08:18 (video 1)	Each group play the instruments again and select a colour, picking a coloured paper card
6	08:18/11:30 (video 1)	The tutor shows a short video that illustrates the functioning of Shybo
7	11:30/15:20 (video 1)	The tutor introduces Shybo to children
8	15:20/21:55 (video 1)	The tutor demonstrates live how to train Shybo
9	00:00/13:00 (video 2)	Each group, one by one, is invited to train Shybo by recording sounds and associating them with colours. During this action one child manage Shybo while another plays an instrument
10	13:00/18:58 (video 2)	After each group trained Shybo, the tutor switches Shybo in play mode and ask for children to play the instrument again. If the robot lights up in the colours associated during the training, it is working
11	18:58/21:56 (video 2)	Children are invited to leave the instruments on the front table, where they found them
12	00:00/04:17 (video 3)	The tutor introduces to children a small paper board of colours and explains briefly what are primary, secondary and complementary colours. She also asks questions to verify that children are understanding
13	04:17/05:51 (video 3)	The tutor introduces a board game that can be played with Shybo. She also explains the rules of the game, especially regarding the game cards and how to move ahead on the table
14	05:51/13:48 (video 3)	Children are divided randomly into four groups, and each group receive a pawn
15	13:48/16:31 (video 3)	The game is set up on the table
16	16:31/21:59 (video 3)	The game starts
17	00:00/20:10 (video 4)	The game goes on
18	20:10/21:57 (video 4)	The game ends when a group arrive at the last box of the board game

. In the first step, children were invited to play musical instruments, create groups of similar sounds and choose a colour for the group. In the second phase, the robot was introduced to children and every child, group by group, was invited to train it by making sounds. In the third step, a researcher, who was leading the experience, introduced children to simple principles of colour theory and then introduced the elements of the game and the rules. Finally, children, divided into teams, played the board game. The procedure’s subsection provides a detailed list of actions that characterised the activity.

Setting and Materials

Although lab experiments allow a greater level of control, wild studies are preferable in those projects in which the interest is in getting results about the effective applicability of the proposed solution [44]. For this reason, the playtest with children was run at Yon Hu Qu Experimental Primary School, in Yuncheng, China. The activity was carried out in the conference room of the school, where a set of tables and other materials were arranged. The materials consisted of twelve musical instruments, five paper colour-cards, four boards of colours, a game board, a set of game cards, four pawns, an hourglass, a laptop and the robot.

The setup was organized as follows. At the centre of the dedicated area was located a group of six small school’s tables, used to create a big table for group activity. In front of this was placed a long table, used to arrange the musical instruments, before and during the activity. On the right side of the central table was located another table were the elements of the game were kept until their use. On the left side, instead, a lectern was placed. This was used to support a laptop connected to Shybo, taking advantage of the height to prevent children from being distracted from the running software. The robot was also placed on the lectern at the beginning of the activity, hidden from children.

7. Results

7.1. Questionnaire

The questionnaire submitted to the parents revealed a general agreement toward the proposed solution, and, except for few cases, the answers were entirely positive. Especially regarding the general aspects of the project (see Figure 3), received very few negative answers. Relating to the purpose of the project (Q1), namely promoting children reasoning and motivation, half of the parents (6) said that it is relevant and two that it is very relevant. Only one parent affirmed that it is not at all relevant. About the appropriateness of using a robot for such a purpose, parents were even more positive: ten out of twelve were positive (8 yes; 2 absolutely yes), while only one was extremely negative. The third question (Q3) also gave a very similar result: nine out of 12 parents find interesting the trainability of the robot (8 yes, 1 absolutely yes), while two said that it maybe interesting and one said not so much.

Regarding other more specific aspects of the project, in Figure 4, the agreement decreased slightly. Parents gave very positive feedback to the question about the likeability of the solution, namely if their children would like Shybo and to play with it (Q4). In fact, ten out of twelve were positive, and none was negative. Conversely, when parents were asked to say if, in their opinion, children may learn from observing the robot’s reactions in terms of color-sound associations (Q5), three answers were totally negative. However, the majority of parents (6 yes; 2 absolutely yes) believe that there is a learning potential. Parents, instead, were quite doubtful when asked to say if they would allow their children to attend a class that employ this robot in a game with rules for the purpose described above. In this case, the positive answers were just four (4 yes). Four other parents stated that maybe they would allow that, while four gave a negative answer (1 not so much; 3 not at all).

Parents were then asked to evaluate the robot on a semantic-differential scale. The descriptors of this questions, were intended to assess, on the one hand, the likeability of the robot and, on the other hand, its suitability for children. As shown in

Table 3. The table presents the mean (M) and the standard deviation (SD) of the six aspects of the robot’s appearance.

Attribute	Descriptor’s Category	M	SD
Likeability	Appeal	4.42	0.79
	Beauty	4.42	0.99
	Friendliness	4.67	0.65
Suitability	Simplicity	4.5	0.8
	Appropriateness	5	0
	Safety	4.42	0.79

, both likeability and suitability aspects gained overall positive feedbacks, with mean (M) values higher than 4.4 for each descriptor. In particular, the mean about the appropriateness of the robot’s appearance for children is equal to the total: every parent considers it appropriate. Another aspect that can be observed is that, although the M of three aspects, namely appeal, beauty and safety, are equal, their standard deviations (SD) reveal greater differences in the evaluation of beauty. One parent, in fact, rated Shybo as almost awful (on the scale from 1= awful and 5 = nice, he rated it 2).

These results suggest that the robot’s appearance is both likeable as well as suitable for children. The open answers at the end of the questionnaire also provide a further chance for understanding the parent’s attitude towards Shybo and the proposed activity. Although not all parents (7 out of 12) answered this question, which was optional, most comments were positive.

The first five comments, listed in

Table 4. The table presents the parent’s open answers about Shybo and the experience.

Parent’s Number	Comment
1	This game can improve children’s observation and reaction ability
2	I really agree that kids can study something at play and all of kids would like to play with Shybo
4	Really like those type of games (game with rules)
5	Its really good for introvert children
7	I hope there are more chance for kids to attend this activity
9	Half of the results with the double of effort, and students should study basic classroom knowledge
11	Good toys and Shybo need to keep on improving and add the anti-interference ability, if not it is just suitable for home play

, manifest the parent’s positive attitude towards Shybo and the proposed experience, and some of the highlight that it can be beneficial for children. Parent 9, instead, points out that, in his opinion, the project requires a double effort compared to traditional activities while produces the half of the benefits. He also states that children have to study basic subjects. The last comment from parent 11, instead, seems to be a warning. She points out that a good toy, in this case, Shybo, has to be provided with an anti-interference ability (presumably for noise issues) and needs to keep on improving over time.

7.2. Children’s Activity

The questionnaire’s results appeared not reliable and were excluded from the evaluation. In fact, although figures supported the semantic differential scales, the aspects under evaluation were too abstract for children of this age. All the answers resulted in the same, oriented on the right axis of the scales.

7.2.1. Again and Again Table and Difficultometer

In addition, the other tools, namely the Again and Again table and the Difficultometer, did not provide highly differentiated results. In the first case, in fact, all children said that they would do every phase of the activity again, except one who stated that “maybe” would do the colour–sound association part again.

Regarding the Difficultometer, some differences were noted. As shown in

Table 5. The table presents the means (M) and the standard deviations (SD) of children’s rating about the difficulty of the main activity’s phases.

Phase	M	SD
Play musical instruments, create groups of similar sounds	1.17	0.58
Deciding a colour to associate with the sounds of the group	1.5	1.17
Train the robot by making sounds	2.17	1.46
Playing the game	2.83	1.64

the first phase was considered very easy by almost every child, the SD, in fact, is also small. In the other phases of the activity, instead, the average difficulty increased constantly, together with the SD. Although according to the M the activities resulted overall easy, some children reported difficulties especially in the last two parts. Training the robot, in fact, was considered difficult by two children and very difficult by one, while all the rest considered it easy. Even more difficult resulted playing the game for some (3 very difficult, 1 difficult).

7.2.2. Video-Recording Observation

The whole activity with children was video-recorded using an action camera, mounted on a tripod. The video was automatically saved in four parts, lasting twenty-two minutes each, for a total of eighty-eight minutes (1.5 h). The videos do not include the preliminary part in which children were welcomed in the room and the final part in which the filled the questionnaire and the other forms.

The video recordings were edited and subdivided according to the procedure listed in Table 2 and subsequently coded using Boris [47], a free and open-source software for video-coding and live observations. Some of the steps of the activity (1, 2, 3, 5, 11, 14, and 15) were excluded from the coding action, after a brief observation. In fact, these parts can be described as transition and propaedeutic steps. Thus, the observation addressed the most salient parts of the activity. The observation was integrated with the transcription and translation of children’s comments.

By observing the recordings, a set of fourteen recurring and relevant behaviours were identified and used for the video coding. Due to the difficulties of observing a group interaction in such a context (e.g., some children cover the others, some parts are too far from the camera, etc.), some of the reported behaviours refer to the overall group’s behaviours. Others, instead, were pointed out when at least one child showed a certain behaviour. These behaviours, listed in

Table 6. The table presents the list of behaviours emerging from the video-coding observation.

Name	Event Type	Description	Reference Sample	Valence
Jumping	Point event	At least one child is jumping in a joyful and excited way.	At least one	Positive
Smiling	State event	Children are overall smiling, there is a joyful atmosphere.	Group behaviour	Positive
Laughing	Point event	At least one child is audibly laughing.	At least one	Positive
Wow	Point event	At least one child express surprise by saying “wooow”.	At least one	Positive
Focused	State event	Children are overall focused on the activity, they look at it, stay close to the tutor A and respond promptly.	Group behaviour	Positive
Concentrated	State event	Children have a serious and concentrated face while following the activity.	Group behaviour	Positive
Asking questions	State event	At least one child ask questions related to the activity.	At least one	Positive
Silence	State event	When required by the activity, children keep silence.	Group behaviour	Positive
Verbal interaction	State event	Children are overall discussing, answering tutor’s questions and commenting.	Group behaviour	Positive
Distracted	State event	At least one child is not focused on the activity, look at something else.	At least one	Negative
Moving around	State event	At least one child is not focused on the activity and move away from the tutor’s position.	At least one	Negative
Instruments noise	State event	At least one child plays instruments while it is not required by the activity.	At least one	Negative
Scratching head	Point event	At least one child is scratching its head because he/she does not know what to do.	At least one	Negative
Bored	State event	At least one child is low reactive, the face is serious and the body is relaxed.	At least one	Negative

, differ also in terms of event type and valence. Some events, such as smiling or verbal interaction, that are prolonged in time, were coded as a state event, indicating a start and stop time. Conversely, others behaviours, characterised by a limited length, such as jumping, were coded as point event.

The behaviours are also divided into positive and negative, according to the activity’s valence they manifest.

The output of the video coding, consisting of ten plot graphs and time budget excel files, allowed to notice the overall feel of the experience, as well as some differences among the various phases. By looking at Figure 5, it is possible to observe the general trend of the four main group behaviours: smiling, focused, concentrated, and silence. Children resulted overall engaged by the playtest. The level of focus on the activity, in fact, was over 90% for almost all phases, while just in phase 12 the focus has fallen sensibly. This decline of attention was reaffirmed by the solo and small groups behaviours illustrated in Figure 6. However, by cross-checking with the video, the distracting element in this phase was represented by the robot. In fact, after the tutor has introduced Shybo to children, this was left on the central table, while the activity moved to the right table focusing on the introduction of some concept about colour theory. Moreover, the introduction of Shybo in phase 7 had an immediate effect. Children, who were very concentrated during the video about Shybo, when the robot was physically presented to them lost their serious and concentrated expression and started laughing frequently. Figure 7 shows a peak in the laughing frequency, which grew from 0.5 events/minute in phase 6 to 2.25 events/minute in phase 7.

Less regular was also the level of smiling behaviour. During the first three phases, in fact, children appeared very smiling, while in phase 8, 9, and 10 this behaviour has fallen dramatically, to rise again in phase 12. However, the declining trend of smiling was compensated by the rising trend of concentration, which greatly varied among the phases. This inversion of the two trends manifests the nature of the activities carried out in those parts. In these phases, in fact, the tutor explained to children how to train Shybo, and let them do it. Furthermore, especially in phase 9 and 10, another behaviour rose together with the concentration: silence. Children were making silence and appeared serious because they were paying attention to the tutor’s explanations and because of the robot’s sensitivity to sound. Both silence and concentration decreased considerably in the following phases, after the training of the robot was concluded and children became familiar with its functioning and the game. This is also noticeable by looking at the trends in Figure 6, where in phase 7 the general verbal interaction decreased considerably while some children started to ask questions about the robot. Nevertheless, the fall of smiling in phase 8 and 9 is not only traceable to the normal robot’s training activity. Looking at Figure 6, in fact, it becomes evident that, in these two phases, some problems emerged. On the one hand, one of the two tutors did not properly connect Shybo to the software that was running on the laptop to perform the training. As a result, children started to get distracted and to appear bored. On the other hand, especially phase 9 in which children trained Shybo, the activity involved children at turns and required them to be quiet, resulting in a decrease of excitement and a rose of boredom.

The level of verbal interaction, as shown in Figure 6, experienced a fall in the central part of the activity, in which children were mostly asked to observe and listen to the tutor explanations, and rose again from phase 12 to the end. On the contrary, the level of instrument noise, namely children playing the musical instruments when the activity did not require it, presents an opposite trend. This, in fact, increased significantly from phase 6 to 10, and ended in phase 12, since, in this phase, children had to leave the instruments on the table and in the following phases they were used for the game. Especially in phase 8 and 9, while the tutor was demonstrating how to train Shybo, children were often playing the instruments to see its reactions.

The final phases of the experience, from phase 12 to the end, were characterized by a very joyful atmosphere. In fact, although Figure 5 shows a slight decrease in the focus towards the activity and in smiling, Figure 7 presents an increase in laughing and a dramatic rise of jumping behaviours. From the beginning of the activity, in fact, children were sometimes jumping, manifesting excitement and joyful impatience. However, when the final game started, in phase 16, the frequency of this behaviour grew by 600%, moving from an average of 0.9 events/minute to 5.61 events/minute. The final phase (17) also reached a peak regarding the excitements, since it represented the end of the game and the victory of one team.

Despite the general excitement and the joyful atmosphere, by looking at Figure 6 it is possible to notice some behaviours that might result in a negative valence of the experience. As already mentioned, the training phase and the error in the setup, in phase 8 and 9, resulted in a rise of boredom. From these phases, the level of distraction also increased slightly. In fact, although in phase 12 this was due to the robot, in these phases, some children started to move around the room, not very far but anyhow distracted from the activity. This was mostly happening when children had to wait for their group’s turn to play. Other children, however, approached these waiting phases by collaborating with the other teams to find the right instruments, or just watched and incited the game staying around the table.

Another important point event illustrated in Figure 7 is “wow”. This represents the actual occurrence of children expressing a positive surprise by saying “wooooow”. This mostly happened in the phases in which new elements were introduced, that are the robot and the game elements. Especially regarding the robot, children expressed curiosity by also asking questions (Figure 6).

The comments of children that were transcribed and translated, highlighted some children’s expectations toward the robot. In fact, a girl asked why Shybo, as a robot, cannot talk and has no legs. Another girl answered her that Shybo cannot talk at the beginning. Some children say that they would like Shybo to be able to reproduce the sounds they play with the instruments or their voices. Another interesting comment is about Shybo getting scared. Rather than thinking that it is getting scared, some of them thinks that it gets angry. In fact, during phase 12, in which children were constantly getting distracted and making noise to see the robot’s reactions, a boy was playing aggressively by walking fast toward the robot with a fist pointed at it. With this behaviour, he was simulating a sort of fight in which he was acting aggressively to make Shybo calm rather than angry.

Finally, Figure 7 shows a point event that occurred only in one phase, namely when children started to play the game. A boy, who was entrusted of finding the right instruments to play, started to scratch his head manifesting a difficulty and making evident that he did not remember which object was associated with which colour. In fact, at the beginning of phase 16, some other children also commented that playing the game was challenging and that they forgot the colour–sound associations.

Another difficulty observed in the game regarded the game cards. Some of the cards were more difficult than others because these asked children to reflect on primary, secondary and complementary colours, rather than showing the colours directly. After some children did this kind of cards, the tutors decided to remove them from the game, because of the difficulty and the time required to children to play with these.

8. Discussion and Conclusions

The evaluation stage of the project allowed getting feedback on the overall validity of the project, on the robot and the playful learning experience.

Regarding the project in general, the parent’s questionnaire allowed investigating two main aspects: the perceived usefulness of the proposed solution, and the compatibility of this with the existing practices. Regarding the perceived usefulness, the parent’s answers confirmed the relevance of the purpose, which is promoting children’s reasoning through play, as well as the learning potential and appropriateness of robots, including Shybo, to support this purpose. Regarding the compatibility, instead, the questionnaire evidenced some resistances of parents towards letting children attend a class that employs a robot such as Shybo. This is probably motivated by the fact that very often there is a fear of adversely affecting the education of children. A comment from a parent, in fact, explicitly mentioned that children have to focus on traditional subjects.

Regarding the robot, instead, both parents and children seemed positive. In particular, regarding the likeability, parents’ results showed a high appreciation of the robot’s appearance and also affirmed that children would like it too. The observation of the activity, in fact, revealed how children were positively impressed and enjoyed by the robot. On the one hand, children were expressing surprise by saying “wow” when the robot was introduced and when it was doing something different. On the other hand, they were constantly looking at the robot, making noise to cause a reaction on it, and some children were also touching it when the group was distracted. Some of these behaviours also revealed that children perceived the robot as animated. Although they had a different interpretation of the scared behaviour, which for some was interpreted as angry, children were acting towards the robot as if it was alive and able to react socially. The observation revealed also that both training and playing with the robot was easy to earn for children, giving a positive feedback in terms of learnability. This was also reaffirmed by the fact that, at the end of the activity, children independently explained to their parents how the robot works by showing them and playing together. Nevertheless, some comments of children highlighted a mismatch between children’s expectations towards the robot’s abilities and the actual abilities of Shybo.

The observations also allowed us to notice a general positive valence of the experience. Regarding the enjoyment, this was revealed by a general smiling atmosphere and the constant increase of the laughing and jumping frequency that characterized the whole experience. The enjoyment of the experience was also confirmed by the Again and Again table, in which children stated that they would like to attend all phases of the experience again. An exception to this general mood was represented by phase 8 and 9 which, however, were compensated with a high level of concentration. Children, in fact, appeared highly focused on the activity for the entire duration of the experience and in many phases they also appeared very concentrated, maintaining silence if necessary. This resulted in a high level of engagement. In spite of this, some issues also emerged regarding the engagement. In fact, on the one hand, when the activity required interaction in turns, and the groups had to wait, some children started to be distracted and to move around. On the other hand, during the explanation of the colour theory’s principles, children were distracted by the robot.

To sum up, the questionnaire and the playtest’s observations highlighted four main issues, regarding both the robot and the organization and communication of the experience. Regarding the robot, children have expectations towards the robot’s abilities that the current prototype does not meet. About the experience, instead, the first issue is that parents may not be in favour of letting children attend a class that implies such experience. With respect to the experience’s activity flow, an issue is represented by the activities in turn, which may lead children to boredom and distraction. In addition, the robot may represent a very distracting element if not integrated in the activity.

Thus, to address these issues, the future steps of the project will consist of a redesign of the robot according to children’s comments and to a reflection on how new features may allow more learning opportunities. In future iterations, the robot’s interactive abilities will be improved by providing more sensory capabilities, larger set of responses, and more motor abilities, such as the ability to recognize colours, emit sounds and move around. This might result not only in more learning opportunities, but also in higher level of stimulation and engagement for children.

The experience will also be improved, focusing on strategies for making explicit the learning potential of such experiences. For instance, the next activities might be designed to be explicitly bind to the school curricula. The redesign of the experience, will also need to pay attention to the issue of waiting times, trying to avoid them, and to one emerging requirement: once the robot is introduced to children, it has to be part of the activities.

Finally, the understanding of the design implications of such projects would greatly benefit from more extensive evaluative studies in which different conditions are tested. For instance, running a same activity about sounds and colour theory, once with Shybo and once with a different “tool” with the same function, might provide a deeper understanding of the actual benefit of having a robot. Similarly, a further understanding of the potential and the limitations of both Shybo and its intended use might result from testing it in different kind of playful learning experiences or in the same experience, but carried out by different educators, rather than the researchers. Accordingly, in the next steps, the project will involve educators.