Abel: Integrating Humanoid Body, Emotions, and Time Perception to Investigate Social Interaction and Human Cognition

: Humanoids have been created for assisting or replacing humans in many applications, providing encouraging results in contexts where social and emotional interaction is required, such as healthcare, education, and therapy. Bioinspiration, that has often guided the design of their bodies and minds, made them also become excellent research tools, probably the best platform by which we can model, test, and understand the human mind and behavior. Driven by the aim of creating a believable robot for interactive applications, as well as a research platform for investigating human cognition and emotion, we are constructing a new humanoid social robot: Abel. In this paper, we discussed three of the fundamental principles that motivated the design of Abel and its cognitive and emotional system: hyper-realistic humanoid aesthetics, human-inspired emotion processing, and human-like perception of time. After reporting a brief state-of-the-art on the related topics, we present the robot at its stage of development, what are the perspectives for its application, and how it could satisfy the expectations as a tool to investigate the human mind, behavior, and consciousness.


Introduction
In recent decades, robots with human-inspired appearance, expressiveness, and cognitive capabilities have been used in the field of healthcare [1], education [2,3], and therapy [4,5]. The encouraging results in these fields of application highlighted that the involvement of humanoids become particularly fruitful in specific scenarios where the creation of an empathic bond is crucial. Accordingly, the perception of robots has also changed over time, together with definitions: no longer Robota (Robota derives from Czech, meaning "labor". The term was coined in the drama of K.Čapek R.U.R. "Rossum's Universal Robots" (1920).) but companions [6,7], assistants [8], tutors [9], or-citing Brooks et al.-"Alternative Essences of Intelligence" [10]. These last words well introduce the other side of the coin: beyond their application as socially interactive agents, many researchers in the humanoid robotics community see humanoid robots as a tool for better understanding humans [11,12]. We do agree with Kemp et al. claiming that humanoid robots offer an avenue to test understanding through construction (synthesis), and thereby complement the careful analysis provided by researchers in disciplines such as cognitive science [13]. This approach, which sees humanoids as testbeds for testing cognitive models by the means of embodied artificial intelligence, highly resembles the "understanding by building" approach [14]. It is related to the intrinsic value of every modeling process: firstly, it is necessary to divide the complex system that must be modeled into simpler functional parts, then the interactions between these parts producing an observable behavior must be highlighted; a priority is given to each part and mechanism in order to decide what is needed for the model and how to make it work. The model then is tested, errors or imperfections are encountered, and finally the model is improved with corrections. Regardless of what motivates this iterative modeling and improvement process (e.g., imitating human motion, cognitive capabilities, emotion processing, personality, or consciousness) it will still produce a two-fold effect: better synthesis and better analysis. The former leads to more believable and skilled humanoids; the latter leads to a better comprehension of human nature.
The scope of this paper is not to provide definitive answers about optimized robot design in the form of a series of guidelines for roboticists, neither to discuss ethical issues, which have been optimally debated in the recent IEEE Global Initiative for Ethical Considerations in Artificial Intelligence and Autonomous Systems [15], but to present an overview, based on our works and scientific literature, on some aspects that appear to be fundamental for the development of social humanoids as research tools for understanding humans. In particular, three factors, which we address to be useful both for the synthetic and the analytic intent, are here discussed: • a believable aesthetics and expressivity of the robot for better conveying and expressing emotions • an embodied cognitive system, with internal emotional representation, that can flexibly adapt its behavior basing both on the robot's internal and external scenario • the possibility to endow the robot with its subjective time In the next section, we discuss why these aspects can be important for the next generation of humanoids, stressing their interconnection and their effect on human-robot interaction. We report a brief state-of-the-art for each topic, lessons learned from previous experiments, and the subsequent reflections that determines the design of a new humanoid that is currently under construction: Abel.
Abel is a humanoid adolescent built in London at the Biomimic Studio by one of the authors (GH). It is conceived by the authors for eliciting an empathic bond with human interlocutors. Abel is covered with flesh-like skin and equipped with sensors and actuators to detect and express emotions at a high level of realism. The humanoid is here introduced with its mechatronic features. Then, a specific section of the paper is dedicated to the robot cognitive system. We present how it is designed, how the features discussed in the introduction can be implemented, and how this integration on the robot would allow its usage both as a platform for cognitive robotics and as a tool for emotional human-robot interaction. Finally, we discuss potential improvements and limitations of the presented system.

Humanoid Body
Fifty years ago, the Japanese roboticist Masahiro Mori proposed the Uncanny valley hypothesis, which predicted a nonlinear relation between robots' perceived human-likeness and their likability [16]. The unpleasant reaction of a human to a particularly humanlike robot, described by Prof. Mori in 1970 [17] was later proven to be dependent on several factors, such as the habituation and exposure to robots, which depends on cultural context, and the disappointment of expectations caused by the misalignment between the aesthetic realism and the actual capabilities and movements of the robot [18,19]. All these factors have inevitably changed in the following years. In fact, humans have continued to pursue their innate tendency to imitate nature and transfer knowledge from biology to the machine [20].
Based on the same principles, the anthropomorphicness of humanoid robots has also increased over the years and continues to improve (examples of humanoid robots with different levels of human-likeness are shown in Figure 1, together with related descriptive publications). A satisfactory review of human-likeness in the design of robots and humanrobot interaction was reported in [21], where the author claims that one approach to enhance people's acceptance of robots is the attempt to increase a robot's familiarity by using anthropomorphic design and "human social" characteristics. Moreover, sharing a similar morphology, robots can communicate in a manner that supports the natural communication modalities of humans [22]. Chaminade et al. investigated the effects of humanoid appearance in human-robot interaction by the means of behavioral and neuroimaging experiments, finding the involvement of an important aspect of social cognition, particularly important in automatic and unconscious perception of another agent, i.e., motor resonance [23]. In their paper, it was highlighted how motor resonance was present only (or increased) in human-robot interactions in which the robot had some level of humanness, in its appearance and motion. They discussed how this may be linked with the activation of social resonance, empathy, and social bonding, which are highly linked with motor resonance in humans [24].  [25] and Pepper [26]; Kaspar [27]; Zeno [28]; iCub [29]; Bottom row: Nexi [30]; Face [5]; Sophia [31]; GeminoidF [32].
Another reason to design anthropomorphic robots is related to the concept of embodiment [33]. It is a consolidated idea in the scientific community that intelligence cannot merely exist in the form of an abstract algorithm, on the contrary, it requires a physical instantiation, a body [34][35][36].
In "Internal Robotics", Parisi described how an organism's body determines the organism's behavior, reporting several robotics examples [37]. In Figure 2, taken from the article, the body is represented as the interface dividing two worlds with which the nervous system interacts: the inner world and the external world. The only possible representation that a mind can have of the world is determined by the possibilities of perception and action with which the body is equipped. Figure 2. As a physical system the nervous system affects and is affected by two components of the environment that lies outside itself: (a) the environment inside the organism's body (internal environment) and (b) the environment that lies outside the organism's body (external environment). Image taken by [37].

©Taylor & Francis 2004.
It is therefore clear that the aim of obtaining an agent with human-like intelligence and behavior cannot be separated from the objective of providing the agent with a body and with the instruments of perception and actuation that are inspired by those of a human being. As a consequence, the body of a robot should be considered not only for its aesthetic appearance but as the specific interface that allows the robot to internalize the information on which to build any abstraction, reasoning, and feeling of what happens.

Humanoid Emotions
The peculiar way in which emotions are recognized, internalized, and exploited by the brain is one of the characteristics that distinguishes human beings from all other beings [38]. As a consequence, it can be said that emotions are essential to define a form of life as a form of human life. In [39], the philosopher Bennett W. Helm claimed that we can understand a creature as an agent in this minimal sense only if it both exhibits goal-oriented behavior structured in accordance with instrumental rationality and displays a pattern of noninstrumental rationality characteristic of significance and of the emotions. Dennett described emotions as the main tools, given by evolution, for the generation of intentionality and consciousness [40,41]. Accordingly, neuroscientists demonstrated how emotions are linked with the body and the sense of self [42,43], their role in optimizing human decision making [44], and their indispensability for the emergence of human consciousness [45].
Based on their demonstrated relevance, emotions had a prominent role in social robotics, and they were integrated in humanoids with two different aims: imitating the human empathic interaction and modeling the human mind-body emotion process. In order to establish an empathic connection with its interlocutor-what Paiva calls an affective loop (see Figure 3)-the robot must be equipped with a various range of sensors, particularly for audio-visual perception and an acquiring system capable of recognition and classification of high-level data (e.g., words, facial expressions, gestures). This information is needed by the robot to be constantly aware of the user's affective state. Likewise, to elicit emotion in the user, the robot should be able to respond accordingly through human-like facial expressions, gestures, and utterances. These aspects of emotion improve the naturalness of the human-robot interaction (e.g., [46,47]) and concern external robotics. In case the robot is aimed to be more than an engaging emotional character, but also a research tool for modeling human emotions and studying their implication with cognition (e.g., mood, personality, consciousness), then emotions should be available also in the form of internal values and then abstract representations. It follows that, in order to generate human-like behavior and cognition, the human emotional process must be modeled and integrated in embodied cognitive architectures [49].
Although not all of them have been conceived to be embedded in humanoid robots, Bio-Inspired Cognitive Architectures (BICA) [50] are successful examples of imitation of functions and structures of the human brain and often include emotional behavior generation. In particular, the ones that in [51] are classified as Hybrid Architectures (e.g., LIDA [52], CLARION [53], DUAL [54], MicroPsi [55], and OpenCog [56]). These architectures are characterized by having both a sensorimotor connection and a deliberative path to manipulate symbolic concepts and planning ponderate actions. These two coexistent connections, i.e., reactive and deliberative, are equally important for emotional behavior generation: the former guarantees the direct and continuous link between the environment and the body, e.g., the fast and unweighted reaction happening when basic emotions are induced and expressed; the latter is necessary for symbolic abstraction, e.g., the mental representation of secondary emotions, the generation of beliefs, and the construction of desires and intentions.
For what concerns the modeling of emotion in the robot cognitive systems, instead of viewing emotions in terms of categories (happiness, disgust, fear, etc.) the most widespread school of thought is that of conceptualizing the dimensions that could span the relationship between different emotions (valence and arousal, for instance) [22]. This concept, coming from psychology with different variants [57][58][59][60], was well accepted by social roboticists, who needed a numerical representation and saw the possibility of shifting the emotional state as a movement of a point in an emotional space, which form and dimensions may vary depending on the model. An artificial agent, built according to the materials and methods outlined so far, would be a robot with a humanoid body endowed with social perception, human-like expressive and communication capabilities, controlled by a bioinspired hybrid cognitive system that allows the representation of emotions also as abstract constructs.
If this is sufficient to obtain a likable emotional conversational agent, it would be not enough for the goal of creating an artificial lifeform that is useful for human understanding. For this scope, a third component is missing.

Humanoid Perception of Time
An artificial agent that has to model and interact with a human must be designed to share both the same spatial environment and the same temporal dimension of humans. For example, primary and secondary emotions, as well as personality and consciousness, evolve (or change) in different timescales. Nonetheless, humans do not have any perceptive apparatus capable of directly measuring time; therefore, the brain makes an indirect reconstruction of time [61,62], through a coding very similar to that discovered by O'Keefe in the hippocampal neurons implicated in spatial navigation [63]. This internalization of time described by neuroscientists has been confirmed by behavioral psychology experiments, which demonstrated also that humans have subjective time perception that is affected by their emotional state [64,65]. In addition, attention has a role in time estimation [66]. Differences emerged when subjects were asked to perform time judgments under prospective conditions (in which subjects are instructed to attend to time) and retrospective conditions (in which subjects are unaware that they will be required to judge time) [67]. Another condition in which time duration perception may be biased is when decision-making tasks have to be performed under time stress [68].
All these results demonstrated that human time perception is naturally subjective and strictly connected with the emotional context that is present in the body and in the environment of the agent who judges time [69].
In the literature, there are rare cases of implementation of cognitive systems that take time perception into account, e.g., the DESTIN cognitive architecture [70,71], which is a spatiotemporal inference network capable to inherently capture both spatial and temporal regularities in the data. Even more rare are the cases of embodiment of time perception models in artificial agents, for which TimeStorm [72] must be cited. TimeStorm is an H2020 EU Project aimed at investigating the temporal attributes of human-machine synergetic interaction, adopting a multidisciplinary research approach that involves developmental studies, among which they consider embodied experiments.
Without any doubt, there is a lot to do in social robotics on this side. The emulation of the distortions found in human perception of time, once applied to humanoid robotics, would lead to a more natural human-machine alignment in the temporal dimension. Furthermore, this could be a first step towards the possibility to create some sort of artificial phenomenal time, i.e., the perceived qualitative awareness of the relationship between futurity and pastness, that is a fundamental element of human awareness and their sense of self. This kind of approach in humanoid robotics is still unexplored, and an embodied cognitive system with human-inspired emotional processes and time perception has not yet been developed.

The Making of Abel
Abel is a new generation humanoid robot, conceived to be a research platform for social interaction, emotion modeling, and studies on embodied intelligence. The name Abel derives from the Hebrew Hevel, or Havel, which means 'breath', 'vital breath', or 'vapour'. Its appearance resembles that of an 11-12 year old boy. It is a unique piece, resulting from the collaboration between the Enrico Piaggio Research Center of the University of Pisa [73] and Gustav Hoegen, from Biomimic Studio [74], based in London. Abel is a humanoid resulting from the merge between researchers with bioengineering and robotics background and animatronics artists. The collaboration between engineers and creatives with such an artistic inspiration is fundamental for the creation of a humanoid like Abel, i.e., an emotional robot. Actually, the importance of being emotionally evocative [75] drove the design of the Abel's body (Figure 4). The robot design has been chosen after years of experience with the FACE robot (Facial Automaton for Conveying Emotions-bottom row, second from the left, in Figure 1), a social robot with human-inspired facial expressiveness that has been used in therapy with children affected by Autism Spectrum Disorder (ASD) [5] and in educational contexts as synthetic tutor [3,9].
For what concerns the mechatronic design, a creative approach is adopted by GH, who was responsible on this side. His design method does not start from an a priori idea of the mechanisms that will move the robot. On the contrary, the mechanics are dictated by the shape of the face and the body of the robot, as well as by its conceptual design. In this approach, it is the creature's body itself that communicates to the creator the most natural and effective way to control it, in order to make it expressive to the maximum of its possibilities. The realism of the movements is obtained by entrusting the tough part of this task to particular mechanical transmissions, which translate a simple linear movement of an upstream servo motor into a sophisticated movement downstream, i.e., movements of the skin or the frame of the android. The mechatronics of the head confers to Abel the possibility to express a wide spectrum of emotions by means of facial expressions, which are accompanied by a body which is also designed to represent emotional and meaningful body gestures.
Because of their relevance in generating the illusion of life in the human interlocutor, a particular focus has been given to eyes and mouth movements [76], that have been designed to perform a believable gaze movement, and lip-sync for verbal communication (a detail of Abel's head mechatronics is shown in Figure 5, on the right).

Figure 5.
A detail of Abel's head. On the left the head is covered with bioinspired skin-like material; on the right the internal mechatronics exposed, designed to perform facial expressions, gaze behavior, and lip-sync speaking.
Abel is physically made up of the head and the upper part of the torso with arms and hands, all of these are robotic parts moved by the latest generation of Futaba, MKS, and Dynamixel servo motors. Twenty-one servo motors are inside the Abel's head, they are dedicated to the movement of the facial expression, to perform gaze, and simulate speaking: four move the brow, eight move the eyes, one moves the jaw, and eight are for the movement of mouth, lips, and cheeks. Five motors are dedicated to neck and head movement. Then, five servo motors are mounted in each arm (three for the shoulder, one for the elbow, one to twist the arm), and three servo motors are in each hand, for a total of 42 degrees of freedom.
The humanoid is equipped with an integrated camera into the torso and integrated binaural microphones, which are specifically designed to emulate the acoustic perception of the human listener. The robot also has an internal speaker to reproduce its voice.

The Mind of Abel
The cognitive system of Abel is an adaptation and extension of SEAI (i.e., Social Emotional Artificial Intelligence), a bioinspired hybrid cognitive architecture we devised and implemented, based on Damasio's theory of mind and consciousness [77]. SEAI is a modular architecture, conceptually divided in Figure 6 between the characteristic Sense, Plan, and Act functional blocks. Modules are grouped forming services, which are designed as standalone applications. A service can be an application for image analysis, an animator of a robot part, or a component of the cognitive system which is dedicated to the processing of a specific domain of information. These services can be distributed in different computers and the communication among them is implemented with the YARP middleware [78], by which we create a local network dedicated to robot control. The framework allows both low-level reactive control, by means of direct connection between perception and actuation control services and a high-level deliberative control, which includes the possibility to plan context-dependent actions and to perform abstract reasoning on the acquired sensory data by means of symbolic manipulation. Figure 6. A functional framework of the SEAI cognitive architecture (Social Emotional Artificial Intelligence). Image taken from [77]. More details on single services reported in the figure can be found in [79][80][81].
Scene Analyzer is the social perception system responsible for processing the environmental information gathered by the available sensors and extract data with social or emotional significance, e.g., 3D position of human interlocutors, gestures, facial expressions, speaker probability, estimated age, and gender [80].
Such information detected by the perception system can lead to an immediate physical reaction, like a movement or a facial expression, in the body of the robot [81] (i.e., the reactive path) and/or constitute a trigger for emotional or reasoning processes that will lead to more complex behavior (i.e., the deliberative path).
In this latter case, the information passes through the symbolic part of the cognitive architecture: the I-CLIPS brain [79] implemented on CLIPS as a rule-based expert system [82]. Including this approach in the cognitive architecture allows for the easy simulation of human expertise in certain domains, manipulate abstract concepts, and perform deductive and inductive logical reasoning by using forward and backward chaining in the rules engine. Moreover, it is possible to implement patterns of behavior, for example, programming rules that determine the modification of robot internal values depending on external events affordable by the perception system or on previous internal values, and rules that determine how the change of internal values influence the cognitive processes of the robot, and then its behavior.
SEAI has been successfully exploited to implement the influence of emotion in decision-making described by Bechara and Damasio as the mechanism of the somatic marker [44]. Implementation details and results are available in [77,83], where this mechanism has been tested in a real context of human-robot interaction using the FACE Robot. Furthermore, a model of subjective time perception, which takes into consideration the influence of the emotional state of the agent has been recently implemented in SEAI and presented at the 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN 2019) [84].
Our perspective is to implement all these features in the Abel robot, optimizing both its role as a research platform to model human cognitive processes and as an empathic social robot able to interact more and more naturally with humans.

Body
Although we have highlighted the importance and reasons for using an anthropomorphic corporeity, many studies must be conducted regarding the details of this aesthetics. For example, the perceived gender and age of the humanoid may influence the induction of trust, empathy, and the perceived level of competence of the robot. Future studies based on the comparison among different embodiments in equal interaction scenarios are probably a good approach to find (if they exist) correlations that determine which aesthetic or physical variants are better in different applications and contexts.
Moving to the involvement of the body in emotion and cognition, our intention is to provide Abel with a deeper link to its body, completed by a proprioception system. We are considering the application of ink-based conductive material for the sensorization of the skin of the robot, a method that allows extrapolating position and movements from the stretch of the sensorized skin. This would make it possible also to discern tactile interactions with emotional content.

Emotion
For what concerns emotion recognition, we are studying how to extend Abel perception connecting its sensory apparatus to wireless and wearable affective computing devices. The program application of each device should be integrated into the SEAI cognitive architecture as additional services of the SENSE block in the YARP network. This extended perception of the robot on the body of the human interlocutor would augment intensively the reliability of the robot's beliefs about the other's emotions. A preliminary experiment of a similar integration was made in [85].
Some limitations, then, regard the emotion modeling, for which we refer to the mentioned Russell's Circumplex Model of Affect [86]. The bidimensional model which describes emotions by means of the two coordinates of valence and arousal is useful and probably sufficient for guiding the facial expressions of social robots, while it is certainly too reductive if we move to a higher level of emotional states, such as the mood of the robot. In the future, we will try other methods of spatial representation of the emotion, which would make a strong contribution to the possibilities of generating complex emotional behavior.
Moreover, the mechanism of somatic markers, included in SEAI, should be appliedas in humans-not only to label external entities but also in the labeling of concepts and decisions, based on the emotional consequences perceived in the subsequent events, therefore influencing the possibility for the robot to choose or not the same decision branches in future similar situations. A similar application would lead to something that resembles the formation and development of a personality. Indeed, the robot would have available both universal laws of social contexts (in the form of behavioral rules in SEAI) but it would also be able to autonomously develop opinions, reticences, preferences, and preferred modalities of behavior, according to its subjective past experience.
Finally, since we included verbal communication capabilities in Abel, a speech recognition and analysis software capable of extracting both the textual content and the emotional content (i.e., prosody) of the speakers' utterances must be developed and integrated into the perception system of the robot. Likewise, an emotion-based speech synthesis program modulating the voice of the robot according to its internal emotional state must be included.

Time
Introducing human-like time perception in humanoid social robotics is a very recent and largely unexplored objective. We do believe that time will have a central role in the future of social robotics.Especially if we think about emotions, a lot has been done in terms of recognition, synthesis, expressiveness, but what do we know about the dynamics of emotions? Furthermore, how is the evolution of emotions over time implemented in robots? Time is a key factor, especially when considered in association with emotions and social behavior [87].
For an extensive discussion on the different implications of time on cognition, that should be implemented in social robotics, we refer to [88], from which we report here: temporal degradation of memory, memory reconsolidation, estimating time duration, and time-warping emotions (e.g., depression, sadness, urgency, anger, pleasure).

Conclusions
We do expect that humanoids, in the next future, will be greatly improved in terms of their acceptability, believability, and efficiency in the mentioned fields of application and many others not yet foreseen, but they also will prove to be a key tool for a better understanding of human nature, behavior, mind, and consciousness. They will be a mirror, which we can reflect on and understand. What has been remarked in this paper is that a humanoid artificial intelligence would greatly benefit from a higher integration of mind and body, both human-inspired in form and functions; that this mind-body creature should have the autonomy of interact with the same world in which humans are living, using similar ways of extracting information from it and similar ways of processing and use this gathered information; and that this mind-body agent should be also aligned in the same temporal dimension in which humans live, in order to understand their emotions and have similar emotional dynamics. Emotions are essential for the generation of personality, character, and consciousness itself. This means that, if an artificial consciousness can ever exist, it cannot disregard body, emotion, time, and their mutual interaction. Abel has then been presented at its actual stage of development. The authors constructed this new social robot in order to explore the topics discussed in the present paper. The SEAI cognitive system will lead to the possibility both to implement cognitive models from mind theories and to design robot behavior in therapy or interaction contexts. The robot will be used also to test the possibility for an artificial agent to perform human-like temporal distortion according to its internal and external emotional environment, which we consider the first step towards a deeper alignment of human and robot time dimensions.
The future studies that will be conducted on Abel and on this new generation of robots, on how to better model and replicate human properties, will undoubtedly bring results to the double intention of humanoid robotics: producing robots that will be able to contribute more effectively and more naturally to the life of humans and at the same time bringing us deeper knowledge of ourselves.  development of Abel. In particular, we would like to thank Pasquale Scilingo, Arti Ahluwalia, Giovanni Vozzi, and Caterina Giannetti.

Conflicts of Interest:
The authors declare no conflict of interest.