A Review of the Practical Applications of Pedagogic Conversational Agents to Be Used in School and University Classrooms

: Pedagogic Conversational Agents (PCAs) can be deﬁned as autonomous characters that cohabit learning environments with students to create rich learning interactions. Currently, there are many agents reported in the literature of this fast-evolving ﬁeld. In this paper, several designs of PCAs used as instructors, students, or companions are reviewed using a taxonomy to analyze the possibilities that PCAs can bring into the classrooms. Finally, a discussion as to whether this technology could become the future of education depending on the design trends identiﬁed is open for any educational technology practitioner, researcher, teacher, or manager involved in 21st century education.


Introduction
In the last decades, there has been a great deal of research into how computer systems can be applied to assist education. E-learning courses are more common, evolving from transcriptions of textbooks and traditional lecturers to sophisticated environments. Moreover, Blended Learning, which can be defined as the combination of face-to-face lessons with other computer-based instruction methods, can provide even more benefits [1].
Some educational computer systems have integrated Pedagogic Conversational Agents (PCAs), which can be defined as, "lifelike autonomous characters that cohabite the learning environment creating a rich interface face-to-face with students to create rich learning interactions" [2]. Some reviews of PCAs can be found in [3][4][5][6].
According to [7], students communicate with the agents through conventional input channels such as speech, keyboard, gesture, or touch panel screen, and the agents communicate with the students through speech, facial expression, gesture, posture, and/or other embodied actions. Whenever the agent has a body, it can be called Pedagogical Embodied Conversational Agent [8,9].
Moreover, from a pedagogic point of view, the agents can take different roles when interacting with the students, such as mentors, tutors, peers, players, or avatars in virtual worlds. Depending on the pedagogic role of the agent, they can be called Teachable Agent when they take the role of a student who learns from the user [10,11], and Pedagogic Agent as Learner Companion (PAL) when they take the role of a peer student [12,13].
Several benefits of the use of agents in the learning systems have been published since 1997, when the Persona effect [14], according to which, just the presence of an interactive agent in an educational computer environment has a positive influence in the students' perception of the learning experience; the Proteus effect [15], according to which, students are motivated to achieve the features of the agents to become more like them; and the Protégé effect [16], according to which students can make greater efforts to teach their agents than to study on their own, were discovered. More than twenty years later, how pedagogical agents can be beneficial in learning environments has been re-examined depending on the learning problem, application, and population. In particular, some of the capabilities of PCAs that have proven beneficial in learning environments are interactive demonstrations, navigational guidance, gaze and gesture as attentional guides, nonverbal feedback, conversational signals, conveying and eliciting emotion, virtual teammates, and adaptive pedagogical interaction [17]. The positive effects of increased motivation, sense of ease and comfort, stimulation of essential learning behaviors, enhanced flow of information and communication, gains in terms of memory, problem solving and understanding, and fulfilling the need for deeper personal relationships in learning have been highlighted when using pedagogical agents in learning environments [18].
However, it is still not clear that all are benefits, and the full integration of agents in e-learning environments keep debatable as too humanlike agents and non-task interactions in the dialogue can create an undesirable "uncanny valley" [19][20][21]. More research should be provided to overcome the limitations found [22]. Students may distrust the computer when the agent repeats the same sentence twice, or when the agent generates a sentence about something completely unrelated to the previous students' utterances. Therefore, many of the existing PCAs are currently being used only for research.
In this paper, several trends in designing PCAs are reviewed. To do that, the literature has been searched in three phases (see Figure 1):   A taxonomy is used as a tool to analyze the possibilities that PCAs can bring into the classrooms. Finally, a discussion as to whether PCAs could become the future of education, depending on the design trends identified, is open for any educational technology practitioner, researcher, teacher, or manager involved in the 21st century education.
The structure of the paper is as follows: Section 2 reviews several PCAs taking the role of instructor (teaching and assessing); Section 3 reviews several PCAs taking the role of student; Section 4 reviews several PCAs taking the role of companion; Section 5 provides the discussion of the design trends found in the review; and, finally, Section 6 ends the paper with the main conclusions and the possible future of PCAs to be used in school and university classrooms.

Herman the Bug
In 1997, 100 students used the insect shape Herman agent [14] to design a plant. Herman the Bug has different combinations of gestures and verbal advice. Students were split into groups of 20 to use a different mode of the agent. It was observed that just the presence of the agent was positive for all the students (even the ones using the muted agent). However, after the research carried out at the end of the nineties, no more studies have been found indicating the use of Herman in class as an educational tool in class.

Steve
In 1999, the PCA Steve [23] was designed and trialed as a human shape agent who can collaborate with the students represented with their avatars in a 3D virtual world. The world represents a ship to teach navy training by asking students to complete certain tasks. Steve avoids giving orders to the students. Instead, he uses a mixed-interaction dialogue (i.e., both the agent and student can start the dialogue) to help the student to complete the task together with some basic emotional support. Steve has been used in several experiments. However, it is not reported that it is currently being used in class.

Guilly
In 2002, the PCA Guilly [24] was designed and trialed as a worm shape agent to teach children how to recycle. It talks to the students who must complete the tasks requested. As the students can successfully complete the tasks, the environment gets greener to show the benefits of recycling to them. However, Guilly does not seem to be longer available. It is not currently being used in class.

Sam
In 2003, the PCA Sam [25] was designed and trialed as a human child shape agent. The experiments carried out with Sam have shown that children are able to learn from Sam's storytelling skill to improve their own storytelling skills. Sam was projected behind a big toy castle on a wall to engage children in telling stories. However, Sam is no longer available. It is not currently being used in class.

Autotutor
In 2005, the PCA Autotutor [26] was designed and trialed as a human shape agent that uses Latent Semantic Analysis [27], and a curriculum script database with the questions and dialogue moves to manage the conversation with the student. The dialogue is always oriented to build an answer to a question between the agent and the student according to a constructivist model. According to its authors, Autotutor works well when the shared knowledge between the student and the agent is low or moderate. On the other hand, when the shared knowledge between the student and the agent is high, the conversation may become erratic. The agent is still under study.

Baldi
In 2005, the PCA Baldi [28] was designed and trialed as a human shape agent able for language training. Baldi can achieve satisfactory results to accomplish his goal, provided that the students drive the conversation by clicking on the objects to be pronounced. The agent is still under study.

Overview of PCAs as Students
As a representative of the agents assuming the role of students following the "Learning by Teaching" paradigm, Betty will be described [29]. Betty is an agent with human shape designed and trialed since 2009. She is represented as a little girl who wants to learn about science. Therefore, she waits that children teach her, and later, she answers questions asked by the children to check whether she has understood the lessons. Therefore, the conversation is driven by the students, and it is supported by a concept map [30]. Betty has been designed to be used with a computer with graphics and sound animation in English.
If the information provided to Betty is incorrect, another supervisor agent called Mr. Davis indicates when there is something wrong. That way, children are not allowed to lie to Betty (whether intentionally or not).
The use of agents following the "Learning by Teaching" paradigm is still under study, currently exploring the benefits of integrating contextualized conversational feedback [31], positive feedback [32], and collaboration possibilities [33] in the student-agent dialogue.

Overview of PCAs as Companions
In this section, the agents assume the role of companions following the "Learning by Doing" paradigm. That is, the agent is not designed to teach or to act as a student, but it serves to provide emotional support to motivate the students to keep studying and to solve certain tasks in the environment. In these cases, the agent-student natural language conversation is usually less important, even replaced by gestures or static menu-based choices. On the other hand, the 3D animated graphical environment is improved, and more effort is placed on trying to empathize with the students' feelings.

SBEL Agents
In 2007, the SBEL human shape agents [34] were designed and trialed to serve as companions to teach Brazilian Portuguese to children and adults in different contexts. However, this work seems no longer under study, since there are no new papers published with the results of using SBEL agents in class.

Crystal Island Agents
In 2009, the Crystal Island immersive 3D learning environment with sound and several animated human companion agents, the Crystal Island agents, was designed and trialed to help students (children and adults) to solve certain tasks [35]. The published results seem to validate the hypothesis that emotions serve to foster learning with emotional companions. However, these agents are still under study.

Jake and Jane
Similarly, as in the case of the Crystal Island agents, Jake and Jane [36] were agents designed and trialed since 2009 to try to foster learning by providing emotional support. In this case, by empathizing with 3D graphics, sound, and the taxonomy of emotions provided by [37] with a male agent (Jake) and a female agent (Jane). The hypothesis is that female students would learn more with a male agent. However, that hypothesis is still under study [38,39].

MyPet
In 2009, the PCA MyPet [40] was designed and trialed as a pet shape agent focused on improving Chinese students' motivation. The core hypothesis is that if you make an effort you can achieve success. Therefore, MyPet measures the effort (how many exercises the students have tried) to show students that success or failure is dependent on effort. However, authors have still not found any evidence of learning improvement when analyzing pre-post test data in students using MyPet. This issue is still under study.

BILAT Agents
In 2009, the BILAT agents [41] were designed and trialed with human shape as characters in a Serious Game. It is a similar approach to the Crystal Agents but with the goal of training negotiation skills. The conversation, however, is limited to choosing sentences from menus. All in all, evidence was reported of an improvement in the training skills of the students who used the BILAT agents. However, no more research using the BILAT agents has been found.

Discussion
From the review of the literature carried out, with the taxonomy presented in [42][43][44], with social [45] and Human-Computer Interaction (HCI) [46][47][48][49][50] criteria have been chosen and revisited as a tool to analyze the possibilities that PCAs can bring into the classrooms.
Sections 5.1-5.3 describe the pedagogic, social, and HCI criteria included in the taxonomy, and Section 5.4 ends the paper with the open discussion of whether PCAs could become the future of education. It is important to highlight that many criteria can be interrelated and that the possible values of the criteria are not binary as possible gradations can be found even within the cross-branch relationships and combinations.

Pedagogic Criteria
According to [43], a classification of PCAs should start from a learning and teaching model. Therefore, the first criterion is role and attitude of the agent, i.e., depending on the pedagogic goal pursued, agents can be classified as taking several roles [5,44].
Originally, Ref. [50] distinguished between the roles of authoritarian teachers (tutor, coach, and guide) as the traditional role of the teacher in a class, learning companion or co-learner (peer tutor, tutee, simulated student, collaborator, competitor, troublemaker, critic, and clone) which can be a friend, a competitor, critic, etc., or a personal assistant (teacher assistant or student assistant).
However, since 1999, IT teaching has evolved so that it is more student-centered and authoritarian roles, albeit possible, are just one possibility that coexists with the trend of assistant agents whose pedagogic role is to serve as a guide (coach). Moreover, a new pedagogic role has appeared: agents as students who learn from the student [29,31,33], and the role of companions is kept [16,40], adding more emotional support (see Section 5.2). Given that it is possible that in the future more roles appear, criterion 1.4 has been added for other possible future roles.
To sum up, the role focuses on the function assumed by the PCA. It can be as a source of knowledge and evaluator in 1.1, as a recipient to be taught in 1.2, or as a person, robot or animal with whom to spend time and get help if needed. The attitude of the PCA focuses on its way of thinking or feeling. In the role of instructors/lecturer/tutor, PCAs can be authoritarian and try to impose their way of thinking to the students. Students using authoritarian PCAs must obey the rules provided and do as they are told. On the other hand, students using coach PCAs are guided in their learning by being asked questions. PCAs in the role of students can be on their own (individual) or to work in pairs or groups (collaborative). It is possible to have more than one role when there is more than one PCA. One could have an authoritarian instructor role and another PCA could be a student. Finally, three types of attitude can be distinguished for companion PCAs: peer companion that works as an equal, i.e., another student that also wants to learn with sentences such as, "What would you say if you are asked [ . . . ]?"; emotional assistant that do not want to learn or teach but just to be there with sentences such as, "How are you doing? Congratulations! You did it! Keep it on!"; and the troublemaker attitude that is adequate for students who need to be challenged with sentences such as, "Is this the best you can do?", always trying to create conflicts to the student so that they try to improve, even telling them that something they have said is wrong (albeit it was right) just to find out how sure the students are of their answers.
Hence, the first criteria would be: The second pedagogic criterion (Interaction Modality) is based on the teaching modes described by [33]: teach/explain, observe/help, or cooperate/support. In this case, besides taking a role, the kind of interaction changes, because when the teacher is explaining, s/he is starting the conversation as in 2.1; when the student is talking, the teacher is listening as in 2.2; and sometimes both teacher-student are talking in a mixed cooperative dialogue as in 2.3. Hence, the second criterion would be:

2.
Interaction modality 2.1. The agent starts the conversation 2.2. The student starts the conversation 2.3. Mixed (both the agent and the student can start and continue the conversation) The third pedagogic criterion (Domain) is based on the statement claimed by [33]: "The model (of interactive educational software) separates dimensions for (meta) contents, students' activities and teaching." Therefore, it is important to take into account that it is not the same to design agents for a general domain as in 3.1 or more specific domain as in 3.2, the contents may change, the students' activities and the teaching. Hence, the third criteria would be:

Social Criteria
According to [44], when dealing with PCAs, social factors should also be taken into account. Therefore, the fourth criterion is affective possibilities. It could be no affective possibilities as in 4.1, some basic emotional support because people tend to respond to communication technologies as they would to another person [23,32,36,48,51] even with some small talk [52] as in 4.2, or even reaching rapport levels [26]

Human-Computer Interaction Criteria
According to [45][46][47][48], there are several HCI criteria that should be taken into account when working with interactive software. Moreover, interactive agents can get benefit from applying User-Centered Designs, UCD [53]. UCD consists in taking into account the needs of the students and teachers in the design of the agent, i.e., the type of character, adaptationevolution possibilities, ubiquity, type of animation, language, and students' features. Therefore, a fifth criterion that has been included in the taxonomy is the type of virtual character. According to [14] the possibility of Herman the Bug of pointing with fingers at objects had a positive effect in the student-agent interaction. Moreover, agents able to produce specific non-verbal feedback cues, such as nodding or shaking its head, and display facial expressions such as smiling or raising its eyebrows have also a positive effect in the student-agent interaction [44].
Furthermore, the likeability of the agent leads to a higher motivation and increases transfer performance [54]. Something similar was reported when the Proteus effect was discovered [15], since the type of virtual character has an effect of the interaction, and students tend to feel more confident if their agent shows a confident attitude. To sum up, the PCA could take a human shape without animation just a face for instance in the corner of the e-learning system as in 5.1, or it could be a human shape animated with gestures and even breathing as in 5.2. It is also possible that the PCA does not have a human shape such as an animal or robot without animation as in 5.3, or with animation as in 5.4. Hence, the fifth criterion would be:

5.
Type of virtual character 5.1. Human shape without animation 5.2. Human shape with animation 5.3. Non-human shape without animation 5.4. Non-human shape with animation The sixth criterion is the adaptative-evolution possibilities. It has been included from the inspiration of reading the work of Liebermen [55], in which he claimed that autonomous agents can show adaptive and evolution possibilities, i.e., the agent adapts its behavior to each student, and learns how to evolve to interact better depending on previous interactions [56][57][58][59]. PCAs may not have either adaptive or evolution possibilities as in 6.1, to have adaptive possibilities to modify their dialogue and/or gestures to the student as in 6.2, to have evolution possibilities and to learn from the previous sentences of the dialogue to generate different interactions (verbal and or non-verbal) as in 6.3, and even to have adaptive and evolution possibilities so that from the learning of previous and present interactions the behavior and/or dialogue of the agent changes as in 6.4. Hence, the sixth criterion would be:

6.
Adaptive-evolution possibilities 6.1. None 6.2. Adaptive possibilities 6.3. Evolution possibilities 6.4. Adaptive and evolution possibilities The seventh criterion is the ubiquity, i.e., the possibility that the agent is not only used in a computer but in other devices too. In [60] claimed that where teaching and learning takes place is relevant. Teaching and learning are highly social activities [52].
Moreover, the design of the interface for a computer is different than for mobile devices with smaller screens, lower resolutions, and maybe used in places where the sound should be disabled [48]. The criterion when the PCA is to be used with a computer would be 7.1; it would be 7.2 with mobile devices; 7.3 with robots; and 7.4 with responsive design that allows the synchronization of the PCA across the students' devices. Hence, the seventh criteria would be: The eighth criterion is the type of animation, which is related to the type of virtual character and the ubiquity. This is because if the virtual character does not have a body, then some animations would not be possible. Similarly, it the animation is for mobile devices in which processors could be slower, it should be less hardware demanding.
Ref. [44] also highlighted that the voice of the agent is important both for text-based interaction and in general, for multi-mode interactions. Therefore, the use of sounds and graphics should be taken into account when designing pedagogic agents. It could happen that the PCA did not have any type of animation as in 8.1, that it only includes graphics as in 8.2, that it includes both 2D graphics and sound as in 8.3; that it includes 3D graphics and sound as in 8.4; and to create a more or less immersive experience by using virtual reality as in 8.5. Hence, the eighth criteria would be:

8.
Type of animation The ninth criterion is the language, written and/or spoken by the agent. It is because depending on the language of the agent, different Natural Language Processing tools should be used. As [28] claimed, "It is valuable to add new languages to extend speech and language research." It should be considered whether the interaction is only written, for instance, by typing in a chat with the agent in one language as in 9.1; it could also be possible that the interaction is more natural as it involves written and spoken language even with speech intonation and recognizing the voice of the student as in 9.2; it could also be possible that more than one language is used only written as in 9.3; or that more than one language is used both written and spoken as in 9.4. Hence, the ninth criterion would be: Finally, although more features may appear in the future, the tenth criterion would be the features of the students. According to User-Centered Design (UCD), the agent should be designed taking into account the type of users, in this case the type of students [53]. In [38,39] studied the impact of learner attributes and learner choice in an agent-based environment. Hence, the tenth criterion would be:     Table 1 shows a comparison of the twelve Pedagogic Conversational Agents reviewed in this paper according to the criteria identified in the taxonomy. The agents have firstly been ordered according to their role, and secondly by chronological order.

Comparison
As can be seen, no agent is able to teach/be taught in open domains, but the trend is to focus on specific domains. Moreover, only 3 out of the 12 reviewed agents interact with the student in languages different to English, and 5 out of the 12 reviewed agents support mixed-initiative dialogues, in which both the agent and the student can change the turn of the conversation.
Thus, the most common combination is that the conversation is driven by the agent in a specific domain in English. However, it could diminish the believability of the dialogue since in human-human conversations, we can change the turn of the conversation, and even introduce other related topics and some small talk [52]. It would be advisable to have mixed-interaction dialogue with some small talk as well as talking about the knowledge domain.
Affective computing is also important to create realistic human-computer interactions. In the case of PCAs, affective computing could be even more important because students may feel alone studying, and they can appreciate the emotional support provided. Companion agents that provide this type of support are flourishing in the literature of the field [13,43,[61][62][63]. It is highly probable that the future of Pedagogic Conversational Agents for education will keep that emotional support line, as can also be seen in the workshops and conferences on the field. It is expected that it increases the motivation of children and adolescents to learn with their agents [62]. In the future, students could even develop social friendship relationships with their agents [63][64][65][66][67]. Regarding the type of character, 8 out of the 12 reviewed agents have a human shape, usually for older students. The other two shape possibilities were animals or toys agents, usually for younger students. It seems that the current trend to design PCAs for children is to keep the human shape for older students, and other shapes for younger students. For instance, the teddy toy of RoDy [67] a recent agent to teach sharing at Preschool as can be seen in Figure 5.
Ten out of the 12 reviewed agents do not have either adaptive or evolution possibilities. However, the increasing use of Embodied Pedagogical.
Agents and advances in Natural Language Processing could improve the interaction with the students showing more complex animation, so that agents become able to adapt their conversation and behavior to the conversation and behavior to the students [57,58,61].
Agents could learn from previous mistakes in the dialogue, and give the impression that they are actually talking to the student (as in a human-to-human dialogue in which we remember what we said and adapt ourselves to the context and dialogue of the person to whom we are talking). Advances in Natural Language Processing could improve the student-agent dialogue and avoid the frustration feeling that some students can get if the agent makes mistakes because it does not understand what the student is saying, or it is in a language that the agent cannot process [28,59]. Nine out of the 12 reviewed agents have been designed to be used with a PC. However, the advent of ubiquitous computing in the last years and responsive design makes this author believe that more agents will be created to be used from any device, with synchronization of the data across devices, and taking into account that the language used should be considered depending on the context, i.e., when the agent is used at home with a mobile device could be a friendlier conversation than when students are at a lab room in the school. Ubiquity factors such as displaying the interface of the agent in a smaller screen, or in places in which sound is not allowed, should also be taken into account. Given the relevance of mobile learning, having the possibility of interacting with the agents in different devices could also open many promising possibilities in the field.
Five out of the 12 reviewed agents have 3D graphics options. In some cases, the lack of flexibility of the natural language dialogue is compensated by 3D graphics animation similarly to the use of gestures by humans in non-verbal communication. Moreover, the use of gestures can serve not only as hints to support the content of the conversation, but they can also serve as powerful indicators of emotions. 3D animations could keep the students' learning, motivation and satisfaction levels even during pandemic situations like COVID-19 [61] (see Figure 6) in which the PCA could keep teaching to the students when no face-to-face lessons are possible.
Finally, it is important to highlight that the main design options that could be expected to evolve over time towards better practices in the future could be having mixed-interaction multi-language spoken and written dialogue with some small talk; 3D animations and sound; affective, adaptation, evolution, and collaborative possibilities and, responsive design to be used from any device (including robots) with synchronization of the data across devices. Now the discussion is open. Figure 6. Alcody emotional learning companion for teaching programming to Primary Education even during COVID-19 (source: [61]).

Conclusions and Future Work
The main contribution of this paper is the following set of PCA design recommendations to promote the use of PCAs in school and university classrooms grounded in the review of the literature: (1) The first step when a designer or a teacher wants to integrate a PCA to design learning activities would be to think about the instructional method embedded in the agent [5]. That is, the first criterion of the taxonomy (role and attitude), if the paradigm to follow is "Learning by Teaching" or "Learning by Explaining", a Teachable Agent should be chosen, while if the paradigm to follow is "Learning by Doing", a Companion Agent should be chosen. Otherwise, the agent can be used as a teacher, coach or instructor. Pedagogic agents can be beneficial in education taking the role of teachers, students, or companions [42] according to a certain pedagogic goal pursued. For instance, [26] reported an increase of one point in the final score of students using Autotutor, and when no improvement in the score is achieved, even basic agents such as Herman the Bug [14], allow students to have a more satisfactory experience of studying with the computer, and students can get more encouraged to help the agent than to study on their own [29].
(2) The second and third design choices should also be made regarding pedagogic criteria, given that they are highly relevant for the kind of agent to use. The interaction modality and the domain may change the type of agent. Currently, agents are limited to specific domains with limited interaction possibilities. However, experiments with more self-regulation [31], change of turns [26], and more general domains could extend the use of agents in class. It would be advisable to have mixed-interaction dialogue with some small talk as well as talking about the knowledge domain. Social factors should also be taken into account [44]. Agents with affective possibilities seem to provide better results [16,36,61] not only in the role of companion, but also in other roles [26,32]. It has also been noted that it is important to choose the features of the agent correctly, i.e., the voice of the agent seems important as well as the type of animation [44].
(3) The final set of design choices should be based on HCI guidelines. It is advisable to follow a UCD to adapt the design of the PCA to the needs of the students, which could be the key to overcome many limitations found (i.e., interaction modalities, specific domains, limited language interaction, etc.). It is not the same to design an agent for little children than for High School or University students [39,53,59,62].
The main limitation of this paper is that these recommendations are based on the current literature review as described in Section 1. It has been noted how design choices evolve over time to better practices so they should be taken as evolving recommendations that will change as technology and society changes. It is possible that new items should be added to the taxonomy, while others are no longer relevant and, they should be removed. As future work, I intend to keep the taxonomy updated with all these changes and to extend it to other PCAs applications more than their use in school and universities classrooms.