Negotiation is a social influence process of interaction by two or more parties (conversational agents) making decisions, allocating resources, or resolving disagreement in a mutually interesting, and possibly cross-cultural, context. It reflects differing preferences and decision-making between partners and engenders mutual effects on one another with a view to work together to achieve a satisfactory agreed outcome, that is, an effective negotiation [1
]. Negotiation may take place between human agents as well as between human and virtual agents. The negotiation process can be manifested in intelligent interaction, such as serious games, intelligent tutoring systems, chatbot interaction and human-robot interaction systems. Embodied conversational agents (ECAs) are used across diverse health (e.g., cognitive disability), academic (e.g., STEM education), business and training settings [2
]. In human-to-human negotiation, issues regarding interaction may be addressed and resolved either with more integrative, collaborative-associated (i.e., win-win) or more competitive-oriented (i.e., win-lose) approaches [6
]. In ECAs, in particular, the latter are programmed not only to facilitate negotiation by enabling positive climate, aligned consensus, favourable attitudes and learning experience [7
]. They can also be used to set the negotiation procedure in a rather more equal basis of reference, given that status and power differentials rather pertinent in human-to-human interaction, for example one partner may dominate over the other, risking trust between negotiating parties [10
], tend to become more easily absent in human-to-agent interaction [11
]. As such, the negotiation process may seem to fall short of being an easy-managed, fully predictable and consistently successful social interaction procedure [12
The social nature of negotiation between humans, as an interaction process, maps into the need for metacognitive skill training [13
]. This is apparent in any given social, educational and business exchange (interaction) which requires knowledge (awareness), consideration (reflection) and control (regulation) over ones’ objectives in a way that directly relates to successful task performance [14
] and decision-making improvement [15
]. In cases when humans do perform well, they (a) are more likely to indicate higher levels of self-efficacy (i.e., the positive motivational attitude of accomplishing challenging tasks), (b) differentiate and self-regulate their learning approach for subsequent activity and vice versa (i.e., self-regulation) [16
], (c) seem to practice their skill to use logical reasoning for problem-solving and communicate successfully with others to evaluate the relevant context (i.e., interpersonal and problem-solving skills), (d) are oriented toward adopting mastery-as-a-goal in their choices (i.e., mastery goal motivational disposition) and choose assignments that adhere to inspiring and advocate additional training and learning ones [18
]. In taking an active social presence role when relating to others [5
], facilitating rapport and fostering cooperation and coordination [19
], they are expected to benefit from activities to engage with the community by proactively assisting others (i.e., civic action) and exhibit positive reactions towards the deployment of changes (i.e., readiness to change) [20
]. Similar human-to-human interaction social presence and intelligence attributes are replicated in ECA setups (human-to-virtual agent), as the latter are likewise modelled to be socially intelligent multimodal systems that demonstrate human-matching physical appearance and behavioural characteristics, treated by their human counterparts like real humans [21
]. As the employment of ECAs tends to increase, enabling (a) successful interaction between humans and ECAs, in terms of their negotiation (training) functional capabilities by gaining the users’ perceptions regarding natural human–computer interaction (macrolevel; multimodal system performance) and (b) assess the contribution of metacognitive-, individual- and community-associated skills and attitudes (microlevel; users’ dispositions) as an integral element for mastering successful user learning experience exercised within ECAs across diverse cultural context. An advanced research interface of such an ECA system [22
] is the one that employs multimodal interaction for instructing metacognitive knowledge and skills of both application and users [25
] by modelling human negotiation behaviour and being evaluated across diverse cultural context [26
]. Such multimodal metacognitive training system allows (a) extended cooperation between negotiating agents (human and virtual) in order to reach a designated consensus over negotiation issues, (b) natural language interaction by both interacting agents (human and virtual) and (c) demonstration of expressive quality of verbal and nonverbal characteristics generated by the conversational agent.
Based on the above reasoning, therefore, the research illustrated in the current paper pursues: (a) to extend previous evidence regarding ECA functional interaction (negotiation training) abilities [27
] (i.e., macrolevel; dialogue system performance), (b) to further explore users’ metacognitive-individual and community-related attitudes and skills embedded within ECA negotiation training, for the first time (i.e., microlevel; users’ attitudes and skills) and (c) to integrate ECA functionality (macrolevel) with users’ metacognitive-motivational and behavioural indicators (microlevel), also for the first time, as an essential element of carefully designed actions in human to virtual agent immersive learning and negotiation training environment, where social, affective and functional properties of agents, as well as user attitudes and behaviour, tend to serve as critical attributes in unravelling human–artificial interactions successfully [28
]. In accordance with the aforementioned rationale, therefore, the research questions that reflect the scope of the current research are the following:
[RQ1] Do self-efficacy, self-regulation, interpersonal and problem-solving skills, individual readiness to change, mastery goal orientation and civic action attitudes and skills associate significantly with negotiation training?
[RQ2] Does the interaction with the ECA training system lead to improved metacognitive skill attributes (as listed above) when measuring them before and after the interactive training?
[RQ3] Does a prominent metacognitive attitude change (pre- and postinteraction) exist and how do the other examined attitudes affect it?
[RQ4] Are there any differences in the perceptions of users regarding the ECA usability for evaluation based on gender and country of origin?
The paper is divided as follows. Section 2
presents the related work in negotiation-related, metacognitive, motivational and behavioural attitudes and skills within the intelligent virtual agent context. Section 3
presents the research design. Section 4
reports the results on metacognitive skill training, while Section 5
elaborates on the user study results, focusing on user gender and origin. Section 6
summarizes the results and illustrates their theoretical and practical implications. Section 7
concludes the paper and presents future research ideas on ECA negotiation-associated learning and training.
2. Related Work
Advances in intelligent interaction have been developed by using either narrower or richer modalities, (multimodal) designs pending on their context of relevance to support real-life human–computer interactions. Bickmore and Cassell claim that “as computers begin to resemble humans, the bar of user expectations is raised” [29
]. An enormous progress has been propelled. However, building intelligent virtual agents with advanced verbal and nonverbal skills altogether to fully address consistent, coherent, realistic and desirable functional and behavioural responses interpreted by the human users as humanlike and supporting long-term engagement in the endorsed behaviour posits a considerable ongoing challenge for both academic and commercial artificial intelligence designers to navigate [30
]. Given the broad range of conditions in which intelligent agents or ECAs may operate, exploring the way that certain design options might influence usability, interaction and learning experience outcomes across diverse situations and cultural context, is a considerable research issue to investigate [31
]. An emerging need is the extension of modelling of intelligent agent-to-agent negotiations (within multiagent environments) toward human-like negotiation behaviour as embedded in the training context [24
Existing research has stressed the importance of investigating the antecedents and outcomes of metacognitive and individual and community-associated attitudes and skills across human-to-human (face-to-face) and human-to-virtual (human-to-agent) interactions during instruction and learning [32
]. Therefore, the challenge of designing context-specific intelligent virtual systems and ECAs to support learner motivation, involvement, knowledge, attitudes and skills remains an explicitly open demanding perspective aimed to design approaches that are capable of gaining user rapport, establishing a long-term interaction in a personalized and immersive way guided by intelligence (i.e., cognitive abilities) to build positive learning achievement, learning attitudes and training efficiency outcomes (i.e., knowledge, problem-solving, logical thinking), as related work indicates below.
Teaching and learning within sophisticated interaction applications illustrates that intelligent systems appear to rely on positive outcomes for motivational and problem-resolution intentions and capabilities [33
], accompanied by facilitating student accessibility of instructional (educational) content [34
] and supporting integrative potential during interaction including satisfaction and connectedness with the virtual agent during training interaction [7
]. Inducing motivation and user self-confidence [35
] and increased e-learning performance [36
], aims to result in effective interaction and user engagement with the (training) virtual agents and task fulfilment by elderly individuals including users with diagnosed cognitive impairments [2
]. Enhancing learners’ positive judgments regarding their ability to organize and accomplish the required courses of action to achieve successful learning performance after training (i.e., self-efficacy [37
]) positively affects the inclination to participate in the learning activity at hand [4
The knowledge acquisition of the trainees is focused on exercising critical thinking [40
] and increasing the instructional and engaging power of the training environment [41
]. Although the majority of the above reported results seems to indicate rather favourable metacognitive and individual-level related effects for users in intelligent systems training situations, in terms of building effective learning performance indicators, relative evidence tends to be rather mixed. Holmes reports improvement in learning performance indicators for learners after interaction with intelligent tutoring system [43
]. Mayer and DaPra [44
] improved knowledge transfer when interacting with intelligent virtual agents, while van der Meij [38
] reports lowered learning gains, that is decreased task retention, for vocational education training students interacting with agents, in contrast to human interaction partners. Similar inconclusive findings are found in regard to gender differences for users interacting with ECAs. Guadagno et al. reported that both male and female users indicated similar attitudes and higher attitude change when interacting with an ECA of the same gender as theirs [45
]. This effect was noticeable for female participants, potentially attributed to more powerful identification (matching) with their own gender in comparison to male counterparts (i.e., Self-Categorization and Social Identity Theories [46
]). In other cases, male users exhibited more positive behavioural change after interacting with an ECA [49
]. Within human–robot interaction, in particular, female users did not exhibit any attitude change in response to the robot’s gender identification during interaction, while their male peers demonstrated different attitudes based on matching to their own gender [50
]. Along with the mixed evidence findings, therefore, further exploration of learners’ metacognitive-individual and community-related attitudes and skills within natural-language human–agent negotiation training of diverse cultural and gender-specific interaction, is required.
5. Usability Evaluation
The distribution (%) of the total sample (N = 40), Greek (N = 30) and German (N = 10) users’ responses regarding the usability evaluation questions postinteraction by gender and origin, are reported below. The full data can be found in the Appendix A
The participants originated from Greece (75.0%) and Germany (25.0%), representing 60.0% male and 40.0% female learners, respectively. Table 5
indicates the percentage distribution of the user perceptions regarding metacognitive and negotiation training, scope of the system and suggestions for future improvement.
The higher percentage of participants (45.0%) indicated that they did not really know what metacognitive skills were, followed by those who claimed lack of familiarity (37.5%) and awareness of metacognitive skill training (17.5%), respectively. 47.5% of users indicated that the system scope was negotiation skill training, followed by 40% of those reporting that the system was about artificial intelligence skill training, 10.0% metacognitive skill training and 2.5% responded they did not know what the system was about. Regarding the user perception on the “current” functionalities and suggestions for future versions, 67.5% of them reported features add-ons and 32.5% learning progress feedback for the ECA.
There are specific differences regarding the perception of the interaction between the two genders (see Table A1
of the Appendix A
). For example, female participants reported that it was moderately easy to interact with the system, while the male participants indicated a perceived slightly artificial interaction. The male participants considered the approach as quite promising to become a fully-fledged skill training application. On the other hand, the female participants were positive but more reserved. In sum, both genders overall seemed to indicate moderate to higher satisfactory perceptions regarding timely communication flow, useful provision of information, self-confidence in their knowledge of the training application functionalities, interesting idea to pursue, easy setup to understand, flexible to use a simplified version and pre- and postinteraction feedback as moderately valuable to facilitate learning performance awareness.
Greek users tended to exhibit moderate to higher favourable perceptions regarding accurate communication flow, helpful delivery of information, self-confidence in their understanding of its’ functionalities, useful idea to adopt, straightforward setup to comprehend, adaptable enough to exercise a simplified version and pre-, during and postinteraction feedback as moderately valuable to facilitate learning performance (see Table A2
of the Appendix A
). However, female Greek participants indicated rather moderate to neutral perceptions concerning task fulfilment during interaction with the system, in relation to their male Greek counterparts and the total sample’s related attitudes of both genders, accordingly.
The German participants exhibited rather mixed responses in relation to the ones received by both genders of the total sample and their Greek counterparts, respectively (see Table A2
of the Appendix A
). Although the majority of female German users reported absolute satisfaction regarding the utility of the information provided and a moderate extent of agreement with the fast speed of interaction. They reported experiencing rather less accurate actions of the system, greater difficulty in multimodal interaction and fairly moderate perceptions as to the overall usefulness of the interactive system’s role.
The prototype, implementation and evaluation of our negotiation training multimodal system aimed to seize interactive learning and training of metacognitive-related essentials for both system and participants, demonstrated significant positive findings in relation to the favourable attitudes and skills that learners improved after they interacted with the ECA during our assessment workshop. Our usability evaluation (macrolevel) corroborated by integration and additional exploration of metacognitive and individual- and community-level attitudes and skills embedded in multi-issue ECA negotiation learning context (microlevel) revealed certain user perception as the main results reported in this paper. ECA training and usability assessment incorporated gender differences and, for the first time, with the conceptual and practical exploration of metacognitive, individual- and community-linked attitudes and skills, such as self-efficacy, self-regulation, individual readiness to change, mastery goal orientation and facets of civic attitudes and skills like civic action and interpersonal and problem-solving skills they convey and mirror.
Such theoretical whilst also evidence-based joint examination of the associations between the aforementioned concepts is explored (a) within multimodal natural language negotiation environment, (b) extends relevant results from service, computer-supported learning, business and intelligent tutoring systems alike [18
] and (c) signals further noteworthy contributions and designates beneficial prospect for both comprehension and improvement of such positive attitudinal constructs and civic-associated attitudes and skills for future activities aimed to foster learners to delve into and expand those attitudes within artificial intelligence negotiation learning and training context, to a greater extent (RQ1).
The significant positive correlations obtained pre- and postinteraction seem to capture the exploration of favourable metacognitive, individual- and community-associated attitudes and skills not only prior but also post-negotiation context, thus, relating and transferring beneficial learning experience outcomes to computer-human interaction context mapped into the ECA negotiation environment. In addition, the higher levels of self-efficacy, civic action and individual readiness to change that Greek users exhibited postinteraction (t
-tests), tend to indicate that our training approach appears to fuel such metacognitive, individual- and community-based interaction learning attitudes and skills in ECA negotiation context, per se. Thus, lending support to previous related findings in both human-to-human and intelligent settings alike [37
], validates the extension of prior favourable virtual learning and training experience [42
] to integrate additional positive civic and change-related attitudes and skills outcomes as transferred into the ECA as a natural negotiation learning experience (RQ2).
The study participants who had a high self-reported logical reasoning and analytical thinking, listening to other people’s opinions, planning to do volunteering work and become involved in community (regressions), were more positive in facilitating and being more competent and confident in accomplishing their goals and handling whatever comes their way pre-interaction. Postinteraction, similar findings were obtained only for those bearing stronger ability to cooperate with others, thinking logically in solving problems and listening to others’ opinions. This may have to do with the capability of the ECA to detect this interaction, lending support to previous findings which postulate that virtual intelligent systems seem to facilitate learners in logical reasoning, decision making and interpersonal and problem-solving tasks [40
]. Based on the significant contributions indicated above, therefore, the proposed multimodal training approach appeared to be advantageous in (a) fuelling, mastering and associating both metacognitive, individual- and community-related interaction-based learning attitudes and skills and (b) expanding prior related findings from human-to-human [69
] into agent-to-human in intelligent virtual negotiation training settings [27
] to intelligent virtual natural negotiation learning context. Stressing further, the helpful learning experience that ECAs may bear in challenging learners and users to be proactive in advancing their reflection, lead to understanding and mastery of their current and future learning performance within intelligent virtual natural language negotiation per se. Self-efficacy was found to be the metacognitive skill attitude that most prominently described the change of the users’ skills after the training (RQ3).
Following the usability evaluation responses postinteraction based on gender (distribution of users’ responses based on gender), it seems that, overall and across Greek and German users, both male and female participants did perceive the system with moderately clear role, use, easiness to complete tasks and understanding its’ functionalities, timely communication, helpful information provided and an interesting idea for future training development. The female Greek users, in particular, seemed to indicate rather moderate to neutral perceptions concerning task fulfilment during interaction in comparison to their male Greek counterparts and the overall sample’s related attitudes of both genders, respectively (RQ4). Similar mixed results were also reported by German genders. Although the majority of female participants seemed to be absolutely pleased with the usage of the information offered and at a moderate degree of agreement with the fast pace of interaction perceived, they reported rather less correct actions of the system and lower self-confidence in their knowledge of using the system on their own, increased difficulty in multimodal interaction and moderate perceptions as regards the overall helpfulness of the system (RQ4). The aforementioned gender differences concerning interaction performance indicators, especially within Greek and German participants, (a) tend to corroborate previous findings, illustrating more favourable attitudes for male users after interacting with intelligent virtual agents [49
], and (b) contrast those reported by [30
], who demonstrated no gender differences between participants in their interaction with the intelligent virtual character (agent) at hand. This contradiction perhaps reinforces the significance of the findings especially within the diverse cultural background that characterizes our users. This poses an appealing question for future exploration and is additionally discussed later.
It could be also noted, as shown by the research findings, that the majority of users perceived the system as that of negotiation skills training and suggested further features add-on and learning progress feedback for future system improvement. In that sense, therefore, and in conjunction with their moderately favourable usability experience attitudes postinteraction already illustrated, it could be argued that our approach appeared to be also promising in expanding negotiation and aligned with metacognitive, individual- and community-related skills training beyond traditional and intelligent tutoring learning context, as introduced, exercised and assessed within an ECA environment.
A longitudinal research design and intervention over longer cohorts and follow-up evaluation across different real-life learning environments (e.g., industry, education) or academic context (e.g., win-win negotiation, metacognitive, individual and civics-related attitudes and skills), field (e.g., leadership, project management, organizational behaviour, civics) and (or) diverse user groups based on their generational (i.e., millennials vs. older populations) and cultural background differentials, might permit greater generalizability of our current findings. Such longitudinal replication not only would clarify and promote (a) the positive effects of metacognitive, individual and community-related attitudes and skills within intelligent virtual natural language agents’ long-term relatedness further, but would also (b) explore the persistency of similar favourable user perceptions gained, as useful feedback obtained for the future improvement of our training system. The latter might also further be facilitated by an additional potential exploration of direct personalization features embedded, following corresponding research [71
]. Nevertheless, the hands-on issues and built-in difficulties associated with the extremely demanding complexity of the ECA design, execution and assessment, need to be taken into account.