You are currently viewing a new version of our website. To view the old version click .
Informatics
  • Editor’s Choice
  • Article
  • Open Access

16 March 2023

Affective Design Analysis of Explainable Artificial Intelligence (XAI): A User-Centric Perspective

and
Industrial and Systems Engineering Department, De La Salle University—Manila, 2401 Taft Ave, Malate, Manila 1004, Philippines
*
Author to whom correspondence should be addressed.
This article belongs to the Special Issue Feature Papers in Human-Computer Interaction

Abstract

Explainable Artificial Intelligence (XAI) has successfully solved the black box paradox of Artificial Intelligence (AI). By providing human-level insights on AI, it allowed users to understand its inner workings even with limited knowledge of the machine learning algorithms it uses. As a result, the field grew, and development flourished. However, concerns have been expressed that the techniques are limited in terms of to whom they are applicable and how their effect can be leveraged. Currently, most XAI techniques have been designed by developers. Though needed and valuable, XAI is more critical for an end-user, considering transparency cleaves on trust and adoption. This study aims to understand and conceptualize an end-user-centric XAI to fill in the lack of end-user understanding. Considering recent findings of related studies, this study focuses on design conceptualization and affective analysis. Data from 202 participants were collected from an online survey to identify the vital XAI design components and testbed experimentation to explore the affective and trust change per design configuration. The results show that affective is a viable trust calibration route for XAI. In terms of design, explanation form, communication style, and presence of supplementary information are the components users look for in an effective XAI. Lastly, anxiety about AI, incidental emotion, perceived AI reliability, and experience using the system are significant moderators of the trust calibration process for an end-user.

1. Introduction

Recent breakthroughs in algorithmic techniques and the complementary development of more capable computing tools have exponentially progressed the artificial intelligence (AI) field. These advancements have boosted AI’s analytical power, enabling convolution to take on more cognitively demanding tasks [,]. As an effect, AI can now be seen powering different systems, for multitudes of uses, and at varying levels of human augmentation.
While the added utility that AI can potentially bring is undeniably beneficial, unfortunately, it also detriments the possible adoption of users to it. Why? Because these advancements were realized by sacrificing AI’s interpretability or the ability to understand how and why the AI came up with its recommendations. This stems from using complex algorithms (i.e., machine learning—ML and deep learning—DL) that are inherently incomprehensible []. In return, it restricts how trust [,,,,] and subsequent reliance [] can be calibrated accurately, given that users are unaware of AI’s actual inner workings. This often leads to over-trusting an incapable AI or, at worst, abandoning a reliable AI, which is a pressing issue considering how society frames AI’s role in the future [,].
So, why do the experts that developed the algorithms not just provide the explanations? As simple as it seems, this will not work as AI functions similarly to human brains []. Evolution will happen as it learns, creates rules, contextualizes, and eventually adapts to gain performance. Because of this, new parameters (“hidden layers” in the context of ML) that are not originally part of the understandable design are added to the system, making it inexplainable to all users, including the developers []. For this reason, the issue is often tagged as the black box paradox of AI [].
As a workaround solution, the innovative field of explainable AI (XAI) has been introduced. Rather than entirely unpacking the algorithms that the AI uses, which is difficult, it focuses on providing human-level explanations conceptualized from the comprehensible facets of the AI []. It centers on the AI’s purpose, process, and performance to grant cognitive resources [,] to help developers [] (e.g., troubleshooting, understanding limitations, and improving performance) and to answer critical questions raised by an end-user []. Often, this is in the form of visualization tools embedded in the AI interface for interaction [].
Take image recognition, for instance, which is one of the first, most prominent, and well-researched AI domains. XAI is being used as an alternative in explaining the thinking process of the AI (e.g., heat maps, feature examples, rule explanations, etc.). This makes the complex analysis per bitmap (i.e., geometric coding and pattern analysis) [] comprehensible and deducible for trust and reliance assessment for adoption [,]. As a result, more demanding tasks (e.g., self-driving cars with visualization, facial recognition with saliency maps, etc.) are being unlocked and adopted across different functions []. This goes the same with other domains, which saw positive changes upon using XAI [].
The benefits from XAI have encouraged many scholars to expand the field by developing newer techniques, with the majority taking on the challenge of providing a deeper and more accurate representation of AI’s complex inner workings (e.g., Deterministic Local Interpretable Model-Agnostic Explanations, Shapley Additive Explanation, and Quantitative Testing with Concept Activation Vectors) [,]. As a result, most of the research has centered on what information XAI should carry, or focused on its role as a cognitive repository of explanation [,]. Although this is theoretically useful given the field’s infancy and rapidly expanding AI use cases, the spotlight from current research has underrepresented the end-users and centered more on the developers [,,].
Three foundational aspects can be synthesized from the current research stream that supports the claim. First, expertise with ML and DL is needed to understand current XAI techniques, which a typical end-user does not have [,]. Next, the goal is often directed at improving AI’s algorithm [,] and not on an end-user’s fundamental goal of triggering adoption and its prerequisite of trust. Lastly, current techniques were mainly viewed as a cognitive resource rather than a bridge in human–computer interaction (HCI) [,]. This means that little to nothing is known about how XAIs are perceived by an end-user when embedded in an AI, considering its inherent characteristics. With the limited understanding of the end-user context, this can cause a significant threat to the trajectory of AI’s role in society, viz., possibly halting the advantages AI can bring to the day-to-day life of humans.
Considering the gap in the context of end-user XAI, this study is proposed to answer the question: “How to create an end-user-centric XAI?”. As pointed out by recent works, people are approaching XAI with the same attitude of expectation they are employing towards another human [,,]. This entails that, in using XAI, interaction process (how the stimuli were used), structure (how it is presented), and variability (possible external influences) can play essential roles in making a judgment (i.e., trust and reliance) [,,,,,]—aspects that have been limited in existing XAI research. With that, the problem will be examined following the objectives:
  • Determine how an end-user calibrates trust from XAI.
  • Identify the factors that can significantly change how trust can be calibrated.
  • Examine possible moderating factors that can affect the calibration process.
  • How do external factors moderate the effect of XAI in the calibration process?
The remainder of this paper is divided into six sections. The next section presents the background of the study. Section 3, Section 4 and Section 5 confer the experiment design, give the data-gathering results, and discuss the findings with respective recommendations based on the study’s objectives. Finally, the last section closes the paper with the conclusion.

3. Materials and Methods

To test the proposed hypothesis, viz., to analyze the objectives of the study, an asynchronous experiment was designed and carried out. The goal was to simulate an interaction with XAI by using an existing AI-powered system. This was performed using an experimental testbed where different design combinations were prompted to the user in a between-subject design. Data that were recorded were based on the independent variables from the XAITC model.
To guide the development, a pre-study was conducted with four primary goals: (1) to identify the most familiar AI to avoid alienation, (2) to decide the design setup to minimize the undesirable effect of negative user experience, (3) to determine the design attributes of XAI considered to be necessary by users, and (4) to identify the configurations of the moderators. Data were collected from an online survey answered by 312 current AI users. For idea validation and feasibility assessment, results were run through a focus group discussion comprised of six AI developers and six user experience designers.
The conclusion was to use image classification AI as a use-context, with the Google Lens application as the template for the overall composition and logic flow. For the XAI, three design attributes were selected. Notably, (1) the explanation form or how XAI is presented, (2) the communication style for the explanation, and (3) the presence of supplementary information. Two levels per design attribute were considered based on the enumeration of Jian et al. [].
For the AI features, reliability was set to differ from 70% for low and 90% for high. Learning capability defaulted to pure conditions. As for the brand, Google was selected to be the name to represent high-reputation AI. Lastly, for the time experience, two days was deemed optimal. All in all, 64 designs (26) were tested. Take note that time experience was not considered as part of the factorial computation given that it is not a design input but a moment of recording. Table 1 summarizes all design configurations for the experiment.
Table 1. Experimental design configurations.

3.1. Participants

The snowball convenience sampling method was used for the data gathering. Initial leads were generated from direct invitations of peers and promotional ads posted in various social networking groups. Qualifications were set as being able to communicate in English, being at least 18 years old, having a normal or corrected-to-normal vision without any color blindness issues, and with experience using any AI-powered system. Considering the nature of the study, being not emotionally depressed was also a requirement to limit bias and skewness. This, however, was not posted as part of the advertisement material to avoid discrimination. Further discussion on how these requirements were confirmed will be given in the next section.
Technical requirements were also specified to facilitate the experiment effectively in remote conditions. Particularly, having a smartphone with at least 1080 × 1920 pixel resolution without any screen issues, internet connectivity of at least 5 Mbps, an updated web browser app, and availability of area conducive to the experiment. Additionally, they were required to have at least 30 min of uninterrupted time to perform the test for two days. A token worth 100 PHP (~2.00 USD) plus a performance bonus ranging from 25 PHP to 50 PHP was guaranteed in exchange for their complete involvement.
Considering the overarching goal of the paper of testing the relationships involved in the trust calibration of an end-user from XAI through the proposed XAITC model, as well as to attain a statistically capable data set, the minimum sample size was set to 152 participants. This was evaluated based on the principles set by Kline [] for structural assessment and the result of the priori-power sample computation established from the guidelines of Westland [] and Cohen []; the setup was at 0.3 anticipated effect size, 0.85 desired statistical power level, and 0.05 alpha level.

3.2. Measurements

Data were captured through an online questionnaire and an experimental testbed. The former requested the control (i.e., demographics, disposition, and situational factors) and independent variables (i.e., depressive state and incidental emotions), while the latter enabled the manipulation of XAI design attributes to measure changes from the interaction. Both tools were designed to be accessed through a web browser, with English being its default language.

3.2.1. Online Questionnaire

A three-section online questionnaire was developed and hosted through Google Forms. The first section functioned as the preliminary, where the overview, general instruction, and data consent clause were detailed. It also carried the yes-or-no screening test for English language competence, age restriction, visual acuity, and AI experience. As for the depressive state, the Mood and Feeling Questionnaire (MFQ) developed by Angold & Costello [] was used. Moving on, the second part requested demographic and dispositional information. Age, gender, educational attainment, occupation, and income were the identified controls. For the disposition, trust assessment by Frazier et al. [] and AI anxiety measurement by Wang & Wang [] were contextually used. Finally, the last section inquired about AI plus XAI experiences and incidental emotions. The years of experience were in a multiple-choice form, while four new seven-point Likert questions were developed to measure incidental emotions. The XAI emotion set (XES) of Bernardo & Seva [] was used as the reference for the different emotion groups.

3.2.2. Experiment Testbed

The three-sectioned testbed was built using Figma and was hosted through the Quant-UX prototyping site. The first section handled the instruction and examples, with three practice trials available. This was followed by the second section, which carried the main image classification task. The general workflow is as follows: (1) the participants will first select the image from the gallery, (2) the AI will generate its recommendation and explain its decision-making, and (3) the participants will decide on whether to agree or provide their classification as a measurement for their reliance. Take note that the correctness of recognition was not disclosed as the testbed’s purpose was limited to recommendation and machine learning. Finally, the third section features the rating scale sliders for the integral emotions, measured like the incidental emotions mentioned earlier. Figure 2 presents sample screenshots of the testbed.
Figure 2. Sample screenshot of experiment testbed: (a) AI classification and XAI presentation of the image being recognized; (b) dependent variable sliders to be answered after classification.
The user experience, grammar, spelling, and interface of both tools were pre-tested with nine current AI users, three English language experts, and four app developers. Recommendations were implemented before its use for the main experiment. In addition, factor consistency and validity of the newly introduced questionnaire for incidental emotions were checked.

3.3. Procedure

There are three phases in data-gathering: pre-experiment onboarding, main experiment, and post-experiment analysis.

3.3.1. Pre-Experiment Onboarding

The experiment started with the participants attending a synchronous online onboarding via Facebook Group Call. The focus was to relay the general instructions, check the setup requirements, present the data confidentiality, and explain the priming condition. In particular, the scenario was that an NGO hired the participants to help recognize pictures of different species in the Philippines saved in their database. To aid them, an image recognition AI system was developed that could give recommendations on what species the photos contain. Participants were allowed to use it or provide their own. The onboarding ended with the measurement tool links shared with the participants for asynchronous access.

3.3.2. Main Experiment

The main experiment started with the participants accessing the online questionnaire. They were allowed to do it anytime as long as they finished it uninterruptedly. Upon access, participants were prompted with the data agreement and screening questions. Those who agreed and qualified were the only ones allowed to continue. Demographic information was then requested from the participants, together with the rating for AI, XAI, and incidental emotions-related questions. Once completed, participants were forwarded randomly to any of the 24 designs of the XAI testbed.
The use of the XAI testbed started with the preliminaries: application information, general instructions, and the recap of the priming scenario. Each participant was instructed to classify 50 random photos available in the application. This was performed on two consecutive days (25 photos per day), with scores recorded to measure the performance bonus for the compensation. After completing the required task, additional instructions were given to the participants, plus the list of available schedules for the voluntary post-experiment interview and mode of call.

3.3.3. Post-Experiment Analysis

After all the data were collected, the post-experiment analysis started. It centered on analyzing experiment results, interviews, and token distribution. Initially, the data were assessed for completeness and performance. Those who garnered at least 40 correct classifications were tagged to receive an additional 25 PHP (~0.5 USD), while 50 PHP (~1 USD) was intended for those who got all correct. Once finalized, the evaluation was sent to the participants via their social media accounts and email. This contained information on how the token will be distributed, the interview schedule for those who volunteered, and the meeting access links. The interview focused on the reasoning for the answers on the dependent variables.

3.4. Technique of Analysis

The analysis was principally driven by the two-stage methodology proposed by Lowry & Gaskin [] for Structural Equation Modeling (SEM) under a covariance-based optimization. This technique was selected, primarily, because of its ability to deduce causal relationships proposed in the objectives. In addition, it can estimate model parameters that minimize residual variance [], is insensitive to parametric conditions [], and is suitable for simultaneous analysis of the design constructs [].
Analysis was segmented into two main phases. The first part is for the confirmation of the rigidity of the tool and the data it gathered. This was performed via factor analysis and measured against convergent, discriminant, and fit measures. After confirmation, SEM was performed which was further divided into mediation analysis to confirm the path of calibration, direct analysis for the relationship between the design component and integral emotions, and moderation analysis for the effect of incidental emotions and AI reliability. All tests were assessed based on their statistical significance and rigidity.
As for the data management, representative figures were computed based on the aggregate measurement from the initial trial up to the time considered. This adheres to the recommendations by Yang et al. [] on the detectable moment of difference (i.e., the area under the curve). Since the method of use was SEM, design elements were coded dichotomously (e.g., +1, −1) to represent change. For moderators, the multigroup test was the approach used to parallel the objectives of the study. Particularly, for factors that used the Likert scale, conversion is based on the midpoint.
The main program used was IBM’s Analysis of Moment Structure (AMOS) graphics version 24. In addition, Design Expert (DX) version 13 was used to generate the design of the experiment, and IBM’s Statistical Package for the Social Science (SPSS) version 25.0 was used for all statistical tests outside SEM. For consistency, testing was held constant under a p < 0.05 significance.

4. Results

The data gathering lasted for 15 days. Seven onboarding sessions were conducted, with at most 40 participants per session. The cumulative time in the main experiment lasted 40 min, with access happening between 11:00 a.m. and 10:00 p.m. There were no recorded ethical concerns or testbed issues. As for the post-interview, 22.27% of the participants joined, with an average call lasting 10 min. Lastly, those who scored 12 or above on the MFQ were notified at most two days after the experiment and were referred to a professional health organization.

4.1. Data Screening

All in all, 234 participated in the data gathering. After filtering, only 202 were considered for analysis as the data from those who failed the requirements, tested positive for the depression test, and did not finish the experiment were removed.
The summary of the demographics is presented in Table 2. Structurally, the gender count for those who disclosed was relatively the same (male—40.59% and female—44.55%), with the majority belonging to the millennial age group (58.42%), followed by generation X (22.77%), and generation Z (16.34%). Most were degree holders (vocational—9.41%; college—68.81%; postgraduate—15.84%) and part of the working class (67.33%). Looking at AI-related factors, the majority have at least five years of experience (more than 5 years—71.78%; 3 to 5 years—25.25%; less than 2 years—2.97%), with almost all having previous interaction experience with an XAI (90.10%). For the moderators, most of the participants reported positive incidental emotions (71.78%), used the high AI reliability version (62.87%), and were recorded at the later stage of use (56.44%).
Table 2. Summary of subjects’ demographics.

4.2. Exploratory Factor Analysis

Excellent results from the exploratory factor analysis (EFA) supported the use of the designed questionnaire to capture the latent variables of AI anxiety, incidental emotion, trust disposition, perceived usefulness, and perceived trust.
Primarily, the 0.919 Kaiser–Meyer–Olkin (KMO) measure and significant Bartlett’s test of sphericity (p < 0.001) highlighted the high proportion of variance among variables [] (see Table 3). This was further proven by the high communality extraction ranging between 0.947 and 0.958 and high cumulative variance for five component eigenvalue at 88.710%. As for loadings, the proposed five dimensions were cleanly factored with high intercorrelation scores (minimum of 0.731) and no significant cross-loadings under a principal component analysis extraction at a varimax normalization (see Table 4). This highlighted that the structure of the questions does not overlap due to its validity securing a highly stable analysis [].
Table 3. Internal consistency measures.
Table 4. Rotated component matrix.

4.3. Confirmatory Factor Analysis

Same with the findings from the EFA, confirmatory factor analysis (CFA) also attests to the structure of the dimensions (see Table 5). From the model validity test, Cronbach’s alpha and average variance explained (AVE) showed high reliability and convergent validity, respectively, as all are above the threshold of 0.70 []. Moreover, divergent validity also follows the same trend with minimum shared variance (MSV) being below AVE and maximum reliability (MaxR(H)) being above 0.70 []. All of these were achieved at an excellent fitted confirmatory model as highlighted in Table 6. Collectively, the measures validated the questionnaire’s ability to explain incidental emotion and soundness to be used for the hypothesis testing.
Table 5. Model validity measures.
Table 6. Confirmatory factor analysis fit estimates.

4.4. Structural Equation Modelling

A significant and good-fitting model was achieved from the 2000 bootstrapped SEM run. As summarized in Table 7, all representative measures from the three types of fit belong to the threshold limit. This highlighted the consistency of the data and its ability to reproduce the hypothesized relationship. Further, there were no suggested additional structural links from the main variables symbolizing a rigid model structure.
Table 7. Structural equation model fit estimates.

4.4.1. Mediation Effect Analysis

Results identified that both affective and cognitive elements function as mediators in the trust and reliance calibration process from XAI (see Table 8). For affect, anxiously suspicious was the only insignificant mediator. Interestingly surprised mediates the appreciation of the explanation form, trusting mediates explanation form and supplementary information, and communication style and supplementary information for fearfully dismayed. Relating to reliance, the insignificance of anxiously suspicious was still seen with the path coming from it when mediated by perceived trust. As for the cognitive paths, all proposed mediation of perceived trust and usefulness from the design elements were significant. Overall, these findings highlight the initial proposed idea from the study that affective path calibration exists in the use of XAI.
Table 8. Mediation effect analysis.

4.4.2. Direct Effect Analysis

Considering the results from the mediation analysis, relationships and design recommendations were successfully drawn from the direct effect analysis. Of the 21 hypothesized direct relationships, only 12 were identified to be statistically supported (see Table 9). As for the design to emotions group, explanation form significantly relates to interestingly surprised and trusting emotions. Communication style was significant to affect fearfully dismayed and anxiously suspicious. Lastly, the presence of supplementary information decreases fearfully dismayed.
Table 9. Direct effect analysis.
For emotion to trust, all emotions aside from anxiously suspicious have a significant relationship to perceived trust. Particularly, users that felt interestingly surprised and trusting reported higher perceived trust, while reporting the opposite when fearfully dismayed was felt. As for the cognitive-based groups, perceived usefulness has a positive relationship with perceived trust. This stemmed from explanation form and communication style design elements being significant. Supplementary information, although identified be part of the mediated path, was insignificant for the direct test showing the possibility of partial mediation []. Lastly, perceived trust has a significant positive relationship with reliance. Given that each design was dichotomously coded for the analysis, translating the results to design means
  • Example-based explanation increases interestingly surprised, trusting emotions, and perceived usefulness, while feature- and rule-based explanation decreases them.
  • Logic robotic increases fearfully dismayed plus anxiously suspicious emotions and decreases perceived usefulness, while humanized communication functions the opposite.
  • The presence of supplementary information decreases fearfully dismayed emotions, while the absence of it increases the effect.

4.4.3. Moderation Effect Analysis

Only half of the moderators were identified to have a significant effect. As shown in Table 10, the nested comparisons for AI anxiety and incidental emotions for human factors, while reliability and experience for the AI factors received a p-value less than 0.05, highlighted the substantial concurring difference from the full default model. With these results, deeper analysis was performed to identify the source of moderation on a per-path basis.
Table 10. Global multi-group moderation effect analysis.
The results from the individual fit test for all moderation runs also favored the global results. As shown in Table 11, the CFI, SRMR, and PClose all passed the thresholds assuring capability for further exploration. Take note that these are the measures used considering the data were truncated due to the stratification. More so, these are insensitive with low sample size and corresponding degrees of freedom [,].
Table 11. Local moderation test fit scores.
The local effect surfaced varying degrees of moderation per relationship, as presented in Table 12. For AI anxiety, the difference seen was mainly that values are higher with the low group than with the high group in three key areas. First, when fearfully dismayed emotions were felt, users with high AI anxiety experienced lower perceived trust than those who have low AI anxiety (β = −0.059 vs. β = 0.021, z-score = 2.195 and p-value = 0.014). Second, when exposed to a logic-robotic communication style, users in the high group have lower perceived usefulness than the low group (β = 0.804 vs. β = 1.185, z-score = 1.762 and p-value = 0.039). Lastly, for the relationship between perceived trust and reliance, users in the high group have lower reliance than the low group (β = 0.804 vs. β = 1.185, z-score = 1.762 and p-value = 0.039).
Table 12. Local multi-group moderation difference effect analysis.
For incidental emotions, six paths were identified to be significantly moderated. In terms of dampening effect, when supplementary information was provided, users with positive incidental emotions experienced lower interestingly surprised and trusting emotions than those with negative (β = −0.979 vs. β = 0.457, z-score = 2.513 and p-value = 0.006 and β = −1.001 vs. β = 0.137, z-score = 1.868 and p-value = 0.031). Next, the effect of perceived trust is lower for the positive group than the negative group when trusting and fearfully dismayed emotions were felt (β = −0.017 vs. β = 0.174, z-score = 4.081 and p-value = 0.000 and β = −0.045 vs. β = 0.023, z-score = 2.071 and p-value = 0.019). Both conditions highlight that positive incidental emotions resulted in a decrease in effect towards supplementary information and perceived trust. In contrary, there are also amplification effects that surfaced from the moderation. First when logic-robotic communication style was used, the negative group had a higher perception of usefulness than the positive group (β = −0.875 vs. β = −1.058, z-score = 2.096 and p-value = 0.018). In addition, perceived trust was higher in the positive group relative to perceived usefulness (β = 0.196 vs. β = 0.028, z-score = 1.780 and p-value = 0.038).
Aside from incidental emotion, the same moderated paths for reliability and experience were determined. Specifically, these are the affective paths from trusting and fearfully dismayed emotion to perceived trust. For both instances, low AI reliability and short experience have higher perceived trust when trusting (β = 0.223 vs. β = 0.064, z-score = 3.416 and p-value = 0.000; β = 0.184 vs. β = 0.095, z-score = 1.859 and p-value = 0.032) and fearfully dismayed emotions (β = 0.027 vs. β = −0.073, z-score = 2.479 and p-value = 0.007; β = 0.018 vs. β = −0.046, z-score = 1.987 and p-value = 0.023) were experienced.

5. Discussion

Considering the quantitative results from the experiment and subsequent interviews, the study was able to successfully analyze the postulated objectives for the end-user XAI consideration.

5.1. Objective 1: Confirmation of Affective Trust Calibration for XAI

Affect was determined to be a variable route for trust calibration. This was established through the causal relationship test, anchored on the mediation analysis of the SEM. Notably, emotions belonging to the group of interestingly surprised (e.g., interested, excited, surprised, pleased, and amazed), trusting (e.g., happy, confident, secure, proud, and trusting), and fearfully dismayed (e.g., dismayed, afraid, fear, angry, and sad) were identified to be significant mediators for trust and reliance in a behavioral and use change view. On the other hand, anxiously suspicious (e.g., suspicious, concerned, confused, nervous, and anxious) emotions were unsupported. In terms of relationships, both interestingly surprised and trusting emotions have a positive relation to trust, while fearfully dismayed observe a negative stance. All of these were validated jointly with the significant mediation of the cognitive route showing that trust calibration can happen on the two routes.
Insights from the interview also second the confirmation of the variability of the affective route alongside the cognitive route. By synthesizing the claims, distinctively, participants can be divided into two types: people who value emotions and people who value information. The former works when intuition or perception were the mode of assessment, while the latter when information quality deliberation happens. This observation resonates with the elaboration likelihood model (ELM) of persuasion of Petty and Cacioppo [], which is essentially a theory about the thinking process in the context of persuasion variables. Contextually, this is called the central route when high elaboration happens on the detail presented, while peripheral when low. Further, the interview also uncovered the same consideration of motivation and ability from ELM which dictates what route of trust calibration the user will follow. If they have high motivation (i.e., invested in the task reward, high curiosity with the XAI, aiming for a high score in the experiment) and high ability (i.e., understanding of the information on XAI, perceived expertise with the task), they will follow a cognitive path or central route, and affective or peripheral for the opposite.
Overall, these findings offset the running hypothesis on how XAI induces trust as determined in previous review studies [,]. This shows that XAI works similarly to other cues in the field of HCI that transverse on both trust calibration routes. Considering this study is a pioneer for testing the affective route, further studies can work on measuring such elaboration to analyze the differences in various contexts to ultimately check the variability of each route.

5.2. Objective 2: Effect of Different XAI Designs and Importance of End-User Centric Approach

For the design, two important observations were recognized. First, the pre-study was able to identify that certain design elements are being considered in the utility assessment of an XAI. From the survey, explanation form, communication style, and use of supplementary information rest on the top of the perceived important design element list. Notably, reasoning from the interview showed that these are selected as they provide the ability to see the rules AI rules, give an idea of the information used in such decision-making, and learn the details of the process. This shows that perceived important XAI design mainly echoes the original purpose of XAI among other things. For example, aesthetics can be considered. Text size and color are design elements but only very few noted their importance as they do not perform a critical role in explaining.
The next observation is that XAI design elements play a significant role both for the affective and cognitive trust calibration route, with changing each configuration producing different results. As deduced via SEM, design change will yield distinctive effects on emotions and cognitive evaluation (see Table 13 for the summary of effects). The varying effect theoretically implies that for XAI, design functions on a deeper level—a micro view as configuration rather than a macro view through categorical effect.
Table 13. Summary of affect and cognitive change per design element.
The key findings support and explain the significant elaboration routes and ELM parallelism reflected in the first objective. The second objective unfolds that in the development of XAI, aside from the information it carries, the design also matters. More so, the common consensus highlights the subsistence of a general structure for possible design prioritization when developing an XAI. This means that the identified effects can be leveraged to create a targeted XAI for an optimized utility function for an XAI. For example, to increase trust and reliance, an XAI featuring an example-based explanation form can be tapped to have a positive change to interestingly surprised and trusting emotions, or can use human-like communication to simply remove the feeling of fearfully dismayed and anxiously suspicious emotions. Overall, the observations shed light on the importance of design and its variability for an end-user-centric XAI.

5.3. Objective 3: External Factors Delimiting XAI Effect

The results also identified that external factors are a point worth noting when developing an XAI, with both human (i.e., AI anxiety and incidental emotions) and AI factors (i.e., reliability and experience) being viable. This implies that XAI utility effectiveness works beyond the design and information as situational factors are also significant. For example, supplementary information will only cause a significant negative affective change on interestingly surprised if they have positive incidental emotions. Viewing the XAI development in this light further denotes that to mitigate transparency, design should be contextualized on the situational need of the implementation.
In addition, commentaries from the interview suggest that the moderation works on a per AI-type experience basis. For example, a user with a bad experience with image recognition carried this in the experiment with higher anxiety, negative incidental emotion, and lower perception of the AI reliability, even though they have a good experience relative to the other types. This infers that XAI, being a complementary element, was evaluated after the AI system was assessed, opening the idea that the XAI effect might differ for other AI use cases. A possible extension of the study can be performed to check this idea.

6. Conclusions

Explainable artificial intelligence (XAI) has successfully addressed the black box trust problem of artificial intelligence (AI) by allowing users to gain a human-level understanding of how AI works, even if they have limited knowledge on the complex machine learning algorithms that power it. However, concerns that current XAI techniques have been delimited in terms of their applicability and impact have been highlighted in the recent years. Particularly, the exploration has determined that it focuses more on the needs of developers and not end-users, putting AI adoption in a critical position. To provide the necessary viewpoint, this study aimed to explore end-user-centric XAI.
For the first objective, which aims to determine how an end-user calibrates trust from XAI, it was identified that it not only serves as a cognitive resource but also as an affective cue for trust and reliance change. Effectively, this study argues that XAI can be used as an information resource and irrationally via emotions through its affective contributions. Continuing that, another claim identified in the study is that information carried out by the XAI is not the only determinant for both routes. The study tested and identified that design—or manner by which XAI is presented—can also alter its effectiveness. This answers the second objective regarding the factors that can viably change trust from an XAI. Lastly, as for the third and fourth objective, which looked on the moderating factors in the trust calibration process, evidence showed that human and AI factors were capable to influence the effect. This includes anxiety and incidental emotion for the former, while AI reliability and experience for the latter. Overall, the study successfully filled in the theoretical gap acknowledged from the research stream. It opened a new understanding of the routes by which XAI calibrates trust, the importance of its design, and external factors that may alter its effectiveness.

6.1. Implications

Considering the findings from the study, several implications can be drawn both in theoretical and managerial landscapes. For theoretical, as this study viably determined that XAI also functions in the affective trust route, it opens a new path of research regarding XAI effectiveness and subsequently poses important delimitation on previous research that only approached calibration through cognitive route. Possibly, revisiting such studies can be carried out to firm up the findings relative to the importance of integral emotions. In terms of development, the study also creates a new paradigm on the course of XAI improvement research.
For the managerial, the results create a better position for the implementors and users. For the former, developers and designers can use the findings on how to better leverage the effect of XAI towards trust. For instance, they can reorient the XAI to induce more positive emotions or use XAI more on systems where the primary customers are those with high anxiety (e.g., telehealth applications, banking, security, driving). Moving on, for the users, if the results are operationalized, they can have a better interaction with AI systems, possibly allowing effective adoption.
Aside from the positive implications, another side that can be drawn from the results is the possibility of misusing XAI. From the findings, it highlights that not all users use XAI as an information resource for mental model calibration; some only use it as a cue. In addition, design can manipulate emotions that later affect trust and reliance. In this chain, faulty and manipulative XAI can be simply shown in the system and be effective if the design can produce a positive effect on the user.

6.2. Limitations and Direction for Future Research

Although the study has filled in an important research gap and has followed a well-planned methodology, there have been limitations that should be addressed for future research. First, the study can be extended and retested under different domains. This is recommended to strengthen the claims from the study and to identify the limitations of the relationship. Possibly, different purpose, level of sensitivity, and stakeholders can be viewed. In line with that, secondly, the study can be expanded to check on other types of XAI other than the tested type in the study. In the experiment, the focus has been on visual imagery type since it complements the nature of the AI system selected (i.e., image recognition). Other types such as tree diagrams or textual can be tested. Third, moderator testing can be improved by expanding its limits. For instance, the experience can be viewed for a longer time frame (e.g., 2 weeks) to have a broader and more realistic view of the relationships. This is recommended as the post-interview surfaced it on multiple occasions. Lastly, a cross demographical view can be conducted to check on differences between less and highly adopted AI communities as this directly affects the use of XAI.

Author Contributions

Conceptualization, E.B. and R.S.; methodology, E.B. and R.S.; experiment testbed, E.B.; validation, E.B. and R.S.; formal analysis, E.B.; investigation, E.B.; data curation, E.B.; writing—original draft preparation, E.B.; writing—review and editing, E.B.; visualization, E.B.; supervision, R.S.; project administration, E.B. and R.S.; funding acquisition, E.B. and R.S. All authors have read and agreed to the published version of the manuscript.

Funding

This research and the APC were funded by De La Salle University—Manila.

Institutional Review Board Statement

The ethical aspects of this research have been approved by the Social Science Ethics Review Board (SSERB) of the Philippine Social Science Council (Reference Code: CB-22-20 on 27 June 2022).

Data Availability Statement

Data are not publicly available, though the data may be made available on request from the corresponding author.

Acknowledgments

The research team would like to acknowledge 80/20 Design Labs for their help in the development of the experiment testbed, Angelimarie Miguel and Wira Madria for their aid in the data curation and processing, and Naomi Bernardo, Edgardo Bernardo, Noel Bernardo, and Christiane Willits for their technical support and material donations used in the experiment.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

References

  1. Lu, Y. Artificial Intelligence: A Survey on Evolution, Models, Applications and Future Trends. J. Manag. Anal. 2019, 6, 1–29. [Google Scholar] [CrossRef]
  2. Jordan, M.I.; Mitchell, T.M. Machine Learning: Trends, Perspectives, and Prospects. Science 2015, 349, 255–260. [Google Scholar] [CrossRef] [PubMed]
  3. Rai, A. Explainable AI: From Black Box to Glass Box. J. Acad. Mark. Sci. 2020, 48, 137–141. [Google Scholar] [CrossRef]
  4. Doshi-Velez, F.; Kim, B. Towards A Rigorous Science of Interpretable Machine Learning. arXiv 2017, arXiv:1702.08608. [Google Scholar]
  5. Castelvecchi, D. Can We Open the Black Box of AI? Nat. News 2016, 538, 4. [Google Scholar] [CrossRef]
  6. Schmidt, P.; Biessmann, F. Quantifying Interpretability and Trust in Machine Learning Systems. arXiv 2019, arXiv:1901.08558. [Google Scholar]
  7. Kliegr, T.; Bahník, Š.; Fürnkranz, J. A Review of Possible Effects of Cognitive Biases on Interpretation of Rule-Based Machine Learning Models. Artif. Intell. 2021, 295, 103458. [Google Scholar] [CrossRef]
  8. Linardatos, P.; Papastefanopoulos, V.; Kotsiantis, S. Explainable AI: A Review of Machine Learning Interpretability Methods. Entropy 2020, 23, 18. [Google Scholar] [CrossRef] [PubMed]
  9. Weitz, K.; Hassan, T.; Schmid, U.; Garbas, J.-U. Deep-Learned Faces of Pain and Emotions: Elucidating the Differences of Facial Expressions with the Help of Explainable AI Methods. TM Tech. Mess. 2019, 86, 404–412. [Google Scholar] [CrossRef]
  10. Preece, A. Asking ‘Why’ in AI: Explainability of Intelligent Systems—Perspectives and Challenges. Intell. Sys. Acc. Fin. Manag. 2018, 25, 63–72. [Google Scholar] [CrossRef]
  11. Venkatesh, V. Adoption and Use of AI Tools: A Research Agenda Grounded in UTAUT. Ann. Oper. Res. 2022, 308, 641–652. [Google Scholar] [CrossRef]
  12. Chowdhary, K.R. Fundamentals of Artificial Intelligence; Springer: New Delhi, India, 2020; ISBN 978-81-322-3970-3. [Google Scholar]
  13. Lewis, M.; Li, H.; Sycara, K. Deep Learning, Transparency, and Trust in Human Robot Teamwork. In Trust in Human-Robot Interaction; Elsevier: Amsterdam, The Netherlands, 2021; pp. 321–352. ISBN 978-0-12-819472-0. [Google Scholar]
  14. Savage, N. Breaking into the Black Box of Artificial Intelligence. Nature 2022. [Google Scholar] [CrossRef]
  15. Mohseni, S.; Zarei, N.; Ragan, E.D. A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems. ACM Trans. Interact. Intell. Syst. 2021, 11, 1–45. [Google Scholar] [CrossRef]
  16. Barredo Arrieta, A.; Díaz-Rodríguez, N.; Del Ser, J.; Bennetot, A.; Tabik, S.; Barbado, A.; Garcia, S.; Gil-Lopez, S.; Molina, D.; Benjamins, R.; et al. Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI. Inf. Fusion 2020, 58, 82–115. [Google Scholar] [CrossRef]
  17. Singh, A.; Sengupta, S.; Lakshminarayanan, V. Explainable Deep Learning Models in Medical Image Analysis. J. Imaging 2020, 6, 52. [Google Scholar] [CrossRef]
  18. Miller, T.; Howe, P.; Sonenberg, L. Explainable AI: Beware of Inmates Running the Asylum Or: How I Learnt to Stop Worrying and Love the Social and Behavioural Sciences. arXiv 2017, arXiv:1712.00547. [Google Scholar] [CrossRef]
  19. Lopes, P.; Silva, E.; Braga, C.; Oliveira, T.; Rosado, L. XAI Systems Evaluation: A Review of Human and Computer-Centred Methods. Appl. Sci. 2022, 12, 9423. [Google Scholar] [CrossRef]
  20. Alicioglu, G.; Sun, B. A Survey of Visual Analytics for Explainable Artificial Intelligence Methods. Comput. Graph. 2022, 102, 502–520. [Google Scholar] [CrossRef]
  21. Zhang, Z.; Zhao, L.; Yang, T. Research on the Application of Artificial Intelligence in Image Recognition Technology. J. Phys. Conf. Ser. 2021, 1992, 032118. [Google Scholar] [CrossRef]
  22. Arun, N.; Gaw, N.; Singh, P.; Chang, K.; Aggarwal, M.; Chen, B.; Hoebel, K.; Gupta, S.; Patel, J.; Gidwani, M.; et al. Assessing the Trustworthiness of Saliency Maps for Localizing Abnormalities in Medical Imaging. Radiol. Artif. Intell. 2021, 3, e200267. [Google Scholar] [CrossRef]
  23. Zhang, J.; Chao, H.; Dasegowda, G.; Wang, G.; Kalra, M.K.; Yan, P. Overlooked Trustworthiness of Saliency Maps. In Medical Image Computing and Computer Assisted Intervention—MICCAI 2022; Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S., Eds.; Lecture Notes in Computer Science; Springer Nature: Cham, Switzerland, 2022; Volume 13433, pp. 451–461. ISBN 978-3-031-16436-1. [Google Scholar]
  24. Adadi, A.; Berrada, M. Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI). IEEE Access 2018, 6, 52138–52160. [Google Scholar] [CrossRef]
  25. Haque, A.B.; Islam, A.K.M.N.; Mikalef, P. Explainable Artificial Intelligence (XAI) from a User Perspective: A Synthesis of Prior Literature and Problematizing Avenues for Future Research. Technol. Forecast. Soc. Chang. 2023, 186, 122120. [Google Scholar] [CrossRef]
  26. Shin, D. The Effects of Explainability and Causability on Perception, Trust, and Acceptance: Implications for Explainable AI. Int. J. Hum. Comput. Stud. 2021, 146, 102551. [Google Scholar] [CrossRef]
  27. Rudin, C.; Radin, J. Why Are We Using Black Box Models in AI When We Don’t Need To? A Lesson From An Explainable AI Competition. Harv. Data Sci. Rev. 2019, 1. [Google Scholar] [CrossRef]
  28. Förster, M.; Hühn, P.; Klier, M.; Kluge, K. User-Centric Explainable AI: Design and Evaluation of an Approach to Generate Coherent Counterfactual Explanations for Structured Data. J. Decis. Syst. 2022, 1–32. [Google Scholar] [CrossRef]
  29. Ferreira, J.J.; Monteiro, M. Designer-User Communication for XAI: An Epistemological Approach to Discuss XAI Design. arXiv 2021, arXiv:2105.07804. [Google Scholar] [CrossRef]
  30. Silva, A.; Schrum, M.; Hedlund-Botti, E.; Gopalan, N.; Gombolay, M. Explainable Artificial Intelligence: Evaluating the Objective and Subjective Impacts of XAI on Human-Agent Interaction. Int. J. Hum. Comput. Interact. 2022, 1–15. [Google Scholar] [CrossRef]
  31. Cirqueira, D.; Helfert, M.; Bezbradica, M. Towards Design Principles for User-Centric Explainable AI in Fraud Detection. In Artificial Intelligence in HCI; Degen, H., Ntoa, S., Eds.; Lecture Notes in Computer Science; Springer International Publishing: Cham, Switzerland, 2021; Volume 12797, pp. 21–40. ISBN 978-3-030-77771-5. [Google Scholar]
  32. Chari, S.; Seneviratne, O.; Gruen, D.M.; Foreman, M.A.; Das, A.K.; McGuinness, D.L. Explanation Ontology: A Model of Explanations for User-Centered AI. In The Semantic Web—ISWC 2020; Pan, J.Z., Tamma, V., d’Amato, C., Janowicz, K., Fu, B., Polleres, A., Seneviratne, O., Kagal, L., Eds.; Lecture Notes in Computer Science; Springer International Publishing: Cham, Switzerland, 2020; Volume 12507, pp. 228–243. ISBN 978-3-030-62465-1. [Google Scholar]
  33. Chromik, M.; Butz, A. Human-XAI Interaction: A Review and Design Principles for Explanation User Interfaces. In Human-Computer Interaction—INTERACT 2021; Ardito, C., Lanzilotti, R., Malizia, A., Petrie, H., Piccinno, A., Desolda, G., Inkpen, K., Eds.; Lecture Notes in Computer Science; Springer International Publishing: Cham, Switzerland, 2021; Volume 12933, pp. 619–640. ISBN 978-3-030-85615-1. [Google Scholar]
  34. Liao, Q.V.; Varshney, K.R. Human-Centered Explainable AI (XAI): From Algorithms to User Experiences. arXiv 2021, arXiv:2110.10790. [Google Scholar]
  35. Gan, Y.; Ji, Y.; Jiang, S.; Liu, X.; Feng, Z.; Li, Y.; Liu, Y. Integrating Aesthetic and Emotional Preferences in Social Robot Design: An Affective Design Approach with Kansei Engineering and Deep Convolutional Generative Adversarial Network. Int. J. Ind. Ergon. 2021, 83, 103128. [Google Scholar] [CrossRef]
  36. Nawaratne, R. Human-Centric Product Design with Kansei Engineering and Artificial Intelligence. Available online: https://towardsdatascience.com/human-centric-product-design-with-kansei-engineering-and-artificial-intelligence-f38cb3c0f26d (accessed on 21 December 2021).
  37. Wang, D.; Yang, Q.; Abdul, A.; Lim, B.Y. Designing Theory-Driven User-Centric Explainable AI. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow Scotland, UK, 2 May 2019; ACM: New York, NY, USA, 2019; pp. 1–15. [Google Scholar]
  38. Lee, J.D.; See, K.A. Trust in Automation: Designing for Appropriate Reliance. Hum. Factors 2004, 46, 50–80. [Google Scholar] [CrossRef]
  39. Hoff, K.A.; Bashir, M. Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust. Hum. Factors 2015, 57, 407–434. [Google Scholar] [CrossRef] [PubMed]
  40. Kramer, R.M. Trust and Distrust in Organizations: Emerging Perspectives, Enduring Questions. Annu. Rev. Psychol. 1999, 50, 569–598. [Google Scholar] [CrossRef] [PubMed]
  41. Lewis, J.D.; Weigert, A. Trust as a Social Reality. Soc. Forces 1985, 63, 967. [Google Scholar] [CrossRef]
  42. McAllister, D.J. Affect- and Cognition-Based Trust as Foundations for Interpersonal Cooperation in Organizations. Acad. Manag. J. 1995, 38, 24–59. [Google Scholar] [CrossRef]
  43. Panksepp, J. Affective Consciousness: Core Emotional Feelings in Animals and Humans. Conscious. Cogn. 2005, 14, 30–80. [Google Scholar] [CrossRef]
  44. Schwarz, N.; Bless, H.; Bohner, G. Mood and Persuasion: Affective States Influence the Processing of Persuasive Communications. In Advances in Experimental Social Psychology; Elsevier: Amsterdam, The Netherlands, 1991; Volume 24, pp. 161–199. ISBN 978-0-12-015224-7. [Google Scholar]
  45. Forlizzi, J.; Battarbee, K. Understanding Experience in Interactive Systems. In Proceedings of the 2004 Conference on Designing Interactive Systems Processes, Practices, Methods, and Techniques—DIS ’04, Cambridge, MA, USA, 1–4 August 2004; ACM Press: New York, NY, USA, 2004; p. 261. [Google Scholar]
  46. Van Gorp, T.; Adams, E. Design for Emotion; Morgan Kaufmann: Waltham, MA, USA, 2012; ISBN 978-0-12-386531-1. [Google Scholar]
  47. Madsen, M.; Gregor, S. Measuring Human-Computer Trust; Australasian Association for Information System: Wales, Australia, 2000; Volume 53, pp. 6–8. [Google Scholar]
  48. Myers, C.D.; Tingley, D. The Influence of Emotion on Trust. Polit. Anal. 2016, 24, 492–500. [Google Scholar] [CrossRef]
  49. Jin, N.; Merkebu, J. The Role of Employee Attractiveness and Positive Emotion in Upscale Restaurants. Anatolia 2015, 26, 284–297. [Google Scholar] [CrossRef]
  50. Jensen, T.; Khan, M.M.H.; Albayram, Y.; Fahim, M.A.A.; Buck, R.; Coman, E. Anticipated Emotions in Initial Trust Evaluations of a Drone System Based on Performance and Process Information. Int. J. Hum. Comput. Interact. 2020, 36, 316–325. [Google Scholar] [CrossRef]
  51. Albayram, Y.; Khan, M.M.H.; Jensen, T.; Buck, R.; Coman, E. The Effects of Risk and Role on Users’ Anticipated Emotions in Safety-Critical Systems. In Engineering Psychology and Cognitive Ergonomics; Harris, D., Ed.; Lecture Notes in Computer Science; Springer International Publishing: Cham, Switzerland, 2018; Volume 10906, pp. 369–388. ISBN 978-3-319-91121-2. [Google Scholar]
  52. Guerdan, L.; Raymond, A.; Gunes, H. Toward Affective XAI: Facial Affect Analysis for Understanding Explainable Human-AI Interactions. In Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Nashville, TN, USA, 19–25 June 2021; Volume 10, pp. 3796–3805. [Google Scholar]
  53. Phillips, R.; Madhavan, P. The Role of Affective Valence and Task Uncertainty in Human-Automation Interaction. Proc. Hum. Factors Ergon. Soc. Annu. Meet. 2013, 57, 354–358. [Google Scholar] [CrossRef]
  54. Gompei, T.; Umemuro, H. Factors and Development of Cognitive and Affective Trust on Social Robots. In Social Robotics; Ge, S.S., Cabibihan, J.-J., Salichs, M.A., Broadbent, E., He, H., Wagner, A.R., Castro-González, Á., Eds.; Lecture Notes in Computer Science; Springer International Publishing: Cham, Switzerland, 2018; Volume 11357, pp. 45–54. ISBN 978-3-030-05203-4. [Google Scholar]
  55. Buck, R.; Khan, M.; Fagan, M.; Coman, E. The User Affective Experience Scale: A Measure of Emotions Anticipated in Response to Pop-Up Computer Warnings. Int. J. Hum. Comput. Interact. 2018, 34, 25–34. [Google Scholar] [CrossRef]
  56. Bernardo, E.; Tangsoc, J. Explanatory Modelling of Factors Influencing Adoption of Smartphone Shopping Application. IEMS 2019, 18, 647–657. [Google Scholar] [CrossRef]
  57. Chen, Q.Q.; Park, H.J. How Anthropomorphism Affects Trust in Intelligent Personal Assistants. Ind. Manag. Data Syst. 2021, 121, 2722–2737. [Google Scholar] [CrossRef]
  58. Helander, M.G.; Khalid, H.M. Affective and Pleasurable Design. In Handbook of Human Factors and Ergonomics; Salvendy, G., Ed.; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 2006; pp. 543–572. ISBN 978-0-470-04820-7. [Google Scholar]
  59. Khalid, H.M. Embracing Diversity in User Needs for Affective Design. Appl. Ergon. 2006, 37, 409–418. [Google Scholar] [CrossRef] [PubMed]
  60. Lottridge, D.; Chignell, M.; Jovicic, A. Affective Interaction: Understanding, Evaluating, and Designing for Human Emotion. Rev. Hum. Factors Ergon. 2011, 7, 197–217. [Google Scholar] [CrossRef]
  61. Gasah, M.; Mat Zain, N.H.; Baharum, A. An Approach in Creating Positive Emotion for Children’s e-Learning Based on User Interface Design. IJEECS 2019, 13, 1267. [Google Scholar] [CrossRef]
  62. Isbister, K. How Games Move Us: Emotion by Design; Playful Thinking; MIT Press: Cambridge, MA, USA, 2016; ISBN 978-0-262-03426-5. [Google Scholar]
  63. Gutierrez, A.M.J.; Chiu, A.S.F.; Seva, R. A Proposed Framework on the Affective Design of Eco-Product Labels. Sustainability 2020, 12, 3234. [Google Scholar] [CrossRef]
  64. Dy, A.K.; Lazo, M.; Santos, A.G.; Seva, R. Affective Trash Bin Signage to Promote Waste Segregation. In Proceedings of the 21st Congress of the International Ergonomics Association (IEA 2021), Online, 13-18 June 2021; Black, N.L., Neumann, W.P., Noy, I., Eds.; Lecture Notes in Networks and Systems. Springer International Publishing: Cham, Switzerland, 2022; Volume 223, pp. 20–30, ISBN 978-3-030-74613-1. [Google Scholar]
  65. Norman, D.A. Emotional Design: Why We Love (or Hate) Everyday Things; Basic Books: New York, NY, USA, 2004; ISBN 978-0-465-05135-9. [Google Scholar]
  66. Jordan, P.W. Designing Pleasurable Products; CRC Press: Boca Raton, FL, USA, 2000; ISBN 978-1-135-73411-4. [Google Scholar]
  67. Khalid, H.M.; Helander, M.G. A Framework for Affective Customer Needs in Product Design. Theor. Issues Ergon. Sci. 2004, 5, 27–42. [Google Scholar] [CrossRef]
  68. Bernardo, E.; Seva, R. Explainable Artificial Intelligence (XAI) Emotions Set. Appl. Sci. 2022, submitted.
  69. Albayram, Y.; Jensen, T.; Khan, M.M.H.; Buck, R.; Coman, E. Investigating the Effect of System Reliability, Risk, and Role on Users’ Emotions and Attitudes toward a Safety-Critical Drone System. Int. J. Hum. Comput. Interact. 2019, 35, 761–772. [Google Scholar] [CrossRef]
  70. Du, N.; Zhou, F.; Pulver, E.M.; Tilbury, D.M.; Robert, L.P.; Pradhan, A.K.; Yang, X.J. Examining the Effects of Emotional Valence and Arousal on Takeover Performance in Conditionally Automated Driving. Transp. Res. Part C Emerg. Technol. 2020, 112, 78–87. [Google Scholar] [CrossRef]
  71. Jian, J.-Y.; Bisantz, A.M.; Drury, C.G. Foundations for an Empirically Determined Scale of Trust in Automated Systems. Int. J. Cogn. Ergon. 2000, 4, 53–71. [Google Scholar] [CrossRef]
  72. Kline, R.B. Principles and Practice of Structural Equation Modeling, 4th ed.; Methodology in the Social Sciences; The Guilford Press: New York, NY, USA, 2016; ISBN 978-1-4625-2335-1. [Google Scholar]
  73. Westland, C. Lower Bounds on Sample Size in Structural Equation Modeling. Electron. Commer. Res. Appl. 2010, 9, 476–487. [Google Scholar] [CrossRef]
  74. Cohen, J. Statistical Power Analysis for the Behavioral Sciences; Routledge: London, UK, 1988; ISBN 978-0-203-77158-7. [Google Scholar]
  75. Angold, A.; Costello, E.J. Short Mood and Feelings Questionnaire; APA PsycNet: Washington, DC, USA, 1987. [Google Scholar] [CrossRef]
  76. Frazier, M.L.; Johnson, P.D.; Fainshmidt, S. Development and Validation of a Propensity to Trust Scale. J. Trust. Res. 2013, 3, 76–97. [Google Scholar] [CrossRef]
  77. Lowry, P.B.; Twyman, N.W.; Pickard, M.; Jenkins, J.L.; Bui, Q. “Neo” Proposing the Affect-Trust Infusion Model (ATIM) to Explain and Predict the Influence of High and Low Affect Infusion on Web Vendor Trust. Inf. Manag. 2014, 51, 579–594. [Google Scholar] [CrossRef]
  78. Hsu, S.-H.; Chen, W.; Hsieh, M. Robustness Testing of PLS, LISREL, EQS and ANN-Based SEM for Measuring Customer Satisfaction. Total Qual. Manag. Bus. Excell. 2006, 17, 355–372. [Google Scholar] [CrossRef]
  79. Henseler, J.; Ringle, C.M.; Sinkovics, R.R. The Use of Partial Least Squares Path Modeling in International Marketing. In Advances in International Marketing; Sinkovics, R.R., Ghauri, P.N., Eds.; Emerald Group Publishing Limited: Bingley, UK, 2009; Volume 20, pp. 277–319. ISBN 978-1-84855-468-9. [Google Scholar]
  80. Chin, W.W. The Partial Least Squares Approach for Structural Equation Modeling. In Modern Methods for Business Research; Methodology for Business and Management; Lawrence Erlbaum Associates Publishers: Mahwah, NJ, USA, 1998; pp. 295–336. ISBN 0-8058-2677-7. [Google Scholar]
  81. Yang, X.J.; Unhelkar, V.V.; Li, K.; Shah, J.A. Evaluating Effects of User Experience and System Transparency on Trust in Automation. In Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction, Vienna, Austria, 6 March 2017; ACM: New York, NY, USA, 2017; pp. 408–416. [Google Scholar]
  82. Vogt, W.P.; Johnson, R.B. The SAGE Dictionary of Statistics & Methodology: A Nontechnical Guide for the Social Sciences, 5th ed.; SAGE: Los Angeles, CA, USA, 2016; ISBN 978-1-4833-8176-3. [Google Scholar]
  83. Hair, J.F. (Ed.) Multivariate Data Analysis; Prentice Hall: Upper Saddle River, NJ, USA, 1998; ISBN 978-0-13-894858-0. [Google Scholar]
  84. Taber, K.S. The Use of Cronbach’s Alpha When Developing and Reporting Research Instruments in Science Education. Res. Sci. Educ. 2018, 48, 1273–1296. [Google Scholar] [CrossRef]
  85. Hu, L.; Bentler, P.M. Cutoff Criteria for Fit Indexes in Covariance Structure Analysis: Conventional Criteria versus New Alternatives. Struct. Equ. Model. A Multidiscip. J. 1999, 6, 1–55. [Google Scholar] [CrossRef]
  86. Schreiber, J.B.; Nora, A.; Stage, F.K.; Barlow, E.A.; King, J. Reporting Structural Equation Modeling and Confirmatory Factor Analysis Results: A Review. J. Educ. Res. 2006, 99, 323–338. [Google Scholar] [CrossRef]
  87. Cangur, S.; Ercan, I. Comparison of Model Fit Indices Used in Structural Equation Modeling Under Multivariate Normality. J. Mod. Appl. Stat. Meth. 2015, 14, 152–167. [Google Scholar] [CrossRef]
  88. Baron, R.M.; Kenny, D.A. The Moderator–Mediator Variable Distinction in Social Psychological Research: Conceptual, Strategic, and Statistical Considerations. J. Personal. Soc. Psychol. 1986, 51, 1173–1182. [Google Scholar] [CrossRef]
  89. Shi, D.; Lee, T.; Maydeu-Olivares, A. Understanding the Model Size Effect on SEM Fit Indices. Educ. Psychol. Meas. 2019, 79, 310–334. [Google Scholar] [CrossRef]
  90. Petty, R.E.; Cacioppo, J.T. The Elaboration Likelihood Model of Persuasion. In Advances in Experimental Social Psychology; Elsevier: Amsterdam, The Netherlands, 1986; Volume 19, pp. 123–205. ISBN 978-0-12-015219-3. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Article Metrics

Citations

Article Access Statistics

Multiple requests from the same IP address are counted as one view.