Enriching Artificial Intelligence Explanations with Knowledge Fragments

Abstract

Artificial intelligence models are increasingly used in manufacturing to inform decision making. Responsible decision making requires accurate forecasts and an understanding of the models’ behavior. Furthermore, the insights into the models’ rationale can be enriched with domain knowledge. This research builds explanations considering feature rankings for a particular forecast, enriching them with media news entries, datasets’ metadata, and entries from the Google knowledge graph. We compare two approaches (embeddings-based and semantic-based) on a real-world use case regarding demand forecasting. The embeddings-based approach measures the similarity between relevant concepts and retrieved media news entries and datasets’ metadata based on the word movers’ distance between embeddings. The semantic-based approach recourses to wikification and measures the Jaccard distance instead. The semantic-based approach leads to more diverse entries when displaying media events and more precise and diverse results regarding recommended datasets. We conclude that the explanations provided can be further improved with information regarding the purpose of potential actions that can be taken to influence demand and to provide “what-if” analysis capabilities.

1. Introduction

The industry is the part of the economy that is concerned with a highly mechanized and automated production of material goods [1]. Since the beginning of industrialization, technological breakthroughs have allowed to achieve increasing manufacturing efficiencies and have led to paradigm shifts. The latest such shifts are known as Industry 4.0 and Industry 5.0 [2,3], and aim to foster and increase the digitalization, networking, and automation of the manufacturing processes. Furthermore, these, in turn, enable tighter integration between physical and cyber domains (e.g., through cyber–physical systems [4], and digital twins [5]), increased deployment of intelligence (e.g., through artificial intelligence solutions [6]), to achieve autonomy and automation. These changes are expected to shorten the development and manufacturing periods, increase the manufacturing efficiency and sustainability [7], and achieve greater flexibility. Furthermore, emphasis on making such technologies and applications safe, trustworthy, and human-centric is a crucial characteristic of the nascent Industry 5.0 paradigm [8,9].

The increasing digitalization and the democratization of artificial intelligence have enabled such models to automate and provide assistance on a wide range of tasks. Machine learning models can learn from historic data to predict future outcomes (e.g., estimate future demand [10]), or perform certain tasks (e.g., identify manufacturing defects [11] or automate manual tasks with robotic assistance [12]). Nevertheless, it is crucial to establish a synergic and collaborative environment between humans and machines, where humans can flourish in developing their creativity, while delegating monotonous and repetitive tasks to machines [9]. Furthermore, such collaboration requires transparency to develop trust in artificial intelligence applications. Research exploring the models’ rationale behind the predictions and how the insights must be provided to the user as an explanation is known as explainable artificial intelligence.

Insights obtained regarding the models’ rationale behind a prediction can be enriched with domain knowledge to provide a better-contextualized explanation to the user [13,14]. To encode domain knowledge, researchers have resorted to using semantic technologies, such as ontologies and knowledge graphs. In order to integrate semantic technologies into the explanation crafting process, the structure of the explanation must be defined, and a procedure established to (a) extract semantic meaning from the explanation, (b) use such information to query external sources (e.g., open knowledge graphs), and (c) enrich the explanation with new information and insights obtained from the external sources.

In this research, we extend the work performed by [15,16]. Given a set of demand forecasts, a mapping between features and concepts that define an ontology and a hierarchy of concepts, and a set of keywords associated with each concept, we recourse to a wikification process to extract wiki concepts based on the keywords mentioned above. The wiki concepts are then used to query external sources (e.g., open knowledge graphs or open datasets) and rank query results based on their semantic similarity, computing the Jaccard distance. Our approach was tested in the domain of demand forecasting and validated on a real-world case study, using models we developed as part of the European Horizon 2020 projects FACTLOG (https://www.factlog.eu/, (accessed on 27 April 2022)) and STAR (http://www.star-ai.eu/, (accessed on 27 April 2022)). We evaluate our results through three metrics (precision@K and Ratio of Diverse Entries (RDE@K), and coverage regarding five questions proposed in [17]) to assess how precise and diverse the knowledge fragments are.

The rest of this paper is structured as follows: Section 2 presents related work, Section 3 describes the use case we used and the implementation we followed to test our concept, and Section 4 provides the results we obtained and their evaluation. Finally, in Section 5, we provide our conclusions and outline future work.

2. Related Work

2.1. Industry 4.0 and Industry 5.0

Industry 4.0 was coined in 2011 by the German government initiative aimed at developing advanced production systems to increase the productivity and efficiency of the manufacturing industry. The initiative was soon followed by national initiatives across multiple countries, such as the USA, UK, France, Canada, Japan, and China, among others [18,19,20]. Industry 4.0 was conceived as a technological revolution adding value to the whole manufacturing and product lifecycle. Part of such a revolution is the concept of a smart and integrated supply chain, which aims to reduce delivery times and reduce information distortion across suppliers and manufacturers [21]. The benefits mentioned above are achieved by enhancing the demand forecasting and optimizing the organization and management of materials, suppliers, and inventory [22,23]. To that end, digital twins of existing processes can be created to simulate, test what-if scenarios, and enhance them without disruption of the physical operations [24,25].

On top of the aforementioned integrated supply chain, the Industry 4.0 paradigm emphasizes the redesign of the human role in manufacturing, leveraging new technological advancements and capabilities [21]. This aspect is further evolved in Industry 5.0. Industry 5.0 is a value-driven manufacturing paradigm that underscores the relevance of research and innovation to develop human-centric approaches when supporting industry operation [26]. Human-centricity in manufacturing must take into account the skills that are unique to the human workers, such as critical thinking, creativity, and domain knowledge, while leveraging machine strengths (e.g., high efficiency in performing repetitive tasks [3,27,28]). Human-centricity can be realized through (a) a systemic approach focused on forging synergic, two-way relationships between humans and machines, (b) the use of digital twins at a systemic level, and (c) the adoption of artificial intelligence at all levels [9,29].

Artificial intelligence can be essential to achieving human–machine collaboration. In particular, active learning and explainable artificial intelligence can be used to complement each other. Active learning is the sub-field of artificial intelligence, concerned with retrieving specific data and leveraging human knowledge to satisfy a particular learning goal. On the other hand, explainable artificial intelligence is the sub-field of artificial intelligence concerned with providing insights into the inner workings of a model regarding its outcome so that the user can learn about its underlying behavior. This way, the human can act as a teacher to artificial intelligence models and learn from them through explainable artificial intelligence. This two-way relationship can lead to a trusted collaboration [30,31].

2.2. Demand Forecasting

Demand forecasting is a key component of manufacturing companies. Precise demand forecasts allow to set correct inventory levels, price the products and plan future operations. Any improvements in such forecasts directly translate to the supply chain performance [32]. Demand depends on characteristics that are intrinsic to the product [33] (e.g., elasticity or configuration), and external factors [34] (e.g., particular sales conditions).

Researchers have developed multiple schemas to classify demand according to its characteristics. Among them, we find the ABC inventory classification system [35], the XYZ analysis [36], and the quadrant proposed by Syntetos et al. [37]. The ABC inventory classification system classifies the items based on their decreasing order of annual dollar volume. The XYZ analysis classifies items according to their consumption patterns: (X) constant consumption, (Y) fluctuating (usually seasonal) consumption, and (Z) irregular consumption. Finally, Syntetos et al. divide demand into four quadrants based on demand size and demand occurrence variability. It was recognized that artificial intelligence models can provide accurate demand forecasts based on past demand and complementary data. Different methods are appropriate for demands with different characteristics [38]. Demand forecasting models are usually framed as time series forecasting problems using supervised regression models or specialized models to learn patterns and forecast future values. Research related to the automotive industry has reported using complementary sources, such as the unemployment rate [39], inflation rate [40] or gross domestic product [41], and a variety of algorithms, such as multiple linear regression [42], support vector machine [43], and neural networks [44].

The ability to leverage a wide range of complementary data sources is a specific advantage of such models against humans in light of their visual working memory, short-term memory, and capacity to process variables [45]. Furthermore, artificial intelligence models avoid multiple cognitive biases to which humans are prone to [46]. Nevertheless, planners must approach such forecasts with critical thinking since they hold responsibility for decisions based on such forecasts. They must understand the models’ rationale behind the forecast [47,48,49], take into account information that could signal that adjustments to the forecast are needed and make such adjustments when needed [50,51,52]. Furthermore, while transparency regarding the models’ underlying rationale can be, in some cases, required by law [53] (e.g., the general data protection regulation (GDPR) [54] or the artificial intelligence act [55]), it also provides a learning opportunity, which is key to the employees’ engagement [56].

2.3. Explainable Artificial Intelligence

Explainable artificial intelligence is a sub-field of artificial intelligence research, concerned with how the models’ behavioral aspects can be translated into a human interpretable form to understand causality, enhance trustworthiness, and develop confidence [57,58]. Techniques are usually classified based on their complexity (degree of interpretability), their scope (global or local), and whether they are model-agnostic [47]. Regarding their degree of interpretability, models are usually considered black-box (opaque) or white-box (transparent). The source of models’ opacity can be due to the inherent properties of the algorithms, their complexity, or due to the explicit requirement to avoid exposing its inner workings (e.g., trade secret), [59,60].

Demand forecasting can be framed as a regression problem, and thus explainability techniques developed for this kind of supervised learning can be used to unveil the models’ inner workings. One such technique is the Local Interpretable Model-agnostic Explanations (LIME) [61], which provide a model-agnostic approach to estimate features’ relevance for each forecast. LIME creates a linear model to approximate the model’s behavior at a particular forecasting point. Then, it estimates the features’ relevance by measuring how much predictions change upon features’ perturbation. Similar approaches were later developed to ensure that explanations were deterministic [62], or take into account non-linear relationships between the features [63,64]. Other frequently cited methods are the Shapley additive explanations (SHAP, which leverages coalitional game theory to estimate the contributions of individual feature values) [65,66], Local Agnostic attribute Contribution Explanation (LACE, which leverages SHAP and LIME to provide an explanation through local rules) [67], Local Rule-based Explanation (LoRE, which crafts an explanation extracting a decision rule and a set of counterfactual rules) [68], anchors [69] (which establishes a set of precise rules that explain the forecast at a local level), and local foil trees [70] (which identifies a disjoint set of rules that result in counterfactual outcomes).

Understanding why the model issued a particular forecast is of utmost importance [71]. Explanation crafting can be guided using domain knowledge, enriched with complementary data or insights. An approach to enhance explanations given specific domain knowledge was developed by Confalonieri et al. [72], who demonstrated that decision trees built for explainability were more understandable if built considering domain knowledge. Enrichment with complementary data was researched by Panigutti et al. [73], where high fidelity to the forecasting model was achieved by enriching the explainability rules with semantically encoded domain knowledge. Semantic enrichment or recourse to graph representations to bind multiple insights was explored by several authors [14,15,74]. Finally, Rabold et al. [75] devised means to create enhanced, multimodal explanations by relating visual explanations to logic rules obtained through an inductive logic programming system.

While many metrics have been devised to assess the quality of the outcomes produced by explainability techniques [76,77,78,79], explanations must also be evaluated and measured from a human-centric perspective. Attention must be devoted to ensuring the explanations convey meaningful information to different user profiles according to their purpose [80,81,82] such that they promote curiosity to increase learning and engagement [83], and provide means to develop trust through exploration [84]. Furthermore, it is desired that the explanations are actionable [85,86], and inform conditions that could change the forecast outcome [87]. The aspects mentioned above are frequently evaluated through qualitative interviews and questionnaires, think-aloud, or self-explanations [83,88]. Methods such as tracking participants’ eye movement patterns, measuring users’ response time to an explanation, the quantification of answers provided, or the number of explanations required by the user to learn have also been proposed [17,83,88,89].

3. Use Case

This research was developed for a demand forecasting use case, exploring how to provide comprehensive forecast explanations. In particular, we consider the most relevant features regarding a particular forecast and enrich it based on domain knowledge and information retrieved from external sources. The explanations aim to (a) provide context regarding real-world events that could describe feature values observed for a particular forecast, (b) assist in exploring datasets that can be used to enrich the dataset used to train the model, and (c) discover new relevant concepts by integrating with knowledge graphs. Knowledge regarding which factors drive a particular forecast and real-world events provides the user with valuable context for responsible decision making. Exploring complementary datasets is valuable to data scientists and machine learning engineers, who can iterate upon the existing model and enhance it. Furthermore, such information can also be valuable to other profiles, widening the understanding of forces driving the forecast or the influence of such particular demand. Such understanding can also be widened with insights obtained from knowledge graphs.

This research extends work performed in [15,16,90], taking into account four data sources: (i) data provided by a European original equipment manufacturer targeting the global automotive industry market; (ii) news media entries provided by Event Registry [91], a well-established media events retrieval system that provides real-time mainstream media monitoring; (iii) the EU Open Data Portal [92]; and (iv) the Google Knowledge Graph (Google KG) [93]. We detail the procedure in Figure 1.

$$JaccardIndex(A,B)=\frac{|\mathit{A}\cap \mathit{B}|}{|\mathit{A}\cup \mathit{B}|}$$

(1)

$$JaccardDistance=1-JaccardIndex$$

(2)

In [15], we studied how explanations given to the user could be enriched with external sources, searching for news media entries and metadata regarding datasets. Searches were performed based on keywords that described features’ semantic abstractions and new keywords identified from retrieved news media entries. The entries were then ranked based on the word movers distance ([94]) between embeddings. In this research, we adopted a different strategy and (i) computed wiki concepts for the features’ abstraction keywords (see

Table 1. Mapping between feature keywords and wiki concepts. Wikification was performed invoking the service available at https://wikifier.org (accessed on 27 April 2022). Note that while most concepts were accurate, the wikification of the Purchasing Managers’ Index produced a wrong concept.

Feature Keywords	Wiki Concepts
Car Sales Demand	Car
	Demand
New Car Sales	Car
	Sales
Vehicle Sales	Vehicle
Car Demand	Car
	Demand
Automotive Industry	Automotive Industry
Global Gross Domestic Product Projection	Gross Domestic Product
	Gross World Product
Global Economic Outlook	Economy
	World economy
Economic Forecast	Forecasting
	Economy
Unemployment Rate	Unemployment
Unemployment Numbers	Unemployment
Unemployment Report	Unemployment
Employment Growth	Employment
Long-term Unemployment	Unemployment
Purchasing Managers’ Index	Manager (Gaelic games)

), (ii) ranked news media entries based on the Jaccard distance (see Equation (2)) between relevant reference wiki concepts and the ones obtained by wikifying the news media events [95], (iii) queried and ranked external data sources based on reference wiki concepts and the most important concepts that emerged from news media events, and (iv) further enriched the explanations by adding most relevant entries from the Google KG queried with the most relevant wiki concepts obtained from news media events (Media Events’ Keywords and Wiki Concepts (Media Events’ K&WC)). However, we found that the most frequent wiki concepts in media news referred to persons and places, which were not informative to the task at hand. Therefore, we decided to filter them out to enhance the quality of the outcomes.

4. Evaluation and Results

Our primary interest in this research was to use semantic technologies and external data sources to enrich the explanations. We consider that explanations can be enriched with (a) information contextual to existing domain knowledge, (b) metadata regarding datasets, which can be used to enrich future models, and (c) relevant concepts obtained by integrating with semantic tools and knowledge graphs. We realized (a) by querying events informed in media news, which could potentially explain a given forecast. Furthermore, we realized (b) by querying open dataset portals, and (c) by retrieving wiki concepts associated with media news pieces of interest and the results obtained from the Google KG.

$$RDE=\frac{\mathit{Unique}\mathit{Entries}}{\mathit{Total}\mathit{Listed}\mathit{Entries}}$$

(3)

To evaluate the outcomes of such enrichment, we used two metrics to assess whether the entries were precise and diverse: (a) Average Precision@K and the RDE@K (see Equation (3)). The first metric allowed us to measure how much of the information we displayed was related to the underlying model’s features. Given that the users rely on trusted automation [96], we consider that exact results are required to increase the users’ trust in the underlying application. Furthermore, considering that curiosity is related to the workers’ engagement, we consider a diverse set of entries is preferred to foster it. We quantify such diversity through the RDE@K metric. While the entries do not repeat themselves in a single forecast explanation, nothing prevents having the duplicate entries listed through different forecast explanations. We consider that the best case could avoid repeated entries to maximize users’ learning. Nevertheless, different strategies could be adopted. One such strategy could be to frame the entries displayed as a recommender system problem and consider users’ implicit and explicit feedback to rank and decide whether and where to display them [97,98,99].

We present our results in

Table 2. Results we obtained by analyzing the forecast explanations created for 56 products over three months. The best results are shown in bold. Media events, Media events’ keywords, external datasets and Google KG correspond to contextual information displayed for each forecast explanation. NA is used for entries regarding the results concerning the Google KG for the embeddings-based approach, given that the research we compare against did not provide such results.

	Metric	Embeddings-Based Approach	Semantics-Based Approach
Media Events	Average Precision@1	0.97	0.95
	Average Precision@3	0.97	0.95
	RDE@1	0.30	0.38
	RDE@3	0.11	0.14
Media Events’ K&WC	Average Precision@1	0.77	0.71
	Average Precision@3	0.78	0.72
	RDE@1	0.14	0.01
	RDE@3	0.09	0.01
External Datasets	Average Precision@1	0.56	0.68
	RDE@1	0.41	0.43
Google KG	Average Precision@1	NA	0.76
	Average Precision@3	NA	0.46
	RDE@1	NA	0.15
	RDE@3	NA	0.09

. We compare the results obtained in this research (the semantics-based approach described in Section 3) with those obtained in our previous work (embeddings-based approach [15]). We found that the former approach had better performance when considering the media events’ diversity, trading a slight decay in precision compared with the embeddings-based approach. Regarding the Media Events’ K&WC, the embeddings-based approach achieved better performance for both diversity and precision. While differences in precision were contested, they were pronounced regarding diversity. We consider it is natural to have a less numerous set of wiki concepts compared to keywords. Nevertheless, we consider that the metric values could be improved. On the other hand, the semantics-based approach was much more precise when recommending datasets and displayed a slightly higher diversity. Finally, when evaluating the entries related to the Google KG, we observed that the diversity was similar to the one obtained for media news with the embeddings-based approach. The first results were precise, but the precision dropped by 0.30 points when considering k = 3. Furthermore, we found that those considered erroneous were mostly related to the economy or automotive industry when analyzing the entries. Nevertheless, they were not useful for the explanations at hand since they referenced banks (e.g., Bank of America or Deutsche Bank) or prominent figures (e.g., Edward Fulton Denison, who pioneered the measurement of the United States’ Gross Domestic Product, or Kathleen Wilson-Thompson, independent director at Tesla Motors Inc. at the time of this writing). On the other hand, we considered that the entries were accurate when they referenced companies from the automotive sector (e.g., Faurecia, Rivian Automotive Inc., Polestar or Vinfast) or related to it (e.g., plug power which develops hydrogen fuel cell systems to replace conventional batteries, or the Flinkster carsharing company).

Considering the results regarding Media Events’ K&WC and external datasets, we consider that the embeddings-based approach is best to obtain new keywords and concepts later displayed to the users. In contrast, the semantics-based approach provides the best concepts that lead to better results when searching for external datasets’ metadata.

Finally, based on the work of [17], we evaluated the goodness of the overall explanations provided to the user based on a score similar to the degree of explainability, computed as the coverage of five questions related to the main archetypes from the abstract meaning representation theory [100]: (i) Why? (ii) How? (iii) What for? (iv) What if? (v) When? Our explanations achieved a score of 0.6, providing information regarding (i) the factors leading to such a forecast (why is such a forecast issued?), (ii) how such an outcome can be changed (relevant, actionable aspect), and (iii) when will the forecast demand take place and when did the relevant events influencing the demand occur. The explanations can be improved with information regarding the purpose of the action (increasing or decreasing the demand) and providing capabilities to explore ”what-if” scenarios.

5. Conclusions and Future Work

Along with the increasing adoption of artificial intelligence in manufacturing, explainability techniques must be developed to ensure that users’ can learn the models’ behaviors. Furthermore, explanations provided to the user can be enriched with additional insights that foster the users’ curiosity, resulting in an exploratory dynamic toward artificial intelligence application and domain-specific problems, enabling the development of trust toward such applications. One way to achieve this goal is to enrich the explanations with information obtained from external sources to augment users’ knowledge and help them make responsible decisions.

This research explored augmenting explanations by incorporating media news, datasets’ metadata, and information queried from open knowledge graphs. Furthermore, we compared two approaches (based on embeddings and wiki concepts) to rank the data sources’ entries and retrieve new concepts and keywords from them. We found that results were similar when considering media events but differed for new keywords and concepts (embeddings-based approach was best) and external datasets (semantic-based approach was best). We found that the embeddings-based and semantic-based approaches could complement each other regarding keywords and concepts extracted from media events. While the embeddings-based approach resulted in better scores regarding keywords and concepts shown to the user, the semantics-based approach led to better results when searching for datasets’ metadata. Finally, while the integration with the Google KG proved informative, new strategies must be explored to increase the precision of results when showing multiple entries.

We consider that such an approach toward building explanations can be extended to other use cases. Among its strengths, we consider the capability to explore real-world events impacting the forecasting model and the ability to identify complementary data that can be used to enhance the forecasting model. On the other hand, the approach requires manually associating each feature with higher-level concepts and a set of keywords characterizing them. This requires that the features used to train the model have a meaning intelligible to the human—a requirement always satisfied by handcrafted features. Another limitation is the need to specify a time frame within which feature-related events are expected to happen, given the features’ time of occurrence.

In future work, we would like to explore how critical components of an explanation encoded in a graph structure can be used to enhance the explanations and whether users’ feedback can lead to better explanations if we frame them as a recommender system problem.