In this Section we describe the main learning principles behind deep learning models (connectionist AI) and KGs and ontologies (symbolic AI). Understanding their differences is a crucial requirement to realize how symbolic systems can be integrated within connectionist approaches to build a more comprehensible AI.
The movement of connectionism
, also known as parallel distributed processing, correspond to “the second wave” (the first wave dates back to the 1940s and it was based on a neural perspective of AI [17
]) of research in neural networks. Born in the 1980s, the central idea of connectionists is that a network of simple computational units, defined as neurons
, excite and inhibit each other in parallel to achieve an intelligent behavior. As underlined by Rumelhart et al. [18
], the resulting knowledge consists of the connections between these computational units distributed throughout the network. Two key concepts of the connectionism have been borrowed by modern deep learning techniques. The first concept is represented by the distributed representation
]. According to this principle: (i) each input of a system has to be represented by multiple features; (ii) each of these multiple features needs to be adopted in the representation of many possible inputs. The second concept consists in the adoption of the back-propagation
algorithm to train deep neural networks [20
]. The goal of the back-propagation is to efficiently compute the gradient of the loss function with respect to the weights, or parameters, of a neural network. Through this gradient computation, it is possible to update the network weights to minimize the loss value. As a consequence, the learning process can be reduced to an optimization problem, finding a function that produce the minimal loss. Such methodological advances, in combination with the increasing of computational resources and the availability of large datasets, make the modern deep learning techniques very powerful in being trained without supervision, extracting patterns and regularities from the input data. Making this acquisition process explainable and transparent is still an open research issue in the connectionist community, which is now trying to fill the gap between black-box systems [21
] to more understandable systems. Nevertheless, as recognized by some authors (e.g., Lecue [22
]), this community is far from creating tools that in terms of explainability adapt to different domains and applications.
Beside the connectionist approach, the movement of symbolic AI, also known as GOFAI (Good Old-Fashioned Artificial Intelligence) [23
], adopts an opposed paradigm: the knowledge about the world is not acquired deriving a mathematical model through optimization techniques, but it is hard-coded in the system exploiting formal languages. Therefore, the system is able to reason on the statements expressed in these formal languages through logical inference rules. The modern implementation of symbolic systems is represented by ontologies, formal representation of domain conceptualizations, and Knowledge Graphs (KGs), large networks of entities and relationships relevant to a specific domain, where each node of the graph is an entity and each edge is a semantic relationship connecting two different entities. KGs are explicitly designed to capture the knowledge within domains, integrating and linking data from different phenomena, or different types of representation [24
]. The reasoning over these artifacts, as reported by Hoehndorf et al. [25
], enables (i) consistency checking
(i.e., recognizing contradictions between different facts), (ii) classification
(i.e., defining taxonomies), (iii) deductive inference
(i.e., revealing implicit knowledge given a set of facts). Considering their features, symbolic methods are not robust to noise and can not be applied to non-symbolic context where the data is ambiguous. Nevertheless, they offer a data-efficient process by which models can be trained to reason on symbolic contexts and are able to provide background knowledge for deep learning models [25
]. Lecue [22
] stresses the opportunity of the usage of KGs to encode the semantic of input and output data, considering also their specific properties. Moreover, KGs could play a central role in designing novel deep learning architectures that incorporate causation [12
], supporting also other tasks, from data integration and discovery to semantic fragmentation and composition [22
]. All these features represent an essential condition to create, as defined by Doran [4
], a system that is able to emit comprehensible symbols apart from its output. This condition enables the opportunity for the user to interact with the system, exploiting her tacit form of reasoning and knowledge [4
] that relies on shared symbols.