Integrating Deep Learning and Knowledge Graphs for Natural Language Inference

Abstract

Natural Language Inference (NLI) deals with the problem of logical entailment between two sentences. It is evidenced that apart from interpretation of words and syntactic structures, use of external relevant knowledge is also required. This paper investigates whether the integration of external knowledge, represented as triplets and extracted from Knowledge Graphs (KGs), can improve the performance of a pretrained language model on NLI tasks. It also investigates how the source of knowledge affects performance. Six benchmark datasets, namely RTE, QNLI, SICK, SNLI, DNLI, and SciTail, were enriched with knowledge extracted from AristoTuple and WordNet, while ConceptNet was additionally used for the SNLI and SciTail datasets. Enrichment was applied only to the training data through a strict relevance-based selection mechanism using paraphrase-MiniLM-L6-v2 embeddings and cosine-similarity filtering, and the resulting datasets were used to fine-tune BERT-base-uncased. Evaluation included method-specific configurations selected through validation-based hyperparameter exploration and a controlled multi-seed setting using the vanilla baseline hyperparameters with matched seeds across baseline and enriched datasets. The best observed validation accuracy was 95.24% on SciTail with AristoTuple, while the highest test accuracy, also on SciTail, was 93.70% with ConceptNet. Across both evaluation settings, the effect of knowledge enrichment varied across datasets and knowledge sources, with the multi-seed analysis further assessing the robustness of these effects under controlled training conditions. Overall, the proposed data-level approach achieved competitive performance compared with more complex knowledge-enhanced NLI methods. A taxonomy of existing NLI approaches is also presented.

1. Introduction

With modern technology, the human approach to language understanding and inference is being transferred to machines. The field known as Natural Language Inference [1] aims to teach machines to “understand” sentences and investigate logical entailment, as humans do. NLI deals with the problem of estimating the logical relationship between two sentences: the premise and the hypothesis. The central question is whether the hypothesis can be logically inferred from the premise. This investigation can result in one of three possible outcomes: entailment, where the hypothesis logically follows from the premise; contradiction, where the hypothesis and premise are in direct conflict; and neutral, where the truth or falsity of the hypothesis is independent of the premise. In some cases, a simplified binary approach is used, limited to entailment and not entailment cases.

The development of this technology is no longer merely a subject of academic research; it reflects a critical need in our era. The more accurately machines can understand and interpret language as humans do, the more reliable, useful and safer their applications become. NLI technology is now present in search engines, digital assistants, machine translation systems, sentiment analysis, AI-assisted diagnostics and recommendation algorithms. Thus, progress in this field is not only a research achievement but also a prerequisite for the effective utilization of artificial intelligence in everyday life. The pivotal importance of NLI is highlighted by the fact that its study and development have not remained stagnant; on the contrary, NLI has followed a trajectory of continuous evolution over the years.

In early attempts, prior to 2015 [2,3,4,5,6,7,8], addressing the problem relied mainly on lexical analysis and surface-level features. Systems utilized resources such as stemming, WordNet, FrameNet and syntactic parsers, while training was performed with basic machine learning techniques like support vector machines and decision trees. Those methods focused either on the conceptual coverage of the hypothesis by the premise or on the logical entailment between sentences. Despite those efforts, overall system performance remained limited, highlighting the need for more complex and effective approaches.

This need was addressed during the period 2015–2017, when the introduction of deep learning, combined with the availability of larger datasets such as SNLI [9] and MNLI [10], led to remarkable progress. New models, such as LSTM, BiLSTM and attention-based architectures [11,12,13], were developed, enabling automatic feature extraction and significantly increasing accuracy. Systems thus gained the ability to recognize relationships between sentences without extensive manual analysis.

Building on this momentum, the period 2017–2019 marked a true revolution with the advent of transformers [14] and the creation of pretrained language models [15,16]. Through techniques such as transfer learning and fine-tuning [17], models were able to leverage general knowledge acquired from large datasets and adapt it to specialized tasks. Particularly decisive was the introduction of contextualized embeddings [18], a novel technique that allowed words to be represented according to their context, improving accuracy and enabling the recognition of complex logical relationships.

Subsequently, from 2019 to 2021, research focused on the integration of external knowledge [19] and addressing issues of model robustness and interpretability [20]. The use of information from external sources, such as knowledge graphs, enhanced the systems’ ability to draw conclusions with logical consistency. Challenges such as models’ sensitivity to minor data modifications and limited generalization to new or unseen domains were also highlighted [21,22].

From 2022 onward, the emergence of Large Language Models (LLMs) such as GPT-3.5, LLaMA-2 (7B and 13B), PaLM and ChatGPT enabled NLI tasks to be addressed even without specialized training, utilizing few-shot and zero-shot learning approaches. Despite impressive results, challenges remain, including response instability, the generation of inaccurate information, and limited capacity to mimic human reasoning, emphasizing the need for continuous optimization and the development of more reliable, interpretable and generalizable systems [23,24].

Considering the above developments, pretrained transformer-based models remain a strong and practical foundation for NLI. However, model performance and reliability can still be improved, especially in cases where sentence understanding requires information that is not explicitly present in the input text. For this reason, the integration of external knowledge represents an important research direction. Previous studies have incorporated external knowledge in different forms, such as lexical relations from resources like WordNet, syntactic information from dependency parsers, semantic role labeling, domain-specific knowledge, and structured knowledge graphs.

From the reviewed literature, knowledge can be integrated into NLI models in different ways. One option is to enrich the input data with additional knowledge before training. Another option is to modify the final classification or feed-forward layers of the model, so that external knowledge is combined with the learned sentence representations before prediction. A third option, followed by several knowledge-enhanced approaches, is to modify or extend the internal architecture or representation-learning mechanisms of the model, for example, by incorporating external knowledge into attention-based components or intermediate representations.

Existing KG-based or knowledge-enhanced methods, such as [19,25], often follow the latter direction, introducing knowledge through changes inside the model architecture or attention mechanism. Although such approaches can be effective, they also make the system more complex and may require model-specific modifications. In contrast, the present study investigates a simpler but strict data-level strategy. Instead of modifying the internal layers of the downstream NLI model, the proposed approach introduces knowledge directly into the training data by enriching the examples with relevant triplets extracted from knowledge graphs. In this way, the original architecture of the downstream model remains unchanged, while the training data are enhanced with additional semantic information.

To this end, a two-stage strategy was adopted, which combines dataset enrichment with relevant external knowledge and the subsequent training and evaluation of a pretrained transformer-based language model on the enriched data. In the enrichment stage, n-gram phrases within the sentences are identified and matched with the subjects of triplets in the knowledge graphs. The corresponding candidate triplets are converted into natural-language representations, and their relevance to the original sentence is assessed using cosine similarity and graph-specific thresholds, based on contextual embeddings produced by a Sentence Transformer model. This strict selection criterion mitigates the noise that could arise from the inclusion of irrelevant knowledge, while applying enrichment exclusively to the training set ensures that the learning phase is reinforced without altering the validation or test data.

The enriched datasets are then used for the training and performance analysis of a pretrained transformer-based NLI model, using a framework that supports the integration of external datasets and the evaluation of multiple NLI configurations. The application of this strategy in the present study primarily aims to investigate whether, and to what extent, the integration of knowledge from different knowledge graphs can improve performance compared to previous approaches with or without external knowledge. The analysis also examines why the effect of knowledge enrichment varies across datasets. This variation appears to depend not only on the source and relevance of the integrated knowledge, but also on dataset-specific characteristics, such as sentence length, content, linguistic structure, and the way each dataset was constructed.

The contributions of this work are summarized as follows:

• A structured overview of the evolution of NLI approaches, from earlier lexical and neural methods to pretrained language models and knowledge-enhanced approaches, also depicted on a unique taxonomy.
• A lightweight data-level enrichment strategy that incorporates knowledge graph triplets into the training data without modifying the internal architecture of the pretrained transformer-based models.
• A strict relevance-based triplet selection mechanism using n-grams, Sentence Transformer embeddings, cosine similarity, and threshold.
• An experimental evaluation across multiple NLI datasets and knowledge graphs, aiming to examine when external knowledge is beneficial, and which factors affect its contribution.

2. Related Work

2.1. NLI Approaches Without External Knowledge

An early approach to the NLI task is presented in [26], where the authors employ SVM classifiers on the RTE-3 dataset, which comprises four subcategories: Question Answering (QA), Information Retrieval (IR), Multi-document Summarization (SUM) and Information Extraction (IE), using lexical and syntactic features. Their method extracts features from multiple normalized versions of each Text–Hypothesis pair and applies various similarity metrics. To overcome the limitations of strict lexical matching, the approach also incorporates “partial matches” that capture more flexible morphological or semantic relationships, along with additional features such as negation indicators and length ratios. Altogether, these procedures produce a total of 128 features. System performance varies across the RTE-3 subcategories, achieving higher accuracy in QA and IR, and lower performance in SUM and particularly IE, highlighting the limitations of relying solely on surface-level similarities.

With the advent of neural networks and the introduction of the SNLI dataset, deep learning approaches became prominent in addressing the NLI task. Within this context, the Decomposable Attention Model (DAM) [11] was proposed as an efficient and conceptually simple architecture that avoids complex sentence-level representations. The model decomposes the inference process into three stages—Attend, Compare and Aggregate—focusing on word or phrase-level interactions through attention mechanisms. By relying on feed-forward networks and largely ignoring word order, DAM achieves high parallelization and low computational complexity. An optional intra-sentence attention component further enriches word representations with limited structural information. Despite its strong performance and efficiency, the emphasis on local, word-level alignments may limit the model’s ability to capture the global sentence meaning, which is often crucial for more complex forms of reasoning in NLI.

The introduction of the Transformer [14] and its self-attention mechanism marked a shift from traditional neural networks to models capable of processing complex relationships between words and sentences with greater efficiency and parallelization. In this context, Bidirectional Encoder Representations from Transformers (BERT) [15] emerged as one of the most influential approaches, introducing bidirectional contextual representations that significantly improved sentence-level natural language understanding. BERT relies exclusively on the encoder component of the Transformer architecture and is pre-trained in an unsupervised manner on large-scale corpora, such as the BooksCorpus and English Wikipedia. The pre-training process involves two main tasks: Masked Language Modeling (MLM), which allows the model to learn bidirectional context by predicting masked words, and Next Sentence Prediction (NSP), which strengthens the understanding of relationships between consecutive sentences. Inputs to BERT are represented using token embeddings, positional embeddings and segment embeddings, facilitating the simultaneous processing of sentence pairs within a unified framework. During the fine-tuning stage, the pre-trained model is adapted to specific supervised tasks, such as NLI, via a task-specific output layer. In this phase, all model parameters are updated end-to-end, allowing the general representations learned during pre-training to be tailored for the target task. Despite its strong performance, BERT relies primarily on the internal knowledge it acquires during pre-training. To further improve results, techniques such as combining multiple Transformer models and modifying attention mechanisms have been explored, aiming to increase prediction accuracy and reliability.

A prominent example of this multi-model approach can be found in the study by [27], which proposes a stacking ensemble composed of three high-performing Transformer models: ALBERT-XXLarge, RoBERTa-Large, and DeBERTa. Each architecture provides complementary strengths: ALBERT offers efficiency with a reduced parameter footprint, RoBERTa provides robust contextual representations, and DeBERTa introduces disentangled attention mechanisms alongside improved embeddings. The prediction outputs produced by the three Transformer models are used as inputs to different meta-models, including Logistic Regression, XGBoost, feed-forward neural networks, LSTMs, and graph-based networks. Depending on the ensemble configuration, the meta-model also receives additional features derived from these predictions, such as prediction difference, entropy, or confidence margin, and uses all these inputs to produce the final NLI label. Sequential experiments demonstrate that the ensemble consistently outperforms individual Transformer models. Furthermore, the application of explainability techniques, such as LIME and SHAP, reveals which words and contextual cues most influence inference decisions. Nevertheless, despite these architectural advancements, challenges remain in cases requiring deeper inferential reasoning, such as metaphorical relations.

Since 2022, research in NLI has been significantly influenced by the emergence of Large Language Models (LLMs), which introduced new evaluation paradigms such as zero-shot and few-shot learning, in contrast to traditional fine-tuning models like BERT. A recent study [24] systematically evaluates LLM performance on Semantic Textual Similarity (STS) and NLI tasks across ten datasets from general, clinical and biomedical domains, comparing zero-shot ChatGPT, few-shot LLaMA-2 (7B and 13B) and fine-tuned BERT-base. A preliminary analysis examines different prompting strategies, including annotation guidelines and chain-of-thought instructions, leading to the selection of zero-shot prompting for ChatGPT and few-shot prompting for LLaMA-2. The generated responses are converted into numerical similarity scores for STS and class labels for NLI. The study also evaluates calibration using different confidence-estimation methods: ChatGPT is treated as a black-box model, with confidence estimated from repeated generations, whereas LLaMA-2 is used as a white-box proxy, allowing confidence to be derived from label-token probabilities. Expected Calibration Error is then used to assess the correspondence between confidence and prediction correctness. Finally, ChatGPT is prompted with different system roles to examine whether it can simulate individual judgements and reproduce collective human opinion distributions. The results show that fine-tuned BERT remains stronger in most settings, while the LLMs exhibit overconfidence, and the persona-based ChatGPT predictions fail to capture human judgements reliably.

In addition to clinical and biomedical domains, NLI research has expanded into more areas such as mathematics. The Math NLI task [28] investigates whether LLMs can reliably perform entailment, contradiction and neutral reasoning on research-level mathematical texts. The authors construct a balanced, human-annotated seed corpus and evaluate ten LLMs using prompts that request both an NLI label and an explanation. During corpus validation, pairs showing strong disagreement between the models and human annotations are manually re-examined by mathematically trained researchers, revealing difficulties related to ignored context, vague quantifiers, lexical ambiguity and unfamiliar mathematical concepts. The study also uses GPT-4 to generate an additional Math NLI corpus and applies diagnostic tests for self-entailment, contradiction symmetry and hypothesis-only artifacts. Although the models achieve moderate to high overall accuracy, they struggle particularly with neutral pairs, which are frequently misclassified as entailments, indicating persistent limitations in logical consistency and mathematical inference.

2.2. NLI Approaches with External Knowledge

As the field of NLI has evolved, it has become evident that pre-trained models (PTMs), such as BERT, capture extensive contextual information but lack direct access to external knowledge. To address this limitation, methods have been proposed to integrate external knowledge into neural architectures, aiming to enhance logical reasoning and comprehension in more complex or knowledge-intensive scenarios.

A relevant approach in this direction is presented in [25], where the integration of lexical knowledge from WordNet into language models is explored. The authors propose a method that directly modifies the multi-head attention mechanism of Transformer models to incorporate word-level relations such as synonymy, antonymy, hypernymy, and hyponymy. By adjusting the similarity scores between word embeddings according to these relations, the model becomes more robust to lexical noise and better captures the relevant semantic relationships. Performance is evaluated on both clean and adversarial datasets, showing improvements particularly in cases containing lexical traps. Two architectures are investigated. Model I follows a decomposable structure with separate encoders for the premise and hypothesis and cross-encoder layers that compute interactions between the two sentences. External knowledge is injected into the multi-head attention of selected cross-encoder layers and the resulting sentence representations are fed into a multi-layer perceptron classifier. Model II is a simpler Transformer encoder that concatenates the premise and hypothesis, processes them through N encoder layers and uses the [CLS] token representation for classification. This technique is also applied to pre-trained BERT-base, integrating knowledge directly into the attention layers and further improving performance. Although the approach does not introduce additional trainable parameters, it still requires modifications to the models’ internal attention mechanism.

Building on the same principle of structured knowledge integration, ExBERT [19] enriches BERT-base-uncased with information from knowledge graphs such as AristoTuple and ConceptNet. After stop words are removed, candidate triplets are retrieved when tokens from the premise or hypothesis occur in the head entity of a knowledge graph triple. The triplets are converted into natural language sentences and ranked according to their contextual relevance using cosine similarity between their BERT representations and the representations of input bigrams, trigrams, four-grams, and the complete sentence. The highest-ranked triplets are retained for its gram, while duplicates are removed. The premise–hypothesis pair and the selected triplets are encoded separately using BERT. Their representations are then combined through an additional multi-head attention mechanism, where the premise–hypothesis representations act as queries and the triplet representations as keys and values. A mixture mechanism balances the original context with external knowledge before the combined representations pass through composition, pooling, and a multilayer perceptron classifier. The results indicate that performance depends more on the contextual and domain relevance of the selected triplets than on their quantity.

Building upon these approaches that leverage structured knowledge from knowledge graphs, external knowledge in NLI can also take alternative forms. For instance, another line of research is presented in [29], which incorporates external knowledge in the form of syntactic information. In this method, Syntactic Word Representations (SWRs) are extracted from a pre-trained dependency parser for both the premise and the hypothesis. These representations can be integrated into NLI models either through late fusion (+LF), by concatenating them with the final representation before classification, or through syntactic attention (+SA), by augmenting the attention mechanism to guide alignments between subphrases. This approach can be applied to a variety of models, including Decomposable Attention, ESIM, BERT, and MT-DNN, providing additional syntactic context that enhances the model’s understanding of sentence structure and dependencies. Further analysis through ablation experiments and diagnostic evaluation on HANS shows that the gains are not merely due to additional parameters but are associated with a reduced reliance on shallow syntactic heuristics.

Another syntax-oriented approach is Syntax-BERT [30], which integrates syntactic tree information directly into pre-trained Transformer models. The method modifies the self-attention mechanism through syntax-based masks derived from dependency and constituency trees. These masks decompose the original fully connected self-attention into multiple sparse sub-networks, each capturing a specific syntactic relation, such as parent, child, sibling, or cross-sentence interactions in pairwise inference tasks. The representations produced by these sub-networks are then combined through a topical attention layer, which learns the relative importance of different syntactic views for each downstream task. A key advantage of Syntax-BERT is its plug-and-play design, since it can be applied to existing pre-trained Transformer checkpoints without pre-training from scratch. The framework is evaluated with several backbones, including BERT, RoBERTa, T5 and their variants, and shows consistent improvements on NLI datasets, as well as on broader GLUE tasks. Ablation experiments show that syntax-based masks, especially parent and child relations, contribute to the model’s performance, while structural probe analysis indicates that syntax-aware models encode more syntactic information than their vanilla counterparts.

In addition to syntactic knowledge, in the medical domain, external knowledge can take a domain-specific and sentiment-aware form [31]. In this study, structured knowledge from the Unified Medical Language System (UMLS) is combined with sentiment information. The approach leverages the MedNLI dataset, where each word in the sentences is first tokenized and then processed using MetaMap to extract relevant UMLS concepts along with their associated sentiment annotations. To represent the information, BioELMo embeddings are used to capture contextual word representations, while UMLS concepts are encoded via MultE embeddings. Additionally, sentiment vectors are incorporated to indicate positive or negative references to medical concepts. These embeddings are combined using a weighted sum, allowing the model to dynamically adjust the relative importance of each type of information. The combined representations are fed into a BiLSTM-based architecture with attention mechanisms, enabling the model to perform more accurate and informed reasoning. Experimental results demonstrate that the integration of structured domain knowledge and sentiment information significantly improves both the reasoning capabilities and the accuracy of NLI models in the medical domain.

Another approach that incorporates explicit semantic knowledge is presented in [32], where Semantic Role Labeling (SRL) is used to enhance text comprehension and inference. The method is motivated by the fact that NLI and reading comprehension require understanding predicate–argument structures, such as who did what to whom. An SRL labeler assigns semantic role labels to the input tokens, which are converted into SRL embeddings and concatenated with the original word representations, providing downstream models with explicit contextual semantic cues. The approach is evaluated on textual entailment using the SNLI dataset and on machine reading comprehension using SQuAD. In both tasks, SRL is integrated into strong baseline models, including an ELMo-based architecture, a BiDAF-style model, BERT-base and BERT large. The results show consistent improvements, while ablation studies confirm that the gains come from the semantic role information itself rather than merely from stronger word representations. Comparisons with POS and named-entity tags further suggest that SRL provides more task-relevant semantic cues for inference.

Extending the use of semantic role information, Semantics-aware BERT (SemBERT) [33] integrates explicit contextual semantics directly into a pre-trained BERT architecture. The method uses a pre-trained SRL labeler to extract predicate-centered semantic structures from each input sentence, capturing multiple semantic perspectives such as who did what to whom. These SRL labels are mapped into semantic embeddings and encoded with a BiGRU. Since BERT operates on subword units while SRL labels are word-level annotations, SemBERT reconstructs word-level contextual representations from BERT subwords using a convolutional layer with max pooling. The resulting BERT-based word representations are then concatenated with the SRL-based semantic representations to form a joint semantics-aware representation for downstream tasks. The SRL labeler remains fixed, while the SemBERT model is fine-tuned in the same way as BERT. Experimental results, reported for both base and large model variants, show that this SRL-based integration improves over the original BERT baseline across NLI and broader NLU tasks. Further analysis indicates that the gains come not only from adding SRL information, but also from the proposed integration mechanism, as the full architecture outperforms simple BERT + SRL concatenation. Qualitative examples show more semantically complete predictions, while robustness tests with corrupted SRL labels suggest stability under noisy annotations. The approach is also potentially extensible to other languages through multilingual SRL resources such as the CoNLL-2009 treebanks.

Beyond explicit forms of external knowledge, implicit knowledge acquired through large-scale pre-training and transfer learning has also been shown to benefit NLI and broader NLU tasks. MT-DNN [34] combines pre-trained language modeling with multi-task learning to improve natural language understanding. It extends the earlier MT-DNN architecture by replacing its previous shared representation layers with a pre-trained BERT encoder, thereby combining the linguistic knowledge captured during large-scale pre-training with the transferable knowledge learned jointly from multiple supervised NLU tasks. The model is trained across four task categories: single-sentence classification, pairwise text classification, text similarity scoring and relevance ranking. Its architecture consists of shared lower layers, including a lexicon encoder and a Transformer encoder, together with task-specific output layers tailored to each task type, such as classification, regression-based similarity scoring, multi-step reasoning for NLI and relevance scoring for ranking. The model is considered in both base and large variants, showing that the proposed multi-task learning framework can be applied at different model scales. Evaluations on different datasets show that multi-task learning improves performance across several benchmarks and supports more effective domain adaptation, requiring fewer in-domain labeled examples when adapting to new datasets or domains.

2.3. A Taxonomy of NLI Approaches

Based on the preceding analysis, we developed a taxonomy of approaches (Figure 1) used to address the NLI problem. At the highest level, the approaches are divided into two broad categories, according to whether they rely on external knowledge in addition to the information contained in the input sentences and the knowledge acquired during model training.

Figure 1. Classification of NLI approaches according to the use of external knowledge. Classical machine-learning approaches [26]; deep-learning approaches [11,15,27]; LLM-based approaches [24,28]; lexical knowledge [25]; syntactic knowledge [29,30]; semantic knowledge [32,33]; general knowledge graphs [19]; domain-specific knowledge graphs [28,31]; and implicit knowledge approaches [34].

Approaches that do not use external knowledge can be further distinguished according to the main computational design on which they are based. The first subcategory includes classical machine learning techniques, such as SVM-based systems that rely on manually designed lexical and syntactic features [26]. The second includes deep learning approaches, ranging from neural architectures such as the Decomposable Attention Model to pretrained Transformer-based models and ensemble methods [11,15,27]. The third subcategory comprises approaches based on LLMs, which are commonly evaluated through zero-shot or few-shot settings rather than through conventional task-specific fine-tuning [24,28].

Approaches that use external knowledge are divided into those that employ explicit knowledge and those that exploit implicit knowledge. Implicit knowledge is acquired indirectly through pretraining, transfer learning, or multi-task learning rather than being provided through a separate external resource during the NLI task. MT-DNN [34] is placed in this category because it combines knowledge acquired during BERT pretraining with transferable representations learned from multiple supervised natural language understanding tasks.

Explicit knowledge, in contrast, originates from identifiable external resources and is directly incorporated into the NLI process. Depending on its nature, it may be lexical [25], syntactic [29,30], semantic [32,33], or represented through knowledge graphs [19,28,31]. Knowledge graphs may provide either information associated with a specific domain or general-purpose knowledge. Our approach belongs to that last category.

3. Materials and Methods

3.1. System Development

3.1.1. System Design

The proposed system follows a modular pipeline that separates knowledge enrichment from downstream NLI training and evaluation. As illustrated in Figure 2, an NLI sentence pair and an external knowledge graph are provided as input, and an enriched version of the training example is produced before model training. The enrichment component operates directly on the textual input, after which the resulting data are used for model training and evaluation within a common experimental framework.

Figure 2. Overview of the proposed system.

The enrichment process augments the original premise and hypothesis with external semantic information while preserving their original content and class labels. Although the enriched datasets are configured as separate tasks within the experimental framework, they retain the same premise–hypothesis–label structure as their non-enriched counterparts. Therefore, the same downstream NLI model architecture can be used for both baseline and knowledge-enriched configurations.

To identify the optimal configuration, experiments were conducted with different combinations of hyperparameters, including batch size, learning rate, number of epochs and evaluation-related parameters such as early stopping and evaluation frequency. This procedure aimed both at optimizing final performance and preventing overfitting. Overall, the system design was based on simple and extensible principles, enabling the effective integration of external knowledge into the data while ensuring a stable training process for the language model.

3.1.2. System Implementation

To enable the integration of heterogeneous external knowledge sources within a common pipeline, all knowledge graphs were represented in a unified subject–predicate–object (S–P–O) format. Each triplet was stored as a tab-separated record in an .spo file, with the subject and object corresponding to graph nodes and the predicate representing the semantic relation connecting them. This common representation allowed knowledge originating from structurally different resources to be processed by the same enrichment procedure.

The preparation process differed according to the original structure of each knowledge source. For resources distributed in structured tabular form, the relevant subject, predicate, and object fields were extracted and reorganized into the common S–P–O representation. Other resources required additional preprocessing, such as filtering by language, normalization of URI-based concepts into readable textual forms, and removal of duplicate triplets. For lexical resources accessed programmatically, semantic relations were extracted through the corresponding Python interfaces and converted into explicit triplets. These relations included synonymy, antonymy, hypernymy, hyponymy, meronymy, holonymy, entailment, similarity, and attribute-based relations.

The NLI datasets were also converted, where necessary, into formats compatible with the Jiant framework. Datasets directly supported by the experimental framework were obtained through its built-in data-loading functionality, whereas datasets not natively supported were downloaded from their respective official sources and transformed into a common premise–hypothesis–label structure. Dataset-specific preprocessing was applied before enrichment to ensure a consistent input representation across experiments. Depending on the original dataset format, this included retaining only the fields required for the NLI task, converting sentence pairs and labels into the expected structure, and applying sequence-length filtering when necessary. In cases where sentence-length constraints were relevant to the enrichment model, examples whose premise or hypothesis exceeded the predefined token limit were excluded before knowledge integration.

Following the preparation of the knowledge sources and the NLI data, the next stage focused on the integration of external knowledge into the textual input. The objective of this procedure was not to append arbitrary knowledge-graph information to the sentences, but rather to identify and retain only triplets that were sufficiently related to the semantic context. To achieve this, the premise and hypothesis were processed independently, allowing the relevance of external knowledge to be assessed separately for each component of the NLI pair.

The knowledge-retrieval process begins with the identification of expressions within each sentence that may correspond to subjects represented in the knowledge graph. Each sentence is divided into whitespace-separated tokens and, for a sentence containing N tokens, all contiguous n-grams from length 1 to N are generated. Therefore, the search is not restricted to individual words or to a predefined maximum n-gram size, but also considers multi-word expressions and, where applicable, the complete sentence. In total, a sentence of N tokens can generate

$${\displaystyle \frac{N\left(N+1\right)}{2}}$$

contiguous n-grams. Each generated n-gram is compared with the subjects represented in the corresponding knowledge graph. When a match is identified, the relations originating from the matched subject are retrieved and reconstructed as candidate subject–predicate–object triplets.

The identification of a graph subject within a sentence does not, however, necessarily imply that all triplets connected to that subject are relevant to the particular context in which it occurs. A concept may participate in several semantic relations, only a subset of which may be useful for the meaning expressed by the current sentence. For this reason, an additional semantic-selection stage is introduced. The complete sentence and each candidate triplet are independently encoded into contextual vector representations using the same sentence-embedding model. The semantic correspondence between the sentence and each candidate triplet is then quantified using cosine similarity:

$$\mathrm{sim}\left(s,t\right)={\displaystyle \frac{e_s\cdot e_t}{\Vert e_s\Vert \Vert e_t\Vert }}$$

where $e_s$ denotes the embedding of the complete sentence and $e_t$ the embedding of the candidate triplet. Importantly, similarity is evaluated against the complete sentence rather than only against the matched n-gram. In this way, triplet selection takes into account the broader semantic context in which the matched concept occurs.

The resulting similarity score constitutes the main filtering criterion of the enrichment procedure. Candidate triplets whose cosine similarity falls below a predefined threshold are discarded, whereas those satisfying the criterion proceed to the ranking stage. The choice of this threshold is particularly important because it directly determines the amount and semantic specificity of the external knowledge introduced into the training examples. A relatively low threshold increases knowledge coverage by allowing a larger number of triplets to pass the filtering stage; however, it also increases the possibility of introducing weakly related or contextually irrelevant information. Conversely, a high threshold favors more strongly related triplets and therefore provides more conservative knowledge selection but may exclude potentially useful information and reduce the overall degree of enrichment. The threshold therefore regulates the trade-off between knowledge coverage and semantic relevance, and its value must be selected carefully. The graph-specific threshold values and their selection are discussed in Section 3.2.2.

After threshold filtering, the remaining candidate triplets associated with each matched n-gram are ranked in descending order according to their cosine-similarity scores. Up to the two highest-scoring triplets are retained for each matched n-gram and appended to the corresponding sentence. If only one candidate satisfies the threshold, only that triplet is incorporated, whereas no external knowledge is added when none of the candidates meets the filtering criterion. Because this Top-2 selection is performed separately for each matched n-gram, a sentence containing multiple matched expressions may ultimately receive more than two triplets.

The selected triplets are appended to the original sentence while preserving its initial textual content, thereby enriching the input with additional semantic information derived from the external knowledge source. The resulting enriched representations are subsequently used during model training, allowing the downstream NLI model to process the original NLI statements together with the selected external knowledge.

Following the enrichment of the datasets with external knowledge, the next stage involved integrating the resulting datasets into the experimental framework as separate NLI tasks in order to evaluate the impact of knowledge enrichment on model performance. The framework supports the definition and execution of custom tasks, allowing both the original and enriched dataset configurations to be incorporated within a common training and evaluation environment.

The integration of each task required the definition of its task type, label space, and the procedures used to load the corresponding training, validation, and test splits. The framework subsequently handled the conversion of the input data into model-compatible examples, together with the preprocessing operations required for model input, including tokenization, feature construction, and batching. Each task was registered within the framework and associated with the corresponding configuration information, enabling it to be recognized and executed as an independent experimental setting.

Accuracy was used as the primary evaluation metric, in accordance with the binary or ternary classification formulation of the examined NLI datasets. Each task configuration also specified the corresponding dataset locations and the information required for training and evaluation, thereby providing a consistent experimental workflow across the different baseline and knowledge-enriched configurations.

The pretrained transformer-based models required for the experiments were downloaded from an external repository, in which the framework is connected, and prepared in a format compatible with it. This design allows different pretrained models or checkpoints to be incorporated into the same experimental pipeline according to the requirements of each experiment. The training, validation, and test splits were subsequently tokenized and cached, allowing the resulting representations to be reused across experimental runs and reducing repeated preprocessing.

Model training and evaluation were controlled through the configuration mechanisms provided by the framework. The corresponding settings included parameters such as learning rate, batch size, number of training epochs, evaluation frequency, early stopping, and prediction generation. During training, model performance was periodically assessed on the validation data, while the final trained configuration was subsequently used to generate predictions for the corresponding evaluation sets.

In the final stage, the generated model predictions were used for test-set evaluation and subsequent analysis. The evaluation procedure depended on whether gold-standard labels for the corresponding test split were publicly available.

For datasets with publicly available test labels, the gold-standard annotations were loaded and converted into the numerical representation required for comparison with the model outputs. The stored predictions were then aligned with the corresponding test instances, and their number was verified against the number of reference labels to ensure a one-to-one correspondence between predictions and examples. Accuracy was subsequently computed locally as the proportion of correctly classified instances over the total number of test examples.

For benchmark datasets whose official test labels are not publicly released, direct local evaluation was not possible. In these cases, the generated predictions were transformed into the submission format required by the corresponding benchmark. The predicted class identifiers were mapped to the expected label representation and associated with the original test-instance indices. The resulting predictions were then stored in the required output format and ordered according to the original test-set sequence before being submitted to the official online evaluation platform, from which the final test performance was obtained.

3.2. Experimental Setup

The experimental procedure was designed to evaluate the effect of external knowledge integration across different NLI settings. Six datasets were selected to provide diversity in terms of structure, content, and linguistic characteristics, while three knowledge graphs with a common triplet-based representation but different types of information were employed to examine the contribution of different external knowledge sources. A pretrained sentence-embedding model was used during the knowledge-selection stage to estimate the semantic relevance of candidate triplets, whereas two pretrained language model checkpoints were subsequently used for NLI training and evaluation. This combination of multiple datasets, heterogeneous knowledge sources, and pretrained models provided a common experimental basis for assessing the proposed enrichment approach.

The experiments were implemented in Python (v3.11)within a virtual environment to ensure controlled dependency management and library compatibility. Jiant (v2.2.0) was employed as the main experimental framework for organizing the NLI tasks and supporting the model training, evaluation, and prediction procedures. The experiments were executed within the Windows Subsystem for Linux (WSL 1) [35], providing a Linux-based environment within Windows, while code development was carried out using the Spyder IDE. The computationally intensive stages of knowledge enrichment and downstream model training were performed using NVIDIA GeForce RTX 3080 Ti and NVIDIA GeForce RTX 2080 Ti GPUs (NVIDIA Corporation, Santa Clara, CA, USA).

3.2.1. Datasets

The RTE (Recognizing Textual Entailment) challenges, introduced within the framework of the PASCAL program, were among the first benchmarks for evaluating textual entailment, aiming to determine whether a hypothesis can be inferred from a given text. A total of seven editions (RTE-1 through RTE-7, [2,3,4,5,6,7,8]) were conducted, during which the datasets evolved in size, sentence length and complexity, progressing from small pairs of sentences with binary labels to more realistic scenarios with longer texts and additional entailment categories. Subsequently, the data from RTE-1, RTE-2, RTE-3 and RTE-5 were incorporated into the GLUE benchmark [36], which provides a unified evaluation framework for contemporary language understanding models. The GLUE-RTE dataset comprises a total of 2490 sentence pairs for training, 277 for validation and 3000 for testing, with all examples converted into a binary classification: entailment or not entailment, where the original contradiction and neutral categories were merged into a single not entailment category.

The SICK (Sentences Involving Compositional Knowledge) dataset [37,38] was designed to evaluate models of semantic understanding and compositional language. It comprises 9927 English sentence pairs, annotated both for semantic relatedness on a 5-point scale and for logical entailment with the labels entailment, contradiction and neutral. The sentences were derived from the 8K-ImageFlickr and SemEval 2012 STS MSR-Video Description datasets and were subsequently normalized and recomposed to introduce a range of semantic and syntactic phenomena while limiting the presence of idiomatic expressions and domain-specific terminology. The annotations were collected through crowdsourcing, with multiple annotators evaluating each sentence pair and quality-control procedures applied during the annotation process. The class distribution is imbalanced, with 5611 neutral pairs (56.5%), 2857 entailment pairs (28.8%), and 1459 contradiction pairs (14.7%). The dataset is divided into 4500 training pairs, 500 validation pairs, and 4927 test pairs.

The QNLI (Question-Answering Natural Language Inference) dataset is part of the GLUE benchmark and is designed to assess a model’s ability to perform NLI in a question–answering setting. Its construction is based on SQuAD (Stanford Question Answering Dataset) [39], one of the most widely used datasets for text comprehension, which contains over 100,000 questions derived from passages taken from Wikipedia articles. QNLI reformulates the original answer-extraction task as a binary inference problem by pairing each question with a sentence from the corresponding context. The objective is to determine whether the sentence contains sufficient information to support the answer to the question, resulting in the labels entailment and not entailment. The QNLI dataset contains approximately 105,000 sentence pairs in the training set and around 5500 pairs in both the validation and test sets.

DNLI (Dialogue Natural Language Inference) [40] is a dataset designed to evaluate models on consistency and contradiction detection within dialogue contexts. It is constructed from the Persona-Chat corpus, where each participant in a dialogue is associated with predefined personal attributes expressed as persona sentences. Each such sentence is converted into an entity–relation–entity triplet, providing a structured semantic representation. Pairs are then formed either between dialogue utterances and persona sentences or between pairs of persona sentences. Since the original Persona-Chat data does not include explicit logical relationships, all labels require human annotation. Each pair is classified into one of three categories—positive (entailment), neutral, or negative (contradiction)—based on specific semantic criteria. The final DNLI dataset consists of 343,110 annotated pairs, divided into 310,110 training, 16,500 validation and 16,500 test examples.

The SNLI (Stanford Natural Language Inference) dataset [9] is one of the most widely used resources for NLI. It contains 570.000 sentence pairs, each consisting of a premise and a hypothesis labeled as entailment, neutral, or contradiction. The premises were derived from image captions in the Flickr30k dataset, while the hypotheses were generated through crowdsourcing. For each premise, annotators were asked to produce three hypotheses corresponding to the entailment, neutral, and contradiction classes. On average, hypotheses are shorter than premises, with approximately 7 and 14 words, respectively, reflecting a preference for concise but grammatically correct statements. To improve annotation reliability, a subset of the examples (10%) was independently evaluated by additional annotators, and gold labels were assigned on the basis of annotator agreement. Examples for which a valid gold label (−1) was not available were excluded from the final labeled dataset. The resulting dataset contains 549,367 training examples, 9842 validation examples, and 9824 test examples.

The SCITAIL dataset [41] was constructed to support NLI derived from science question-answering contexts. It was derived from multiple-choice science questions collected from standardized examinations and the SciQ dataset. Its creation followed a multi-stage pipeline involving question collection, hypothesis generation, evidence retrieval and human annotation. Annotators judged whether the premise entailed the hypothesis or was neutral with respect to it. The original SciTail split contains 23,097 training examples, 1304 development examples, and 2126 test examples. During preprocessing for the knowledge-enrichment stage, 9 training instances exceeded the input-length constraints of the sentence-embedding model and were therefore excluded. This corresponds to approximately 0.04% of the original training set, resulting in 23,088 training examples. To ensure a consistent comparison, the same filtered training split was used for both the baseline and knowledge-enriched configurations, while the validation and test sets remained unchanged.

3.2.2. Knowledge Graphs

AristoTuple [42,43] was developed by the Allen Institute for Artificial Intelligence (AI2) as a comprehensive knowledge base for extracting structured scientific information from textual sources, intended for use in natural language understanding applications requiring scientific reasoning. The construction of the dataset relied on three main inputs: a large volume of scientific sentences retrieved from the web using the Bing search engine, a specialized scientific vocabulary and predefined types of entities (e.g., animals, objects, body parts) to facilitate concept recognition. These inputs were processed to extract subject–predicate–object triplets using OpenIE 4.2.

The reliability of the extracted triplets was ensured through manual annotation of a representative subset using Amazon Mechanical Turk, combined with a logistic regression model to predict the validity of the remaining triplets based on features such as pointwise mutual information (PMI) and semantic patterns. Subsequently, the triplets were enriched and normalized using the Canonical Schema Induction (CASI) method, which identifies groups of equivalent or similar relations and maps them to a generalized schema, producing a coherent and consistent knowledge representation. The resulting AristoTuple knowledge base contains 282,594 triplets (see Table 1 for examples) and provides a robust and extensible resource suitable for applications in natural language understanding and scientific reasoning.

Table 1. Examples of AristoTuple triplets.

Subject	Predicate	Object
bacterium	data	human disease
person	has-part	human body
project leader	conduct	project team

For AristoTuple, a cosine-similarity threshold of 0.8 was adopted for triplet filtering. The threshold was determined empirically through preliminary experiments conducted on the RTE validation set, where different candidate values of 0.6, 0.7, and 0.8 were explored. Among the examined settings, 0.8 produced the highest observed validation performance and was therefore selected for the subsequent experiments. A comparatively stricter threshold was also considered appropriate for AristoTuple because of its predominantly scientific and domain-oriented content, where retaining only triplets with stronger semantic correspondence can help limit the incorporation of loosely related information. Once selected, the threshold of 0.8 was kept fixed for AristoTuple across all datasets.

WordNet [44] is an extensive lexical database for the English language, developed at Princeton University. Unlike conventional dictionaries, which organize words alphabetically, WordNet is structured around meanings rather than word forms. Words that express similar concepts are grouped into synsets, with each synset representing a distinct semantic concept and accompanied by a definition as well as example sentences demonstrating usage.

The database encodes a rich set of semantic relationships between synsets. These include traditional relationships such as synonymy, antonymy, hypernymy and hyponymy, as well as part–whole relations (meronymy/holonymy). Additionally, WordNet contains other semantic connections, such as similar-to, entails and has-attribute, forming a highly interconnected network of concepts.

Due to this organization, WordNet functions not only as a lexical resource but also as a structured semantic knowledge graph. After preprocessing it for use, these synsets and their relations are converted into triplets of the form (subject, predicate, object). The generated dataset contains approximately 789,739 triplets (see Table 2 for examples). It provides a stable framework for computational tasks in natural language understanding, including textual entailment.

Table 2. Examples of WordNet triplets.

Subject	Predicate	Object
Extract	synonym	take out
Playback	has hypernym	electronic equipment

For WordNet, the filtering threshold was likewise determined through preliminary experiments on the RTE validation set, where candidate values of 0.6 and 0.7 were compared. Of the examined settings, 0.7 produced the highest observed validation performance and was therefore retained for the subsequent experiments. WordNet contains broad lexical–semantic relations, such as synonymy, hypernymy, and hyponymy, for which a slightly less restrictive threshold was considered appropriate to preserve potentially useful lexical information. Following its selection, the threshold of 0.7 was applied consistently across all datasets.

ConceptNet [45,46] is a multilingual knowledge graph that collects and organizes commonsense knowledge from a variety of sources, including Open Mind Common Sense (OMCS), Wiktionary, WordNet, DBPedia, OpenCyc and Games With A Purpose. The graph consists of nodes, representing words or phrases in natural language and edges, which connect two nodes through general relations that capture semantic, functional or causal connections between concepts.

Term representation is standardized: words are normalized, tokenized and embedded in URIs that include their language. The vocabulary includes common words, phrases and, where available, disambiguated forms, while different forms of the same word are linked to each other to enable consistent search and traversal.

Because ConceptNet is multilingual and this work focuses only on English, only the English triplets were retained from the original graph. Since ConceptNet is updated over time and different studies may use different versions or filtering criteria, the number of triplets may vary. In this work, the latest available version was downloaded and filtered for English entries, resulting in approximately 3,339,418 triplets (see Table 3 for examples) covering over 1.5 million nodes. ConceptNet is widely used to enhance NLP systems with commonsense knowledge, measure semantic similarity, support reasoning, and create enriched embeddings, such as ConceptNet Numberbatch.

Table 3. Examples of ConceptNet triplets.

Subject	Predicate	Object
ruby	IsA	jewel
digital	RelatedTo	computer
uncle	Antonym	aunt

For ConceptNet, a cosine-similarity threshold of 0.7 was adopted as a fixed filtering parameter rather than being independently tuned. ConceptNet contains broad and heterogeneous commonsense relations covering everyday concepts and associations; therefore, a moderately permissive threshold was considered appropriate for retaining potentially useful commonsense information while excluding candidates with low semantic correspondence. The value of 0.7 was fixed from the outset and used consistently in all ConceptNet enrichment experiments.

3.2.3. Models

BERT-base-uncased (Hugging Face checkpoints: howey/bert-base-uncased-mnli (accessed on 12 October 2026) for RTE; google-bert/bert-base-uncased (accessed on 9 September 2026) for the other datasets) [15] is one of the most widely adopted pretrained language models based on Transformer architecture. This variant consists of 12 encoder layers, 768 hidden units and 12 self-attention heads, amounting to approximately 110 million parameters. Its architecture enables effective modeling of long-range linguistic dependencies, providing strong contextualized representations of natural language.

In this study, BERT-base-uncased serves as the foundational pretrained model that is subsequently fine-tuned on datasets enriched with external, structured knowledge extracted from knowledge graphs. This integration aims to enhance the model’s capability in logical inference, combining contextual language understanding with higher-level semantic information derived from external knowledge sources.

For integrating external knowledge, we generated semantic embeddings of both dataset sentences and knowledge graph triplets to measure their semantic relatedness. We used the pretrained Paraphrase-MiniLM-L6-v2 Sentence-Transformer model (https://huggingface.co/sentence-transformers/paraphrase-MiniLM-L6-v2 (accessed on 9 September 2026)) [47], which has 6 hidden layers and produces 384-dimensional sentence embeddings, with a maximum input length of 128 tokens. Sentence embeddings are obtained via mean pooling over token embeddings and semantic similarity is measured using cosine similarity.

The model was pretrained on a variety of natural language tasks, including paraphrase identification, question matching and text similarity, using datasets such as AllNLI, MSMARCO, QuoraDuplicates and SimpleWiki. These pretrained embeddings are then used to select the most relevant knowledge triplets for each sentence, which are incorporated into the original sentences.

3.2.4. Experimental Framework

The experiments were conducted using Jiant [48], an open-source framework designed for modular and configuration-driven experimentation on natural language understanding tasks. Jiant supports a broad collection of NLU benchmarks, including the GLUE and SuperGLUE tasks, and provides integration with Transformer-based pretrained language models through the Hugging Face Transformers library [49]. In the present study, Jiant was used as the main framework for organizing the NLI tasks and managing the downstream training, validation, and prediction procedures. Datasets directly supported by the framework were handled through the corresponding built-in task definitions, whereas datasets not natively available were incorporated through custom task implementations while preserving a common premise–hypothesis–label structure. The baseline and knowledge-enriched versions were configured as separate experimental tasks, allowing the same downstream model architecture and evaluation procedure to be applied to both settings. Training parameters, including batch size, learning rate, number of epochs, validation frequency, and early-stopping criteria, were specified through the corresponding experimental configurations and varied during the hyperparameter exploration.

4. Results

In this section, we present the results obtained from the knowledge-enriched configurations and examine their performance across the considered NLI datasets. For each dataset and knowledge source, the reported results correspond to the best-performing configuration identified among the experimental settings examined, based on validation performance. These results therefore represent the best observed performance under the selected configuration for each enriched setting. In addition to model performance, the enrichment process is examined in terms of the number of knowledge triplets incorporated into the training data.

Table 4 illustrates representative examples of sentence pairs: the first column shows the knowledge graph used, the second column presents the original sentence pair, and the third column depicts the enriched version after the incorporation of knowledge triplets from the corresponding knowledge graph. These examples illustrate how external knowledge is added to the original textual input and provide a qualitative view of the information introduced during the enrichment process.

Table 4. Examples of original and knowledge-enhanced sentence pairs.

Graph	Original Pair	Enhanced Pair
AristoTuple	Premise: Rabies virus infects the central nervous system, causing encephalopathy and ultimately death. Hypothesis: Rabies is fatal in humans.	Premise: Rabies virus infects the central nervous system, causing encephalopathy and ultimately death. Hypothesis: Rabies is fatal in humans. (Rabies kill human)
WordNet	Premise: When is Carnival celebrated? Hypothesis: Alpine festivals vary from country to country and often include the display of local costumes such as dirndl and trachten, the playing of Alpenhorns, wrestling matches, some pagan traditions such as Walpurgis Night and, in many areas, Carnival is celebrated before Lent.	Premise: When is Carnival celebrated? (Carnival has hypernym festival) Hypothesis: Alpine festivals vary from country to country and often include the display of local costumes such as dirndl and trachten, the playing of Alpenhorns, wrestling matches, some pagan traditions such as Walpurgis Night and, in many areas, Carnival is celebrated before Lent.
ConceptNet	Premise: Dietary fiber is the foodstuff that is not digested by the stomach or small intestine. Hypothesis: The small intestine begins in the stomach.	Premise: Dietary fiber is the foodstuff that is not digested by the stomach or small intestine. Hypothesis: The small intestine begins in the stomach. (small intestine RelatedTo stomach) (small intestine PartOf intestinal tract) (intestine RelatedTo stomach) (intestine RelatedTo small intestine)

4.1. RTE

The RTE dataset poses a challenging benchmark due to relatively long sentences, semantic complexity and realistic scenarios. Within this context, the present study investigates the integration of external knowledge from the AristoTuple and WordNet graphs, enabling the model to leverage causal, scientific and general semantic relationships to assess whether this improves entailment recognition between sentence pairs. For both knowledge sources, the number of triplets incorporated into the training data was relatively small. Specifically, AristoTuple contributed 28 triplets across 16 of the 2490 training examples, corresponding to a coverage of 0.64% and an average of 0.011 triplets per training example. In comparison, WordNet contributed 18 triplets across 13 examples, resulting in a coverage of 0.52% and an average of 0.007 triplets per training example.

Extensive hyperparameter tuning was conducted for both scenarios. For AristoTuple, several hyperparameter configurations were examined, with the highest validation accuracy obtained using a training batch size of 32, an evaluation batch size of 16, a maximum of 10 epochs, and a learning rate of 4e-5. Validation was performed every 50 steps, with an early-stopping patience of 3, and the reported run used a random seed of 96621154. During training, evaluations were conducted on up to 500 validation examples, which covered the entire RTE validation set. The selected configuration achieved a validation accuracy of 84.83%. As shown in Figure 3, performance improved gradually during training, reaching a peak around step 450 before exhibiting a slight decline. Early stopping was subsequently applied when further improvement was no longer observed. To assess performance on the official benchmark, the corresponding predictions were submitted to the GLUE evaluation server, where the AristoTuple-enhanced model achieved a test accuracy of 76.0%.

Figure 3. Training performance curves for RTE-AristoTuple.

For WordNet, the highest validation performance was obtained with a training batch size of 64, an evaluation batch size of 16, a maximum of 15 epochs, and a learning rate of 2e-5. The model was evaluated every 30 steps on the complete 500-example validation set, while early stopping was applied with a patience of 4. Using a random seed of 678171624, this configuration achieved a final validation accuracy of 84.11%. The training curves in Figure 4 reveal a temporary decline around step 250, followed by a recovery until approximately step 300, before the early-stopping criterion was eventually met. Evaluation through the official GLUE benchmark yielded a test accuracy of 77.5%. Although AristoTuple achieved a higher validation accuracy, WordNet produced the higher test accuracy.

Figure 4. Training performance curves for RTE-WordNet.

4.2. SICK

The SICK dataset presents a demanding benchmark for semantic understanding, featuring varied sentence structures and a rich mix of semantic and syntactic phenomena, while deliberately excluding idiomatic expressions and domain-specific terminology. In this study, we probe the incorporation of external knowledge from the AristoTuple and WordNet graphs, allowing the model to exploit structured scientific and lexical–semantic relationships to enhance its capacity to identify entailment and other semantic relations between sentence pairs. The incorporation of external knowledge affected only a small portion of the training set. AristoTuple contributed 138 triplets to 86 of the 4500 training examples, corresponding to a coverage of 1.91% and an average of 0.031 triplets per training example. WordNet contributed 95 triplets to 57 examples, resulting in a coverage of 1.27% and an average of 0.021 triplets per training example.

For the AristoTuple-enhanced scenario, an extensive hyperparameter exploration was carried out. The highest validation performance was obtained with a training batch size of 16, an evaluation batch size of 16, a maximum of 15 epochs, and a learning rate of 4e-5. Validation was performed every 100 steps using a subset of 500 validation examples, while early stopping was applied with a patience of 5. The reported run used a random seed of 351642494. Final evaluation on the complete validation and test sets yielded accuracies of 88.40% and 87.78%, respectively. As shown in Figure 5, the training trajectory remained relatively stable, with only limited fluctuations before early stopping was activated when no further improvement was observed.

Figure 5. Training performance curves for SICK-AristoTuple.

For the WordNet-augmented setup, the highest validation performance among the examined configurations was obtained using a training batch size of 32, an evaluation batch size of 16, a maximum of 15 epochs, and a learning rate of 4e-5. Validation was conducted every 40 steps on a subset of 500 validation examples, with an early-stopping patience of 5. The reported run used a random seed of 1304729058. Evaluation on the full validation and test sets resulted in accuracies of 88.60% and 87.62%, respectively. The learning curves shown in Figure 6 exhibit only minor variations during training, with performance eventually reaching a plateau before early stopping was applied. In these runs, WordNet achieved slightly higher validation accuracy, whereas AristoTuple produced marginally higher test accuracy.

Figure 6. Training performance curves for SICK-WordNet.

4.3. QNLI

Here we examine the results of integrating external knowledge into the QNLI dataset using the AristoTuple and WordNet knowledge graphs. This enrichment provides the model with structured scientific knowledge and lexical–semantic relationships, aiming to enhance its ability to determine whether a sentence answers a given question, with triplets potentially linking the question and answer directly to facilitate inference. The amount of incorporated knowledge differed noticeably between the two sources. AristoTuple contributed 839 triplets to 576 of the 104,743 training examples, corresponding to a coverage of 0.55% and an average of 0.008 triplets per training example. WordNet resulted in a larger amount of enrichment, with 2786 triplets incorporated into 1746 training examples, corresponding to a coverage of 1.67% and an average of 0.027 triplets per training example.

For the AristoTuple-integrated setting, extensive hyperparameter tuning was conducted. The highest validation performance was obtained using a training batch size of 32, an evaluation batch size of 16, 3 epochs, a learning rate of 2e-5, and validation every 500 steps. The reported run used a random seed of 651321276. During training, performance was monitored on a subset of 500 validation examples, where the model reached a peak accuracy of 90.8%, as shown in Figure 7. Final evaluation on the complete validation set yielded an accuracy of 91.68%. Since QNLI does not provide publicly available test labels, the corresponding predictions were submitted to the official GLUE evaluation server, where the AristoTuple-enhanced model achieved a test accuracy of 90.8%.

Figure 7. Training performance curves for QNLI-AristoTuple.

In the WordNet-enhanced setting, the contribution of lexical–semantic knowledge was assessed. WordNet relations, including hypernymy, antonymy and synonymy, can provide critical context to support entailment when connections between question and sentence are not explicit. After hyperparameter tuning, the highest validation performance among the examined configurations was obtained using a training batch size of 32, an evaluation batch size of 16, 3 epochs, a learning rate of 2e-5, and validation every 500 steps. The reported run used a random seed of 2749343491. Evaluation during training on the 500-example validation subset reached an accuracy of 91.6%, as shown in Figure 8, while final evaluation on the complete validation set resulted in 91.56% accuracy. Submission to the official GLUE evaluation server produced a test accuracy of 90.7% for the WordNet-enhanced model. AristoTuple achieved a closely higher validation and test accuracy, despite WordNet introducing a considerably larger number of knowledge triplets into the training data.

Figure 8. Training performance curves for QNLI-WordNet.

4.4. DNLI

The DNLI dataset, which consists of relatively short sentence pairs derived from dialogue-based NLI, provides a useful setting for examining how external knowledge interacts with limited textual context. In this case, AristoTuple and WordNet were used to investigate whether scientific and lexical–semantic relations could provide additional information without introducing excessive noise. The two sources of knowledge resulted in different levels of enrichment. AristoTuple contributed 4394 triplets to 3458 of the 310,110 training examples, corresponding to a coverage of 1.12% and an average of 0.014 triplets per training example. WordNet introduced a larger amount of external knowledge, with 10,638 triplets incorporated into 7824 training examples, corresponding to a coverage of 2.52% and an average of 0.034 triplets per training example.

For the AristoTuple-enhanced configuration, a series of experiments was conducted to explore different hyperparameter settings. The highest validation performance was obtained using a training batch size of 32, an evaluation batch size of 16, one training epoch, and a learning rate of 2e-5. The reported run used a random seed of 3637980206. During training, evaluation was performed every 500 steps on a subset of 500 validation examples. The model reached its highest accuracy on this subset at step 4999, achieving 90.4%. Final evaluation on the complete validation set resulted in an accuracy of 89.29%, while performance on the labeled test set reached 90.59%. As shown in Figure 9, the training curve exhibits a temporary increase around the peak point, followed by a decline, indicating that the highest intermediate validation score was not maintained throughout the remainder of training.

Figure 9. Training performance curves for DNLI-AristoTuple.

In the WordNet-enhanced scenario, we examined whether the integration of lexical–semantic relations could improve the model’s inferential accuracy. After tuning the hyperparameters, the highest configuration comprised a training batch size of 32, an evaluation batch size of 16, one training epoch, and a learning rate of 3e-5. The reported run used a random seed of 3067464431. As in the AristoTuple setting, evaluation was carried out every 500 steps on a 500-example validation subset. The highest subset accuracy, 90.6%, was observed at step 6499. Final evaluation yielded 89.41% accuracy on the full validation set and 90.82% on the test set. The learning curves in Figure 10 show a temporary rise around the best intermediate checkpoint, followed by a decrease and subsequent fluctuations across evaluations. WordNet yielded a small improvement over AristoTuple on both evaluation sets, while also introducing a larger number of triplets into the DNLI training data.

Figure 10. Training performance curves for DNLI-WordNet.

4.5. SNLI

The SNLI dataset, composed of premise–hypothesis pairs with generally longer premises and shorter hypotheses, is a widely used and influential benchmark for NLI. In this context, we investigate the integration of external knowledge from AristoTuple, WordNet and ConceptNet, aiming to enrich sentence pairs with structured scientific data, lexical–semantic relations and commonsense knowledge, and to examine whether this additional information improves the model’s ability to detect entailment and other semantic relationships. The amount of incorporated knowledge varied across the three sources. AristoTuple added 8752 triplets to 7214 training examples from 549,367, corresponding to a coverage of 1.31% and an average of 0.016 triplets per training example. WordNet resulted in 6381 triplets across 4629 examples, with a coverage of 0.84% and an average of 0.012 triplets per example. ConceptNet produced the largest enrichment, contributing 14,691 triplets to 10,204 examples, corresponding to a coverage of 1.86% and an average of 0.027 triplets per training example.

For AristoTuple, the configuration with the highest validation performance among those examined used a training batch size of 32, an evaluation batch size of 16, 2 epochs, and a learning rate of 2e-5. Performance was monitored every 500 steps on a subset of 500 validation examples, and the reported run used a random seed of 396880983. The highest accuracy observed on the validation subset was 91.0%, while evaluation on the complete validation and annotated test sets yielded 91.01% and 90.71%, respectively. As shown in Figure 11, performance increased rapidly during the initial stages of training and subsequently reached a plateau.

Figure 11. Training performance curves for SNLI-AristoTuple.

For the WordNet-enhanced scenario, we explored whether integrating lexical–semantic information could boost the model’s entailment detection. The best hyperparameters were a training batch size of 32, an evaluation batch size of 16, 1 epoch and a learning rate of 5e-5, with evaluation every 500 steps. The corresponding run used a random seed of 1915683226. The maximum accuracy recorded on the 500-example validation subset was 89.6%, whereas evaluation on the full validation and test sets resulted in 90.70% and 90.68%, respectively. Figure 12 shows a generally upward training trajectory with limited fluctuations.

Figure 12. Training performance curves for SNLI-WordNet.

In the ConceptNet-augmented setting, the best results were obtained with a training batch size of 32, an evaluation batch size of 16, 2 epochs, and a learning rate of 3e-5. Validation was again performed every 500 steps on 500 examples, and the reported run used a random seed of 961590061. The validation subset reached a peak accuracy of approximately 91.0%, while final evaluation produced 90.84% accuracy on the complete validation set and 90.69% on the test set. The learning curves in Figure 13 display a marked improvement during the earlier stages of training, followed by stabilization around 90% with relatively small fluctuations.

Figure 13. Training performance curves for SNLI-ConceptNet.

Despite the substantial differences in the amount of knowledge introduced by the three graphs, their test results were remarkably close, ranging from 90.68% to 90.71%. AristoTuple attained the highest test accuracy, whereas ConceptNet incorporated more than twice as many triplets as WordNet and substantially more than AristoTuple.

4.6. SciTail

The SciTail dataset constitutes a challenging benchmark for the task of textual entailment, as it is characterized by a high degree of overlap between the premise and hypothesis sentences, as well as by relatively long sentence lengths. These characteristics make it difficult to distinguish between true semantic entailment and mere lexical similarity. To this end, experiments were conducted to evaluate the impact of three different external knowledge sources: AristoTuple, WordNet and ConceptNet. Compared with the other datasets examined in this study, external knowledge was incorporated into a substantially larger proportion of the SciTail training set. AristoTuple contributed 5216 triplets to 2337 training examples from 23,088, corresponding to a coverage of 10.12% and an average of 0.226 triplets per training example. WordNet added 5735 triplets across 3064 examples, resulting in a coverage of 13.27% and an average of 0.248 triplets per example. The largest amount of enrichment was obtained with ConceptNet, which contributed 6274 triplets to 3378 examples, corresponding to a coverage of 14.63% and an average of 0.272 triplets per training example.

In the case of enrichment with AristoTuple, the best configuration included a training batch size of 16, an evaluation batch size of 16, 5 epochs, and a learning rate of 3e-5. Evaluation during training was performed every 500 steps on a subset of 500 validation examples, and the reported run used a random seed of 3204807276. Final evaluation yielded an accuracy of 95.24% on the complete validation set and 93.37% on the labeled test set. As illustrated in Figure 14, the training curve generally followed an upward trajectory, with intermediate fluctuations and values reaching approximately 95.6% before settling slightly lower toward the end of training.

Figure 14. Training performance curves for SciTail-AristoTuple.

For WordNet, the configuration with the highest validation performance was a training batch size of 64, an evaluation batch size of 16, 3 epochs, and a learning rate of 3e-5Performance was again monitored every 500 steps on a 500-example validation subset, with a random seed of 2221893396. Evaluation on the complete validation set resulted in an accuracy of 94.63%, while test accuracy reached 93.41%. The learning curves presented in Figure 15 show comparatively limited variation throughout training.

Figure 15. Training performance curves for SciTail-WordNet.

In the ConceptNet-augmented scenario, the best results were obtained with a training batch size of 32, an evaluation batch size of 16, 4 epochs, and a learning rate of 4e-5. The reported run used a random seed of 1830463577, with validation performed every 500 steps on the 500-example subset. Intermediate validation performance reached approximately 95.2% around step 2499 before decreasing. Final evaluation on the complete validation set yielded 94.78%, while the corresponding test accuracy was 93.70%. Figure 16 shows greater variation during training than that observed for WordNet, followed by performance around the 0.95 level during the later evaluations.

Figure 16. Training performance curves for SciTail-ConceptNet.

Among these configurations, ConceptNet produced the highest test accuracy, followed closely by WordNet and AristoTuple. AristoTuple, however, achieved the highest validation accuracy.

5. Comparison and Discussion

After presenting the results of the individual knowledge-enriched configurations in Section 4, this section examines the findings from two complementary perspectives. Section 5.1 evaluates the contribution of knowledge enrichment through a controlled multi-seed comparison between the vanilla and enriched configurations under common training conditions, considering both average test performance and statistical evidence across repeated runs. Section 5.2 places the best validation-selected configurations in the broader context of related knowledge-enriched BERT-based NLI approaches reported in the literature. Together, these analyses distinguish the within-study effect and consistency of the enrichment process from the broader benchmark-level competitiveness of the proposed approach.

5.1. Comparison with Baseline

To assess the effect of knowledge enrichment under common training conditions, an additional multi-seed evaluation was performed for the baseline and knowledge-enriched configurations. For each dataset, the hyperparameter configuration selected for the vanilla model on the validation set was kept fixed and subsequently applied to both the baseline and the corresponding enriched variants. Each configuration was evaluated using the same set of multiple random seeds, allowing every enriched run to be directly paired with a baseline run initialized with the same seed. This experimental setting complements the method-specific results reported in Section 4 by examining the consistency of the enrichment effect while controlling for differences in training configuration.

For each model, test performance is reported as the mean accuracy and standard deviation across the repeated runs. Statistical analysis was performed on the paired accuracy differences between each enriched configuration and its corresponding baseline. For each comparison, the mean difference in percentage points, its 95% confidence interval, and the two-sided paired t-test p-value were calculated. The results of this controlled multi-seed comparison are summarized in Table 5.

Table 5. Controlled multi-seed comparison between the vanilla and knowledge-enriched configurations across the evaluated NLI datasets.

Dataset	Configuration	Mean Accuracy (%)	SD	Δ vs. Vanilla (pp)	95% CI	p-Value
RTE	Vanilla	76.80	0.96	—	—	—
RTE	+AristoTuple	76.90	0.99	+0.10	[0.01, 0.19]	0.034
RTE	+WordNet	77.12	0.63	+0.32	[−0.82, 1.46]	0.480
QNLI	Vanilla	90.42	0.35	—	—	—
QNLI	+AristoTuple	90.62	0.35	+0.20	[−0.53, 0.93]	0.490
QNLI	+WordNet	90.70	0.27	+0.28	[0.10, 0.46]	0.013
SICK	Vanilla	86.33	1.49	—	—	—
SICK	+AristoTuple	86.66	0.81	+0.35	[−1.45, 2.12]	0.630
SICK	+WordNet	85.83	1.96	−0.50	[−3.86, 2.86]	0.700
DNLI	Vanilla	90.19	0.36	—	—	—
DNLI	+AristoTuple	90.49	0.12	+0.29	[−0.20, 0.78]	0.170
DNLI	+WordNet	90.28	0.19	+0.10	[−0.35, 0.52]	0.610
SNLI	Vanilla	89.51	0.78	—	—	—
SNLI	+AristoTuple	89.72	0.88	+0.21	[−0.18, 0.59]	0.210
SNLI	+WordNet	90.26	0.28	+0.75	[−0.16, 1.65]	0.080
SNLI	+ConceptNet	89.83	0.9	+0.32	[−0.41, 1.04]	0.280
SciTail	Vanilla	92.78	0.27	—	—	—
SciTail	+AristoTuple	92.82	0.42	+0.04	[−0.71, 0.80]	0.880
SciTail	+WordNet	92.81	0.23	+0.04	[−0.34, 0.41]	0.790
SciTail	+ConceptNet	92.53	0.30	−0.25	[−0.41, −0.08]	0.014

The same five random seeds (7777, 8888, 9999, 11111, and 22222) were used for all vanilla and knowledge-enriched configurations. For each dataset, the hyperparameter setting selected for the vanilla model according to validation performance was subsequently kept unchanged for all corresponding enriched variants. For RTE, the configuration used a training batch size of 64, an evaluation batch size of 16, a maximum of 6 epochs, a learning rate of 2e-5, evaluation every 30 steps, and early stopping with a patience 3. QNLI used batch sizes of 64 and 16 for training and evaluation, respectively; 4 epochs; a learning rate of 3e-5; and evaluation every 500 steps. For SICK, the respective batch sizes were 64 and 16, with a maximum of 10 epochs, a learning rate of 3e-5, evaluation every 20 steps, and early stopping with patience 3. DNLI used training and evaluation batch sizes of 32 and 16, one epoch, a learning rate of 1 × 10⁻⁵, and evaluation every 500 steps. SciTail used batch sizes of 64 and 16, 4 epochs, a learning rate of 3e-5, and evaluation every 500 steps, whereas SNLI used batch sizes of 32 and 16, one epoch, a learning rate of 5e-5, and evaluation every 500 steps.

On RTE, AristoTuple produced a small but statistically significant positive mean difference of 0.10 percentage points relative to the vanilla configuration. WordNet achieved the highest mean accuracy for this dataset, reaching 77.12% compared with 76.80% for the vanilla model, corresponding to a mean difference of 0.32 percentage points. However, the confidence interval included zero and the difference was not statistically significant.

For QNLI, both enriched configurations achieved higher mean accuracy than the vanilla model. AristoTuple increased the mean accuracy from 90.42% to 90.62%, although the corresponding difference was not statistically significant. WordNet reached 90.70% and produced a statistically significant mean improvement of 0.28 percentage points.

The SICK results present a less consistent pattern. AristoTuple achieved a higher mean accuracy than the vanilla configuration (86.66% versus 86.33%), corresponding to a positive mean difference of 0.35 percentage points, whereas WordNet reached 85.83%, yielding a mean difference of −0.50 percentage points. In both cases, the confidence intervals included zero and the differences were not statistically significant. Therefore, the multi-seed evaluation does not provide sufficient statistical evidence to conclude that either knowledge source consistently improves or degrades performance on SICK.

On DNLI, both knowledge sources produced positive average differences. AristoTuple achieved the highest mean accuracy at 90.49%, compared with 90.19% for the vanilla configuration, while WordNet reached 90.28%. The corresponding differences of +0.29 and +0.10 percentage points were not statistically significant, indicating that the observed average increases were not sufficiently consistent across the repeated runs.

A similar pattern was observed for SNLI, where all three enriched configurations achieved higher mean accuracy than the vanilla model. AristoTuple increased the mean from 89.51% to 89.72%, while ConceptNet reached 89.83%. WordNet produced the largest average increase, reaching 90.26% and a mean difference of +0.75 percentage points. Nevertheless, none of the three comparisons reached statistical significance at the 0.05 level. WordNet showed the closest result to this threshold at p = 0.080, whereas AristoTuple and ConceptNet yielded p = 0.21 and p = 0.280, respectively.

For SciTail, AristoTuple and WordNet produced almost identical mean performance to the vanilla configuration. Their mean differences were both approximately +0.04 percentage points and were not statistically significant. ConceptNet, in contrast, reduced the mean accuracy from 92.78% to 92.53%. The corresponding difference of −0.25 percentage points were statistically significant, indicating a consistent negative effect under the common training configuration used in the multi-seed evaluation.

Overall, the controlled multi-seed analysis provides a more nuanced view of the effect of training-data enrichment than the individual validation-selected configurations reported in Section 4. Statistically significant positive differences were observed for AristoTuple on RTE and WordNet on QNLI, whereas ConceptNet produced a significant negative difference on SciTail. For the remaining comparisons, the enriched configurations often achieved positive average differences, but the corresponding confidence intervals included zero and the paired tests did not provide sufficient evidence of a consistent effect across runs. These findings indicate that the contribution of external knowledge is not uniform across datasets or knowledge sources.

5.2. Comparison with Existing Works

Beyond the comparison with the vanilla baseline, Table 6 compares the best-performing configurations of the proposed approach with representative BERT-based NLI methods that incorporate some form of external or additional knowledge, including lexical, semantic, syntactic, knowledge-graph, and multi-task information. Since the studies compared generally report a single test score for each model configuration, the values reported for the present approach correspond to the best validation-selected single-run configurations presented in Section 4. The comparison therefore provides a benchmark-level view of how the proposed knowledge-enrichment strategy performs relative to other approaches that enhance pretrained language models with additional sources of information.

Table 6. Comparison between the proposed knowledge-enriched configurations and related BERT-based approaches on NLI datasets in terms of test accuracy (%).

Model/Dataset	RTE	QNLI	SNLI	SciTail
MT-DNN base [34]	—	—	91.10	94.10
BERT + LF [29]	—	—	90.50	92.80
MT-DNN + LF [29]	—	—	91.10	94.30
BERT + SRL base [32]	—	—	89.60	—
BERT + WordNet [25]	—	—	90.10	—
SemBERT base [33]	69.30	90.90	91.00	—
SyntaxBERT base [30]	68.90	91.10	87.70	—
SyntaxBERT base + LISA [30]	—	—	87.80	—
ExBERT [19]	—	—	91.50	95.20/95.90
AristoTuple *	76.00	90.80	90.71	93.37
WordNet *	77.50	90.70	90.68	93.41
ConceptNet *	—	—	90.69	93.70

* Results obtained in the present work correspond to the best validation-selected single-run configurations reported in Section 4.

For RTE, the proposed configurations achieve test accuracies of 76.00% with AristoTuple and 77.50% with WordNet, compared with 69.30% for SemBERT base [33] and 68.90% for SyntaxBERT base [30]. The WordNet-enriched configuration therefore obtains the highest RTE accuracy among the approaches included in Table 6. It should also be noted that, similarly to the present RTE experiments, SemBERT benefits from prior fine-tuning on MNLI before adaptation to RTE.

For QNLI, AristoTuple and WordNet achieve 90.80% and 90.70%, respectively, compared with 90.90% for SemBERT base [33] and 91.10% for SyntaxBERT base [30]. Although the proposed configurations remain slightly below these methods, the differences are small. SemBERT incorporates explicit Semantic Role Labeling information through additional semantic representations that are combined with BERT features, whereas SyntaxBERT modifies the Transformer attention mechanism through syntax-based masks derived from dependency and constituency structures. The proposed approach reaches a comparable level of performance without modifying the internal attention mechanism of BERT or introducing additional semantic or syntactic encoding components.

On SNLI, the proposed configurations achieve test accuracy ranging from 90.68% to 90.71%. These results exceed those reported for BERT + LF [29] (90.50%), BERT + WordNet [25] (90.10%), BERT + SRL base [32] (89.60%), SyntaxBERT base [30] (87.70%), and SyntaxBERT base + LISA [30] (87.80%). In particular, the comparison with BERT + WordNet is noteworthy because the latter explicitly injects WordNet relations into the Transformer’s multi-head attention mechanism, whereas the proposed approach incorporates external knowledge only at the data level.

The proposed SNLI results also remain close to SemBERT base [33] (91.00%), MT-DNN base [34] (91.10%) and MT-DNN base + LF [29] (91.10%), with differences of 0.29 and 0.39 percentage points. SemBERT combines BERT representations with semantic-role representations generated by an external SRL system and encoded through additional components, while MT-DNN benefits from joint supervised training across multiple NLU tasks, allowing the shared encoder to acquire transferable information from several sources of supervision. Despite not using additional semantic encoders or multi-task supervision, the proposed approach remains within a narrow performance margin.

ExBERT [19] achieves the highest SNLI result among the knowledge-enhanced approaches shown in Table 6, reaching 91.50%, while the best proposed configuration reaches 90.71%, a difference of 0.79 percentage points. ExBERT employs a dedicated architecture in which retrieved knowledge-graph information is encoded separately and subsequently combined with the premise–hypothesis representations through additional attention and mixture components. By contrast, the present method leaves the downstream BERT architecture unchanged and introduces external knowledge directly into the training examples. The relatively small performance gap therefore suggests that a substantially simpler training-level enrichment strategy can retain competitive performance without dedicated knowledge-integration layers.

A similar pattern is observed on SciTail. The proposed configurations achieve 93.37% with AristoTuple, 93.41% with WordNet, and 93.70% with ConceptNet, all exceeding the 92.80% reported for BERT + LF [29]. The best proposed configuration remains only 0.40 percentage points below MT-DNN base [34] (94.10%) and 0.60 points below MT-DNN + LF [29] (94.30%). MT-DNN derives additional benefit from multi-task learning across several NLU tasks, while the +LF variant further incorporates external syntactic representations. Reaching similar performance through training-data enrichment alone indicates that useful gains can also be obtained without introducing additional task-specific or syntactic modeling components.

ExBERT [19] reports the highest SciTail scores, reaching 95.20% with ConceptNet and 95.90% with AristoTuple. These results are higher than the best score of 93.70% obtained by the proposed method, but ExBERT relies on a specialized knowledge-integration architecture that separately represents and combines external KG information with the NLI input. The proposed approach instead uses the selected knowledge solely to enrich the training text, preserving the original BERT architecture. This comparison highlights the trade-off between architectural complexity and performance: more elaborate integration mechanisms can achieve higher absolute scores, while a simpler data-level strategy can still remain competitive.

The SciTail results should additionally be interpreted considering the preprocessing applied in the present study. Nine training examples were excluded because they exceeded the input-length constraint of the enrichment stage, resulting in 23,088 training instances for the knowledge-enriched configurations. Although this represents only a very small fraction of the original training set, it should be considered when comparing the reported scores.

Overall, Table 6 shows that the proposed approach achieves competitive performance across several NLI benchmarks. It exceeds several methods that incorporate lexical, semantic, or syntactic information and remains relatively close to approaches that rely on more elaborate architectural modifications or multi-task supervision. These results indicate that external knowledge can be effectively exploited through a comparatively simple training-level enrichment strategy, without modifying the internal architecture of the downstream transformer-based model.

6. Conclusions

This work examines the impact of external knowledge integration on handling the NLI problem through a simple data-level enrichment strategy, where selected knowledge graph triplets are added only to the training examples, while the validation and test sets remained unchanged. Taken together, the comparisons in Section 5 show that the proposed approach achieves competitive results in several cases, while remaining simpler than methods that modify the model architecture or rely on additional linguistic information. The results indicate that this strategy can improve NLI performance, but its effect is not uniform across all datasets and knowledge graphs. The usefulness of the added triplets may depend on the characteristics of each dataset, the knowledge source, the relevance of the selected information, and the way this knowledge is incorporated. Therefore, the proposed method offers a lightweight and flexible way to introduce structured knowledge into NLI models, while preserving the original model architecture.

Future work could extend this approach to additional NLI datasets and more recent language models, such as ALBERT, DeBERTa, or larger Transformer-based architectures. It would also be useful to examine alternative sentence embedding models for selecting relevant triplets, especially models that can better handle longer and more complex sentences. Another direction concerns the use of additional knowledge graphs or the combination of multiple knowledge sources, as well as the investigation of adaptive similarity thresholds for each dataset instead of a fixed threshold across all cases.