Diverse Knowledge-Embedded Method for Chemical-Induced Disease Relation Extraction

Abstract

Chemical-induced disease (CID) relation extraction plays an important role for disease treatment, drug repositioning, and drug discovery; thus, researchers employ natural language processing techniques to extract such relationships from the biomedical literature. However, the dispersion of CID relation across various sentences of a document poses a significant challenge for effective extraction. To address this issue, a neural network model is proposed in this paper, and it integrates diverse knowledge and vector cross-self-attention mechanisms for CID relation extraction. The model takes documents, their crucial shortest dependency path trees, and external knowledge as inputs. It utilizes pre-trained language models to generate the basic semantic embedding of all inputs. The proposed vector cross-self-attention is adopted to enhance knowledge representation with the semantics of document. The fusion of document semantics, enhanced shortest dependency path representations, and external knowledge embeddings is used to predict CID relationships. Finally, the proposed model is evaluated on BC5CDR and BioRED datasets. Experimental results demonstrate that diverse knowledge is effective for CID extraction. Chemical–disease pairs derived from both generative LLMs and CTD boost extraction performance. The vector cross-self-attention mechanism effectively enhances knowledge representation by attending to the document representations. This guides the model to identify CID relationships. The proposed model achieves F1-scores of 71.9% and 77.6% on BC5CDR and BioRED, respectively.

1. Introduction

Knowledge of chemical-induced disease relations (CIDs) is crucial for biomedical research such as drug repurposing [1], precision medicine [2], and adverse drug reactions (ADRs) [3]. Therefore, experts curate these relationships and store them in knowledge bases (KBs) such as the Comparative Toxicogenomic Database (CTD) [4], FDA [5]. However, traditional manual methods are difficult to efficiently extract and update CID relations due to the massive volume and rapid updates of the biomedical literature. To address this challenge, researchers have begun to utilize natural language processing (NLP) techniques to automatically extract CID relations from the literature. It is difficult to assess the different techniques owing to the lack of a comprehensive benchmarking dataset [6]. To this end, the BioCreative V [6] competition proposed a challenging task at the right time: automatically extracting CID relations from Medline abstracts. This task is broken down into two key subtasks. The first is Disease Named Entity Recognition (DNER), which involves accurately identifying disease names from the text. The second is CID relation extraction, which aims to identify and extract the association information between a drug and the disease it may induce. This paper focuses on CID relation extraction. This subtask is more challenging due to its nature as document-level relation extraction [7], where the semantics are inherently more difficult to capture accurately.

For the CID task, existing extraction systems can be broadly categorized into three main approaches: rule-based methods, traditional machine learning methods, and deep learning methods. Rule-based methods [8], though simple and effective, rely on rules derived from specific datasets, resulting in poor generalization ability and limited transferability to other datasets. In contrast, traditional machine learning methods, such as support vector machines (SVMs) [9], typically outperform rule-based approaches by leveraging linguistic elements such as lexical [9] and syntactic features [10]. However, the performance of these methods heavily depends on time-consuming manual rule design or effective feature representation. To tackle this challenge, researchers have introduced deep neural networks into the field of natural language processing for CID relation extraction. These methods employ architectures such as convolutional neural networks (CNNs) [11], recurrent neural networks (RNNs) [12], and graph convolutional networks (GCNs) [13] to automatically capture latent semantic features within word sequences, thereby improving overall performance. Moreover, researchers integrate attention mechanisms [14] and incorporate useful information extracted from knowledge bases [15] to improve extraction performance.

Previous studies have shown that the difficulties of the CID task mainly manifest in two aspects. First, CID relations may span sentence boundaries, meaning that the expression of a CID relation is not confined to a single sentence [13]. Consequently, researchers have divided the corpus into intra-sentence and inter-sentence sub-datasets [16]. Their results indicate that CID relation extraction on the inter-sentence dataset is considerably more difficult than on the intra-sentence dataset, as the semantic relationship between chemicals and diseases in cross-sentence instances is challenging to capture. Second, compared to articles in general domains, the biomedical literature tends to contain more sentences that are lengthy and syntactically complex [17], which further complicates CID relation extraction.

Fortunately, three existing facts offer potential solutions to the above-mentioned difficulties. First, two entities may co-occur within the same sentence. Deep neural network methods have demonstrated excellent performance in intra-sentence relation extraction, such as protein–protein interactions (PPIs) [18] and drug–drug interactions (DDIs) [19]. Therefore, it is feasible to construct co-occurrence sentence features within the model. Second, the Shortest Dependency Path (SDP) tree has played a significant role in single-sentence relation extraction [20]. Therefore, it is worth constructing both intra-sentence and inter-sentence SDP structures. Third, the CTD knowledge base provides annotated associations among genes, proteins, references, and toxic substances. Thus, incorporating CTD knowledge can help improve model performance. Furthermore, large generative language models such as DeepSeek [21] and ChatGPT [22], having been trained on massive amounts of data, already possess strong semantic understanding and discriminative capabilities. Therefore, they can be utilized as part of the knowledge resources.

In this paper, a knowledge-enhanced and vector cross-self-attentive document-level relation extraction model (VCA-DocRE) is proposed based on the above-mentioned facts. First, SDPs are extracted from documents as a form of intrinsic knowledge; chemical–disease relation pairs are derived from the CTD according to chemicals and disease mentioned in the documents and different representation forms are designed for them. Additionally, a large language model is employed to generate supplement chemical–disease relation knowledge that does not exist in the CTD. Second, both the documents and the aforementioned knowledge are fed into a pre-trained language model to acquire contextual semantics. Finally, the proposed vector cross-self-attention is employed to enrich knowledge representation and refine SDP through document semantics.

In summary, the contributions in the paper are as follows.

• A diverse knowledge-embedded document-level relation extraction framework is proposed for extracting chemical-induced disease relations. The framework accommodates multiple inputs, including the documents themselves, SDPs as intrinsic knowledge, chemical–disease relation pairs derived from the CTD, and externally sourced knowledge generated by a large generative language model.
• A vector cross-self-attention mechanism with primary-secondary directional awareness is proposed, where document semantics serve as the primary source and the diverse knowledge semantics are enriched as the secondary representations.
• The corpora are divided into inter-sentential and intra-sentential sub-datasets to validate the effectiveness of the proposed framework and the cross-self-attention mechanism. Results indicate that different knowledge contributes differently to relation discrimination for different instances.

2. Materials and Methods

Figure 1 illustrates a schematic overview of our proposed model. It contains four parts. First, an input layer consisting of documents, intrinsic knowledge (i.e., SDP), and external knowledge (i.e., chemical–disease pairs existing in CTD and generated by large generative language models). Second, semantic representation layer of pre-trained models such as PubMedBERT [23]. Third, a semantic features extractor layer, that primarily defines the representations of intrinsic knowledge (i.e., SDP) and external knowledge through vector cross-self-attention from the document. Finally, an output layer that concatenates the document semantics, the refined knowledge semantics, and the averaged entity semantics, and feeds the combined representation into the softmax function for prediction.

Figure 1. The architecture of vector cross-self-attentive document-level relation extraction (VCA-DocRE) neural network, containing three inputs, i.e., documents, SDP knowledge, and external knowledge. They are fed into a pre-trained language model to obtain their semantic representations, and then the representations of SDPs and external knowledge are refined/enriched by the document semantics through vector cross-self-attention. Finally, document semantics, knowledge semantics, and entity-averaged semantics are concatenated and fed into a SoftMax classifier for prediction. Notes: the dashed frame groups related information of the same category.

2.1. Input

The input layer consists of documents, SDPs, and external knowledge. Each document contains the abstract and title from the same literature. There is a natural intrinsic relationship between them. Thus, they are treated as one branch of the input. SDPs contain important information between two entities [20], derived from documents and titles. Thereby they are fed into the proposed network architecture as internal knowledge. Moreover, chemical–disease pairs existing in CTD play an important role in improving the recognition rate of CID [24]. Meanwhile, Generative LLMs like DeepSeek, Kimi, and ChatGPT yield credible knowledge [25] for CID recognition, which can be utilized to augment external knowledge sources. Therefore, external knowledge (chemical–disease relations) from CTD and generative LLMs serves as the third input branch of our model.

2.1.1. Documents

A document consists of a title and an abstract, and the abstract consists of a number of sentences. Thus, a document can be expressed as follows, D = (S_t,S₁,S₂,S_i,…,S_n), where S_t denotes the title, and S_i denotes the ith sentence in the abstract. To directly recognize sentence boundaries, the [SEP] token is placed between the adjacent two sentences following BERT’s sentence separator. The final sentence representation is D = (S_t,SEP,S₁,SEP,S₂,SEP,S_i,…,S_n).

2.1.2. Shortest Dependency Path Construction

A shortest dependency path (SDP) is extracted from a sentence containing the two target entities. It is essential for identifying the relation between the entities, as it captures the core syntactic structure that determines their relation [26]. Therefore, the shortest dependency path is incorporated into the proposed model as internal knowledge. However, a document is composed of three types of sentences: (i) co-occurrence sentences, (ii) entity sentences, and (iii) non-entity sentences. A co-occurrence sentence refers to a sentence in which both target entities appear together. An entity sentence contains only one target entity; a non-entity sentence, in contrast, includes neither of the target entities. Consequently, SDPs are constructed using both co-occurrence sentences and entity sentences for the target document to capture the semantic relation between the two target entities. The construction process is illustrated in Figure 2.

Figure 2. The process for constructing the multiple SDPs from co-occurrence sentences and entity sentences.

For the target document D = (S_t,S₁,S₂,S_i,…,S_n), the spaCy library [27] is firstly employed to split the document into individual sentences, and the target drugs and diseases in each sentence are masked with $CHEMICAL$ and #TARGET#, respectively. Then, the generation of SDPs is presented in two categories: one for co-occurrence sentences and the other for entity sentences.

Part 1: SDP Generation for Co-occurrence Sentences.

Step 1: All co-occurrence sentences are first retrieved from the document;

Step 2: For each such sentence, the SDP is extracted using the Stanford Parser [28];

Step 3: Finally, all extracted SDPs are concatenated with semicolons (“;”) to construct the document-level SDP knowledge.

Part 2: SDP Generation for entity Sentences.

Step 1: Sentences containing either $CHEMICAL$ or #TARGET# are identified and arranged in sequential order as ${\mathrm{D}}^c=(S_1^c,S_2^c,\dots ,S_k^c)$, ${\mathrm{D}}^t=(S_1^t,S_2^t,\dots ,S_k^t)$, respectively;

Step 2: The distance between the target entities in $S_i^c$ and $S_j^t$ is then calculated, measured by the number of intervening tokens;

Step 3: The pair $S_i^c$ and $S_j^t$ with the minimum distance is concatenated, and the SDP is extracted using the Stanford Parser. This SDP serves as the document-level SDP knowledge.

2.1.3. Chemical–Disease Pair in CTD

Leveraging external knowledge to guide models has been recognized as an important approach for improving performance in natural language processing tasks [24,29]. For the CID task, the most critical external resource is the CTD knowledge base [30]. Therefore, relevant knowledge labels are extracted from the CTD in this paper. It is found that there exist three relationship types between chemicals and diseases via an analysis of the CTD knowledge base, namely, marker/mechanism, therapeutic, and inference. Furthermore, when the relationship type is inference, certain inferred gene symbols are also provided. These gene features are considered to be effective clues for determining CID relationships. Consequently, the CID knowledge constructed in this paper includes not only the relationship types, but also the gene features, as illustrated in Figure 3.

Figure 3. The process for extracting knowledge from CTD. R1, R2, and R3 denote marker/mechanism, therapeutic, and inference, respectively. Moreover, inference relation consists of tag and gene symbols. Notes: R3|S denotes inference and gene list.

First, the external knowledge is formalized as a triple $K=[R_1,R_2,R_3]$, in which $R_1$, $R_2$, $R_3$ correspond to the relation types “marker/mechanism”, “therapeutic”, and “inference”, respectively.

Second, for an inference relation, the triple is composed of two parts: the inference type and the gene features. It is represented as ${“\mathrm{infer}:\mathrm{g}}_1{,\mathrm{g}}_2{,\mathrm{g}}_3\dots {\mathrm{g}}_{\mathrm{n}}”$, where ${\mathrm{g}}_{\mathrm{i}}$ denotes the $i$-th gene feature.

Third, for each <$CHEMICAL$, #TARGET#> pair in the BC5CDR dataset, the $i$-th element of the triple is assigned the corresponding relation name if it exists in the CTD. Otherwise, generative large language models are employed to extract the knowledge, as described in the next section. In addition, if neither source provides a relation, the position is filled with “null”. If no relation is found for the document, the corresponding knowledge representation is denoted as $K=[null,null,null]$.

2.1.4. Chemical–Disease Pair Generated by Generative Large Language Model

It is not guaranteed that the <$CHEMICAL$, #TARGET#> knowledge pair can be retrieved from the CTD for every instance in the BC5CDR dataset. According to statistical analysis, the chemical–disease co-occurrence coverage between CTD and the BC5CDR dataset is 72.53%. Meanwhile, generative LLMs such as ChatGPT [22] and DeepSeek [21] have increasingly become important tools in research due to their powerful representation capabilities, making them well-suited for knowledge acquisition in scenarios where CTD coverage is insufficient.

First, we build a prompt template to guide the large language model for knowledge extraction. The template is defined as follows: {Given document [], from the semantic level, infer whether there is an [Relation Desc] between $CHEMICAL$ and #TARGET#. Reply template: Exist/Not exist, Probability: high, medium, low, none; No detailed explanation is required}. Here, “Probability” refers to four discrete confidence levels (high, medium, low, and none) rather than a numerical probability between 0 and 1. The LLM directly returns a categorical judgment (Exist/Not exist) together with one of these four levels, and these outputs were used directly to construct the LLMPair representation without any probability conversion.

Second, $K=[R_1,R_2,R_3]$ is extended to a 4-tuple, denoted as $K=[R_1,R_2,R_3,R_4]$, where $R_4$ represents the prediction made by the generative LLMs (named LLMPair), which consists of a binary indicator (“Exist” or “Not exist”) and a discrete confidence level.

2.2. Semantic Representation

Biomedical documents often contain sentences with complex syntactic structure [17]. Hence, it is challenging to extract the full semantic information from these sentences. Bidirectional Encoder Representations from Transformers (BERT) [31] has become a fundamental paradigm for semantic representation, owing to its powerful semantic encoding capabilities, and achieves surprising performance in multiple NLP tasks. Furthermore, generative language models such as ChatGPT [22] are also grounded in the Transformer framework. However, large-scale generative models with massive parameters are not well-suited for the CID task. On one hand, the training process demands substantial time and computational resources; on the other hand, these models suffer from the inherent hallucination issue. Therefore, BERTs like BioBERT [32] are employed to represent the semantics of documents, SDPs, and external knowledge. The semantic segments are separated by two consecutive [SEP] tokens.

2.3. Semantic Feature Extraction

Semantic features are extracted by (i) vector cross-self-attention (VCA), and (ii) average semantic representation of target entity (ASR). VCA is used to enrich the semantic information of SDPs and external knowledge with document-level context, and ASR is a natural approach for extracting the full semantic information of all target entities within the document.

2.3.1. Vector Cross-Self-Attention

Previous studies have shown that semantic interaction is beneficial for improving model performance [7]. However, such interaction may also introduce noise. Allowing valid information to flow in while automatically filtering out invalid information may further improve model performance. Therefore, a vector cross-attention (VCA) mechanism is employed to enrich the knowledge representations with document-level semantics, aiming to capture more sufficient semantic information of the target entity pair and thereby improve document-level relation extraction performance. VCA is a directional cross-attention mechanism inspired by capsule networks [33], whose general formulation is given in Equation (1).

$$VCA(V_2\to V_1)=c_{s}soft\max ({\displaystyle \frac{{(\mathrm{W}}_1{V}_1)·{(W_2{V}_2)}^T}{\sqrt{d_2}}})(W_o{V}_2)+(1-{\mathrm{c}}_s{)\mathrm{V}}_1$$

(1)

where the equation represents the cross-self-attention from $V_2$ to $V_1$, which is designed to refine/enhance the representation of $V_1$. $d_2$ denotes the dimensionality of $V_2$, while $W_1$, $W_2$, $W_o$ are learnable weight matrices, and ${\mathrm{c}}_s$ denotes routing coefficient. It determines which information from v2 is used for injection into v1 to refine v1.

To obtain ${\mathrm{c}}_s$, $V_1$ and $V_2$ are first projected into a shared space.

$$P_1=V_1{W}_1^a,P_2=V_2{W}_2^a$$

(2)

where $W_1^a$ and $W_2^a$ are learnable projection matrix. Then, mean pooling is applied to generate fixed-length vectors.

$$p_1=\overline{P_1},p_2=\overline{P_2}$$

(3)

The agreement between the two representations is measured by the inner product of their normalized vectors, and the routing coefficient ${\mathrm{c}}_s$ is calculated by Equation (4).

$$c_s=\sigma (⟨norm(p_1),norm(p_2)⟩)$$

(4)

where $norm\left(.\right)$ denotes L2 normalization and $\sigma \left(.\right)$ is the sigmoid function.

Following the definition of vector cross-self-attention, the attention of SDPs toward the document and that of external knowledge K toward the document are given by Equations (5) and (6), respectively.

$$VCA(S\to D)=c_s^{s,d}soft\max ({\displaystyle \frac{({\mathrm{W}}_D^{\alpha }D)·{(W_{S}S)}^T}{\sqrt{d_S}}})(W_o^{S}S)+(1-c_s^{s,d})\mathrm{D}$$

(5)

$$VCA(K\to D)=c_s^{k,d}soft\max ({\displaystyle \frac{(W_D^{\beta }D)·(W_{K}K)}{\sqrt{d_K}}})(W_o^K\mathrm{K})+(1-c_s^{k,d})D$$

(6)

After the VCA is applied, we obtain the enhanced semantic representation of the SDP and external knowledge, as shown in Equation (4).

$$D_o={\alpha }_1{D}_{BERT}+{\alpha }_2\mathrm{VCA}(\mathrm{S}\to D)+{\alpha }_3\mathrm{VCA}(\mathrm{K}\to D)$$

(7)

where ${\displaystyle \sum {\alpha }_i}=1$.

2.3.2. Average Sematic Representation of Target Entity

The document-level relation extraction system aims to determine whether a given relationship exists between two target entities within the document. However, in the pair <$CHEMICAL$, #TARGET#>, $CHEMICAL$ is mentioned across multiple sentences in the document. In other words, the determination of whether a relation exists between $CHEMICAL$ and #TARGET# is not made by a single entity alone. Consequently, a natural choice is to adopt the averaged semantic embeddings of all $CHEMICAL$ as the global representation for $CHEMICAL$, and to apply the same procedure to #TARGET#, as shown in Equations (8)–(11).

$$H_{p_c^i}={\displaystyle \sum _{t=i}^{i+N_c^t}{H}_t}/(N_c^t)$$

(8)

$$H_c=({\displaystyle \sum _{i=1}^m{H}_{p_c^i}})/m$$

(9)

$$H_{p_d^k}={\displaystyle \sum _{t=k}^{k+N_d^t}{H}_d}/(N_d^t)$$

(10)

$$H_d=({\displaystyle \sum _{i=1}^n{H}_{p_d^i}})/n$$

(11)

where $H_{p_c^i}$ denotes the semantic representation of the i-th $CHEMICAL$; $H_c$ denotes the semantic representation of target entity $CHEMICAL$ in the document; $H_{p_d^k}$ denotes the semantic representation of the k-th #TARGET#; and $H_d$ denotes the semantic representation of target entity #TARGET# in the document.

2.4. Output

The semantic representations of a <CHEMICALCHEMICAL, #TARGET#> pair are concatenated and fed into a Softmax layer for relation prediction (Equation 12).

$$\hat{y}=\arg \max (soft\max ([H_c;H_d]))$$

(12)

3. Experiments

In this section, the corpus, evaluation metrics, baseline methods, and training details used in the experiments are introduced.

3.1. Corpus

In the experiments, BC5CDR [34] is chosen as main evaluation dataset, and BioRED [35] is also used to evaluate the proposed method and baseline methods. BC5CDR and BioRED consist of a total of 1500 Medline articles and 600 PubMed abstracts, respectively. To apply our experiments, abstracts unrelated to chemicals and diseases were removed, leaving 390 abstracts for subsequent analysis. A data extractor was built to separate intra-sentence and inter-sentence entity relations. Chemical entity sets and disease entity sets were built independently. For each instance in the annotated corpus, it was annotated as positive if a <chemical, disease> pair was present in the given labeled text, and negative otherwise. The distinction between intra-sentential and inter-sentential entity relations hinges on whether there exists at least one sentence that contains both a chemical and a disease entity. Accordingly, the SDPs were extracted for each instance based on the SDP construction approach introduced in the preceding section. Finally, the statistics of the reconstructed BC5CDR and BioRED corpora are shown in Table 1.

Table 1. Statistics of the reconstructed corpora.

Corpus	Docs	Inter	Intra	All
BioRED	390	129/2665	1061/1663	1190/4328
BC5CDR	1500	855/9097	2261/7279	3116/16,376

3.2. Evaluation Metrics and Baseline Methods

In the following experiments, Precision (P), Recall (R), and F1-score (F) are employed to evaluate the performance. In addition, multiple methods with KB are employed, which evaluate chemical–disease relation extraction at the inter-/intra-sentential level as baselines, briefly introducing these baselines as follows.

BioREDirect [36] is a BioLinkBERT-based multi-task learning framework with a soft-prompt learning mechanism. The mechanism concatenates three different chunks (prefix, suffix, and infix) and feeds them into the model to focus on task-relevant knowledge, thereby enabling the joint optimization of three objectives: relation type, novelty status, and directionality.

Two-step Inference [37] is a prompt engineering approach that decouples reasoning from structured output generation to address LLM challenges in biomedical text mining. The mechanism first enables free reasoning without forced formatting, then converts the results into schema-compliant outputs. This approach resolves the conflict between formatting constraints and reasoning capacity, thus achieving competitive performance without fine-tuning.

KGAGN [24] is a deep learning framework that combines prior knowledge with syntactic dependency information for chemical–disease relation extraction. An attention mechanism with trained entity and relation embeddings captures weighted contextual information, while a graph convolutional network encodes document-level syntactic dependency graphs to capture long-distance cross-sentence dependencies. The model obtains competitive performance on the BC5CDR dataset.

KRC [30] is a framework that reformulates biomedical relation extraction as a question answering task to leverage machine reading comprehension (RC) for context understanding. To integrate prior knowledge, an efficient knowledge-enhanced attention interaction mechanism is introduced to incorporate structured knowledge base representations into the RC framework. The model effectively captures both intra-sentential and inter-sentential relations, achieving competitive performance on biomedical relation extraction benchmarks.

SKEoG [38] is an edge-oriented graph neural network that leverages both document structure and external knowledge for document-level medical relation extraction. The framework integrates structural information from documents and knowledge from graph-based external knowledge, enabling effective reasoning across sentences. The model achieves state-of-the-art performance on the BC5CDR and CHR datasets.

KCN [15] is a knowledge-guided convolutional network that leverages prior knowledge from structured knowledge bases for chemical–disease relation extraction. The framework learns entity and relation embeddings from knowledge bases, then employs gated convolutions and shared attention pooling to control context feature propagation and capture weighted contextual information. The model fuses prior knowledge into textual context, achieving competitive performance on the BC5CDR dataset.

CDR-KB [16] is a document-level neural model that integrates domain knowledge for chemical-induced disease relation extraction. A knowledge attention mechanism distinguishes the influences of different knowledge based on article representation, enabling effective knowledge integration for CID recognition on the BC5CDR dataset.

NAM [39] is a neural network-based attention model that leverages prior knowledge from knowledge bases for chemical–disease relation extraction. An attention mechanism selects important context words with respect to relation representations learned from knowledge bases, and integrates textual context and structured knowledge. The model achieves competitive performance on the BC5CDR dataset.

The data-centric approach [40] is an ensemble learning system for biomedical relation extraction that leverages multiple pretrained BERT models with diverse preprocessing inputs, including prompt questions, entity ID pairs, and co-occurrence contexts. Special tokens and boundary tags are incorporated to enhance model comprehension, and a Max Rule ensemble mechanism amalgamates outputs from diverse classifiers. The system prioritizes high-quality data instances to enhance model performance, achieving competitive results on the BioRED benchmark.

LEAP [41] is a framework that investigates demonstrations in large language models for biomedical relation extraction via three adaptive tuning methods: instruction adaptive tuning, example adaptive tuning, and instruction-example adaptive tuning. The framework systematically explores both adaptive task descriptions and adaptive examples within the demonstration, thus enhancing LLM performance without extensive fine-tuning. Therefore, the model achieves competitive results on the DDI, ChemProt, and BioRED datasets.

3.3. Training Details

All models were implemented with PyTorch environment (https://pytorch.org/, accessed on 8 October 2026) and Python 3.13. PubMedBERT [23] used in our experiments is gained from https://microsoft.github.io/BLURB/ (accessed on 8 October 2026). We trained five epochs and performed batch training with a batch size of 16. The sequence lengths of the document, SDP, and external knowledge were set 320, 128, and 64, respectively. Meanwhile, the model was trained using the Adam optimization algorithm, with the learning rate and dropout rate set to 1 × 10⁻⁵ and 0.5, respectively. A validation pass on the development set was performed every 100 effective batches. In addition, due to access restrictions in our experimental environment, GPT-4 and some other overseas LLMs were not available to us. The LLMs used in this study were limited to those accessible in our setting, namely, DeepSeek-V4.1-Flash. All experiments were conducted on NVIDIA GeForce RTX 4060 Laptop GPU with five random seeds (2024–2028).

4. Results and Discussion

4.1. Overall Results

In this section, our proposed VCA-DocRE model is compared with the baseline methods described above on BC5CDR and BioRED datasets, as shown in Table 2 and Table 3, respectively.

Table 2. Main results on the BC5CDR.

Methods	P (%)	R (%)	F (%)
VCA_DocRE	69.1 ± 3.1	75.3 ± 5.0	71.9 ± 1.2
BioREDirect [36]	72.0	71.0	71.4
Two-step Inference [37]	76.1	83.3	78.1
KGAGN [24]	72.7	73.0	72.9
KRC [30]	71.9	70.5	71.1
SKEoG [38]	64.5	75.5	69.6
KCN [15]	69.7	73.0	71.3
CDR-KB [16]	63.6	76.8	69.6
NAM [39]	62.1	75.1	67.9

Table 3. Main results on the BioRED.

Methods	P (%)	R (%)	F (%)
VCA_DocRE	72.6 ± 1.9	83.5 ± 0.8	77.6 ± 0.9
Two-step Inference [37]	61.3	52.8	56.7
Data-centric approach [40]	61.2	57.9	59.5
LEAP [41]	-	-	75.2

Note: The results of the baseline models are taken from their original papers. Our dataset consists of abstracts related to chemicals and diseases.

Table 2 shows the performance of the existing systems and our model on the BC5CDR corpus. NAM [39] employs relation representations learned from external knowledge to obtain contextual word weights, attaining 62.1% precision, 75.1% recall, and 67.9% F1-score. CDR-KB [16] and KCN [15] construct effective knowledge representations and integrate them into their respective models, achieving F1 scores of 69.6% and 71.3%, respectively. Among all methods, CDR-KB [16] ranks second in recall (76.8%), following only Two-step Inference [37]. The KGAGN model, proposed by Sun et al. [24], leverages a knowledge-guided graph convolutional neural network to balance precision and recall, improving the F1-score to 72.9%. Meanwhile, Two-step Inference [37] achieves the best overall performance among all compared models, with a precision of 76.1%, a recall of 83.3%, and an F1-score of 78.1%.

Compared with the baseline models, our proposed VCA_DocRE achieves 69.1 ± 3.1% precision, 75.3 ± 5.0% recall, and 71.9 ± 1.2% F1-score over five random seeds. The F1-score is comparable to KRC [30] (71.1%), KCN [15] (71.3%), and BioREDirect [36] (71.4%), while remaining below KGAGN [24] (72.9%) and Two-step Inference [37] (78.1%). For recall, VCA_DocRE attains 75.3%, comparable to CDR-KB [16] (76.8%) and higher than KRC [30] (70.5%), KCN [15] (73.0%), and KGAGN [24] (73.0%). Against NAM [39] (67.9% F1), VCA_DocRE improves F1 by 4.0 points, which exceeds the seed-level standard deviation (1.2). Nevertheless, its precision (69.1%) remains lower than that of KGAGN [24] (72.7%) and Two-step Inference [37] (76.1%), suggesting room for improvement in suppressing false positives. Overall, VCA_DocRE achieves competitive performance, validating the effectiveness of the proposed model for document-level CID relation extraction.

Table 3 reports the performance comparison of various models on the BioRED corpus. Our proposed VCA_DocRE achieves 72.6 ± 1.9% precision, 83.5 ± 0.8% recall, and 77.6 ± 0.9% F1-score over five random seeds, outperforming both Two-step Inference [37] and the Data-centric approach [40] in precision, recall, and F1. Two-step Inference [37] employs dynamic instruction extraction and LLM-based generative reasoning, achieving 61.3% precision, 52.8% recall, and 56.7% F1-score. The Data-centric approach [40] focuses on data quality control strategies, attaining 61.2% precision, 57.9% recall, and 59.5% F1-score. LEAP [41] utilizes large language models for biomedical relation extraction, achieving an F1-score of 75.2% (with precision and recall not reported in the original paper).

Compared with the baseline models, our proposed VCA_DocRE substantially outperforms Two-step Inference [37] (52.8%) and the Data-centric approach [40] (57.9%) in recall (83.5%), with improvements of 30.7 and 25.6 points, respectively. This indicates that the VCA module captures CID relations more comprehensively through fine-grained contextual semantic modeling and external knowledge. For precision, VCA_DocRE (72.6%) also clearly surpasses Two-step Inference [37] (61.3%) and the Data-centric approach [40] (61.2%), indicating a lower false positive rate in predicting CID relations. Notably, although LEAP [41] achieves a competitive F1-score of 75.2%, our VCA_DocRE still outperforms it with an F1-score of 77.6%. Moreover, it is based on large language models (e.g., MedLLaMA_13B) and does not report precision or recall, making a full comparison difficult and fundamentally different from our PubMedBERT-based VCA_DocRE in methodology. Nevertheless, the strong recall of VCA_DocRE (83.5%) demonstrates its advantage in covering positive relation instances, which is of practical value in biomedical text mining. Overall, VCA_DocRE achieves competitive performance on both the BioRED and BC5CDR datasets, validating the generalization ability and effectiveness of our method across different datasets.

4.2. Impact of VCA

To investigate the contribution of each input component to VCA_DocRE, we conducted ablation experiments by removing SDPs (syntactic dependency paths) and external knowledge, respectively, to analyze their impact on our proposed model performance. The input of VCA_DocRE consists of three parts: document (doc), SDPs, and external knowledge (KB). Accordingly, the VCA module incorporates two attention pathways: SDP → Doc and KB → Doc. The experimental results are presented in Table 4.

Table 4. Impact of VCA on the BC5CDR.

Methods	P ± std (%)	R ± std (%)	F ± std (%)
VCA_DocRE	69.1 ± 3.1	75.3 ± 5.0	71.9 ± 1.2
-VCA (KB to Doc)	62.7 ± 1.9	64.0 ± 4.1	63.3 ± 1.8
-VCA (SDP to Doc)	68.8 ± 1.8	76.3 ± 3.3	72.3 ± 1.2
-VCA (KB to Doc) & (SDP to Doc)	62.2 ± 2.7	68.0 ± 4.1	64.8 ± 0.8

First, when the VCA (KB → Doc) and external knowledge are removed, precision decreases from 69.1% to 62.7% (a drop of 6.4 points) and recall drops from 75.3% to 64.0% (a drop of 11.3 points), resulting in an F1-score of 63.3%. This indicates that external knowledge and its attention over the document are crucial, demonstrating that external knowledge effectively guides the model in identifying CID relations.

Second, when the VCA (SDP → Doc) and SDPs are removed, precision decreases slightly from 69.1% to 68.8% (a drop of 0.3 points), while recall increases from 75.3% to 76.3% (an increase of 1.0 points), leading to a slight F1-score increase from 71.9% to 72.3%. This suggests that removing SDPs alone does not substantially degrade performance, but SDPs still serve as an important supplement for guiding CID relation identification when combined with external knowledge.

Finally, when both external knowledge and SDPs are removed, the F1-score drops to 64.8%. Compared with removing KB alone, precision decreases slightly from 62.7% to 62.2% (a drop of 0.5 points), while recall increases from 64.0% to 68.0% (an increase of 4.0 points). Compared with removing SDPs alone, precision decreases from 68.8% to 62.2% (a drop of 6.6 points), while recall decreases from 76.3% to 68.0% (a drop of 8.3 points). The resulting F1-score is 64.8%. This demonstrates that the two components are complementary, mutually balancing precision and recall to collectively enhance the overall performance of the proposed model.

To further evaluate the effectiveness of each knowledge-embedded component, we extend the ablation study by reporting the confusion matrix (Figure 4) and ROC curves (Figure 5) of different ablation settings. The removal of external knowledge (K) leads to a substantial increase in false negatives, while the removal of SDPs (S) mainly affects the balance between false positives and false negatives. Specifically, the full model achieves the highest F1-score, and removing K causes the most severe performance degradation. This indicates that external knowledge plays the most important role in distinguishing chemical-induced disease relations, while SDPs provide complementary but less pronounced benefits. Figure 5 presents the ROC curves and AUC values of all ablation variants. The B + K variant achieves the highest AUC (0.9353), slightly higher than the full model B + K + S (0.9306), whereas removing K leads to a clear drop in AUC (B + S: 0.8891; B: 0.8903). These results further confirm that external knowledge is the primary contributing component, while the effect of SDPs is limited and may introduce slight redundancy when combined with external knowledge.

Figure 4. Confusion matrix heatmap. Note: B denotes VCA_DocRE with removed VCA (KB to Doc) and (SDP to Doc); B + K denotes VCA_DocRE with removed VCA (SDP to Doc); B + S denotes VCA_DocRE with removed VCA (KB to Doc); B + K + S denotes VCA_DocRE.

Figure 5. ROC curves and AUC comparison for VCA component ablation. Note: B denotes VCA_DocRE with removed VCA (KB to Doc) and (SDP to Doc); B + K denotes VCA_DocRE with removed VCA (SDP to Doc); B + S denotes VCA_DocRE with removed VCA (KB to Doc); B + K + S denotes VCA_DocRE; the circle denotes operating point at threshold 0.5.

4.3. Impact of External KB

External knowledge [42] (CTD and chemical–disease pair generated by generated LLMs) plays an important role in improving the overall performance. Hence, we analyzed the impact of external KB on the proposed model in this section. Experiments are conducted on inter-/intra-sentential sub-dataset, and the results are shown in Table 5.

Table 5. The effect of external KB on the BC5CDR.

Methods	Inter-Sentential Results			Intra-Sentential Results			Overall Results
	P ± std (%)	R ± std (%)	F ± std (%)	P ± std (%)	R ± std (%)	F ± std (%)	P ± std (%)	R ± std (%)	F ± std (%)
Doc + S	51.3 ± 2.7	47.9 ± 7.3	49.2 ± 3.4	67.1 ± 1.3	71.0 ± 2.8	69.0 ± 1.5	62.7 ± 1.9	64.0 ± 4.1	63.3 ± 1.8
Doc + S + P	54.6 ± 3.8	58.5 ± 3.9	56.3 ± 2.3	69.3 ± 2.3	75.7 ± 4.5	72.2 ± 1.3	64.8 ± 2.7	70.5 ± 4.0	67.4 ± 1.4
Doc + S + D	60.5 ± 3.3	59.0 ± 7.3	59.6 ± 5.0	74.0 ± 2.1	77.7 ± 2.9	75.7 ± 1.3	70.1 ± 2.1	72.0 ± 3.8	71.0 ± 2.2
Doc + S + P + D	58.0 ± 3.3	62.7 ± 4.9	60.1 ± 2.7	71.7 ± 1.3	80.9 ± 3.4	76.0 ± 1.2	69.1 ± 3.1	75.3 ± 5.0	71.9 ± 1.2

Note: S denotes SDP; P denotes LLMPair; D denotes CTD; Doc denotes document.

LLMPair helps our model increase the overall F1 score by 4.1%, the inter-sentential F1 score by 7.1%, and the intra-sentential F1 score by 3.2%. Furthermore, incorporating CTD further improves the overall F1 score by 4.5% and the intra-sentential F1 score by 3.8%, and also improves the inter-sentential F1 score by 3.8%. This suggests that our designed knowledge representation is effective for CID extraction, and both LLMPair and CTD are beneficial for boosting the extraction performance. Specifically, LLMPair is particularly effective for cross-sentence relation recognition, while CTD provides consistent improvements across both inter- and intra-sentential settings, contributing notably to the overall performance.

4.4. Impact of Routing Coefficient

VCA introduces the routing coefficient r to control how much external information is injected, and Table 6 examines its role. Aggregate performance is essentially unchanged without r (F1 71.9% → 71.7%; ΔP = +0.9, p = 0.47, ΔF1 = +0.2, p = 0.40, paired t-test over five seeds). For a bounded scalar, this null result is expected: the effect of r must be sought where the model is undecided.

Table 6. The effect of routing coefficient on the BC5CDR.

Methods	P ± std (%)	R ± std (%)	F ± std (%)
VCA_DocRE	69.1 ± 3.1	75.3 ± 5.0	71.9 ± 1.2
-routing	68.2 ± 1.4	75.7 ± 2.9	71.7 ± 0.9

Figure 6 stratifies the test set by the confidence of the un-routed model. On the 1472 instances (5.3%) with probability in [0.40, 0.80), r raises accuracy from 50.9% to 59.3% (Δ = +8.4 pooled; +9.4 ± 4.9 per seed, p = 0.013, 5/5 seeds positive); on the remaining 94.7% it is negligible (−0.3, p = 0.28). The improvement peaks where the model is weakly positive (42.7% → 59.2%, Δ = +16.5 points pooled; per-seed Δ = +16.9 ± 4.5, t (4) = 8.44, p = 0.001, 5/5 seeds positive) and is replicated for both intra- and inter-sentence relations. Net effect: 75 fewer false positives at the cost of 21 additional false negatives.

Figure 6. Effect of the routing coefficient across confidence strata on the BC5CDR.

4.5. Statistical Robustness and Significance Analysis

A single run may reflect randomness rather than genuine gains. Therefore, we first assess performance stability across runs, then test whether the improvements are statistically significant in this section, as shown in Figure 7.

Figure 7. Run-to-run variability of the full model retrained from scratch with five different random seeds (2024–2028), all other settings unchanged. (a) shows precision, recall, and F1 as mean ± one standard deviation over seeds, with individual per-seed scores overlaid (each dot denotes the value corresponding to an individual seed.); (b) Shows the per-seed F1 for the overall, intra-sentence, and inter-sentence subsets (dashed lines mark the means).

The complete model was retrained with five random seeds (2024–2028), all other settings unchanged. The overall F1 is 71.95 ± 1.17 (95% CI [70.49, 73.40]) and intra-sentence F1 is equally stable (76.50 ± 1.07), so the result is not driven by a favorable initialization. Inter-sentence F1 varies more (60.92 ± 2.34), largely because one run (seed 2024, inter-sentence recall 52.01) reflects sparse inter-sentence support and threshold sensitivity rather than systematic instability: excluding it shifts the overall mean by only +0.39. Re-running three configurations under a fixed seed (2028) changes F1 by at most 0.37, so differences below ≈1.2 F1 are not interpretable, whereas the +7 to +9 F1 knowledge benefits exceed this noise floor several times over.

4.6. Error Analysis

In this section, we report the prediction error proportions of different instances containing different knowledge tags. Moreover, we present an analysis of the causes of classification errors. All results reported in this section are obtained from a single representative run of VCA_DocRE on the BC5CDR test set with seed 2028.

Figure 8 decomposes the 657 prediction errors on the BC5CDR test set by instance knowledge tag. They are highly concentrated: marker/mechanism instances account for 88.3% (444 FP = 67.6%, 136 FN = 20.7%), followed by LLMPair instances (55 FN, 18 FP). Therapeutic and inference instances contribute at most 0.6% (four FN in total), since their pairs are almost all negative and the model hardly ever labels them as CID. Within the marker/mechanism tag, the error rate is 24.9% (intra-) and 23.5% (inter-sentential), dominated by FP (20.2% and 16.7%), so the residual mistakes are mostly over-triggering rather than misses.

Figure 8. Composition of the 657 prediction errors by instance knowledge.

Figure 9 and Figure 10 contrast the error profile with and without external knowledge. Removing knowledge raises the error count from 657 to 776. The FN rate of marker/mechanism instances grows from 4.8% to 15.8% (intra-) and 6.8% to 15.8% (inter-sentential). Spurious positives reappear for the tags that knowledge had suppressed (inference FP 0.0% to 9.6%, therapeutic 0.0% to 4.0%, LLMPair 2.7% to 13.7%, intra-sentential). Knowledge thus acts as a recall-oriented prior that mainly suppresses misses on knowledge-supported pairs, at the cost of more over-triggering on marker/mechanism instances.

Figure 9. Error rate (%) within each knowledge tag, with external knowledge.

Figure 10. Error rate (%) within each knowledge tag, without external knowledge.

Finally, LLMPair knowledge derived from the generative LLM can misguide the model through hallucination (Figure 11). In all 10 test instances where the LLM denied an existing CID relation, our model missed it too, whereas only 18 of the 309 spurious relation assertions were followed.

Figure 11. An example of prediction error by generative LLM.

5. Conclusions

In this work, we proposed a diverse knowledge-embedded method for extracting complicated document-level chemical–disease relations from biomedical articles. Documents, their crucial SDPs, and external knowledge were input into the proposed model. SDPs were refined by the documents’ semantic representation. External knowledge was employed to guide the model prediction. Specifically, constructing shortest dependency paths from both multiple co-occurrence sentences and the merged sentence for inter-sentential instances further contributed to improving the overall performance of our model. Meanwhile, the knowledge representation obtained by CTD and generated by the generative LLMs can discern non-CID and CID instances and decrease the prediction error rate. Moreover, the proposed vector cross-self-attention operations from the document to SDPs and knowledge were leveraged to extract mutual semantic information, which enriched/refined the knowledge semantic representation, and thus contributed to more comprehensive full-text semantic information. Finally, the proposed model captured more complete semantic information via the average semantic representation of all target entities in the document. Experimental results and analysis indicate that our approach can recognize chemical–disease relations well, achieving F1-scores of 71.9% and 77.6% on the BC5CDR and BioRED datasets, respectively. Furthermore, the designed knowledge representation plays a beneficial role in improving chemical–disease relation extraction performance. Additionally, the knowledge derived from generative LLMs offers some auxiliary support for extraction performance, yet it may also introduce certain misleading effects. In the current implementation, no dedicated filtering mechanism was applied to contradictory or ambiguous LLM outputs, which may introduce noise into the external knowledge. This is a limitation of this study. Therefore, in future work, we plan to design a stricter filtering strategy for contradictory or low-confidence LLM outputs. We systematically compare different generative LLMs, investigate the magnitude of misleading effects from different LLMs, and conduct in-depth cause analysis to further improve the overall performance.