Fd-CasBGRel: A Joint Entity–Relationship Extraction Model for Aquatic Disease Domains

Featured Application

The model is primarily utilized for the task of entity relationship extraction during the construction process of an aquatic disease knowledge graph.

Abstract

Entity–relationship extraction plays a pivotal role in the construction of domain knowledge graphs. For the aquatic disease domain, however, this relationship extraction is a formidable task because of overlapping relationships, data specialization, limited feature fusion, and imbalanced data samples, which significantly weaken the extraction’s performance. To tackle these challenges, this study leverages published books and aquatic disease websites as data sources to compile a text corpus, establish datasets, and then propose the Fd-CasBGRel model specifically tailored to the aquatic disease domain. The model uses the Casrel cascading binary tagging framework to address relationship overlap; utilizes task fine-tuning for better performance on aquatic disease data; trains on specialized aquatic disease corpora to improve adaptability; and integrates the BRC feature fusion module—which incorporates self-attention mechanisms, BiLSTM, relative position encoding, and conditional layer normalization—to leverage entity position and context for enhanced fusion. Further, it replaces the traditional cross-entropy loss function with the GHM loss function to mitigate category imbalance issues. The experimental results indicate that the F1 score of the Fd-CasBGRel on the aquatic disease dataset reached 84.71%, significantly outperforming several benchmark models. This model effectively addresses the challenges of ternary extraction’s low performance caused by high data specialization, insufficient feature integration, and data imbalances. The model achieved the highest F1 score of 86.52% on the overlapping relationship category dataset, demonstrating its robust capability in extracting overlapping data. Furthermore, We also conducted comparative experiments on the publicly available dataset WebNLG, and the model in this paper obtained the best performance metrics compared to the rest of the comparative models, indicating that the model has good generalization ability.

1. Introduction

China leads the world in aquaculture and had a total output of 68.65 million tons of aquatic products in 2022 alone [1]. Of this, 55.65 million tons are directly attributed to aquaculture production, whose ratio to fishing production is at least 81:19. This substantial contribution from aquaculture not only ensures national food security and the supply of vital agricultural products but also plays a pivotal role in boosting farmers’ income [2,3]. However, factors such as high-density stocking and water pollution have led to a high incidence of disease during aquaculture. Accordingly, the diagnosis and prevention of aquatic diseases have emerged as critical bottlenecks hindering the industry’s rapid and sustainable development. It is imperative to leverage modern information technology to empower traditional aquaculture practices and hasten the transition towards digitalization and intelligence in disease diagnosis and prevention techniques.

Relationship extraction is a fundamental step when constructing a knowledge graph [4], which aims to identify entity–relationship ternaries (HEAD, RELATIONSHIP, TAIL) from unstructured text. In the aquatic disease domain, that relationship extraction primarily involves extracting various entities, such as established diseases, control methods, drugs, and pathogens, from textual data related to aquatic diseases, as well as elucidating the relationships among them. These extracted data are then organized into a knowledge graph, which can be readily used in various downstream applications including intelligent Q&A systems, early disease warning systems, and knowledge visualization tools. By integrating these technologies into the aquatic disease control process, digitalization can be enhanced, thereby improving overall disease management efforts.

For the entity–relationship extraction task within the aquatic disease domain, its extensive overlap in relationships poses a significant issue [5] that impacts the effectiveness of ternary group extraction. Further, this domain’s data exhibit notable specialization and imbalance, being characterized by the prevalence of many specialized terms and an imbalance between data categories and classification difficulty levels. Consequently, generalized pretraining language models encounter challenges in extracting semantic features and have poor domain adaptability, thereby adversely affecting extraction performance. In addition, in the process of entity–relationship extraction, many supplementary features beyond the encoded features of sentences and entities remain underutilized, leading to feature loss. To address these challenges, this paper proposes a cascading binary labeling framework based on a fine-tuned pretraining model, a feature fusion module, and a GHM loss function. We show that this framework can effectively resolve the aforementioned issues and bolsters model performance on small-scale, unbalanced aquatic disease-relationship extraction tasks.

2. Related Work

Entity–relationship extraction is instrumental in the construction of knowledge graphs. The traditional pipeline method for doing that divides the entity and relationship extraction task into separate subtasks, initially extracting entity pairs and later categorizing them based on relationship labels. The main drawback of this approach is that it ignores the potential correlation between these subtasks, which leads to error propagation [6]. In contrast, joint entity–relationship extraction integrates entity recognition and relationship recognition, effectively solving the error propagation problem. Early joint entity–relationship extraction methods include feature engineering methods [7], tree-structured joint models [8], and joint extraction methods based on sequence annotation [9,10,11]. These methods have drawbacks such as poor portability and complex data labeling and feature construction. Further, the key problem of relationship overlap, particularly prevalent in specialized domains like aquatic sciences, is hard to solve using the aforementioned approaches.

In this context, Yu et al. [12], Zhuang et al. [13], and Zeng et al. [14] were able to successfully recognize overlapping relation triplets by augmenting the labeling strategy. Yet these approaches still treat relations as discrete labels of entities, which inevitably limits their performance. Wei et al. [15] devised a pointer network to comprehensively model triplets by learning the mapping function linking relations to entities. To address the Casrel model’s low computational efficiency due to relationship redundancy, Zheng et al. [16] introduced the PRGC model, which breaks down the task into three steps: relationship judgment, entity extraction, and entity alignment. This approach effectively enhances computational efficiency. Models for the extraction of overlapping relationships based on copying mechanisms rely on various feature extraction networks to capture semantic features of target entity fragments after localization and then directly copy them to the Decoder for extraction. Representative models for that approach include CopyR [9] and DPointer [17], but they have their own drawbacks, such as complex mechanisms and entity relationship mismatches. The table-filling methods maintain a table for each relation, with each item indicating whether the tagged pair displays that particular relation or not, e.g., TPLinker [18], PFN [19], and UniRel [20]. These models are effective in overcoming the problem of resolving overlapping relationships, but they tend to have high model complexity. In recent years, the emergence of Graph Convolutional Neural Networks (GCNNs) has been monumental since their graph data structure effectively conveys various NLP task features. GCNNs have been applied to the study of overlapping relationship extraction, whereby syntactic dependency trees are converted into adjacency matrices and inputted into the graph neural network for relationship extraction. Noteworthy examples include research by Wang et al. [21], Fu et al. [22], and Duan et al. [23]. Relational extraction models based on pretrained language models mainly utilize generative pretrained language models to extract features. Through data fine-tuning, they are capable of achieving adequate performance in downstream tasks. Representative models include REBEL [24], CGT [25], UIE [26], and SPN [27]. Nevertheless, such pretrained models evidently have non-trivial shortcomings, such as difficulty in handling long text, as well as complex sentence structures, along with high computational and data requirements.

In the field of fisheries, Yang [28] proposed a BERT-BiLSTM-CRF model that incorporates dual attention mechanisms for words and sentences. This model was designed to enhance the effectiveness of relationship extraction by handling key issues, such as semantic loss in lengthy sequences and the irrational weight allocation of vectors, within the fisheries domain. Additionally, Liu et al. [29] developed the CaBiLSTM model, employing a layered approach to improve the recognition accuracy of outer layer entities. This approach involves dimensionality reduction of inner entity features to mitigate nested entity obstacles in Aquatic Disease Named Entity Recognition (NER) and integrates BERT for enhanced performance. Jiang [30] introduced a BIO-based entity relationship joint annotation strategy, which, in combination with the BERT-BiLSTM-CRF model, strengthens the relationship extraction performance. Bi et al. [31] utilized BERT-BiLSTM to carry out feature extraction from lengthy texts in aquaculture. They applied the N-Gram algorithm to segment features for integration into a cascading BiLSTM model, in this way resolving the issues of information loss and misclassification in long texts to bolster the performance of entity-relation joint extraction.

Previous research in the aquatic domain has mainly focused on extracting information from lengthy texts, identifying nested entities, and conducting the joint extraction of entity relationships. However, the persistent problem of relationship overlap within the aquatic disease domain remains largely unaddressed. The prevalence of abundant overlapping triplets often leads to recognition errors and incomplete extractions, substantially diminishing the effectiveness of relationship extraction. Moreover, the specialized nature of the aquatic disease corpus itself poses stark challenges, namely, the presence of domain-specific terms like disease and a plethora of drug names. This complexity makes it very hard for generic preprocessing models to accurately capture semantic features, resulting in poor domain suitability. Compounding that difficulty, the imbalance in aquatic disease data impedes effective model learning. Lastly, during the extraction process, many features beyond the encoded vectors of sentences will go underutilized, leading to both feature loss and an insufficiently nuanced semantic expression, which reduces model performance. Existing research methods have fallen short of soundly addressing these outstanding challenges. That said, the cascade tagging framework of the Casrel model does show promise in effectively identifying overlapping relationship triads, offering a potential solution to the relationship overlap problem in aquatic disease data. Therefore, this paper adopts the cascading binary labeling framework as its foundational structure. Expanding on that, it integrates a fine-tuned pretraining model, a feature fusion module, and the GHM loss function to construct an entity–relationship joint extraction model tailored to the aquatic disease domain. The main contributions of this approach are summarized as follows:

(1)

A textual corpus pertaining to aquatic diseases was gathered, from which a dataset was compiled for the aquatic disease relationship extraction. This dataset entails 33 relationship categories and comprises a total of 10,068 data entries.

(2)

The cascading binary labeling framework was employed to tackle the issue of relationship overlap, while the Roberta-wwm-ext pretrained model was adopted to replace the Bert pretrained model. Next, the model was fine-tuned with domain-specific knowledge related to aquatic diseases. This process of fine-tuning sought to enhance the model’s adaptation to the task of extracting relationships among entities related to aquatic diseases, particularly in scenarios with limited data samples, thus improving its domain suitability.

(3)

To address the issue of data imbalance, the GHM loss function has been incorporated into the model.

(4)

To further enhance the degree of feature fusion and enrich the semantic features of the model inputs, this paper introduces a feature fusion module named BRC into the infrastructure. The comprehensive structure combines a self-attention mechanism, a BiLSTM network, a multi-head attention mechanism with relative position encoding, and a conditional layer normalization layer. This module integrates the extracted sentence context features, head entity features, and head entity position features through the conditional layer normalization module, thereby enriching the semantic representation of the overall input and improving the effectiveness of the model’s extraction capabilities.

3. Aquatic Disease Dataset Construction

Currently, there is a dearth of openly available datasets for the entity–relationship extraction task within the aquatic disease domain. Hence, we relied on paper books and websites as primary data sources to establish a corpus database. Building upon this foundation, we designed the data structure and used Colabeler (v2.0.4) annotation software to perform the manual data annotation. This culminated in the creation of ‘AquaticDiseaseRE’, a dataset specifically tailored for aquatic disease research.

3.1. Data Collection

This paper gathers textual data from aquatic disease books and supplements it with data obtained by crawling and combing through professional websites. A total of 10,068 text entries were collected, these containing approximately 450,000 characters in the corpus. The primary data sources are outlined in

Table 1. Data sources for the corpus of text on aquatic diseases.

ID	Book or Website	Author	Publishers	Year of Publication or Launch
1	Aquatic Animal Disease Diagnosis and Control Technology	Xiaoling Liu	Chemical Industry Press	2009
2	Aquatic Animal Pathology	Wenbin Zhan	China Agriculture Press	2011
3	Aquatic Animal Disease Control Technology	Denglai Li	China Agriculture Press	2019
4	Aquaculture Network	China Society of Fisheries	--	2006
5	Fisheries Expertise Service System	Chinese Academy of Fishery Sciences	--	2018

:

3.2. Pattern Layer Design

The hierarchical structure of aquatic disease data was crafted by a thorough examination of the literature, for instance, Aquatic Animal Pathology, and in consultation with experts on aquatic diseases, as depicted in Figure 1 (Details of specific types and their descriptions can be found in Appendix A.1).

3.3. Dataset Segmentation

In this paper, the dataset was partitioned into a training set, validation set, and test set in an 8:1:1 ratio. The training set consisted of 8208 data samples, while both the validation and test sets comprised 1026 data samples each. Detailed statistics of the relationship categories in the dataset, along with some examples of their data samples, are presented in Figure 2 (details of specific types and their descriptions can be found in Appendix A.2.) and also in

Table 2. Examples of AquaticDiseaseRE data samples.

Data *	Subject	Relation	Object
Treatment of white spot disease: Administer a 1 mg/L chlorine bleach solution throughout the entire pond.	white spot disease	Treatment	Administer a 1 mg/L chlorine bleach solution throughout the entire pond.
The main target of herpesvirus disease of turbot is the large turbot.	herpesvirus disease of turbot	Hazard	turbot
Lamellodiscusiasis occurs frequently in seabreams reared in Qingdao and Guangdong.	Lamellodiscusiasis	Region	Qingdao; Guangdong
Flavobacterium cloumnare belongs to the family myxobacteriaceae.	Flavobacterium cloumnare	Affiliation	myxobacteriaceae
Aeromonas hydrophila has a short rod-shaped body without spores.	Aeromonas hydrophila	Characterization	short rod-shaped body without spores

* The original data examples are in Chinese and have been translated into English to better serve our readers.

.

The dataset developed in this paper generally faces the issue of Single Entity Overlap (SEO), where a single head entity is simultaneously linked to multiple distinct relationships. So, we classified the AquaticDiseaseRE dataset into Normal No Overlap (Normal) and Single Entity Overlap (SEO) categories based on the type of relationship overlap, as shown in

Table 3. AquaticDiseaseRE dataset composition.

Type	AquaticDiseaseRE
	Train	Dev	Test
Normal	6708	835	830
SEO	1500	191	196

:

Additionally, in this paper, the assembled AquaticDiseaseRE dataset was analyzed for sentence length and the number of triplets, as shown in Figure 3 and

Table 4. Counting the number of distinctive triplets in a sentence.

Triplet Number	Train	Dev	Test	All
N = 1	3825	454	448	4727
N = 2	2081	286	268	2635
N = 3	1363	175	171	1709
N = 4	632	66	91	789
N ≥ 5	307	45	48	400

.

4. Joint Extraction Model for Aquatic Disease Entity Relationships

4.1. Current Issues

Currently, within the aquatic disease domain, the chief challenges to its relationship extraction include relationship overlap, a high degree of corpus specialization, insufficient feature integration, and data imbalance.

(1)

Overlapping relationships: This concept, also known as ternary overlap, is split into two key categories: Single Entity Overlap (SEO) and Entity Pair Overlap (EPO). Within the assembled AquaticDiseaseRE dataset—designed for extracting relationships in the context of aquatic diseases—the relationships mainly follow logical knowledge structures, so the consideration of EPO cases is unnecessary. The phenomenon of SEO is more common, being chiefly characterized by linkages between aquatic diseases, pathogens, and their associated subclass concepts. As Figure 4 shows, the term ‘Normal’ designates those instances featuring standard, non-overlapping relationships.

(2)

The text corpus is highly specialized: Evidently, this corpus for the aquatic disease domain exhibits a high degree of specialization, being replete with an extensive array of proper nouns. For instance, the sentence “Infectious pancreatic necrosis disease causes the diseased fish’s body color to turn black, eyeballs to protrude, and congestion and hemorrhage at the base of the ventral fins, which can be prevented and controlled with povidone-iodine and erythromycin” illustrates this point. Certain terms such as “infectious pancreatic necrosis disease”, “abdominal fins”, “povidone-iodine”, and “erythromycin” are quite specific vocabulary within the aquatic domain of research, encompassing disease names, affected body parts, and both preventative and treatment interventions. Traditional BERT-series pretrained language models—developed from general corpus data—struggle to accurately encode the semantics of these specialized terms. This handicap greatly impairs the performance of subsequent entity–relationship extraction tasks.

(3)

Limited integration of features: In this paper, we explore the dependency of the Casrel model’s extraction performance on the semantic richness of input features. The original model merely employs a simplistic aggregation of BERT-encoded sentence vectors and entity representation vectors for input into the relation–tail entity recognition module. This approach proves clearly inadequate as it overlooks several crucial features, including sentence context and entity location attributes. Additionally, this rudimentary method of feature fusion can result in the loss of features, confusion of information, and a lack of adaptability in adjusting fusion strategies according to specific tasks.

(4)

The data is imbalanced: An imbalance in data mainly occurs in two ways: category imbalance and sample difficulty imbalance, with the task of aquatic disease relationship-extraction saddled with both challenges. The category imbalance is evident from the diverse relationship categories within the data and the significant variation in the sample sizes across different relationship categories, as Figure 2 shows. The Casrel model operates on a cascading binary labeling framework, utilizing this architecture for head entity recognition by determining whether each character in a sentence marks an entity boundary. This process of head entity recognition, alongside relationship and tail entity recognition, relies on a similar structural approach, where a ‘1’ label signifies a character as an entity boundary, in contrast to a ‘0’ label assigned to characters lying outside those boundaries. Typically, the ‘0’ label samples greatly outnumber the ‘1’ label samples and are simpler to classify, highlighting the imbalance between these difficult and easy types of samples.

4.2. Model Architecture

We propose an approach for entity–relationship joint extraction that leverages the cascading binary labeling framework. This framework integrates a series of advanced components, namely, a finely tuned Roberta-wwm-ext [32] pretrained model, the Gradient Harmonizing Mechanism (GHM) for loss adjustment [33], the self-attention Mechanism [34], and a Bidirectional Long Short-Term Memory (BiLSTM) network [5]. It also employs a Multihead Attention Mechanism based on relative position encoding (RPE) [35] in addition to conditional layer normalization (CLN) techniques [36]. These elements are synergistically combined in this study to create the Fd-CasBGRel model; it is specifically designed for extracting entity relationships in the aquatic disease domain. This model adeptly addresses the challenges discussed in Section 4.1, enabling better extraction efficiency and accuracy. The model’s architecture is divided into two core sections: the ‘Encoder’ and ‘Decoder’. The Encoder processes and encodes sentences, feeding them into the Decoder, which in turn is tasked with detecting relationship triplets within sentences. The Decoder is built around a head entity tagger and a relationship–tail entity tagger; its detailed architecture is illustrated in Figure 5.

4.2.1. FD–Roberta-wwm-ext Encoder

The Encoder serves as the model’s input layer, primarily tasked with extracting textual features and converting sentences into fixed-dimensional feature-encoding vectors. In this work, we replace the Bert model with the Roberta-wwm-ext pretrained model, which is then fine-tuned for our specific task by using knowledge of aquatic diseases. Unlike the original Casrel model’s Encoder that relies on the Bert pretraining model—trained on an English corpus and, thus, less effective at encoding Chinese text encoding—we employ the Chinese Roberta-wwm-ext model. This substitution improves our proposed model’s coarse-grained semantic modeling capabilities.

Still, both the Bert and Roberta-wwm-ext models are trained on public corpus sets. But the field of aquatic diseases encompasses a rich array of specialized terminology, and since the Roberta-wwm-ext model lacks domain-specific adaptation, a suboptimal encoding of specialized corpora ensues. Gururangan et al. [37] suggested that in cases of sparse samples or domain-specific applications, fine-tuning pretrained language models could significantly enhance performance. Candidate fine-tuning strategies for these models include In-Domain Further Pretraining and Within-Task Further Pretraining. Here, we implement a hybrid strategy that combines these two approaches. We use the aquatic disease dataset developed in this study, augmented with additional aquaculture knowledge gathered via Scrapy, to fine-tune the Roberta-wwm-ext model. This fine-tuning utilized hyper-parameters with a batch size of 8 and an epoch count of 5. The resulting model was named FD–Roberta-wwm-ext.

The input to the encoding layer is an input vector e, which is obtained by summing the token embedding vector e_w and the position embedding vector e_p. This vector is then processed through N-layers of the ‘Transformer’ to yield the sentence’s final vector representation, h_N. The overall structure is depicted in Figure 6, while the computational process is detailed in Equations (1) and (2):

$$h_1=\mathrm{Trans}\left(e\right)$$

(1)

$$h_N=\mathrm{Trans}\left(h_{N-1}\right)$$

(2)

4.2.2. SAB Feature Enhancement Layer

To enhance the semantic representation of input sentence vectors and to extract additional features, this study introduces a feature enhancement module named SAB. It integrates a self-attention mechanism and a BiLSTM (Bidirectional Long Short-Term Memory) network in a sequential manner that follows the FD–Roberta-wwm-ext pretrained model. First, the sentence vectors h_N are processed by a self-attention network to further capture sentence tele-dependencies and local key features. Next, it is inputted into the BiLSTM network to further extract contextual features of the sentence.

Initially, a self-attention mechanism is incorporated to process the sentence vectors encoded by Bert, enhancing the ability to capture distant sentence dependencies and emphasize local key features. Subsequently, these vectors, now processed through the self-attention mechanism, are fed into the BiLSTM network to extract contextual features of the sentences. The integration of the self-attention mechanism not only augments the synergy between Bert and BiLSTM but also improves the model’s capability to capture contextual nuances and strengthens the semantic representation of the input sentences.

The calculation process went as follows (Equation (3)):

$$h_{sa}=\mathrm{Self}\text{-}\mathrm{Attention}\left(h_N\right)$$

(3)

where h_N represents the input sentence vector, and h_sa denotes the feature vector obtained after encoding by the self-attention mechanism.

BiLSTM—a Bidirectional Long Short-Term Memory network—excels at extracting bidirectional contextual features from lengthy sequences and at capturing semantic dependencies in either direction. Hence, to further distill the semantic features of the input sentence, we use the BiLSTM network to encode the feature vector processed by the ECA network. This step provides a contextual representation of the sentence, as depicted in Equation (4):

$$b_N=\mathrm{BiLSTM}\left(h_N\right)$$

(4)

where b_N refers to the sentence vector encoded by the BiLSTM network.

4.2.3. Header Entity Tagger

The head entity tagger primarily serves to identify the head entity within an input sentence. It utilizes the feature vector b_N, produced by the SAB module, as its input. The output is a two-layer, One-Hot Encoding (OHC) vector, which clearly distinguishes the start and end positions of the head entity. Specifically, b_N first traverses two linear layers, each with an output dimension of 1, and later undergoes a sigmoid activation function. Binary classification is then performed based on a predefined probability threshold, whereby the vectors at the start or end positions of the head entity are assigned a value of 1, while all others are assigned a value of 0. This process is detailed in Equations (5) and (6):

$$p_i^{start\_sb}=\mathrm{sigmoid}\left(W_{start\_sb}{x}_i+b_{start\_sb}\right)$$

(5)

$$p_i^{end\_sb}=\mathrm{sigmoid}\left(W_{end\_sb}{x}_i+b_{end\_sb}\right)$$

(6)

where x_i is the encoding vector of the i-th character within the sentence encoding vector h_N; the W and b denote the learnable weights and biases in the linear layer, respectively, with ‘sigmoid’ denoting the activation function. The corresponding maximum likelihood function for the head entity within a given sentence is defined as follows:

$$p_{\theta }\left(s|x\right)={{\displaystyle \prod _{t\in \{start\_sb,end\_sb\}}{\displaystyle \prod _{i=1}^L\left(p_i^t\right)}}}^{I\left(y_i^t=1\right)}{\left(1-p_i^t\right)}^{I\left(y_i^t=0\right)}$$

(7)

where L is the length of the input sentence, and $y_i^t$ represents the positional classification labels for characters at the start/end positions of the head entity, taking values in the set {0, 1}. The parameters that need to be optimized are denoted by θ = {W_{start_sb}, b_{start_sb}, W_{end_sb}, b_{end_sb}}.

4.2.4. BRC Feature Fusion Module

The original Casrel model has a critical deficiency in how it facilitates information interactions and in correlating the head entity tagger and relation–tail entity tagger. In merely summing the vectors of the head entity and the sentence as subsequent inputs, crucial features are lost. In the literature [38], one finds the introduced the PATB relation extraction model, which increases performance by integrating the head entity and tail character vectors as entity location information for feature fusion. However, relying solely on these vectors for entity location features is inappropriate because it neglects the relative positional features among characters so it is less likely to capture sequential associations accurately. In response, this paper introduces a feature fusion module named BRC, which leverages head entity-relative position encoding along with sentence-local and contextual features. This approach aims to not only strengthen the connection between the head entity tagger and the relation–tail entity tagger but also to maximize the utility of extra features for bolstering the entity–relationship extraction. The BRC module incorporates the previously developed EB feature enhancement layer, a multi-head attention network (RPE) with relative position encoding, and conditional layer normalization (CLN), thus presenting a comprehensive framework for improving the relation extraction performance.

The general workflow went as follows. Begin by extracting the sentence enhancement feature vector b_N encoded by the SAB module; then, calculate the relative position encoding for the first and last characters of the head entity; finally, input the sentence-enhancement feature vector, head entity vector, and head entity-relative position vector into the CLN for feature fusion. This fused feature vector then replaces the original Casrel model’s simplistic summing of the head entity and sentence vectors to serve as the new input to the relation–tail entity tagger.

Multihead Attention Networks with Relative Position Computation

After obtaining the first and last word vectors, v_s and v_e, of the head entity, we can calculate their respective relative position representations for each character within the sentence vector b_n = [v₁, v₂, …, v_n]. These word vectors are then weighted accordingly to serve as the ‘Query’, while the sentence vectors are weighted to act as both ‘Key’ and ‘Value’. After doing so, they are fed into the multi-head attention network as parameters to derive the head entity’s relative position-encoding vectors. Taking the computation of the relative position encoding for the head entity’s first character vector, v_s, as an example, the process is detailed in Equations (8)–(14).

$$\mathrm{clip}\left(v,k\right)=\max \left(-k,\min \left(k,v\right)\right)$$

(8)

$$a_{sj}^K=w_{\mathrm{clip}\left(j-s,k\right)}^K$$

(9)

$$a_{sj}^V=w_{\mathrm{clip}\left(j-s,k\right)}^V$$

(10)

$$z_i^s={\displaystyle \sum _{j=1}^n{a}_i^{sj}\left(v_j{W}_i^V+a_{sj}^{V_i}\right)}$$

(11)

$$a_i^{sj}=\frac{\exp \left(e_i^{sj}\right)}{{\displaystyle {\sum }_{k=1}^n\exp \left(e_{sk}\right)}}$$

(12)

$$e_i^{sj}=\frac{v_s{W}_i^K{\left(v_j{W}_i^K+a_{sj}^{ki}\right)}^T}{\sqrt{d_z}}$$

(13)

$$rp{e}_s=\mathrm{Concat}\left(z_1^s,z_2^s,\dots ,z_h^s\right)W^o$$

(14)

Here, $a_{sj}^K$ and $a_{sj}^V$ denote the relative position representation between the first character vector of the head entity, v_s, and the j-th character vector of the sentence vector, b_n, where j-s refers to the relative position between the two characters, and k is the maximum relative position-encoding distance. The terms $e_{ji}^s$, $a_{ji}^s$, and $v_{ji}^s$, respectively, denote the scaled dot-product representation of the Query and Key computed by the i-th head in the multi-head attention mechanism, the multi-head attention calculation score, and the attention-encoding vector. The rpe_s refers to the relative position-encoding vector of the first character of the head entity. The variable h is the number of heads in the multi-head attention mechanism and w^o is the weight associated with that. The relative position-encoding rpe_e of the tail character vector of the head entity is computed in a manner analogous to rpe_s.

Conditional Layer Normalization (CLN) Layer

The CLN’s architecture is inspired by conditional batch normalization (CBN), which is a structure that incorporates conditional multi-information fusion in the field of image processing. We present a multi-conditional CLN module for feature fusion whose main structure is illustrated in Figure 7.

The vector of the head entity, denoted as sub, along with the two relative position-encoding vectors rpe_s and rpe_e—respectively for the first and last characters of the head entity—are concurrently fed into the CLN layer. This input is accompanied by the sentence vector b_N; it is already refined through the self-attention network and the BiLSTM-encoding layer. The process eventually generates feature fusion vectors that amalgamate sentence context features, sentence localization features, and the positional attributes of the head entity. This integration facilitates the directed control of information output. The involved mechanism is detailed in Equation (15):

$$\mathrm{CLN}\left(b_N,c_{sub},c_{rp{e}_s},c_{rp{e}_e}\right)=\frac{b_N-E\left[b_N\right]}{\sqrt{Var\left[b_n\right]+\epsilon }}{W}_1{c}_{sub}+W_2{c}_{rp{e}_s}+W_3{c}_{rp{e}_e}$$

(15)

Here, the ‘c’ represents the conditional function, while E[b_N] and Var[b_n] are the mean and variance, respectively. The term ε is a very small constant > 0, which we introduced to avoid computational errors that may arise from a zero denominator.

4.2.5. Relationship–Tail Entity Tagger

The relation–tail entity tagger identifies both relations and tail entities by utilizing a structure identical to that of the head entity tagger, such that their quantity aligns with the number of relation categories. For this, we use the feature fusion vector, encoded by the BRC module, as the input. The encoding process for each character within the input sentence is detailed in Equations (16) and (17):

$$p_i^{start\_ob}=\mathrm{sigmoid}\left(W_{start\_ob}{c}_i+b_{start\_ob}\right)$$

(16)

$$p_i^{end\_ob}=\mathrm{sigmoid}\left(W_{end\_ob}{c}_i+b_{end\_ob}\right)$$

(17)

where c_i represents the vector representation of the i-th character in the sentence, as encoded by the CLN module, and W and b are the weight and bias terms, respectively, of the tail entity labeling module, with the sigmoid function being the activation mechanism.

4.2.6. GHM Loss Function

Since the model developed in this paper employs a cascading binary pointer network for the entity–relationship extraction, this fundamentally constitutes a binary classification task. Therefore, we applied a binary cross-entropy loss function for the loss computation. To identify entity boundaries and relationship–tail entities, two classification matrices of dimensions l × 1 and l × r were constructed, where l is the sentence length and r is the number of relationship categories. In spite of this setup, the dataset harbors considerable diversity in the relationship categories, leading to a pronounced disparity in sample distribution across them. This discrepancy manifests as a category imbalance during both the head entity labeling and relationship–tail entity labeling phases. Specifically, non-positional labels ‘0’ outnumber the positional labels ‘1’, and negative samples are mostly labeled ‘0’; they constitute a hefty proportion of the dataset. This predominance of easily classifiable negative samples and the resultant label imbalance could potentially impair the effectiveness of model training.

To deal with that category imbalance issue, the Focal Loss function [39] is often employed for the adjustment of positive and negative samples. It uses two adjustment factors, α and γ, to balance the distribution of positive and negative samples as well as that between difficult and easy samples, in this way aiming to rectify the imbalance of data samples. Still, the Focal Loss function is not without its shortcomings. Firstly, it disproportionately focuses on positive samples that are hard to classify while neglecting the training of negative samples, which can lead to reductions in model performance. Secondly, the interaction between the adjustment factors α and γ unavoidably varies across different datasets, so their optimal values must be determined via extensive experimentation. The GHM loss function offers an innovative solution, by assessing the number of samples within specific ranges using a gradient density variable. More precisely, it mitigates loss by considering the volume of samples within a particular confidence interval p, overcoming the drawbacks besetting the Focal Loss function. The methodology for calculating the GHM loss function is given in Equations (18)–(20):

$$g=|p-p*|=\left\{\begin{array}{c}1-p,p*=1\\ p,p*=0\end{array}\right.$$

(18)

where g is the length of the gradient mode, p is the prediction probability, and p* denotes the actual label.

$$\mathrm{GD}\left(g\right)=\frac{1}{l_{\epsilon }\left(g\right)}{\displaystyle {\sum }_{k=1}^N{\delta }_{\epsilon }\left(gk,g\right)}$$

(19)

Here, GD(g) is the gradient density; the term δ_ε(gk, g) corresponds to the count of samples whose gradient modulus falls within the range (g − ε/2, g + ε/2) among samples 1 to N, while l_ε(g) denotes the length of the interval (g – ε/2, g + ε/2). The loss for each individual sample is then calculated using Equation (20):

$$L_{GHM}={\displaystyle {\sum }_{i=1}^N\frac{L_{\mathrm{CE}}\left(p_i,p_i^{*}\right)}{\mathrm{GD}\left(g_i\right)}}$$

(20)

where N is the total number of samples, the term L_CE(p_i, $p_i^{*}$) denotes the cross-entropy loss for the first sample, and GD(g_i) signifies the gradient density for the first sample.

5. Experiments and Analysis

5.1. Experimental Dataset

In this paper, we mainly study the aquatic disease entity–relationship extraction model and apply it to knowledge graph construction, so the main dataset used is AquaticDiseaseRE. However, it is a Chinese domain, so in order to further validate the model’s paradigm in the English domain, we additionally use the English public dataset WebNLG to conduct the comparative experiments. The sentence sample capacity of the training set, validation set, and test set in the WebNLG dataset is 5019, 500, and 703, respectively, and there is a certain amount of triple overlap, which is similar to the self-built AquaticDiseaseRE dataset.

5.2. Experimental Environment

The experiments described here were conducted on a system running Windows 11. It was equipped with an AMD Ryzen 7 5800H processor with Radeon Graphics at 3.20 GHz, along with 16 GB of RAM and an NVIDIA GeForce RTX 3060 graphics card. The software environment consisted of Python v3.9.13 and the PyTorch framework v1.10.1.

5.3. Model Parameters

We use different parameters to conduct experiments for different datasets. The hyperparameters for the experimental model of this study are in

Table 5. Model training parameters used.

AquaticDiseaseRE		WebNLG
Parameter	Value	Parameter	Value
Length of hidden vectors	768	Length of hidden vectors	768
Maximum input sentence length	294	Maximum input sentence length	294
Batch size	12	Batch size	6
Dropout	0.5	Dropout	0.5
Number of iterations	60	Number of iterations	100
Learning rate	1 × 10−5	Learning rate	1 × 10−5
Entity tagging threshold	0.5	Entity tagging threshold	0.5
Optimizer	Adamw	Optimizer	Adam
Maximum distance for relative position encoding	50	Maximum distance for relative position encoding	50
Number of long attention spans	8	Number of long attention spans	8
BiLSTM network layers	3	BiLSTM network layers	3

.

5.4. Evaluation Metrics

For the task of relation extraction, precision (P), recall (R), and the F1 score were the chief evaluation metrics, as given by Equations (21)–(23).

$$P=\frac{TP}{TP+FP}$$

(21)

$$R=\frac{TP}{TP+FN}$$

(22)

$$F1=\frac{2\times P\times R}{P+R}$$

(23)

where TP is the number of positive samples correctly predicted, FP is the number of negative samples incorrectly predicted as positive, FN is the number of positive samples incorrectly predicted as negative, and TN is the number of negative samples correctly predicted as negative.

5.5. Pre-Trained Model Replacement Experiment

To investigate the impact of various Chinese pretrained language model versions on the performance of an aquatic disease relationship extraction model when utilized as the encoding layer and, thereby, select the most suitable pretrained model for in-task fine-tuning, we tested four distinct pretrained models. Specifically, Bert, Bert-wwm, Bert-wwm-ext, and Roberta-wwm-ext were chosen for this coding layer-substitution experiment whose results are presented in

Table 6. Evaluation results of different pretrained models.

Pretrained Model	P	R	F1
Bert	73.17	68.77	70.90
Bert-wwm	74.43	70.14	72.22
Bert-wwm-ext	75.26	71.52	73.34
Roberta-wwm-ext	78.92	74.56	76.68

.

Evidently, the Roberta-wwm-ext model achieves the highest precision rate (78.92%), recall rate (74.56%), and F1 score (76.68%). Compared to the Bert, Bert-wwm, and Bert-wwm-ext models, the F1 score of Roberta-wwm-ext showed an improvement of 5.78%, 4.46%, and 3.34%, respectively. This superior performance of the Roberta-wwm-ext pretrained model can be attributed to its use of a larger training dataset, a larger batch size, and a whole word-masking mechanism that is better suited to the Chinese language. Collectively, these factors contributed to its advantage over other pre-processing models. Therefore, based on the experimental results, this paper decided upon Roberta-wwm-ext as the foundational model for fine-tuning the encoding layer.

5.6. Impact of Loss Functions on Unbalanced Data

In this study, to address the issue of data imbalance, we implemented the GHM loss function and compared its effectiveness against the Focal Loss and cross-entropy (CE) loss functions as applied to the AquaticDiseaseRE dataset and WebNLG dataset. The Focal Loss was parameterized with α = 0.5 and γ = 0.25. Figure 8 shows the outcomes of these comparisons.

Evidently, the GHM loss value exceeds that of both the CE and Focal Loss. This pronounced difference is due to GHM’s strategy of weighting the gradient of each sample, which ensures a more balanced contribution to the overall loss from gradients of samples varying in difficulty levels. The aquatic disease dataset in this study contains many difficult samples due to its wide-ranging categories and uneven distribution of these samples. Consequently, difficult-to-classify samples constitute a larger fraction of the dataset, increasing the overall loss for the model. Incorporating GHM loss results in a notable enhancement of the model’s overall F1 score vis-à-vis both CE and Focal Loss. Specifically, for the test set, the model using the GHM loss achieves the best performance, showing an improvement of 2.77% and 4.35% over CE and Focal Loss, respectively. This suggests the GHM loss function is able to effectively prioritize the difficult-to-classify samples and reduces the weights for easier ones so as to achieve a better balance of loss, thereby enhancing the model’s predictive accuracy for less represented categories. In contrast, the overall performance of Focal Loss diminishes relative to the CE loss function, likely because of Focal Loss’s excessive focus on hard-to-categorize samples. This overemphasis may cause the model to overfit those outliers.

The WebNLG dataset shows the same rules as the AquaticDiseaseRE dataset. This is because the two datasets have similar characteristics, such as the dataset contains more types of relationships and a large number of overlapping relationship triples. In addition, the loss value fluctuation of the model in the WebNLG dataset is larger than that of the AquaticDiseaseRE. This is because the WebNLG dataset is relatively more complex, with 171 predefined relationship categories, and the proportion of overlapping triples is also larger than that of the AquaticDiseaseRE dataset, and the data are more unbalanced. In general, GHM has good results for datasets with limited and unbalanced samples.

5.7. Comparative Experiment

To verify the effectiveness of the model proposed in this paper, we conducted comparative experiments with several other leading entity relationship extraction models on the AquaticDiseaseRE dataset and the WebNLG dataset:

(1)

NovelTagging [14]: A model designed for the joint extraction of entity relations by leveraging innovative annotation strategies;

(2)

CopyR [16]: An end-to-end entity relationship extraction model that utilizes a replication mechanism;

(3)

GraphyRel [22]: A relationship extraction model whose operation is based upon the structure of relationship graphs;

(4)

TPLinker [18]: A labeling framework that utilizes the spans of entity heads and tails;

(5)

CasRel [15]: A pointer network model that employs cascading binary tags;

(6)

PFN [19]: An entity–relationship extraction model that uses partitioned filter networks;

(7)

UniRel [20]: An entity–relationship extraction model that integrates relational semantics.

The experimental findings are presented in Figure 9 and

Table 7. Evaluation results for the 10 experimental models.

Model		AquaticDiseaseRE			WebNLG
	P	R	F1	P	R	F1
NoverTagging	48.66	38.95	43.27	52.58	19.36	28.30
CopyR-Muti	50.07	33.30	40.00	37.76	36.48	37.11
CopyR-One	48.95	31.51	38.34	32.24	28.91	30.48
GraphyRel-1p	61.01	43.13	50.54	42.36	39.24	40.74
GraphyRel-2p	62.36	44.23	51.75	44.73	41.15	42.87
TPLinker	74.29	71.64	72.94	89.14	84.79	86.91
Casrel	73.17	68.77	70.90	94.17	91.36	92.74
PFN	77.31	74.84	76.05	94.72	92.37	93.53
UniRel	76.32	73.68	74.98	94.88	94.63	94.75
Fd-CasBGRel	83.15	86.33	84.71	-	-	-
Wn-CasBGRel *	-	-	-	96.79	95.46	96.12

*: Wn-CasBGRel denotes the English version of the model in this paper, where the pre-trained model of the coding layer was replaced with the English version and fine-tuned within the English dataset used, and the rest is consistent with the Fd-CasBGRel model.

.

According to Table 7, the model introduced in this paper exhibits impressive performance when applied to the aquatic disease relationship extraction dataset. Specifically, its precision (P), recall (R), and F1 score metrics are distinguished by an enhancement of 5.84%, 14.69%, and 8.66%, respectively, in comparison with the next best-performing model, PFN. Furthermore, the results for the test set were consistent with the F1 value curve for the validation set. The challenge of handling many overlapping relationship triplets in this dataset renders NoverTagging less effective; it cannot properly address the issue of overlapping relationship extraction by solely considering those scenarios where an entity is part of only a single ternary relationship. Meanwhile, the CopyR and GraphyRel series models, which utilize GRU or BiLSTM for encoding, fall short in effectiveness when compared to the Bert pretraining model’s encoding. Although the TPLinker, Casrel, PFN, and UniRel models have marked improvements, the pretraining models they employ are not sufficiently fine-tuned for specialized domains, resulting in a constrained enhancement of their effects relative to the models we proposed in this paper. This outcome further emphasizes that, for highly specific domains, employing fine-tuned pretraining models is essential for augmenting the semantic encoding effect.

In addition, the model proposed in this paper also achieved the best performance on the English public dataset WebNLG, with P, R, and F1 values of 96.79%, 95.46%, and 96.12%, respectively, which are 1.91%, 0.83%, and 1.37% higher than the best-performing UniRel model. Since the WebNLG dataset is public domain data and the data are not very professional, the method of fine-tuning the pre-trained model in the data has a general improvement effect. The experimental results fully demonstrate that the model has potential cross-language application capabilities.

5.8. Experiment on Relationship Overlap

Besides improving the relationship extraction, another critical issue that our model addresses is the problem of relationship overlap. To assess the model’s extraction capabilities under conditions of overlapping relationships, we categorized the test set data into two classes, Normal and Single Entity Overlap (SEO), based on the presence of overlapping relationships. The Normal category has 830 data points, while the SEO category has 196 data points (see Table 3). Figure 10 shows the experimental outcomes.

Evidently, the F1 scores of the model proposed in this study are superior to those of the other nine models for both the Normal and SEO categories tested. The performance of those latter models for these two relationship extraction types displays a general reduction, which suggests the task of extracting entity–relationship triplets gets harder with a greater proportion of shared entities in the dataset. Specifically, NoverTagging, CopyR, and GraphRel models undergo a marked decline in F1 scores when dealing with data that includes relationship overlap, indicating their limitations in handling such overlapping forms of data. By contrast, the remaining models actually exhibit slightly improved performance for overlapping relationship data, which we attribute to their respective architectures’ capability to address that key issue.

The Fd-CasBGRel model developed in this study has the best results for both Normal- and SEO-type relationship extraction tasks. It even records higher F1 scores in SEO- than Normal-type data, which could be due to the smaller sample size of SEO-type in the test set. Overall, these experimental outcomes provide compelling evidence that the model presented in this paper is more adept at resolving the challenge of relationship overlap in the task of aquatic disease relationship extraction.

To further validate the model’s extraction performance across scenarios with varying numbers of triplets, we categorized the data into five classes based on their quantity. These results are presented in

Table 8. Comparison of F1 scores for models handling data from various ternary group quantities.

Model	N = 1	N = 2	N = 3	N = 4	N ≥ 5
NoverTagging	47.20	37.63	39.88	43.96	41.86
CopyR-Muti	53.39	34.38	38.02	31.74	30.00
CopyR-One	54.78	34.46	33.70	29.74	22.39
GraphyRel-1p	56.51	43.61	48.60	58.82	45.62
GraphyRel-2p	54.98	42.69	51.10	59.29	47.56
TPLinker	67.88	66.94	72.81	80.04	76.47
Casrel	65.86	66.08	71.67	79.11	75.21
PFN	72.59	68.73	76.74	84.14	79.63
UniRel	71.36	67.85	75.27	82.97	78.24
Fd-CasBGRel	82.21	80.78	84.04	90.58	82.94

.

A discernible trend emerges where the F1 scores of both NoverTagging and CopyR models diminish to varying extents as the dataset’s triplets increase in frequency. This trend suggests a greater difficulty in processing semantic information and navigating a more complex answer space when more triplets are present, leading to potential confusion and errors during extraction. Additionally, the occurrence of overlapping relation triplets further complicates the extraction process. Conversely, models such as TpLinker, Casrel, PFN, and Unirel have significantly improved F1 scores when the triplet count is between 2 and 4. This better performance is ascribed to these models’ capacity to manage overlapping relational triplets, showcasing their exceptional performance within a specific triplet range. However, as the sentence’s triplet count continues to rise, the F1 scores of those models begin to decline in varying degrees.

The model we developed here achieves the highest F1 score across all five different aquatic disease datasets with varying triplet counts, with a peak F1 score of 90.58% at N = 4. This score not only surpasses several comparative models but also exceeds the best-performing PFN model by 6.44%. This superior performance is credited to the cascading labeling framework employed in this study, which adeptly handles overlapping relationship data across multiple triplets. The integration of a fine-tuned pretraining model and a feature fusion module further enriches the sentence semantic expression and enhances extraction effectiveness. In particular, the incorporated GHM loss function balances well the sample weights and ameliorates the imbalanced data in scenarios involving multiple triplets.

In summary, when assessed alongside several baseline comparative models, this study’s proposed model more effectively addresses the complexities of relationship overlap and multiple triplets in the aquatic disease domain.

5.9. Ablation Experiment

To evaluate the impact of the FD–Roberta-wwm-ext pretrained model, the BRC feature fusion module, and the GHM loss function on aquatic disease entity–relationship extraction, module ablation experiments were also carried out. The training outcomes are conveyed in Figure 11. Here, Casrel denotes the foundational framework, Casrel_BRC refers to the incorporation of only the feature fusion module, Casrel_GHM corresponds to the integration of solely the GHM loss function within the base framework, Casrel_FDRobertawwmext represents the sole addition of the fine-tuned pretraining model, and Fd-CasBGRel is the model proposed in this paper.

We can see from Figure 11 that the Casrel model, when augmented with the fine-tuned pretraining model, demonstrates significant improvements over the baseline Casrel model in terms of precision (P), recall (R), and F1 score metrics, as well as in convergence speed. This enhancement can be attributed to the fine-tuned model’s ability to acquire domain-specific knowledge about aquatic diseases and more effectively encode the semantic features of those entities in comparison to the original pretraining model. Including the BRC feature fusion module alone leads to a notable increase in the model’s p-value relative to the baseline model, suggesting that feature fusion enriches the semantic vector representations, thereby augmenting the accuracy of entity recognition. The R-value undergoes slight improvement, while the overall F1 score surpasses that of the baseline model. Not surprisingly, incorporating the GHM loss function also contributes to performance gains to a certain extent by mitigating the data imbalance. Ablation experiments were also made on the test set, with these results presented in

Table 9. Overall results for the ablation experiment.

Model	P	R	F1
Casrel	73.17	68.77	70.90
Casrel_BRC	76.76	74.98	75.86
Casrel_GHM	74.92	72.46	73.67
Casrel_FDRobertawwmext	83.35	82.56	82.95
Fd-CasBGRel	83.15	86.33	84.71

.

Incorporating the feature fusion module, GHM loss function, and fine-tuned pretraining model individually into the model yields F1 scores of 75.86%, 73.67%, and 82.95%, respectively. This corresponds to an enhancement of 4.96%, 2.77%, and 12.05% vis-à-vis the baseline model, indicating that each of the three augmentations is beneficial to the model’s performance. Notably, a greater improvement is gained from fine-tuning the pretraining model than from both the feature fusion module and GHM loss function. This can be ascribed to the pretraining model’s better ability to learn disease text data features through fine-tuning, which provides stronger domain adaptation and more effective encoding of semantic vectors. The adoption of the feature fusion module, with its ensuing 4.96% improvement, demonstrates that fusing the relative position features of the head entity’s first and last characters, local sentence features, and contextual features through CLN enables deeper feature integration and enriches semantic representations beyond the mere addition of head entity and sentence features. This confirms the efficacy of the BRC module. Moreover, integrating the GHM loss function results in a 2.77% improvement over the baseline model; this suggests the GHM loss function rectifies, to some extent, the issue of imbalanced aquatic disease data categories. When these three enhancements are applied in tandem, the model attains its peak F1 score of 84.71% for the aquatic disease dataset, indicating a synergistic effect that collectively boosts the model’s extraction capabilities.

In this paper, the inputs to the proposed feature fusion module are processed through the self-attention mechanism, a BiLSTM network, and a multi-head mechanism that incorporates entity position computation. To further assess the impact of these components, we conducted another ablation experiment as follows: BRC represents the model that entails the complete feature fusion module; -SA refers to the model with the self-attention mechanism removed; -BiLSTM denotes the removal of the BiLSTM network; -RPE corresponds to the exclusion of the relative position encoding, thus relying solely on the head and tail word vectors of the head entity as conditional inputs for the CLN.

As evinced by

Table 10. Results of the BRC module ablation experiment.

Model	P	R	F1
BRC	76.76	74.98	75.86
-SA	74.69	73.71	74.20
-BiLSTM	72.74	71.35	72.04
-RPE	75.08	73.94	74.51

, the removal of various components from the BRC model leads to a reduction in F1 scores to varying degrees. Among them, removing the self-attention mechanism encoding results in a 1.66% decrease in the model’s F1 score. This suggests that the self-attention mechanism enhances the interaction between the BERT encoding layer and the BiLSTM layer, effectively modeling the distant dependencies within the sentence while emphasizing its local key features. This, in turn, strengthens the contextual representation of the BiLSTM encoding. Moreover, eliminating the BiLSTM leads to a significant drop in the F1 score by 3.82% compared to the baseline. This highlights the importance of contextual features for the task of relationship extraction, which necessitates parsing the sentence structure. Additionally, the F1 score decreases by 1.35% after removing the relative coding of entity positions, indicating that incorporating the positional attributes of the head entity into the model can somewhat improve its ability to localize the head entity. This enhancement aids in better extraction of relationships and tail entities, thereby boosting the overall performance of the model. Overall, the experimental results demonstrate that incorporating additional features such as local key features, contextual sentence features, and relative positional attributes of head entities can effectively enhance the model’s performance.

5.10. Construction of the Knowledge Graph

Next, we can use the Fd-CasBGRel model to extract ternary data related to aquatic diseases and store that in a CSV file. By then importing the ternary data into the Neo4j graph database using Neo4j’s import command, we can begin building a knowledge graph. Statistical analysis reveals that this preliminary construction of the aquatic disease knowledge graph has a total of 20,469 ternary data items. It covers 33 types of entity categories and 32 types of relationship categories. Figure 12 shows a portion of the visualized knowledge graph for the aquatic disease domain.

6. Conclusions

This paper addresses the challenges of relationship overlap, corpus specialization, low feature fusion, and data imbalance in the task of extracting entity relationships for aquatic diseases, all of which diminish the efficacy of relationship extraction. To deal with these issues, we adopt a cascading binary labeling framework as the overarching architecture to enhance the extraction of overlapping relationship triplets and overcome the issue of overlapping relationships in aquatic diseases. Further, the pretrained model in the encoding layer is fine-tuned with a corpus related to aquatic diseases, augmenting the model’s capacity for the semantic encoding of aquatic disease texts. Building on this foundation, we introduce the BRC feature fusion module, which deeply integrates the relative position features of the head entity’s first and last characters, local sentence features, and contextual sentence features via a self-attention mechanism, a BiLSTM network, and a conditional layer normalization process. This module significantly strengthens the model’s feature fusion capabilities in the extraction task, enriches semantic representations, and, thus, amplifies the extraction effect. Additionally, the GHM loss function is implemented to help rectify the data imbalance issue. Finally, the refined model is used to successfully initiate the preliminary construction of a knowledge graph for aquatic diseases, evincing the model’s practical applicability and effectiveness in improving the extraction of aquatic disease entity relationships. The experimental findings demonstrate that the model introduced in this study achieves the highest F1 score of 84.71% when run using the dataset for aquatic disease entity–relationship extraction. Notably, it has an F1 score of 86.52% within the category of data involving overlapping entities, greatly outperforming several established mainstream models for entity–relationship extraction. This highlights the model’s comprehensive effectiveness in addressing the aforementioned challenges. In addition to the self-built aquatic disease dataset, comparative experiments were also conducted on the public English dataset WebNLG. The results showed that the model achieved good results and demonstrated its ability to be applied across language fields.

While location features play a pivotal role, this study acknowledges that other features, both lexical and syntactic, remain underexploited. Investigating how to effectively mine and integrate them to bolster performance is now a chief focus of future research. Also, knowledge graphs’ application within the agricultural sector is relatively unexplored, with most attempts centered on question-and-answer scenarios. The potential for utilizing knowledge graphs in conjunction with real-time monitoring of environmental parameters and fish growth to drive early prediction and warning systems for aquatic diseases represents a promising research direction that warrants further investigation.