An Easy Partition Approach for Joint Entity and Relation Extraction

Abstract

The triplet extraction (TE) task aims to identify the entities and relations mentioned in a given text. TE consists of two tasks: named entity recognition (NER) and relation classification (RC). Previous work has either treated TE as two separate tasks with independent encoders, or as a single task with a unified encoder. However, both approaches have limitations in capturing the interaction and independence of the features for different subtasks. In this paper, we propose a simple and direct feature selection and interaction scheme. Specifically, we use a pretraining language model (e.g., BERT) to extract various features, including entity recognition, shared, and relation classification features. To capture the interaction, shared features consist of the common semantic information used by the two tasks simultaneously. We use a gate module to obtain the task-specific features. Experimental results on various public benchmarks show that our proposed method can achieve competitive performance, and the calculation speed of our model is seven times faster than CasRel, and two times faster than PFN.

1. Introduction

Triplet extraction (TE) aims at extracting the entities and relations between entity pairs to form triples in the given raw text. It is useful for many applications, especially those that require extracting the critical information from a given text. It is related to two other tasks: named entity recognition (NER) [1] and relation classification (RC) [2].

Triplet extraction has been widely studied in the existing work from two perspectives. One is multitask learning perspective: entity recognition and relation classification are regarded as two separate tasks [3], and two independent encoders are used to learn the features of NER [4,5,6] and RC tasks [7,8,9]. Despite the effectiveness of these approaches, the correlation between the features of the two tasks is absent, and thus may lead to error propagation between tasks. In order to mitigate these issues, a single-task perspective is proposed, which typically designs a unique structure to represent the two tasks [10,11,12], such as TPLinker [13] and UniRE [14]. These approaches integrate NER and RC as one task, and generally use a unified encoder to learn the unified feature representation. These approaches avoid error propagation, but they may not capture the specific information needed for each task. In addition, a uniform feature was empirically shown to cause the interference issue [3].

To obtain improved task-specific features, the authors of [15] proposed a neuron-level partition filter network that learns three feature representations at the neuron level, and the structure has been proven effective in multiple tasks, but the method is a bit complex. The pretraining language model (PLM) has achieved state-of-the-art performance in various NLP tasks, including machine translation [16,17] and sentiment analysis [18]. To leverage the latent contextual space of PLM, we propose a simple but effective approach, called easy partition approach for relation extraction (EPRE). Unlike state-of-the-art alternative PFN [15], our proposed EPRE is different in the following aspects: (I) We do not construct a unique partition network based on neurons, but directly and simply partitions the results of PLM based on tokens. Specifically, our feature partition leverages the contextual representations of PLMs and performs feature partitioning on the token level to capture task-specific features. (II) We add a gate module to control the information flow between the NER and RC tasks and enhance the model flexibility. We discard the recursive structure and speed up the training process. As a result, our EPRE is simpler and more time-saving. To validate the effectiveness of EPRE, we empirically conduct extensive experiments on various benchmark datasets. Our experimental results show that EPRE is promising. In brief, we summarize our contributions, as follows:

• We propose a simple but effective joint coding relation extraction approach, namely, easy partition approach for relation extraction (EPRE). Particularly, EPRE models the features between NER and RC tasks to ensure the independence and interaction of specific tasks.
• We consider different gate modules to fuse task features to increase the elasticity of the model.
• Our method is superior in speed, and yields the state-of-the-art results of the NER task and RC task on some benchmark datasets.

2. Related Work

The task of TE aims to extract entities and their relations from unstructured text and represent them as relational triples. The traditional TE method is usually divided into two subtasks: NER and RC. NER identifies entity boundaries and types from text, while RC determines the semantic relation between entity pairs. Various methods have been proposed for both subtasks.

Table 1. Common technology paradigm of entity recognition and relation extraction.

Method Type	Meaning of Methods	Related Paper
Span-based method	In NER tasks, it refers to the process of identifying entity spans in text and classifying them. In RE tasks, it is the process of classifying entity span pairs.	[19,20]
Tag-based method	In NER, it refers to assigning a label to tokens in text, often using “BIO” and “BIOS” to label token positions, and combining token positions and entity types as labels for entities, such as B-location. In RE tasks, a combination of token position, relation type, and entity position is often used as labels, such as B-CEO-1.	[21,23]
Table filling method	Usually using T × N × the matrix of N represents the sentence sequence, T represents the entity type or relation type, and N represents the sentence length. In NER, take the diagonal of the matrix as the starting position of the entity, find the corresponding ending position of the entity in the upper triangle of the matrix, and mark it as 1. Determine the entity in the sentence according to the matrix. In RE, find the position corresponding to the token pair of the head or tail of the entity pair in the matrix, mark it as 1, and determine the entity pairs with relation based on the matrix.	[15,24,25]
Graph-based method	The main idea of this method is to represent entities and relationships as nodes and edges in the graph, and then use graph algorithms to process them.	[26,27]

shows the main methods. For NER, there are span-based methods [19,20] that predict entity spans by scoring all possible text segments, and tag-based methods [21] that assign a tag to each token indicating its entity type and position. For RC, there are methods that use independent encoders for each subtask [3,22], treating them as span classification and span pair classification problems, respectively. However, these methods suffer from weak interaction between the two subtasks, which may lead to inconsistent predictions.

To address this limitation, recent works have explored joint extraction methods that combine NER and RC using a single model. These methods can be roughly categorized into three paradigms: 1. Based on the table filling method [15,24,25], which presents entities and relations in tables. For example, Yan et al. [15] used a table for each type of entity and relation. By combining the results from NER and RE, relational triples can be extracted. 2. Based on the method of sequence tagging, the tag system encodes both entity position and relation type for each token [21]. Then, a triplet is formed according to the tag of the token. For example, Zheng et al. [21] proposed a novel tagging scheme and changed the extraction problem to a sequence labeling problem. To solve overlapping relations, Wang et al. [23] designed a handshaking tagging scheme and transformed the task into a token pair linking problem. 3. Graph-based methods [26,27], which build graphs for entities and relations, and perform graph neural network operations on them. For example, Wang et al. [27] constructed an entity-relation graph based on dependency parsing results, and applied graph convolutional networks to capture global semantic information. In the joint extraction method, the two subtasks usually share the encoding layer with weak independence. Therefore, how to obtain a better feature representation suitable for each subtask is a key challenge.

In order to alleviate the issue, the classical network model LSTM [28] used a gating mechanism to control how much information is forgotten or remembered in the hidden state. Another solution is to use attention mechanisms, which can learn to focus on the relevant parts of the input sequence and ignore the irrelevant ones. Attention mechanisms have been widely adopted in most pretrained language models [29,30], which are large-scale neural networks that are trained on massive amounts of text data, and can be fine-tuned for various NLP tasks. The closest work to us is that of Yan et al. [15]. They proposed a partition filtering network for the neuron level, which needs to use a separate network layer to learn the feature representation of each task. However, their method requires additional parameters and computations for each task, which increases the training cost and complexity. Our approach is much more straightforward with less training cost.

3. Method

3.1. Triple Extraction Task

Following Yan et al. [1] and Zhong et al. [2], our framework takes entity recognition and relation classification as two subtasks. To maintain the correlation and independence between the two subtasks of entity recognition and relation extraction, we adopt the feature partitioning method and the feature fusion strategy under the gating mechanism to generate the feature representation for each task. Given a sequence ${\mathit{s}=[\mathit{w}}_{\mathit{1}}{,\mathit{w}}_{\mathit{2}}{,\mathit{w}}_{\mathit{3}}{,\dots \mathit{w}}_{\mathit{n}}]$ with length N, our goal is to identify all entities under each entity type, and judge whether the relation exists when the head of one entity pair matches under each relation. Formally, let ${\mathit{w}}_{\mathit{i}},{\mathit{w}}_{\mathit{j}}$ denote the start and end indices of an entity type $\mathit{k} \in \mathit{K}$, respectively, and ${\mathit{w}}_{\mathit{p}}{,\mathit{w}}_{\mathit{q}}$ denote the start indices of an entity pair in a relation type $\mathit{l}\in \mathit{L}$, respectively; we require EPRE to identify all token pairs ${(\mathit{w}}_{\mathit{i}}{:\mathit{w}}_{\mathit{j}})\in \mathit{K}$ and ${(\mathit{w}}_{\mathit{p}}{,\mathit{w}}_{\mathit{q}})\in$L. K and L represent the entity type set and the relation type set, respectively. At last, we can combine the results of the two tasks to obtain triples.

3.2. EPRE Architecture

An example of EPRE is depicted in Figure 1. EPRE consists of an encoder module, a fused module, and a task module. The feature generation module generates the task features for the NER task and RC task. Then, we take table filling as the solution for two tasks. Next, we describe each component in detail.

3.2.1. Encoder Module

We leverage the PLM to partition the semantic representation information. Additionally, we assume that the contextual representation can encourage learning suitable information for specific tasks in the training process [31,32]. Specifically, according to the number of tasks, we divide the semantic representation information obtained by each token into three partitions: entity partition, sharing partition, and relation partition. The shared partition can be used for both tasks.

We first use the PLM to model the semantic representation ${\mathit{h}}_{\mathit{i}}$ of each token $w_i$. Each partition feature is obtained by being segmented according to the feature dimensions. If the embedding dimension of one token is $d$, then we split it into three parts according to three tasks, and each part’s dimension is $d/3$. We use the chunk function to express this operation. The features of the three parts are defined as the shared feature representation ${\mathit{h}}_{\mathit{share}}$, NER feature representation ${\mathit{h}}_{\mathit{ner}}$, and RC feature representation ${\mathit{h}}_{\mathit{rel}}$:

$${\mathit{h}}_{\mathbf{1}}{,\mathit{h}}_{\mathbf{2}}{,\dots \mathit{h}}_{\mathit{n}}=\mathrm{PLM}\left({[\mathrm{w}}_1{,\mathrm{w}}_2{,\mathrm{w}}_3{,\dots \mathrm{w}}_{\mathrm{n}}]\right),$$

(1)

$${\mathit{h}}_{\mathit{ner}}{,\mathit{h}}_{\mathit{share}}{,\mathit{h}}_{\mathit{rel}}=\mathrm{chunk}\left({\mathit{h}}_{\mathit{i}}\right).$$

(2)

3.2.2. Fused Module

We observe that the semantic information of each part may contribute differently to the task. Therefore, a gate module is used to weigh up this information in different contexts. Considering the feature of entity recognition part, we compute the feature weight of ${\mathit{h}}_{\mathit{ner}}$ and ${\mathit{h}}_{\mathit{share}}$ as follows:

$$g=\mathrm{sigmoid}\left({\mathit{W}}_{\mathbf{1}}·{\mathit{h}}_{\mathit{n}\mathit{e}\mathit{r}}+{\mathit{W}}_{\mathbf{2}}·{\mathit{h}}_{\mathit{s}\mathit{h}\mathit{a}\mathit{r}\mathit{e}}+\mathbf{b}\right),$$

(3)

where ${\mathit{W}}_1$ and ${\mathit{W}}_{\mathbf{2}}$ are trainable matrices, and $\mathit{b}$ is the corresponding offset term. Then, we merge features to form the NER-specific feature ${\mathit{h}}_{\mathit{ge}}$, as follows:

$${\mathit{h}}_{\mathit{ge}}=[g\circ {\mathit{h}}_{\mathit{n}\mathit{e}\mathit{r}}\left]\oplus \right[\left(1-g\right)\circ {\mathit{h}}_{\mathit{s}\mathit{h}\mathit{a}\mathit{r}\mathit{e}}],$$

(4)

where ∘ represents the element-wise multiplication operation, and 1 is a 1-vector with all its elements equal to 1.

The relation classification part is symmetric with the NER process. We first use a gate module to compute the weight of ${\mathit{h}}_{\mathit{rel}}$ and ${\mathit{h}}_{\mathit{share}}$, as follows:

$$g^{\prime }=\mathrm{sigmoid}\left({\mathit{W}}_{\mathbf{1}}^{\prime }·{\mathit{h}}_{\mathit{r}\mathit{e}\mathit{l}}+{\mathit{W}}_{\mathbf{2}}^{\prime }·{\mathit{h}}_{\mathit{s}\mathit{h}\mathit{a}\mathit{r}\mathit{e}}+{\mathit{b}}^{\prime }\right),$$

(5)

where ${\mathit{W}}_{\mathbf{1}}^{\prime }$ and ${\mathit{W}}_{\mathbf{2}}^{\prime }$ are trainable matrices, and ${\mathit{b}}^{\prime }$ is the corresponding offset term. Then, the global feature ${\mathit{h}}_{\mathit{gr}}$ used for the RC task is formed as follows:

$${\mathit{h}}_{\mathit{gr}}=[g^{\prime }\circ {\mathit{h}}_{\mathit{r}\mathit{e}\mathit{l}}\left]\oplus \right[\left(1-g^{\prime }\right)\circ {\mathit{h}}_{s\mathit{h}are}].$$

(6)

To illustrate how our model works, we take the sentence “The current CEO of Apple is Cook” as an example. After feeding this sentence to the encoder module, we obtain the semantic representation ${\mathit{h}}_{\mathit{i}}$ of the corresponding tokens (‘the’, ‘current’, ‘ceo’, ‘of’, ‘apple’, ‘is’, ‘cook’), and the embedding dimension of ${\mathit{h}}_{\mathit{i}}$ is d. Then, we use the chunk function to split it into three parts according to the three tasks, and each part’s dimension is d/3.

In the fused module, taking the entity extraction task as an example, since these two parts of features may have different contributions to the NER task, we use a gating mechanism to control the weights of ${\mathit{h}}_{\mathit{ner}}$ and ${\mathit{h}}_{\mathit{share}}$ features. Specifically, we use a sigmoid function as the gating function, and learn trainable parameters to obtain $g$ and $1-g$ to determine the weights of the two parts of features. Then, we concatenate the weighted features to obtain a new feature vector ${\mathit{h}}_{\mathit{ge}}$, which is used for the NER task in the subsequent task module.

3.2.3. Task Module

In the task module, we apply the features obtained from the previous module to obtain triples by table filling. Specifically, in the NER unit, we need to find the index of the tail token of the entity in the upper triangle part of the table, and mark the corresponding index position as 1. In the RE unit, we need to find the index of the head token pairs of the entity pair in the whole table range, and mark the index position as 1. The specific calculation process is as follows.

Entity recognition unit. To make the model elastic enough, we feed the entity-level feature ${\mathit{h}}_{\mathit{ge}}$ to the fully-connected layer to obtain the token-level entity representation ${\mathit{h}}^{\mathit{e}}$ for identifying each entity type $\mathit{e}$. We take the i-th token to describe the process:

$${\mathit{h}}_{\mathit{i}}^{\mathit{e}}=\mathrm{Linear}\left({\mathit{h}}_{\mathit{i}}^{\mathit{ge}}\right),$$

(7)

where ${\mathit{h}}_{\mathit{i}}^{\mathit{ge}}$ is the entity-level global feature of the i-th token, and ${\mathit{h}}_{\mathit{i}}^{\mathit{e}}$ is the feature of the i-th token for entity type classification.

Assume that the token x with position $i$ is expressed as ${\mathit{h}}_{\mathit{i}}^{\mathit{e}}$, and the token y with position $j$ is denoted as ${\mathit{h}}_{\mathit{j}}^{\mathit{e}}$. We can obtain the entity score $e_{\mathit{ij}}^{\mathit{k}}$, as follows:

$$e_{\mathit{ij}}^k={\left({\mathit{h}}_{\mathit{i}}^{\mathit{e}}\right)}^{\mathrm{T}}{\mathit{h}}_{\mathit{j}}^{\mathit{e}}.$$

(8)

The corresponding loss function is as follows:

$${\mathrm{L}}_{\mathrm{ner}}={{\displaystyle \sum }}_{k\in \mathit{K}}{{\displaystyle \sum }}_{\left(\mathit{i},\mathit{j}\right)\in {\mathit{P}}_{\mathit{k}}}\mathrm{BCELoss}({\mathit{e}}^{{\mathit{s}}_{\mathit{k}\left(\mathit{i},\mathit{j}\right)}}, e^{\widehat{s_k}\left(\mathit{i},\mathit{j}\right)}),$$

(9)

where ${\mathit{s}}_{\mathit{k}}\left(\mathit{i}, \mathit{j}\right)$ represents the predicted entity score of the entity span of type k, which is composed of tokens [i:j], $\widehat{{\mathit{s}}_{\mathit{k}}}\left(\mathit{i},\mathit{j}\right)$ represents the true entity score of the entity span of type k, which is composed of token [i:j], ${\mathit{P}}_{\mathit{k}}$ is the entity span set of type $k$, and K is the type set of entities.

Relation extraction unit. In this unit, we only predict the relation score between the starting indices of each entity pair. The computation of the RE unit is symmetrical to the NER unit. We use the i-th and p-th tokens to denote the starting index of the entities in an entity pair. To obtain the global representation, for example, we take the p-th token global representation ${\mathit{h}}_{\mathit{p}}^{\mathit{gr}}$ to the fully-connected layer to obtain the token representation ${\mathit{h}}_{\mathit{p}}^{\mathit{r}}$. Then we use the dot product to compute the relation score ${\mathit{r}}_{\mathit{iq}}^{\mathit{l}}$:

$${\mathit{h}}_{\mathit{p}}^{\mathit{r}}=\mathrm{Linear}\left({\mathit{h}}_{\mathit{p}}^{\mathit{gr}}\right),$$

(10)

$${\mathit{r}}_{\mathit{pq}}^{\mathit{l}}{=(\mathit{h}}_{\mathit{p}}^{\mathit{r}}{)}^{\mathrm{T}}{\mathit{h}}_{\mathit{q}}^{\mathit{r}},$$

(11)

where ${\mathit{r}}_{\mathit{iq}}^{\mathit{l}}$ is the relation score, which represents the probability of the relation type $\mathit{l}$ between the entity pair. When the relation score is higher than a certain threshold ${\beta }_{\mathit{r}}$, it can be interpreted as entities with start token indices $\mathit{i}$, and $\mathit{p}$ has a relation of type $\mathit{l}$. The loss function ${\mathit{L}}_{\mathit{rel}}$ is as follows:

$${\mathrm{L}}_{\mathrm{rel}}={\displaystyle \sum }_{\mathit{l}\in \mathit{L}}{\displaystyle \sum }_{\left(\mathit{i},\mathit{p}\right)\in {\mathit{L}}_{\mathit{l}}}\mathrm{BCELoss}({\mathit{e}}^{{\mathit{s}}_{\mathit{l}}\left(\mathit{i},\mathit{p}\right)},{\mathit{e}}^{\widehat{s_l}\left(\mathrm{i},\mathrm{p}\right)} ),$$

(12)

where $\mathit{l}$ represents the type of relation, $\mathit{L}$ is the set of relation types, ${\mathit{s}}_{\mathit{l}}\left(\mathit{i},\mathit{p}\right)$ is the predicted relation score and $\widehat{{\mathit{s}}_{\mathit{l}}}\left(\mathit{i},\mathit{p}\right)$ is the true relation score, and ${\mathit{L}}_{\mathit{l}}$ is the entity pairs which have the $\mathit{l}$ relation.

3.3. Training and Inference

The training objective is to minimize the loss function L, which consists of two parts: ${\mathit{L}=\mathit{L}}_{\mathit{ner}}{+\mathit{L}}_{\mathit{rel}}$. During the inference step, we predict the triple ${(\mathit{e}}_{\mathit{ij}}^{\mathit{k}}{,\mathit{r}}_{\mathit{ip}}^{\mathit{l}}{,\mathit{e}}_{\mathit{pq}}^{\mathit{k}})$ by combining the results of the NER unit and the RC unit ($\mathit{k}\in \mathit{E},\mathit{l}\in \mathit{R}$). It should satisfy the following conditions: ${\mathit{e}}_{\mathit{ij}}^{\mathit{k}} \ge { \alpha }_{\mathit{e}}$, ${\mathit{e}}_{\mathit{pq}}^{\mathit{l}} \ge { \alpha }_{\mathit{e}}$, and ${\mathit{r}}_{\mathit{ip}}^{\mathit{l}} \ge { \beta }_{\mathit{r}}$. ${\alpha }_{\mathit{e}},{\mathit{\beta }}_{\mathit{r}}$ are thresholds set to 0.

4. Experiments

4.1. Setup

We evaluate our approach on three publicly available free datasets: ADE [33], SCIERC [34], and WebNLG [35]. Different datasets have different target domains, entity types, and relation types.

Table 2. Statistics of the three datasets. We use ADE, SCIERC, and WebNLG to evaluate our model. In WebNLG, the entity type has no annotation.

Dataset	Entity Type	Relation Type	Sentence
			Train	Test	Dev
SCIERC	6	7	1861	275	551
WebNLG	None	170	5019	500	703
ADE	2	1	3845	-	427

shows the data statistics of each dataset. Since WebNLG datasets do not contain entity type information, we uniformly set their entity label to “None”. A brief introduction to the three datasets is as follows:

• ADE [33] is a dataset for adverse drug events (ADE) extraction, which contains 4272 sentences collected from the medical literature, and each sentence is labeled with three entities, namely, drug, disease, and adverse event. The difficulty of the ADE dataset is the diversity and ambiguity of adverse events, and the complexity and implication of cause–effect relationships.
• SCIERC [34] is a dataset for information extraction in the scientific domain, which contains 2687 sentences collected from computer science papers. The difficulties of the SCIERC dataset are the fine granularity of entities and relations, and the presence of a large number of abbreviations, symbols, and formulas in the text.
• WebNLG [35] is a dataset for knowledge graph to natural language generation, which contains a collection of 23,767 triads extracted from the DBpedia knowledge graph. The difficulty of the WebNLG dataset is the size and diversity of the triad collection, and the fluency and diversity of the natural language descriptions.

4.1.1. Evaluation Metrics

We use standard micro-F1 to evaluate our model performance. We follow the previous evaluation metrics. In the NER unit, if the entity type and the corresponding span of the entity are correct, the entity is correct. In the RC unit, if the relation type and two corresponding entities are correct, the triple is correct.

4.1.2. Implementation Details

We use bert-base-cased as the basic encoder of WebNLG and ADE. We use scibert-scivocab-uncased as the basic encoder of SCIERC. We retrieve 10% of ADE for validation. The initial learning rate is 5 × 10⁻⁵ for all datasets, with a batch size of 12. We use Adam as our optimizer to train our model with 100 epochs.

4.2. Comparison with Previous Work

4.2.1. Main Results

Table 3. F1 scores for WebNLG, ADE, and SCIERC datasets. The SCIERC dataset uses the SCIBERT model, and the rest use the BERT model. The data with ★ on the superscript are the results we reproduce.

Method	WebNLG		ADE		SCIERC
	NER	RE	NER	RE	NER	RE
CasRel [36]	95.5	91.8	-	-	-	-
TPLinker [13]	-	91.9	-	-	-	-
PURE [3]	-	-	-	-	66.6	35.6
SpERT [37]	-	-	89.3	79.2
PFN [15]	98.0★	93.5★	89.7★	80.3★	68.4★	37.5★
EPRE	97.5	92.9	90.0	81.9	67.6	38.3

reports the comparative evaluation of our EPRE with the existing methods. We report the average F1 scores of three runs on each dataset. Compared with the previous methods of directly using independent (PURE) or shared (CasRel/TPLinker) encoding for two tasks, our EPRE performs better. These results are expected due to the interaction between the entity and relation units. Compared with PFN, EPRE has slightly lower performance on the NER and RC tasks. We consider there are two reasons for this: (1) The model’s performance on WebNLG tends to be saturated with only 5019 training data points. We can observe that the model’s F1 scores on both tasks are above 90%, which is well above the manual level. We believe it is difficult for the model to improve substantially with these training data. (2) We compared the data predicted by the two models, and we find that our model tends to miss the selection when the entities in the sentences are connected in parallel, or when the same type of entities appear in the sentences, which we believe is also related to the quality of the dataset. After examining the data, we find that the triples labeled in the dataset are incomplete, leading to our model’s entity confusion in recognition. The general error is that the model extracts entities that also satisfy relation type, or that are juxtaposed with entities in the labeling. In relation classification, the model is prone to errors when dealing with synonymous or similar relationships (such as country and birthplace) because only one relation type is labeled in the dataset. However, our model may recognize relations that are similar or synonymous with the labeled type. In this case, the model will judge it as an error, leading to a decrease in the accuracy of our model. Our model also performs poorly in the NER task on the SCIERC dataset. We analyze the main types of errors. We observe that EPRE and PFN perform poorly when facing short entities (entities with only one word), and the F1 values are 54.9% and 56.31%, respectively. Compared to other entities, EPRE and PFN reach 67.17% and 68.39% F1 values on NER, respectively. Models have difficultly accurately identifying short entities. In addition, we analyze the impact of whether the entity is included in the triple on NER performance, and we divide the entity into In_triple and Out_triple classes. We observe that EPRE and PFN on Out_triple F1 values (46.5% and 48.6%, respectively) were significantly lower than in In_triple (75.4% and 76.2%, respectively), which may be due to less contextual information about the entity. However, we believe that our model speed advantage can make up for the small gap in NER performance. At the same time, our model performs slightly better in the RE task than PFN.

4.2.2. Computation Efficiency

Table 4. Comparison of computational efficiency. Computing time represents the model’s average time (ms) to process a sample.

Model	Computing Time
CasRel [36]	76.8
TPLinker [13]	25.6
PFN [15]	23.8
EPRE	10.2

compares calculation efficiency between CasRel, TPLinker, PFN, and our EPRE. Computing time represents the average time (ms) the model takes to process a sample. In this comparative experiment, we used the WebNLG dataset. The results of CasRel and TPLinker are produced by official implementation and default configuration. The results of PFN and EPRE are produced with RTX A5000. CasRel is limited to processing one sentence at a time, making it seriously inefficient and difficult to deploy. PFN uses a recursive structure to obtain cell-level semantic information, reducing parallelism. Overall, the speed of our EPRE is competitive.

4.2.3. Detailed Results

Detailed results on overlapping patterns and triple numbers. We compare the ability of our model to extract sentences with different numbers of triples and triples with different overlapping patterns in the WebNLG dataset. We refer to the previous work ([13,36]) and divide the sentences into three classes according to three overlapping patterns: normal, SEO (single entity overlap), and EPO (entity pair overlap). The sentences are divided into five subcategories according to the number of triples: 1, 2, 3, 4, and ≥ 5. The results are shown in Figure 2 and Figure 3. We observe that in both cases, the improvement of the F1 score of our model comes from the most challenging subclass (EPO and ≥ 5). The best results are obtained under the conditions of EPO and N ≥ 5. In these sentences, entities and relations become more complex. This proves that our model shows better advantages than other baselines in dealing with complex scenarios.

Detailed results on entity density. We compare the ability of EPRE in terms of extracting entities under different entity densities. Note that the WebNLG dataset adopts the evaluation method of partial matching; thus, for a fair comparison, we use the ratio of the number of entities to the number of words contained in the text as the calculation formula for entity density. For example, in the sentence “The current CEO of Apple is Cook”, we can observe that there are three entities (CEO, Apple, Cook) and the number of words in the sentence is 7, so the density of the sentence is 3/7. The sentences in datasets are divided into three categories: dense ≥ 10%, 10% > dense ≥ 5%, and dense < 5%. Our results are shown in Figure 4. We observe that the effect is best when the density of the entity is the highest. The lower the density of entities in the sentence, the worse the model’s performance.

4.3. Ablation Studies

We perform an ablation study on the ADE dataset to check the contribution of each element to the final performance of EPRE in NER and RC tasks. It involves three aspects: gate module, feature interaction strategy, and partition granularity.

Table 5. Results of ablation study on the ADE dataset.

Ablation	Settings	Entity			Triple
		Prec.	Rec.	F1	Prec.	Rec.	F1
Gate Module	No gate	89.10	90.30	89.34	81.30	81.30	81.30
	Add Gate	87.71	91.29	89.40	80.61	82.99	81.80
	Cat Gate	88.71	91.39	90.00	80.90	82.99	81.90
Partition Proportion	1:1:1	88.71	91.39	90.00	80.90	82.99	81.90
	2:1:2	87.81	90.70	89.20	78.52	81.90	80.21
	1:2:1	88.11	90.40	89.30	81.70	81.80	81.70
Partition Granularity	One Part	88.40	90.00	89.26	81.70	80.41	81.20
	Two Part	87.51	90.50	89.00	80.90	82.70	81.12

shows the results.

Gate module. We compare the results without using the gate module while using the add gate module and the cat gate module. Without the gate module, we directly apply the feature blocks obtained from the PLM for the NER and RC units. In the add gate module, we replace the cat operation with an add operation. Take the NER unit as an example, that ${\mathit{h}}_{\mathit{ge}}=[g\circ {\mathit{h}}_{\mathit{ner}}\left]+\right[\left(1-g\right)\circ {\mathit{h}}_{\mathit{share}})$. Compared with the case without a gate, the experiment shows that the gate module is effective, and the cat gate module is more effective than the add gate module. This may be because the cat module can maintain the independence of each part of the feature than add a module.

Feature interaction strategy. We compare three feature interaction approaches. In Table 4, the proportional value represents the feature proportion of the NER, share, and RE units. For example, 1:1:1 means that the three-part feature dimension is equal to d/3, where d is the dimension of the PLM. We compare the results under three partition proportions, and the effect is best when the proportion is equal. When we assign a large proportion of the features to a specific task, the proportion of shared features shrinks, and the model performance decreases by 0.8% and 1.7% on the NER and RE tasks, respectively. Compared to the model performance in the three-split case, performance is best when the three-part feature dimensions have equal sizes, which indicates the importance of shared features and dependence features in both tasks.

Partition granularity. We try three partition granularity. One partition means directly using the pretraining features to complete two subtasks, and the features of two specific tasks are the same. Different from the former, two partitions divide the output of the pretraining model into two partitions, one for the NER task and the other for the RC task. The features of two specific tasks are not shared. Our model has three parts. As can be seen, the approach of using independent task-specific features is better than the approach of taking shared features. We expect the features required for each specific task to be different, but the sharing method cannot separate the features for two specific tasks. We observe that the performance is significantly better when considering both independent and shared features for each specific task simultaneously.

5. Conclusions

In this paper, we propose a simple and effective feature partition method to ensure the feature independence and interaction of NER and RC tasks. We simply divide the results of the pretraining model into the RC-specific feature, NER-specific feature, and shared feature. This accelerates the whole process. Then, we use the gate module to obtain the NER feature and RC feature, ensuring feature interaction and independence between tasks. We conduct extensive experiments on three datasets to verify the effectiveness of our EPRE. We will further investigate what the PLM has learned so as to make better use of feature information. Furthermore, an adaptive threshold for each relation type and entity type can alleviate the long-tail problem. However, it needs further investigation.