LAI: Label Annotation Interaction-Based Representation Enhancement for End to End Relation Extraction

Abstract

End-to-end relation extraction (E2ERE) generally performs named entity recognition and relation extraction either simultaneously or sequentially. While numerous studies on E2ERE have centered on enhancing span representations to improve model performance, challenges remain due to the gaps between subtasks (named entity recognition and relation extraction) and the modeling discrepancies between entities and relations. In this paper, we propose a novel Label Annotation Interaction-based representation enhancement method for E2ERE, which institutes a two-phase semantic interaction to augment representations. Specifically, we firstly feed label annotations that are easy to manually annotate into a language model, and conduct the first-round interaction between three types of tokens with a partial attention mechanism; Then we construct a latent multi-view graph to capture various possible links between label and entity (pair) nodes, facilitating the second-round interaction between entities and labels. A series of comparative experiments with methods of various transformer-based architectures currently in use show that LAI-Net can maintain performance on par with the current SOTA in terms of NER task, and achieves significant improvements over existing SOTA models in terms of RE task.

1. Introduction

As a core information extraction (IE) task, end-to-end relation extraction (E2ERE) can be split into named entity recognition (NER) subtask for entity identification and relation extraction (RE) subtask for capturing inter-entity relations from plain texts. As postulated by [1], E2ERE is challenging due to its difficulty in capturing affluent correlations between entities and relations. IE research has traditionally converted NER and RE tasks into span-based tasks [1,2,3,4,5]. Though these methodologies have incrementally advanced model performance from various perspectives, they are still impeded by two pivotal limitations: overdetached subtasks leads to insufficient information exchange between entitiy and relation, and the disparity in modeling strategies between entity and relation result in semantic gaps. In this paper, we mainly focus on enhancing semantic interaction during modeling processes to achieve enhanced span representation for E2ERE.

To address the challenges above, prevailing research mainly focuses on reorganizing the input or intermediate network layers of pre-trained language models (PLMs), attempting to enhance the semantic information of representation through the integration of specialized symbols or extrinsic prior knowledge. We roughly divide this into three types (as shown in Figure 1): The vanilla-based method is a straightforward approach to acquiring a given span representation by feeding raw text tokens series into a pre-trained encoder. The marker-based method inserts independent entity markers like $\left[\mathrm{M}\right],[\setminus \mathrm{M}]$ amidst text tokens to highlight the presence of entities, aiming at attracting more model attention. The enumerate-based method enumerates all possible entity candidates from plain text and then concatenates them after text tokens, and entity tokens share the position IDs with candidates as well. For these methods above, the distinguished LSTM-CRF [6] is a typical vanilla-based sequence labeling method, and PURE [7] is a combination of the marker-based method and the enumeration-based method, which adopts the marker-based method during the NER phase and the enumerate-based method for the RE phase, achieving SOTA performance. PL-Marker [8] is a typical enumeration-based method that promoted the SOTA further.

In addition to the above three methods, what we develop in this paper can be classified as the fourth class, named the annotate & enumeration-based method. It is an novel semantic enhancement approach with external knowledge, inspired by external knowledge-based approaches [3,9,10]. We argue that a thorough comprehension of label semantics will significantly enhance the IE model abilities; this serves as the premise for our work.

As shown in Figure 1, our principal improvement over preceding methods lies in the insertion of external prior knowledge (i.e., label annotations) into the PLM input sequence, aiming to leverage PLM’s internal layers to enhance semantic interaction between label and text. This represents the first-round semantic interaction in the LAI-Net framework.

Unlike the lexicon-adapter-based methods [11,12] and lattice-based methods [13,14,15], we manually expand the label information and embed it between text tokens and enumerated candidates then feed the series into a pre-trained model. We further enhance the representation by combining word vectors using downstream neural networks. Formally, the augmented representations derived from the aforementioned methods can be summarized as follows:

$$\begin{array}{cc}\hfill & {\mathbf{h}}_{\mathrm{span}}^V=f\left({\mathbf{h}}_{\mathrm{span}}^s;{\mathbf{h}}_{\mathrm{span}}^e\right)\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill & {\mathbf{h}}_{\mathrm{span}}^M=f\left({\mathbf{h}}_M;{\mathbf{h}}_{\setminus M}\right)\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill & {\mathbf{h}}_{\mathrm{span}}^E=f\left({\mathbf{h}}_{\mathrm{span}}^s;{\mathbf{h}}_{\mathrm{span}}^e;{\mathbf{h}}_M;{\mathbf{h}}_{\setminus M}\right)\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill & {\mathbf{h}}_{\mathrm{span}}^{A\&E}=f\left({\mathbf{h}}_{\mathrm{span}}^s;{\mathbf{h}}_{\mathrm{span}}^e;{\mathbf{h}}_M;{\mathbf{h}}_{\setminus M};{\mathbf{h}}_a^s;{\mathbf{h}}_a^e\right)\hfill \end{array}$$

(4)

where $f(·)$ is an tensor operator, equipped to execute series of operations including tensor addition, tensor multiplication, tensor concatenation, etc., or even could be a neural network. And superscripts $s,e$ denote the commencement and termination tokens of a span or annotation, while the subscripts $\mathrm{span}, M, \setminus M$ represent the disparate token types.

Based on the first-round semantic interaction, we further meticulously crafted with selectively designed downstream network layers. The main innovations and improvements lie in the following aspects:

• We explore a novel two-round semantic interaction approach for enhancing span representations, wherein the first-round interaction reorganizes the PLM input with annotated information in what we term an annotation & enumeration-based method, and the second-round interaction employs a graph convolutional network (GCN) built atop Gaussian graph generator modules to facilitate label semantic fusion.
• We conduct a coarse screening with a entity candidate filter to eliminate out spans that are clearly not real entities, which also promotes the saving of computing resources.
• Experiments demonstrate that our method, while slightly lagging behind current SOTA in NER performance, takes the lead in the downstream RE task, surpassing the current SOTA performance.

2. Related Work

Graph Convolutional Networks. Recently, academic interest in span representation enhancement has surged, providing a substantial impetus to E2ERE. Traditional neural-network-based methods often ignore non-local and non-sequential context information from input text [16], which is precisely the area where GCNs [17,18] excel. GCNs, which our discussion centers on, have been widely used to model the interaction between entities and relations in the text, and have been demonstrated as a typical and effective approach. GCN-based approaches typically leverage a predefined graph structure, constructed from plain text, to facilitate information propagation among nodes, thus capturing text’s non-linear structure and enhancing NER and RE models’ capabilities to capture both global graph structures and representations of nodes and edges. GCN-based methods [2,5,19,20,21,22] mostly utilize different approaches to define nodes (sentences, words, tokens, spans, labels, etc.), and these nodes can be connected by syntactic dependency edges [23,24,25,26], re-occurrence edges [16], co-occurrence edges [27,28], adjacent word edges [16,26,29], adjacent sentence edges [26], etc. Convolution operations are performed on the graph to facilitate the flow of information between nodes, which enables nodes to efficiently acquire both local and global information. This further refines node representations and downstream network performance.

Traditional Entities and Relation Extraction Methods. Research on traditional entities and relation extraction methods has undergone several stages of evolution, progressing from early rule-based approaches [30,31] to classical machine learning techniques [32,33], neural-network-based approaches [34,35,36,37], and, more recently, to PLM-based methods [7,8,38,39,40,41,42,43,44,45,46,47] that have gained popularity in the past few years. Throughout these developments, system performance has continuously improved. Broadly, existing methods can be categorized into two main paradigms: joint extraction and pipeline extraction techniques. Joint extraction approaches integrate the two subtasks (NER and RE) within a unified framework, often adopting a multi-task learning strategy, aim to perform NER and RE in a single model, thereby mitigating the issue of error propagation from the upstream NER task to the downstream RE task, and ultimately enhancing overall performance. In contrast, pipeline extraction methods treat NER and RE as two separate stages. Specifically, a dedicated NER model is first trained to identify and extract relevant entities, and the resulting entity information is then passed to a separate RE model, which infers the relationships between entity pairs.

Prompt-based Entities and Relation Extraction Methods. The advent of GPT-2 [48] catalyzed the emergence of prompt-based algorithms, wherein early approaches primarily concentrated on identifying optimal prompts to be fed into PLMs [49,50]. With the subsequent development of chat-based generative models, the focus of RE research has increasingly shifted toward leveraging LLMs for classification and information extraction tasks. LLMs trained on massive corpora demonstrate the capability to make effective inferences from only a limited number of examples, thereby rendering them promising candidates for few-shot and low-resource scenarios. Consequently, the potential of LLMs in such settings has attracted significant research attention. Some studies report that LLMs can perform relation extraction effectively under few-shot conditions [51,52,53,54,55], which is something that traditional methods struggle to achieve.

Based on these previous studies, we integrate the powerful capabilities of PLM and GCNs to design and develop a hybrid architecture, aiming to enhance the performance of E2ERE through two-phase semantic interaction.

3. Datasets and Preprocessing

We selected two standard corpus (ACE05, SciERC) in terms of the E2ERE task.

(1) ACE05 (https://catalog.ldc.upenn.edu/LDC2006T06) (accessed on 21 May 2025) is collected from a variety of domains (such as newswire and online forums). It includes 7 entity types and 6 relation types between entities. For data processing, we use the same entity and relation types, data splits (https://github.com/tticoin/LSTM-ER/tree/master/data/ace2005/split) (accessed on 21 May 2025), and pre-processing as [56,57] (351 training, 80 development and 80 testing).

(2) SciERC [58] is a scientific-oriented dataset that is built from 12 AI conference/workshop proceedings in four AI communities, and includes 7 entity types and 7 relation types.

(3) ADE [59] consists of 4272 sentences and 6821 relations extracted from medical reports.

In terms of the experiment, for ACE05 and SciERC, we run our model 10 times with different random seeds, and report averaged results of all the runs. And for ADE, we adopt 10 fold cross-validation, respectively, and run each fold 10 times and report averaged results of all the runs.

4. Methodology

4.1. Task Definition

Given a sentence formularized as $S=\{w_1,w_2,\cdots ,w_m\}$$=\{t_1,t_2,\cdots ,t_n\}$ with m words (or n tokens, $n\ge m$ naturally), the goal of the E2ERE task is to recognize a set of entity spans and relationships of entity pairs automatically, which can be written as $(e_i,r_{i,j},e_j)\in \mathcal{T}$. Every entity e, which attaches a specific type (e.g., person (PER), organization (ORG)), is a sequence of tokens. Every relation $r_{i,j}$ represents the relationship between $e_i$ and $e_j$, and also attaches a specific type (e.g., organization affiliation relation (ORG-AFF)) (both entity type sample and relation type sample mentioned above are quoted from ACE05). Formally, we define the set of possible entity types and relation types as $\mathcal{E}$, $\mathcal{R}$, respectively.

4.2. First-Round Semantic Interaction

During data processing, we concatenate text tokens, annotation tokens, and marker tokens sequentially to formulate a unified input sequence (the bottom element of Figure 2). The PLM encoder conducts the first-round semantic interaction, then outputs the encoded representation, which is a semantic amalgamation of the three types of tokens.

• Text Tokens. Our approach breaks down the words from raw text into text token sequences as part of the model input.
• Annotation Tokens. Inspired by [3,60], we augment semantic information by manually annotating the entity (or relation) abbreviated label both in the NER and the RE phase. For example, the abbreviated entity type GPE can be annotated as “geography political entity”, a fully-semantic unbroken phrase. Correspondingly, the abbreviated relation type ORG-AFF can be annotated as “organization affiliation”. Each label is manually expanded to enrich semantic content and then tokenized into annotation tokens (highlighted by red rectangle in bottom of Figure 2), which are appended to the text tokens sequentially.
• Marker Tokens. We enumerate all potential consecutive token sequences (i.e., entity candidates) not exceeding a predefined limitation of length c (with $c\le n$) within a sentence, labeling each with an entity type. If $c=2$, as shown in Figure 2, the set of all the possible spans from sentence “chalabi is the founder and leader of the iraqi national congress.” can be written as $\mathsf{\Psi }=\{$“chalabi”, “chalabi is”, “is”, “is the”, “the founder”, “founder”, “founder and”, “and”, “and leader”$\cdots \}$. The i-th span can be written as ${\mathrm{span}}_i=[{\mathrm{span}}_i^s,{\mathrm{span}}_i^e]$, where ${\mathrm{span}}_i^s$ and ${\mathrm{span}}_e^e$ are indicative of start and end position IDs of entity span, respectively. Therefore, the entity candidate series can also be written as $\mathsf{\Psi }=\left\{\right[0,0],[0,1],[1,1],[1,2],[2,2],[2,3],[3,3],[3,4],[4,$$4],[4,5\left]\right\}$ given position ID perspective. Thus, the number of candidates for a sentence with m words: $\left|\mathsf{\Psi }\right|=m·c+(c-c^2)/2$.

In model input, we define a start marker (M) and an end marker ($\setminus M$), which form a pair of marker tokens, respectively, representing the start and end of an entity span and are appended subsequent to annotation tokens. The start and end markers share the same position embedding with the corresponding span’s start token and end token, respectively, while keeping the position ID of the original text tokens unchanged. From an PLM encoder input perspective, every marker is a token element of token series, called a marker token. As shown in Figure 2, entity chalabi is highlighted by the light yellow bordered square, and its corresponding markers are denoted by the colorful non-bordered square with line frame differing in various entity labels (white means non-entity). In conclusion, the complete input sequence can be represented as Equation (5), where $a_i$ is a token broken from label annotation, and ${\mathrm{M}}_i^s,{\mathrm{M}}_i^e$ represent the start and end marker tokens, respectively.

$$\begin{array}{c}\hfill \tilde{S}=\{\left[\mathrm{CLS}\right],t_0,\cdots ,t_{n-1},\left[\mathrm{SEP}\right],a_0,\cdots ,a_{N-1},\left[\mathrm{SEP}\right],{\mathrm{M}}_0^s,\cdots ,{\mathrm{M}}_{\left|\mathsf{\Psi }\right|-1}^s,{\mathrm{M}}_0^e,\cdots ,{\mathrm{M}}_{\left|\mathsf{\Psi }\right|-1}^e\}\end{array}$$

(5)

• Partial Attention. Although special tokens such as [CLS] and [SEP] serve to isolate different types of tokens, there still exists semantic interference among them. The straightforward blend of annotation tokens and marker tokens with text tokens may disrupt semantic consistency of the raw text. To mitigate this, we devise a partial attention mechanism, allowing selective semantic influence among the different types of token. This mechanism can effectively control the information flow (could be regarded as a kind of visibility) between different tokens, by adjusting the value of elements of the attention mechanism mask matrix. It suppresses the information interaction among tokens that are mutually invisible while enhancing the information interaction among tokens that are mutually visible. Experimental results show that partial attention effectively improves model performance. See Appendix B for more detailed information about partial attention.

4.3. Second-Round Semantic Interaction

To refine semantic integration, we introduce second-round semantic interaction, employing a semantic integrator that explicitly model interactions between entity candidates and label annotations. The semantic integrator consists of multiple GCN layers with randomly generated adjacency matrix, treats both entity spans and label annotations as nodes, and establishes connections between nodes through the construction of a graph $\mathcal{G}$ so that the interactions between nodes can be explicitly modeled. A GCN typically necessitates a manually predefined and fixed adjacency matrix to depict the inter-nodes connections. The fixed adjacency matrix fixes the perspective from which the model understands the semantics. However, it is noteworthy that the inter-nodes connection cannot be predetermined accurately when considering our task. Otherwise, our task would be meaningless. Therefore, inspired by [22,61], we forgo a static adjacency matrix in favor of a multi-view graph, called Gaussian Graph Generator-based Graph Convolutional Networks (${\mathrm{G}}^4\mathrm{CN}$). ${\mathrm{G}}^4\mathrm{CN}$ is unlike vanilla GCN, which takes a fixed adjacency matrix; it gives up fixing the edge weights and sets the edge weights via trainable neural networks during the network initial stage, which allows the model to assimilate semantic contexts from multiple perspectives.

First, we attach every node with a Gaussian distribution $\mathcal{N}(\mu ,\sigma )$, where both $\mu ,\sigma$ are generated by trainable neural networks, formulated as in Equations (6) and (7), and set the activation function $\varphi$ as the SoftPlus function, as the standard deviation of Gaussian distribution is bounded on $(0,+\infty )$. Then, we simulate edge weight by computing the KL divergence $w_{ij}^e$ between two Gaussian distributions of node, where $w_{ij}^e=\mathrm{KL}\left(\mathcal{N}({\mu }_i,{\sigma }_i)\parallel \mathcal{N}({\mu }_j,{\sigma }_j)\right)$. We can obtain a number of Gaussian distributions for the multi-view graph; each ${\mathcal{N}}_i$ here corresponds to a node representation $v_i$. Thus, a vanilla GCN’s value-fixed adjacency matrix $\mathbf{A}={\left(a_{ij}^e\right)}_{k\times k}$ can be modified to be a ${\mathrm{G}}^4\mathrm{CN}$’s value-varied adjacency matrix ${\mathbf{A}}^{\prime }={\left(w_{ij}^e\right)}_{k\times k}$, where k is the number of nodes. The whole process can be formulated as Equation (8), where ${\mathbf{H}}_{\mathrm{span}},{\mathbf{H}}_{\mathrm{anno}}^{\mathrm{ent}}$ are matrixes formed by concatenating multiple nodes (span or annotation) in the constructed heterogeneous graph $\mathcal{G}$, and $\mathbf{GCN}(·)$ is the vanilla GCN (see detailed formulas in Appendix A).

$$\begin{array}{cc}\hfill \{{\mu }_i^1,& {\mu }_i^2,{\mu }_i^3,\cdots ,{\mu }_i^N\}=g_{\theta }\left(v_i\right)\hfill \end{array}$$

(6)

$$\begin{array}{cc}\hfill \{{\sigma }_i^1,& {\sigma }_i^2,{\sigma }_i^3,\cdots ,{\sigma }_i^N\}=\varphi \left({g^{\prime }}_{\theta }\left(v_i\right)\right)\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill {\tilde{\mathbf{H}}}_{\mathrm{span}}& ={\displaystyle \frac{1}{2}}\left({\mathbf{H}}_{\mathrm{span}}+\mathbf{GCN}\left({{\mathbf{A}}^{\prime }}_{\mathrm{ner}},\left[{\mathbf{H}}_{\mathrm{span}};{\mathbf{H}}_{\mathrm{anno}}^{\mathrm{ent}}\right]\right)\right)\hfill \end{array}$$

(8)

4.4. Name Entity Recognition

Span and Annotation Representation. We extract the contextualized representations $\mathbf{h}$ for individual token s from PLM output, and naturally obtain the involved mathematical formulas for spans and annotations as Equations (9) and (10), where ${\mathbf{h}}_{\mathrm{anno}}\in {\mathbb{R}}^d,{\mathbf{h}}_{\mathrm{span}}\in {\mathbb{R}}^d$. ${\mathbf{h}}_a^s,{\mathbf{h}}_a^e$ is the embedding of the first and last token of a certain type of label annotation, respectively. ${\mathbf{h}}_t^s,{\mathbf{h}}_t^e$ is the embedding of the first token and last token of an entity candidate, respectively, and ${\mathbf{h}}_M^s,{\mathbf{h}}_M^e$ indicates the embedding of the start and end token of the marker, respectively. The linear layer $\mathbf{FC}$ is used to harmonize dimensional space.

$$\begin{array}{cc}\hfill {\mathbf{h}}_{\mathrm{anno}}& ={\mathbf{FC}}_a\left([{\mathbf{h}}_a^s;{\mathbf{h}}_a^e]\right)\hfill \end{array}$$

(9)

$$\begin{array}{cc}\hfill {\mathbf{h}}_{\mathrm{span}}& ={\mathbf{FC}}_{\mathrm{span}}\left([{\mathbf{h}}_{\mathrm{span}}^s;{\mathbf{h}}_{\mathrm{span}}^e;{\mathbf{h}}_M^s;{\mathbf{h}}_M^e]\right)\hfill \end{array}$$

(10)

Entity Candidates Filter. Before the development of the entity candidates filter module, we attempted to train the model, but it ended in failure. The reasons can be summarized as follows: (1) During the initialization phase of training, model parameters are randomly assigned values, leading to lots of candidate entities being randomly predicted as real entities in the early phase. This further causes the adjacency matrix of the graph neural network to become excessively large in scale, resulting in extremely slow network training and significant resource consumption. (2) Among the numerous candidate entities enumerated, positive sample entities are extremely few while negative sample entities are abundant. This easily induces the model to tend towards classifying all entities as non-entities to achieve the minimum loss value, which is not the desired outcome. To overcome the aforementioned issues, we devise a binary classifier that acts as a entity filter, performing coarse screening for all enumerated entities by discarding non-genuine entities, thus optimizing subsequent predictions.

As for loss function, the primary aim of the entity filter is to maximize the likelihood function, which drives us follow [2] to adopt the likelihood loss function as per Equation (11), where ${\mathsf{\Psi }}_g\subseteq \mathsf{\Psi }$ indicates a set of real entity spans. In addition to intuitive time consumption optimization, experimental results indicate that the entity filter successfully alleviates model weakening engendered by negative samples and enhances overall performance.

• Span Classifier. We conduct span representations classification through a linear classifier, utilizing cross-entropy loss to direct the learning process. The combined loss function ${\mathcal{L}}_{\mathrm{ner}}={\mathcal{L}}_{\mathrm{filter}}+{\mathcal{L}}_{\mathrm{span}}$ is optimized during training, using ${\mathcal{L}}_{\mathrm{span}}$ from Equations (11) and (12), with dropout layers for regularization.

In addition, Ye D et al.[8] had proved that packing a series of related spans into a training instance can promote the NER model performance, which naturally prompted us to follow the effective measures when reorganizing the input.

$$\begin{array}{cc}\hfill {\mathcal{L}}_{\mathrm{filter}}& =-{\displaystyle \frac{1}{\left|\mathsf{\Psi }\right|}}\sum _{i=1}^{\left|\mathsf{\Psi }\right|}logP({\mathrm{span}}_i\in {\mathsf{\Psi }}_g|{\mathrm{span}}_i\in \mathsf{\Psi })\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill {\mathcal{L}}_{\mathrm{span}}& =-{\displaystyle \frac{1}{|{\mathsf{\Psi }}_g|}}{\sum }_{{\mathsf{\Psi }}_g}log{P}_{\mathrm{span}}\hfill \end{array}$$

(12)

4.5. Relation Extraction

Subject marker. In the RE phase, we design our annotate & enumeration-based method (as demonstrated in Figure 1) to acquire the enhanced representation. Concretely, we adopt the marker-based approach (shown in Figure 1), and insert a pair of subject markers, called solid markers by [8], into the left and right of the subject entity, and the enumerated object candidate spans, following on the heels of annotation tokens to extract relations involving the subject entity.

• Entity pairs representation. We match subject and object representations up pairwise to obtain a series of entity pairs (${\mathbf{h}}_{\mathrm{pair}}=[{\mathbf{h}}_{\mathrm{subj}};{\mathbf{h}}_{\mathrm{obj}}]$). The label semantic confused pair representation formulas can be written as in Equation (13), where ${\mathbf{H}}_{\mathrm{pair}},{\mathbf{H}}_{\mathrm{anno}}^{\mathrm{rel}}$ are matrixes formed by concatenating entity pair or relation label annotation representations.

$$\begin{array}{c}\hfill {\tilde{\mathbf{H}}}_{\mathrm{pair}}={\displaystyle \frac{1}{2}}\left({\mathbf{H}}_{\mathrm{pair}}+\mathbf{GCN}\left({{\mathbf{A}}^{\prime }}_{\mathrm{re}},\left[{\mathbf{H}}_{\mathrm{pair}};{\mathbf{H}}_{\mathrm{anno}}^{\mathrm{rel}}\right]\right)\right)\end{array}$$

(13)

The loss function of the RE phase is the cross-entropy loss.

5. Experiments

Datasets. We utilize three widely used standard corpora: (1) ACE05 spans various domains (newswire, online forums), and contains seven entity types and six relation types between entities. (2) SciERC [58] is a scientific dataset built from AI conference/workshop proceedings across four communities. It includes seven entity types and seven relation types. (3) ADE [59] consists of 4272 sentences and 6821 relations extracted from medical reports.

Metrics. The model with the best F1 performance on the test set will be selected on a fixed number of epochs. Both micro and macro average metrics are used to evaluate the model performance, the former for ACE05/SciERC and the latter for ADE. For the NER task, an entity prediction is correct if and only if its type and boundaries both match with those of a gold entity. For the RE task, a relation prediction is considered correct if its relation type and the boundaries of the two entities match with those in the gold data. We also report the strict relation F1 (denoted RE+), where a relation prediction is considered correct if its relation type as well as the boundaries and types of the two entities all match with those in the gold data. We show detailed experimental settings in Appendix B.

5.1. Main Results

To ensure the credibility of experimental results, we conducted multiple experiments with different random seeds on all datasets, and the stability of runs are presented in Appendix B.9.

5.1.1. Results Against Horizontal Comparison

Table 1. The main overall results, with our new SOTA results highlighted with bold and suboptimal performance highlighted with underline. In addition, $\star ,\circ ,•$, respectively, indicate decoder-only, encoder–decoder, and encoder-only framework. Bold font indicates the optimal performance, while underline indicates the suboptimal performance.

	Models	Backbone	NER				RE				RE+
			P	R	F1		P	R	F1		P	R	F1
ACE05	SPAN (2020) [38]	• Bert-base	89.32	89.86	89.59		-	-	-		71.22	60.19	65.24
	UniRE (2021) [39]	• Bert-base	88.80	88.90	88.80		-	-	-		67.10	61.80	64.30
	PURE (2021) [7]	• Bert-base	-	-	90.20		-	-	67.70		-	-	64.60
	PL-Marker (2022) [8]	• Bert-base	-	-	89.70		-	-	68.80		-	-	66.30
	ASP (2022) [62]	∘ T5-base	-	-	90.70		-	-	71.10		-	-	68.60
	HIORE (2023) [40]	• Bert-base	-	-	89.60		-	-	-		-	-	65.80
	HGERE (2023) [41]	• Bert-base	-	-	89.60		-	-	-		-	-	65.80
	Mirror (2023) [42]	∘ DeBERTa-v3	-	-	86.72		-	-	-		-	-	64.88
	GPT-NER (2023) [63]	⋆ GPT3	72.77	75.51	73.59		-	-	-		-	-	-
	GPT-RE (2023) [64]	⋆ GPT3	-	-	-		-	-	-		-	-	68.73
	ChatGPT (2023) [65]	⋆ ChatGPT	-	-	-		-	-	-		-	-	40.50
	SET (2023) [43]	∘ T5-large	-	-	-		-	-	-		-	-	65.90
	BR (2023) [44]	• Albert	-	-	90.80		-	-	-		-	-	66.00
	ATG (2024) [45]	∘ DeBERTa-v3	-	-	90.10		-	-	68.70		-	-	66.20
	BiDArtER (2024) [46]	• Albert	-	-	89.80		-	-	-		-	-	68.40
	LAI-Net (Ours)	• Bert-base	90.28	90.60	90.44		73.80	70.42	72.06		71.96	68.67	70.27
SciERC	DyGIE++ (2019) [66]	• SciBert	-	-	67.50		-	-	-		-	-	48.40
	Spert (2019) [67]	• SciBert	70.87	69.79	70.33		-	-	-		53.40	48.54	50.84
	UniRE (2021) [39]	• SciBert	65.80	71.10	68.40						37.30	36.60	36.90
	PURE (2021) [7]	• SciBert	-	-	68.20		-	-	50.10		-	-	36.70
	PL-Marker (2022) [8]	• SciBert	-	-	69.90		-	-	52.00		-	-	40.60
	HIORE (2023) [40]	• SciBert	-	-	68.20		-	-	-		-	-	38.30
	Mirror (2023) [42]	∘ DeBERTa-v3	-	-	-		-	-	-		-	-	36.66
	ChatGPT (2023) [65]	⋆ ChatGPT	-	-	-		-	-	-		-	-	25.90
	InstructUIE (2023) [47]	⋆ FlanT5-11B	-	-	-		-	-	-		-	-	45.15
	GPT-RE (2023) [64]	⋆ GPT3	-	-	-		-	-	-		-	-	69.00
	SET (2023) [43]	∘ T5-large	-	-	-		-	-	-		-	-	35.90
	ATG (2024) [45]	• SciBert	-	-	69.70		-	-	51.10		-	-	38.60
	BiDArtER (2024) [46]	• SciBert	-	-	69.40		-	-	-		-	-	39.90
	LAI-Net (Ours)	• SciBert	70.04	69.89	69.94		65.56	68.48	66.99		59.84	62.01	60.88
ADE	Spert (2019) [67]	• Bert-base	89.02	88.87	88.94		-	-	-		78.09	80.43	79.24
	Table-Sequence (2020) [68]	• Bert-base	-	-	89.70						-	-	80.10
	SPAN (2020) [38]	• Bert-base	89.88	91.32	90.59		-	-	-		79.56	81.93	80.73
	LAI-Net (Ours)	• Bert-base	89.78	91.24	90.49		80.48	83.79	82.09		79.37	83.28	81.25

presents a horizontal comparison with baselines, focusing on comparing excellent models developed in recent years, including several previous SOTA models (details are in Appendix B.1). In terms of the NER task, our method is on par with the current SOTA, with the discrepancy across three datasets ranging from 0.39% to 0.1%, which achieves suboptimal performance. Starting from a slightly lower NER performance compared to the SOTA, our method still achieves a performance gain of 2–10% on relation F1 and strict relation F1, consistently outperforming all selected baselines, even with the error propagation between the NER and the RE task. For example, on ACE05, our method exhibits a 0.24% disadvantage in the NER phase, but takes the lead in the downstream RE phase, surpassing the current SOTA by 0.41%, setting a new SOTA record. Similarly, our method, despite having a 0.39% and 0.1% disadvantage in NER performance, respectively, on SciERC and ADE, achievew performance improvements of 10% and 0.52%. All these improvements demonstrate that two rounds of semantic interactions indeed further utilize the predicted entities from the NER phase, significantly improving the performance of relation recognition while slightly compromising NER performance.

5.1.2. Results Against Significant Hyperparameters

Table 2. The ablation F1 result against number of GCN layer, number of attention head, and whether there is an embed entity filter or not. Bold font indicates the optimal performance.

Task		Number of GCN Layer							Number of Attention Head						Entity Filter
		0	1	2	3	4	5		1	2	3	4	6		w	w/o
ACE05	NER	90.23	89.95	90.44	89.92	90.03	89.94		90.16	90.21	90.30	90.44	90.24		90.44	88.70
	RE	68.05	68.38	72.06	69.37	69.26	69.53		71.75	72.06	72.16	71.84	71.37		-	-
	RE+	65.61	65.75	70.27	66.93	66.70	66.88		69.54	70.27	70.03	69.86	69.21		-	-
ADE	NER	90.49	90.18	90.23	90.32	90.28	90.17		-	-	-	-	-		90.49	89.92
	RE	80.99	82.09	81.64	80.71	81.25	81.04		81.42	81.79	82.09	81.21	80.94		-	-
	RE+	80.99	81.25	80.81	80.39	80.95	80.63		80.83	81.02	81.25	80.88	80.55		-	-
SciERC	NER	69.40	69.94	69.47	69.17	69.31	69.40		69.42	69.76	69.32	69.94	69.23		69.94	69.76
	RE	66.08	66.27	66.99	66.43	66.35	66.28		65.80	65.66	65.62	66.99	64.48		-	-
	RE+	60.49	60.57	60.88	59.91	60.82	60.06		60.24	60.15	60.61	60.88	59.56		-	-

delineates the impact of varying significant hyperparameters of ${\mathrm{G}}^4\mathrm{CN}$ on the performance of LAI-Net. For the number of GCN layers, we consider a range from 0 to 5, with 0 indicating the absence of GCN for semantic interaction and more layers corresponding to increased communication times among nodes within the graph. As for the number of attention heads, we opt for values of 1, 2, 3, 4, and 6. The deliberate exclusion of the value 5 is attributed to the fact that the encoder’s hidden dimension is not divisible by 5, a constraint inherent to the multi-head attention mechanism.

The table permits the intuitive observations that (1) an increase in the GCN layers number does not linearly translate to enhanced performance; an excessive GCN layers number can exert a deleterious effect on the model, with the more optimal layers number identified as either 1 or 2; (2) concerning the number of attention heads, the more optimal solution exhibits some variation across different datasets, yet it is unequivocally clear that neither an excessively high (e.g., 6) nor a disproportionately low (e.g., 1) number of attention heads can fully capitalize on the GCN’s capabilities.

5.2. Inference Speed

There is justification to assess whether the adoption of GCNs to enhance F1 performance has made trade-offs respect to inference efficiency, for the reason that GCNs have certain efficiency disadvantages typically. Hence, we conducted a comparison between PL-Marker and LAI-Net in terms of inference speed. The experiment was evaluated on a NVIDIA GeForce RTX 3090 24 GB GPU with a evaluate batch size of 16. We used the Bert-base model for ACE05 and SciBert for SciERC.

As shown in

Table 3. The inference speed comparison results. Bold font indicates the optimal performance.

Task		Metric	PL-Marker	LAI-Net
ACE05	NER	F1	89.70	90.44
		Speed (sent/s)	62.94	35.10 (−44.23%)
	RE	F1	66.30	70.27
		Speed (sent/s)	93.14	43.00 (−53.83%)
SciERC	NER	F1	69.90	69.94
		Speed (sent/s)	54.13	52.17 (−3.62%)
	RE	F1	40.60	60.88
		Speed (sent/s)	93.29	39.57 (−57.58%)

, we indeed sacrificed varying degrees of inference efficiency in exchange for varying degrees of improvement in F1 performance. Overall, the greater the sacrifice, the greater the benefit.

5.3. Ablation Study

5.3.1. Ablations Against Entity Filter

Considering the binomial surge in candidate entity quantity accompanying increased entity length in the NER phase, the relatively paltry positive examples can be easily overwhelmed by a vast array of negative samples, thereby instilling the network a strong propensity to categorize all samples as negative, leading to difficulty in accurately identifying genuine entities. To mitigate this, LAI-Net incorporates a deliberately inserted filter prior to the entity classifier to preliminarily sieve out spans that are clearly non-entity. To validate whether said filter genuinely facilitates the NER process, we devised associated ablation experiments. As depicted in the two rightmost columns of Table 2, the presence of the filter effectively improves NER performance, with advantages most conspicuous on the ACE05 dataset (surpassing no-filter models by 1.74% in terms of F1-scores), and improvements of varying degrees are also observed on other datasets.

5.3.2. Ablation Against Two Rounds of Interaction

We conducted an ablation experiment specifically targeting the two-round semantic interaction. Drawing upon the outcomes of the experiment, we can directly evaluate the extent to which our devised dual semantic interaction genuinely augments the model’s performance. What should be noted is that the second-round interaction is built upon the annotated label information (that is what the first-round interaction does), so the second-round interaction ceases to exist once the label annotation information is no longer injected. However, the existence of the second-round interaction does not affect the first-round interaction. As shown in

Table 4. The ablation against two rounds of interaction. Bold font indicates the optimal performance.

Task		Method	P	R	F1
ACE05	NER	LAI-Net	90.28	90.60	90.44
		w/o 2nd	89.97 (−0.31)	90.50 (−0.10)	90.23 (−0.21)
		w/o 1st	89.72 (−0.55)	90.68 (0.08)	90.20 (−0.24)
	RE	LAI-Net	73.80	70.42	72.06
		w/o 2nd	69.70 (−4.09)	66.49 (−3.94)	68.05 (−4.01)
		w/o 1st	68.64 (−5.16)	67.16 (−3.26)	67.89 (−4.17)
	RE+	LAI-Net	71.96	68.67	70.27
		w/o 2nd	67.20 (−4.77)	64.10 (−4.58)	65.61 (−4.67)
		w/o 1st	66.47 (−5.49)	64.35 (−4.32)	65.39 (−4.89)
ADE	NER	LAI-Net	89.78	91.24	90.49
		w/o 2nd	-	-	-
		w/o 1st	88.94 (−0.84)	91.07 (−0.17)	89.99 (−0.51)
	RE	LAI-Net	79.38	83.29	81.26
		w/o 2nd	79.01 (−0.37)	83.17 (−0.12)	81.04 (−0.22)
		w/o 1st	79.08 (−0.30)	82.99 (−0.30)	80.99 (−0.27)
	RE+	LAI-Net	79.37	83.28	81.25
		w/o 2nd	78.73 (−0.64)	82.64 (−0.64)	80.63 (−0.62)
		w/o 1st	78.47 (−0.91)	82.44 (−0.84)	80.41 (−0.85)
SciERC	NER	LAI-Net	70.04	69.89	69.94
		w/o 2nd	69.82 (−0.22)	68.98 (−0.91)	69.40 (−0.54)
		w/o 1st	69.58 (−0.46)	69.12 (−0.77)	69.35 (−0.59)
	RE	LAI-Net	65.56	68.48	66.99
		w/o 2nd	64.21 (−1.35)	68.07 (−0.41)	66.08 (−0.91)
		w/o 1st	63.96 (−1.60)	67.82 (−0.66)	65.83 (−1.16)
	RE+	LAI-Net	59.84	62.01	60.88
		w/o 2nd	59.79 (−0.05)	61.22 (−0.80)	60.49 (−0.39)
		w/o 1st	59.81 (−0.03)	60.47 (−1.54)	60.14 (−0.74)

, regardless of which round of semantic interaction is eliminated, it invariably leads to adverse effects of varying magnitudes on the newly SOTA that we have developed, encompassing the precision, recall, and F1-score metrics. From the perspective of performance degradation, the detrimental effect of eliminating the second-round interaction on the basis of LAI-Net is significantly more pronounced than the impact of further removing the first round of interaction on the basis of without second-round interaction, with the discrepancy peaking at over 21-fold ($\frac{4.67\%}{4.89\%-4.67\%}$ for RE+ in ACE05).

5.3.3. Ablations Against Attention Mask Matrix

In order to evaluate the efficacy of partial attention matrices, we selected three distinct attention mask matrices for ablation (details explained in Appendix B.10), namely, the full attention matrix, wherein all tokens are mutually visible; the anno-token visible attention matrix, where annotation tokens and text tokens are intervisible; and the anno-token invisible attention matrix, with annotation tokens and text tokens also being intervisible. Irrespective of the attention masking matrix employed, tokens of the same type remain mutually visible during the computation of attention scores.

As

Table 5. The ablation results against attention mask matrix, where Vis. represents visible and Inv. represents invisible.

Task		NER	RE	RE+
ACE05	Inv.	90.44	72.06	70.27
	Vis.	90.36 (−0.08)	68.01 (−4.05)	65.57 (−4.70)
	Full	88.29 (−2.14)	65.79 (−6.28)	64.28 (−5.99)
ADE	Inv.	90.49	82.09	81.25
	Vis.	90.09 (−0.40)	81.26 (−0.83)	80.09 (−1.16)
	Full	89.40 (−1.10)	79.99 (−2.10)	78.92 (−2.33)
SciERC	Inv.	69.94	66.99	60.88
	Vis.	69.13 (−0.81)	66.44 (−0.55)	60.61 (−0.27)
	Full	66.21 (−3.73)	66.06 (−0.93)	60.30 (−0.59)

elucidates, across all three datasets, the anno-token invisible attention matrix exhibits a markedly superior performance with strict F1 metrics compared to the other attention matrix types. The anno-token visible attention matrix comes in second place, with its largest deficit compared to the top-performing technique capping out at 4.7% across the varied tasks encapsulated within the trio of benchmarks. Meanwhile, the fully attention matrix has a performance lagging behind the peak achieved score on each respective dataset (max up to 6.28%), indicative of appreciably inferior capabilities among the range of workloads tested.

We attempt to analyze the underlying reasons, which may lie in the following points: (1) The mutual visibility mechanism between annotation tokens and text tokens establishes a conduit for semantic communication, thereby enhancing the semantic richness of the embeddings for both annotations and text. On the one hand, this enables annotation tokens to fully comprehend background information of text, which directly impacts their high-dimensional semantic representation. On the other hand, text tokens can also fully integrate the information from annotations, which directly facilitates the screening and categorization of candidate entities. (2) Conversely, the full attention matrix allows for the intermingling of semantic information among annotation tokens, text tokens, and entity candidate tokens, which may lead to an excessive infusion of additional semantics, resulting in semantic redundancy, and ultimately causing a decline in model performance.

5.4. Case Study

We conduct qualitative analysis on concrete examples to help understand our model. We take PL-Marker as the baseline, which is the closest model to ours. For a fair comparison, we set BERT-base as the common encoder. The golden data marked with entities and relations is shown in Figure 3, along with the results of LAI-Net and PL-Marker. For the given text, PL-Marker mistakenly predicts “coalition” as an ORG entity, of course missing the PART-WHOLE relation between “coalition” and “force”, while LAI-Net correctly predicts both entity and relation. In addition, PL-Marker captures “They” and “Iraqi” as “ORG” and “GPE”, but loses the “GEN-AFF” relation between them.

The disparity in NER performance between LAI-Net and PL-Marker is relatively marginal and is, thus, not the primary focus of our analysis. Accordingly, we direct our attention to a more in-depth comparison of their performance on the RE task. To illustrate the performance gap, we analyze a representative case drawn from numerous instances where LAI-Net yields correct predictions whereas PL-Marker performs suboptimally. Given the condition that both the head and tail entities are correctly recognized on two models, LAI-Net benefits from two phases of interaction between the label and the entity pair, thereby facilitating a more accurate understanding of the “GEN-AFF” relation. Consequently, it correctly predicts the relationship between “They” (ORG) and “Iraqi” (GPE).

6. Conclusions

Faced with the challenges posed by subtask excessive separation and modeling gaps between entities and relations, we propose LAI-Net, which leverages a two-phase semantic interaction and attains SOTA performance in both NER and RE tasks. The key novelty is the multi-phase semantic interaction framework to effectively inject external knowledge and unify representations for entities and relations. We manually annotate label abbreviations to fully semantic unbroken phrase for expanding lexical semantics, and then leverage ${\mathrm{G}}^4\mathrm{CN}$ to fuse information from latent multi-view. An entity candidate filter is embedded to coarse screening of candidate spans for NER. Experiments on several datasets show that our LAI-Net allows high-level semantic information flow and facilitates the E2ERE task, successfully achieving the SOTA performance. While the approach is effective, it may be influenced by factors such as annotation quality and limited use of external knowledge. Extending the model to better handle unseen entities and more open-domain settings remains a promising direction for future work.