A Chinese Few-Shot Named-Entity Recognition Model Based on Multi-Label Prompts and Boundary Information

Abstract

Currently, few-shot setting and entity nesting are two major challenges in named-entity recognition (NER). Compared to English, Chinese NER not only has issues such as complex grammatical structures, polysemy, and entity nesting but also faces low-resource scenarios in specific domains due to difficulties in sample annotation. To address these two issues, we propose a Chinese few-shot named-entity recognition model that integrates multi-label prompts and boundary information (MPBCNER). This model is an improvement based on a pre-trained language model (PLM) combined with a pointer network. First, the model uses multiple entity label words and position slots as prompt information in the entity recognition training task. Activating the relevant parameters in PLM associated with the corresponding entity labels through the prompt information improved the model’s performance in entity recognition under small-sample data. Secondly, by using a Graph Attention Network (GAT) to construct the boundary information extraction module, we integrated boundary information with text features, allowing the model to pay more attention to features near the boundaries when recognizing entities, thereby improving the accuracy of entity boundary recognition. Experiments on multiple public small-sample datasets and our own annotated datasets in the field of government auditing demonstrated the effectiveness of this model.

1. Introduction

Named-entity recognition (NER) holds a crucial position in the field of natural language processing [1]. In information-extraction tasks, it can accurately identify key information such as the names of people, locations, and organizations from vast amounts of text, transforming unstructured text into structured data, thereby providing strong support for subsequent data analysis and decision making [2].

Traditional named-entity recognition methods based on sequence labeling highly depend on a large amount of labeled data [3]. This data helps the model learn the relationship between entities and context, as well as the characteristics of entity categories, enabling it to perform well when annotating new texts. At this stage, LAFFERTY J et al. [4] have proposed the conditional random field (CRF) model for sequence-labeling tasks. The CRF model can automatically identify and capture features and patterns in sequences by learning the mapping from input sequences to output label sequences. HUANG et al. [5] proposed a model based on bidirectional long short-term memory networks (BiLSTM) and CRF for sequence labeling. This model uses the BiLSTM network as the context encoding layer, which can better capture long-term dependencies in the sequence. MA et al. [6] have introduced CNN units based on the BiLSTM-CRF model to extract morphological information from word characters, thereby improving the model’s final performance.

In recent years, with the successive emergence and development of large-scale pre-trained models (e.g., Bert [7], GPT [8,9,10], RoBERTa [11], T5 [12], ERNIE [13,14,15], etc.), the large-scale training parameters have enabled these large models to possess better knowledge representation and transfer learning capabilities, thereby achieving the latest and optimal results in many natural language processing tasks. Souza et al. [16] proposed a BERT-CRF model for the NER task, combining BERT’s transfer capabilities with CRF’s structured prediction, achieving results better than those of the aforementioned baseline methods. However, both traditional sequence labeling methods and fine-tuning modes of downstream tasks based on large models require a large amount of sample data during the training phase. In practical applications, obtaining large-scale labeled data may face difficulties, requiring industry experts to spend a lot of time and effort to complete it manually [17]. In small-sample scenarios, due to the limited labeled data, it is difficult for the model to learn the complex features and constraints of different entity types, leading to unsatisfactory entity-recognition results.

To harness the potential of large models in small-sample-data scenarios, researchers have adopted a range of methods, including transfer learning [18], contrastive learning [19], knowledge graph injection [20], and prompt learning [21]. These methods have been used to enhance a model’s generalization ability, thereby improving its performance on small samples. Among these, prompt learning, as an emerging model adaptation paradigm, has emerged as a particularly efficient and flexible solution for model applications in small-sample-data scenarios through the embedding mechanism of natural language prompts, garnering widespread attention from researchers in the field. CUI et al. [22] proposed TemplateNER, which was the first to apply the prompt method to NER, predicting entity types by manually constructing discrete templates and enumerating all possible entity spans. The model has been shown to significantly outperform traditional sequence labeling methods in cross-domain and few-shot setting. However, the prediction results are heavily dependent on the quality of the template design, and the introduction of a large number of negative samples leads to severe efficiency issues. Ma et al. [23] introduced the soft template prompt learning method, incorporating a small number of learnable parameters as prompt guidance in the self-attention layer of pre-trained models, compensating for the shortcomings of manually designed prompt templates. Shen et al. [24] designed a dual-slot multi-prompt template for entity types and positions, extracting all entities by predicting each prompt slot. This avoids the efficiency issues of enumerating spans and achieves a single round of prediction for all entities. This method provides a new approach for using prompts in NER tasks, but its model design is relatively simple. It utilizes the positional information of entities without considering the boundary information of the entity heads and tails, leaving room for further improvement in Chinese NER tasks.

Compared to English, Chinese NER faces many unique challenges. The structure of Chinese is complex, and Chinese characters themselves do not have natural delimiters, making the boundaries between words ambiguous [25]. This makes both character-based and word-based sequence labeling quite challenging. If the model cannot effectively handle nested relationships, it is prone to recognition errors. These factors make Chinese named-entity recognition much more difficult than that of English, placing higher demands on the model’s performance. A series of experiments conducted in [26,27,28,29,30] have demonstrated that the incorporation of boundary information into the encoding and decoding layers of the model can effectively enhance the model’s accuracy in Chinese natural language tasks.

Regarding the issues identified in the aforementioned Chinese NER task, after comprehensively considering the advantages and disadvantages of various methods, we propose a Chinese few-shot named-entity recognition model that integrates multi-label prompts and boundary information (MPBCNER). First, the MPBCNER uses a pre-trained large model as the encoding layer, introducing a template composed of entity label prompt words and entity position prompt slots at the model input layer. This better utilizes the prior knowledge of the pre-trained model during training and prediction phases, enhancing the model’s performance in small-sample scenarios. Secondly, we used a graph attention network to construct a boundary information extraction module, allowing the model to focus more on features near the boundaries when recognizing entities, thereby improving the accuracy of entity boundary recognition. Finally, we use a pointer network as the model’s decoding layer, cleverly avoiding the complexity and inefficiency brought by predicting enumerated spans. Meanwhile, the structure of the pointer network has a certain degree of feature similarity with the boundary information network, which allows it to better learn the boundary features of Chinese entities, providing significant advantages in handling Chinese samples and nested entities.

2. Related Works

2.1. Pre-Trained Language Model

Early language models were primarily based on statistical methods and neural networks, such as a recurrent neural network (RNN) [31] and long short-term memory (LSTM) [32]. However, these models had limitations in dealing with long-distance dependencies and semantic understanding [33]. In 2017, Google introduced the Transformer architecture, which addressed the problem of long-distance dependencies through a self-attention mechanism, laying the foundation for the development of pre-trained language models (PLMs) [34].

In general, there are two main types of PLMs: the autoencoding language model represented by Bert [7], and the autoregressive language model represented by GPT-1 [8]. Bert uses the Transformer’s encoder structure and is pre-trained through the masked language model (MLM) and next sentence prediction (NSP) tasks, and performs exceptionally well on various natural language processing (NLP) tasks. GPT, on the other hand, uses Transformer’s decoder structure and is pre-trained by predicting the next word in a unidirectional language model. A commonality between these two categories of pre-trained models is their extensive parameter scale. Through training on extensive corpora, these models are capable of acquiring intricate language structures and semantics. In numerous natural language processing tasks, fine-tuning the model is sufficient to achieve and even surpass traditional baseline methods.

2.2. Fine-Tuning and Prompt Learning

A large language model (LLM) undergoes self-supervised learning on substantial text corpora to acquire extensive linguistic knowledge. Through fine-tuning, these models can achieve optimal performance on associated downstream tasks. However, the fine-tuning paradigm currently faces two major challenges [35]:

• Fine-tuning requires a large amount of manually annotated data to achieve good model performance on large corpora. However, in practice, many fields have limited sample data and low-quality manually annotated data. Consequently, the efficacy of the model is diminished in a few-shot setting;
• The fine-tuned model is usually only suitable for specific tasks and datasets and cannot be directly used for new tasks. The capabilities of the pre-trained language model cannot be fully utilized.

Researchers have found that through prompt learning, large models can perform better on the same downstream tasks [21]. Currently, there are two main methods of prompt learning:

• Hard prompt. The prompt template consists of a discrete sequence of text [22,36];
• Soft prompt. The prompt template is represented by a continuous sequence of learnable features [37,38].

In most cases, a soft prompt performs better than a hard prompt because of their ability to learn and adapt dynamically.

2.3. Chinese Named-Entity Recognition

Chinese named-entity recognition is an important part of the NER field and has significant research implications. Chinese NER suffers from fuzzy boundary information, lack of labeled data, and semantic complexity. Researchers mainly improve it by lexical information enhancement and fusion of Chinese language character features. Zhang et al. [39] proposed a new dynamic embedding method that uses an attention mechanism to combine character and word vector features in the embedding layer. Zhang et al. [40] proposed the Lattice LSTM model, which incorporates lexical information for the first time into a character-based model, which explicitly utilizes word and word sequence information as compared to character-based methods. The model is not affected by segmentation errors as compared to word-based approaches. Wang et al. [41] propose a hierarchical label-enhanced contrastive learning method for Chinese NER, which reduces word segmentation errors and increases word boundary information for character sequences.

3. MPBCNER Model

In this section, we first describe the overall framework of the model, and then introduce the implementation details of each module of the model, including the construction of prompt templates, the encoding and decoding layers, the auxiliary training of the starting position prompt, and the entity boundary feature extraction in the graph attention network implementation. The overall structure of the model is shown in Figure 1.

3.1. Prompt Template

The prompt template affects the quality of the model’s feature output. Hard prompts, entirely designed by humans using natural language, have high interpretability but may not fully align with the PLM’s understanding. Soft prompts, on the other hand, automatically optimize by introducing learnable parameters to adapt to different tasks but cannot explain their rationality. We have combined the characteristics of both soft and hard prompt to design a new NER prompt template.

By default, each prompt consists of three parts: a start position slot $[S]$, an end position slot $[E]$, and learnable contextual tokens $<L>$. Specifically, our model fills in a set of prompts for each type of entity and then concatenates all the prompts with the original text to use as input. For the sentence $X=$ “北京银行在广西设立分行 (Bank of Beijing opens branch in Guangxi)”, we set two entity types: “地点 (Location)”, “组织 (Organization)”. The input sequence $\mathcal{T}$ with prompt is represented as:

$$\mathcal{T}=\begin{array}{c}\{[S_0]\mathrm{地}\mathrm{点}[E_0],[S_1]\mathrm{组}\mathrm{织}[E_1]\}\\ [CLS]\mathrm{北}\mathrm{京}\mathrm{银}\mathrm{行}\mathrm{在}\mathrm{广}\mathrm{西}\mathrm{设}\mathrm{立}\mathrm{分}\mathrm{行}\end{array}$$

(1)

Generally, for a sequence composed of $n$ tokens with $M$ entity types, the input sequence $\mathcal{T}$ with a prompt can be represented as:

$$\mathcal{T}=\begin{array}{l}{\left\{[S_i]<L>[E_i]\right\}}_{i=1,2,\cdots ,M}\hfill \\ [CLS]<x_1{x}_2,\cdots ,x_n>\hfill \end{array}$$

(2)

where “$[S_i]<L>[E_i]$” represents the $i$-th prompt. $<L>$ is the soft prompt consisting of learnable contextual tokens. Specifically, we added a set of trainable embedding matrices $\{{\mathcal{P}}^1,{\mathcal{P}}^2,\cdots ,{\mathcal{P}}^N\}$ for the PLM encode as prompt parameters, where $N$ is the number of transformer layers. In the model-training phase, the PLM parameters are frozen and the prompt parameters are the only trainable parameters. Meanwhile, we used label words embedding as hard prompts to complete the initialization of the soft prompt.

3.2. Encoder Layer

For a given input sentence $\mathcal{T}$ spliced by the sentence $X$ and $M$ prompts, we used PLM to encode and then obtain the hidden representation $H^{\mathcal{T}}$:

$$H^{\mathcal{T}}=PLM(\mathcal{T})$$

(3)

Then, by indexing on the corresponding position of $H^{\mathcal{T}}$, we can obtain the encoding of the sentence $X$ and the encodings of the two prompt slots, denoted as $H^X$, $H^S$, and $H^E$, where $H^X\in {ℝ}^{n\times h}$, $H^S,H^E\in {ℝ}^{n\times h}$.

In the subsequent experiments, we will use different PLMs for encoding to explore the impact of PLMs on the model.

3.3. GAT-Based Boundary Information Fusion Layer

The graph attention network (GAT) is a neural network model that has been specifically designed to process graph-structured data. It introduces an attention mechanism that enables the model to assign different weights to the neighboring nodes of each node during graph data processing. This mechanism facilitates the enhanced capture of the complex relationships between nodes and the local structures within the graph data, thereby improving the model’s ability to understand and represent the intricate patterns present in the data.

While PLM is capable of effectively extracting contextual features, its ability to capture boundary information features within the input text is limited. To address this, the present paper proposes employing GAT to extract features based on the internal dependency relationships present in the input text. This approach involves mining the word and character dependency relationships within the text to extract the boundary features of entities. Consequently, this enhances the accuracy of entity recognition.

In this study, a sequence of Chinese text is treated as a graph-structured network, with each character in the sequence considered a node in the graph. In particular, we use DDParser [42] to obtain the dependencies of the input sequence and then construct a directed graph based on the dependencies, where the directed edges point to the center word of a given word, and the directed edges are labeled with the specific type of dependency, and the words located at the beginning of the directed edges depend on the words located at the end of the directed edges in some kind of dependency relationship. For a Chinese text sequence “北京银行在广西设立分行 (Bank of Beijing opens branch in Guangxi)”, the syntactic dependency relationship graph is shown in Figure 2.

In this sentence, “北京” is a qualifying modifier for “银行”, “北京银行” and “设立” have a subject–predicate relationship, “在” and “广西” are in a verb–object relationship, etc. Since the model in this paper is character-based, each character in a word has the same dependency relationship with the word. Meanwhile, to retain the feature information of the node itself, it is assumed that each word is related to itself. Each node in the graph will have a ring pointing to itself. According to the dependency relationship between nodes, the elements at the corresponding positions in the dependency matrix are set to 1, and the remaining elements are set to 0. According to the above rules, the dependency matrix of the input sequence is constructed.

In a directed graph, nodes that have a dependency relationship with entities reflect the existence of the entities to a certain extent. Meanwhile, directed edges containing dependency information can assist in distinguishing entity boundaries and identifying the categories of entities.

To more clearly extract features related to boundary information between entities, we built a GAT to capture dependencies between words. Firstly, a linear transformation is done on the input features of each node to map the feature dimension from $F$ to $F^{\prime }$:

$$W{h}_i\in {ℝ}^{F^{\prime }}$$

(4)

where $W\in {ℝ}^{F^{\prime }\times F}$ is sharing weight matrices. $h_i$ is the original feature vector of node $i$.

Then, the attention coefficient $e^{ij}$ indicates the importance of node $j$ to node $i$:

$$\begin{array}{cc}\hfill e^{ij}& =att\left(W{h}_i,W{h}_j\right)\hfill \\ \hfill & =LeakReLU({\overrightarrow{a}}^T\left[W{h}_i\parallel W{h}_j\right])\hfill \end{array}$$

(5)

where $att(\cdot ,\cdot )$ is the attention function, implemented by a single-layer feedforward neural network. $\overrightarrow{a}$ is the weight vector.

In order to compare the importance of different neighboring nodes, the $soft\max$ function is used to normalize all attention coefficients:

$$\begin{array}{cc}\hfill {\alpha }_{ij}& =softmax(e_{ij})\hfill \\ & ={\displaystyle \frac{\exp (e_{ij})}{{\displaystyle \sum _{k\in {\mathcal{N}}_i}\exp (e_{ik})}}}\hfill \end{array}$$

(6)

where ${\mathcal{N}}_i$ is the neighborhood nodes of node $i$.

To enhance the model’s representational capacity, GAT introduces a multi-head attention mechanism. A GAT operation with $K$ independent attention heads can be expressed as:

$$h^{\prime }=\sigma \left({\displaystyle \frac{1}{K}}{\displaystyle \sum _{k=1}^K{\displaystyle \sum _{j\in {\mathcal{N}}_i}{\alpha }_{ij}^k{W}^k{h}_j}}\right)$$

(7)

where $\sigma$ is the activation function.

The GAT layer takes the node dependency matrix and the character vector representation that integrates lexical information as input and uses the attention mechanism to calculate each node, thereby obtaining the feature representation of each node. The text dependency relationship features extracted by the GAT layer can effectively distinguish entity boundaries, thereby significantly improving the accuracy of named-entity recognition.

3.4. Feature Fusion Layer

Although GAT has shown significant effectiveness in capturing internal dependencies within entities, there is still room for improvement in clearly defining entity boundaries. To address this shortcoming, we treat entity boundary detection as a binary classification task and train it simultaneously with the NER. This approach allows us to provide clearer and more accurate entity boundary information to the NER model, thereby enhancing the overall performance of the model.

During the training phase, we used two separate MLP networks to predict the start and end positions of entities:

$$H_S^{\prime }=ML{P}_{Start}\left(H^S\right)$$

(8)

$$H_E^{\prime }=ML{P}_{End}\left(H^E\right)$$

(9)

where $H^S$ and $H^E$ are the hidden feature representation of two prompt slot positions.

Then, we added the hidden features of the entity to the output of the GAT layer:

$$H=W_S{H}_S^{\prime }+W_E{H}_E^{\prime }+W_G{H}_{GAT}^{\prime }$$

(10)

where $H$ is the final input for the point network. $W_S$, $W_E$, and $W_G$ are learnable weight matrices.

At the feature fusion layer, by fusing boundary features and entity location features, we obtain a hidden vector representation after multi-feature fusion. It will further improve the model’s recognition of Chinese entities.

3.5. Decode Layer

We chose a pointer network as the model’s decoding layer. On one hand, the pointer network primarily focuses on the start and end positions of entities, which is consistent with the two auxiliary tasks of our method, making it well-suited for our work. On the other hand, the pointer network addresses the issue of entity nesting, which sequence labeling struggles to resolve, making it more practical for Chinese NER tasks.

The core of the pointer network is to select entity boundaries in the input sequence through the attention mechanism. For a feature sequence output $H=\{h_1,h_2,\cdots ,h_n\}$, the attention weight calculation formula for the pointer network is:

$${\alpha }_{ij}={\displaystyle \frac{\exp \left(score\left(h_i,h_j\right)\right)}{{\displaystyle \sum _{k=1}^n\exp \left(score\left(h_i,h_k\right)\right)}}}$$

(11)

$$score\left(h_{i,}{h}_j\right)=h_i^T{W}_s{h}_j$$

(12)

The pointer network uses binary cross-entropy loss to predict whether each position is the start or end of an entity:

$$p_i^{Start},p_j^{End}=sigmod\left(h_i^{Start},h_j^{End}\right)$$

(13)

$${\mathcal{L}}_{Start}=-{\displaystyle \sum _{i=1}^n{y}_i^{Start}\log p_i^{Start}+(1-y_i^{Start})\log }(1-p_i^{Start})$$

(14)

$${\mathcal{L}}_{End}=-{\displaystyle \sum _{i=1}^n{y}_i^{End}\log p_i^{End}+(1-y_i^{End})\log }(1-p_i^{End})$$

(15)

The overall loss function $\mathcal{L}$ of the model consists of ${\mathcal{L}}_{Start}$ and ${\mathcal{L}}_{End}$ together and:

$$\mathcal{L}=\lambda {\mathcal{L}}_{Stard}+\left(1-\lambda \right){\mathcal{L}}_{End}$$

(16)

where $\lambda$ is the weight parameter, and we set it to 0.5.

4. Experiment

4.1. Datasets

This study conducted experiments on a manually collected dataset in the field of Chinese government auditing and three public datasets, OntoNotes V5.0 [43], Weibo NER [44], and MSRA [45].

4.1.1. The Government Auditing Dataset

Due to the particularity of the field of government auditing, there is a lack of relevant publicly available dataset resources. We manually collected and created a small amount of sample data from the publicly available information on various government websites. After data cleaning, we obtained 1000 samples, involving four types of entities: time, unit, project, and amount. In our subsequent experiments, we will refer to it as GA.

4.1.2. OntoNotes V5.0 Dataset

The dataset is a Chinese dataset and consists of texts from the news domain.

4.1.3. Weibo NER Dataset

The dataset contains annotated NER messages drawn from the social media Sina Weibo.

4.1.4. MSRA Dataset

The dataset is released by Microsoft Research Asia, which contains data on multiple entity nested types.

4.2. Experiment Environment

The experimental code is developed based on the PyTorch 2.3.0 framework, and the cloud GPU server is used as the experimental runtime environment. The experimental environment specifics are shown in

Table 1. Experiment environment.

Name	Parameters
Operating System	CentOS 7.7.1908 (Core)
CPU	Intel(R) Xeon(R) Gold 6271C
GPU	Tesla V100
Memory	128 G

.

The experimental parameters setting are shown in

Table 2. Experiment parameters.

Parameter	Value
batch_size	8
epoch	10
max_seq_length	512
learning_rate	1 × 10−5
dropout	0.1

.

In configuring the pre-trained language model, we established a maximum input length for the text sequence of 512 units. We leveraged 12 transformer layers to facilitate semantic encoding and feature extraction. The learning rate was set to 1 × 10⁻⁵, aiming to enhance the model’s learning velocity and convergence. Additionally, to boost training efficiency and the model’s generalization capacity, we set the number of epochs to 8 and the batch size to 10.

4.3. Evaluation Indicators

This paper uses the F1 score as an evaluation metric. Precision ($P$) refers to the ratio of the number of correctly identified named entities to the total number of identified named entities. Recall ($R$) refers to the ratio of the number of correctly identified named entities to the total number of entities. Precision is the ratio of the number of correctly predicted samples to the total number of samples, and $F1$ is the harmonic mean of precision and recall. The calculation formula is:

$$P={\displaystyle \frac{TP}{TP+FP}}$$

(17)

$$R={\displaystyle \frac{TP}{TP+FN}}$$

(18)

$$F1={\displaystyle \frac{2TP}{2TP+FP+FN}}$$

(19)

Among them, $TP$ represents the number of samples predicted as positive and actually positive, $FP$ represents the number of samples predicted as positive but actually negative, $FN$ represents the number of samples predicted as negative but actually positive, and $TN$ represents the number of samples predicted as negative and actually negative. Accuracy and $F1$ score can comprehensively evaluate the performance of the NER model.

4.4. Baselines

In this work, we conducted experimental comparisons with the following baseline methods to verify the performance of the model in the small-sample scenarios NER task, including based on sequence labeling (BERT-CRF [16]), based on span (SpERT [46]), based on machine reading comprehension (BERT-MRC [47]), based on sequence generation (BARTNER [48]), and based on prompt learning (TemplateNER [22], PromptNER [24]).

These methodologies employ distinct pre-trained language models as encoders. Consequently, the experimental results provide the performance of GLK on various PLMs, including Bert, RoBERTa, and ERNIE. Bert is the most classical large language model. RoBERTa is based on the architecture of BERT with key modifications and optimizations. RoBERTa provides richer linguistic contextual information by using larger datasets (e.g., a combination of BookCorpus and OpenWebText) for training, which helps to improve the model’s generalization ability. In addition, it dynamically adjusts hyperparameters, such as the learning rate, batch size, etc., to optimize the learning efficiency and effectiveness of the model. ERNIE is a knowledge-enhanced pre-trained language model. ERNIE focuses not only on word representation but also on phrase, entity, and sentence representation, and this multi-granularity knowledge modeling helps to capture the rich structure in the language and improves semantic comprehension.

5. Results and Discussion

5.1. Results in Standard Chinese Dataset

We first conducted experiments on the standard Chinese dataset, where OntoNotes and MSRA are flat datasets and Weibo and GA contain a large amount of nested entities. The experimental results are shown in

Table 3. Results in standard Chinese flat and nested dataset (F1%).

Model	OntoNotes	MSRA	Weibo	GA
BERT-CRF	87.62	89.56	70.13	78.23
SpERT	88.47	92.26	79.03	83.14
BERT-MRC	84.61	90.11	75.27	82.42
BARTNER	88.24	91.16	79.75	83.86
TemplateNER	84.65	90.32	76.32	79.65
PromptNER	88.71	92.86	78.26	82.16
MPBCNER (BERT)	89.12	92.14	81.35	85.27
MPBCNER (RoBERTa)	90.23	92.75	82.16	85.92
MPBCNER (ERNIE)	92.16	93.01	83.74	88.16

.

From the results in Table 3, the MPBCNER model performed better than baselines on both the standard Chinese flat and nested. The average lead was 1–2% in the Chinese flat dataset and 3–5% in the standard Chinese nested dataset. The results indicate the efficacy of our methodology in incorporating entity boundary information. This capacity facilitates the model’s enhanced precision in handling nested entity data.

It can especially be seen that different PLMs affected the results to some extent. RoBERTa uses more than ten times the data and longer training time than BERT does, which improves model robustness and performance. ERNIE additionally introduces a knowledge-enhanced pre-training task and enhances the model’s ability to learn about the understanding of words and entities by integrating external knowledge such as knowledge graphs. This feature gives ERNIE a clear advantage in Chinese NLP tasks.

5.2. Results in Few-Shot Setting

To assess the performance of the model in scenarios with limited resources, we randomly sampled a fixed number of instances for each dataset as K-shot, where K = 10, 20, 50, 100, 200, 500. The few-shot setting follows [10]. The experimental results in the OntoNotes few-shot dataset are shown in

Table 4. Results in the OntoNotes few-shot dataset (F1%).

Model	K = 10	K = 20	K = 50	K = 100	K = 200	K = 500
BERT-CRF	29.1	47.9	53.1	56.4	61.1	74.1
SpERT	33.1	41.7	50.0	57.9	65.2	75.4
BERT-MRC	42.9	50.3	57.4	61.6	66.1	78.8
BARTNER	41.0	49.1	55.2	60.1	66.9	78.2
TemplateNER	42.4	51.4	59.1	64.1	69.0	79.3
PromptNER	49.1	58.2	64.5	68.4	71.3	78.5
MPBCNER (BERT)	49.3	58.8	64.9	68.7	72.1	79.0
MPBCNER (RoBERTa)	50.2	60.1	65.7	69.2	73.3	79.1
MPBCNER (ERNIE)	51.1	62.4	68.1	71.2	75.2	80.3

. The experimental results in the MRSA few-shot dataset are shown in

Table 5. Results in the MSRA few-shot dataset (F1%).

Model	K = 10	K = 20	K = 50	K = 100	K = 200	K = 500
BERT-CRF	30.5	48.4	53.8	57.2	62.4	74.9
SpERT	34.6	42.5	50.6	58.5	66.1	75.8
BERT-MRC	43.1	51.0	58.2	62.3	67.3	79.1
BARTNER	41.6	50.2	55.9	60.2	67.0	78.8
TemplateNER	43.1	52.8	60.8	65.4	70.3	80.1
PromptNER	52.7	61.4	65.3	68.5	71.1	81.6
MPBCNER (BERT)	54.3	65.1	71.3	77.2	80.4	82.9
MPBCNER (RoBERTa)	55.8	66.2	72.0	78.6	80.9	83.5
MPBCNER (ERNIE)	58.3	69.1	74.7	80.0	82.3	84.9

. The experimental results in the Weibo few-shot dataset are shown in

Table 6. Results in the Weibo few-shot dataset (F1%).

Model	K = 10	K = 20	K = 50	K = 100	K = 200	K = 500
BERT-CRF	22.6	31.5	40.8	51.3	58.6	63.1
SpERT	25.4	38.2	44.5	52.1	60.3	65.2
BERT-MRC	30.1	39.3	45.1	53.2	61.9	66.3
BARTNER	32.2	40.2	47.5	55.1	62.3	66.8
TemplateNER	38.1	44.2	51.9	60.7	65.4	70.5
PromptNER	40.2	46.3	55.7	62.9	68.5	73.7
MPBCNER (BERT)	43.5	50.2	59.8	65.1	70.2	75.8
MPBCNER (RoBERTa)	43.8	50.8	61.0	65.5	70.6	76.3
MPBCNER (ERNIE)	45.1	52.3	62.5	66.1	72.3	76.9

. The experimental results in the GA few-shot dataset are shown in

Table 7. Results in the GA few-shot dataset (F1%).

Model	K = 10	K = 20	K = 50	K = 100	K = 200	K = 500
BERT-CRF	20.1	29.5	37.8	45.6	54.8	60.1
SpERT	24.3	36.5	42.9	50.4	58.5	63.4
BERT-MRC	30.5	38.9	44.1	51.2	60.3	67.9
BARTNER	33.1	41.6	46.3	53.0	61.8	68.2
TemplateNER	40.2	44.3	52.6	60.8	66.1	72.3
PromptNER	42.8	47.5	58.3	65.1	70.2	74.8
MPBCNER (BERT)	45.1	52.6	61.5	67.3	72.1	78.6
MPBCNER (RoBERTa)	45.9	53.5	62.1	67.9	73.0	79.3
MPBCNER (ERNIE)	46.3	55.4	64.8	69.7	75.1	81.5

.

From Table 4, Table 5, Table 6 and Table 7, we can find that the models based on prompt generally exhibited better performance than others did in the few-shot setting. The prompt template designed in this paper could better stimulate the a priori knowledge of PLM and get better results than other models. After fusion with boundary information features, the MPBCNER model achieved optimal results with limited resources.

From the above experimental results, we can get the following three conclusions:

• Comparing the approach based on sequence labeling and the approach based on span, it can be found that the prompt-based model had a significant advantage because the prompt template could provide the necessary context for large-scale pre-training of the language model, helping the model to better understand the downstream task requirements and therefore achieve better performance in a few-shot setting;
• Comparing the approach based on machine reading comprehension and the approach based on sequence generation, it can be found that these two methods, although they performed better than the sequence labeling-based method did in the few-shot setting, did not perform as well as the prompt-based method in the experimental results because of insufficient deep mining of the prior knowledge of the pre-trained model;
• Comparing the approach based on prompts, it can be found that the prompt-based method was more advantageous in the few-shot setting, but due to the lack of feature learning on Chinese entity boundaries, it was less effective on Chinese datasets, especially on nested datasets. MPBCNER, which incorporates boundary features and entity location features, could better handle the NER task in the Chinese small-sample setting.

5.3. Ablation Assay

In this section, we conducted ablation experiments on the GA dataset to analyze the effect of different modules of MPBCNER. We selected BERT as a defult PLM. The experimental results are shown in

Table 8. Ablation experiment.

Model	K = 10	K = 20	K = 50	K = 100	K = 200	K = 500
MPBCNER (BERT)	45.1	52.6	61.5	67.3	72.1	78.6
-GAT lay	43.2	48.9	59.4	65.7	70.1	75.4
-Prompt	37.5	42.5	52.6	59.7	61.8	68.6
-ALL	30.1	36.6	42.7	50.2	58.2	65.1

.

The results in Table 8 show that the F1 score of the model decreased after removing different submodules. In this case, the model performance was affected more by the removal of the prompt module, which indicates that prompt learning can effectively improve the performance of the model in a few-shot setting. Meanwhile, for Chinese NER, incorporating effective entity boundary features into the model improved the model’s entity recognition capability more accurately.

In order to explore the effect of different prompt templates on model performance, we set up relevant comparison experiments. We denoted the prompt template method that only uses the prompt slot as MPBCNER (Slot), the prompt template method that only uses the hard prompt template as MPBCNER (Hard), and the prompt template method that only uses the soft prompt template as MPBCNER (Soft).

The results in

Table 9. Experimental results for different prompts.

Model	K = 10	K = 20	K = 50	K = 100	K = 200	K = 500
MPBCNER (BERT)	45.1	52.6	61.5	67.3	72.1	78.6
MPBCNER (Slot)	44.8	51.5	60.3	65.8	71.2	77.5
MPBCNER (Hard)	40.3	47.3	55.1	61.9	66.5	71.0
MPBCNER (Soft)	44.2	50.9	59.7	65.1	70.6	76.3

show that different prompt templates had different effects on the models. Of all of these, the soft prompt and prompt slots had the best lift on the model. The effect of hard prompts was the worst. The MPBCNER combining all three cueing features performed the best.

6. Conclusions

In this paper, we designed a dual-position slot template combining soft and hard prompts, using entity label words as a hard prompt to complete the initialization of the prompt slots, and the position cue slots and the NER task jointly involved in the training could be learned to correspond to the start and end position features of the labeled entities. We also utilized graph attention networks to complete the extraction of entity boundary features. Compared with the baseline approach, our model achieved better performance in Chinese flat and nested NER and low-resource scenarios after fusing positional and boundary features.

At the same time, the model still has some shortcomings. One is the use of a GAT and a pointer network for feature extraction of data. There is a problem with operational efficiency, and the use of a sparse matrix can be considered to improve the operational efficiency of the model. The other is that for each class of entity type, a corresponding pointer network is needed for decoding, and the use of a global pointer network can be considered to avoid this problem.