CSIER-FM: A Chinese Sensitive Information Entity Recognition Framework with Few-Shot Learning and Multi-Instance Integration

Abstract

The rapid growth of information technology has posed new challenges for recognizing sensitive information in Chinese text. Traditional rule-based, statistical, and machine learning methods face limitations in domain adaptability and Chinese semantic comprehension. Recently, large language models have offered new opportunities to address these challenges with their strong semantic understanding and transfer capabilities. Building on this, we propose the CSIER-FM, which is a Chinese sensitive information entity recognition framework that integrates prompt design, few-shot learning, parameter-efficient fine-tuning, and multi-instance integration. We design multiple prompt templates and incorporate a k-nearest neighbor (k-NN) sample selection strategy to optimize prompts and enhance the effectiveness of few-shot learning. In addition, we apply Low-Rank Adaptation (LoRA) to efficiently fine-tune the locally deployed Qwen2.5-7B model. Finally, a multi-instance integration mechanism is employed to allow different models to focus on specific entity categories, thereby reducing category confusion and enhancing overall F1-score. We evaluate the CSIER-FM on the Chinese ResumeNER dataset. The results demonstrate that fine-tuning the Qwen2.5-7B model raises its F1-score from 0.7106 to above 0.9518. With the addition of multi-instance integration, the F1-score further increased to 0.9553. The findings indicate that the CSIER-FM effectively integrates named entity recognition with Chinese sensitive information detection, enabling efficient recognition of multiple sensitive entity types in Chinese text.

1. Introduction

With the rapid advancement of information technology, modern society is swiftly entering the digital age. While the widespread availability of information facilitates its collection, storage, and utilization, it also significantly increases the risk of data leakage. Consequently, effectively recognizing and protecting sensitive data has become a critical issue in the field of data security. Sensitive data refers to information that, if accessed without authorization or used improperly, may result in security incidents, privacy violations, or business risks. Its disclosure, alteration, or unlawful access can cause significant harm to individuals, organizations, and the country. Current data security governance typically requires the recognition of sensitive data originating from multiple sources, which can generally be categorized into structured and unstructured data. However, as most existing methods are tailored to non-Chinese data, there is an urgent and practical need to develop effective methods for recognizing Chinese unstructured sensitive information.

Structured data is typically stored in two-dimensional tabular formats, for which numerous recognition methods already exist. In contrast, our approach focuses on the recognition of sensitive data in unstructured data, especially in Chinese data. Unstructured data includes a wide range of formats and is characterized by the absence of a predefined structure. Early methods for recognizing unstructured sensitive data primarily relied on rule-based approaches, such as regular expressions. Subsequently, statistical learning methods, such as TF-IDF, LAD, and BoW, gradually replaced rule-based approaches. With the advent of deep learning, models based on LSTM and BERT gradually emerged, such as ExSense and KGDetector. In recent years, the rapid development of large language models has opened new avenues for unstructured sensitive data recognition, leading to the emergence of numerous methods leveraging these models for improved performance. However, most of the aforementioned methods were designed for non-Chinese data and therefore exhibit limitations, such as insufficient semantic understanding when recognizing sensitive data in Chinese. The summary of existing literature on the recognition of unstructured sensitive data is presented in Table 1.

Table 1. Summary of representative studies on sensitive information recognition.

Approach	Authors	Year	Model
-based approaches	Katić et al. [1]	(2018)	BOW
	Xu et al. [2]	(2019)	LDA
	Lu et al. [3]	(2021)	Naive Bayes classifier
Deep learning	Ong et al. [4]	(2017)	LSTM and BiLSTM
-based approaches	Neerbeky et al. [5,6]	(2017, 2018)	RvNN
	Xu et al. [7]	(2018)	Text-CNN
	Lin et al. [8]	(2019)	CNN, BiLSTM
	Pogiatzis et al. [9]	(2020)	BiLSTM, CRF
	Guo et al. [10]	(2021)	BERT-BiLSTM-Attention
	Ding et al. [11]	(2021)	BERT
	Cong et al. [12]	(2022)	Knowledge graph and BERT
	Anand et al. [13]	(2023)	CharacterBERT
	Ren et al. [14]	(2024)	ELECTRA-BiLSTM-CRF
LLM-based	Wang et al. [15]	(2023)	GPT-3
approaches	Ashok et al. [16]	(2023)	GPT-3, GPT-4, T5-Flan
	Li et al. [17]	(2024)	GPT-4
	Anand [18]	(2024)	GPT-4o
	Wen et al. [19]	(2024)	GPT-4 and GPT-4o

In this paper, we propose a Chinese sensitive information entity recognition framework (CSIER-FM) that integrates prompt design, few-shot learning, model fine-tuning, and multi-instance integration techniques. To enhance the F1-score of Chinese sensitive data recognition, we focused on two key aspects: prompt design and model fine-tuning. We designed multiple prompts and employed the k-NN strategy to select most suitable samples for constructing effective prompts. Additionally, we fine-tuned the locally deployed Qwen2.5-7B model and applied multi-instance model integration, which significantly improved F1-score, achieving performance far superior to that of the online DeepSeek-V3 model.

Our main contributions are as follows:

• We designed and evaluated various prompt formats for Chinese sensitive data recognition, and based on experimental analysis, identified the most effective prompt structure for domain-specific Named Entity Recognition tasks.
• We design a k-nearest neighbor (k-NN)-based sample selection strategy that enables semantic-oriented sampling, thereby enhancing the effectiveness of few-shot learning techniques in Chinese sensitive data recognition tasks.
• We fine-tune the Qwen2.5-7B model and apply a multi-instance integration approach, which significantly enhances its F1-score in Chinese sensitive data recognition.
• Our proposed CSIER-FM achieves very high F1-score in Chinese sensitive data recognition and can effectively perform sensitive information entity recognition tasks.

2. Related Work

Unstructured sensitive data recognition methods can generally be classified into 4 categories: rule-based methods, statistical methods, deep learning-based methods, and LLM-based methods.

Early methods for unstructured sensitive data recognition primarily relied on manually defined rules such as regular expressions, keywords, and dictionaries to recognize sensitive information. These approaches suffered from limited recognition capability, high time complexity, poor scalability, and high maintenance costs. With the rise of traditional machine learning algorithms, rule-based methods were gradually replaced by statistical approaches.

Statistical learning-based approaches mainly utilize methods such as Bag of Words (BOW) [1], Latent Dirichlet Allocation (LDA) [2] and Naive Bayes classifiers [3] to represent text and perform sensitive information classification.

Subsequently, the emergence of models such as LSTM, BiLSTM, RvNNs and BERT has provided an effective solution to the aforementioned problems, demonstrating strong capabilities in recognizing and classifying sensitive information entities. Consequently, deep learning-based approaches for unstructured sensitive data recognition have rapidly advanced.

In the DLP system proposed by Ong et al. [4], sensitive word detection at the word level leverages contextual information through models such as Long Short-Term Memory (LSTM) and Bidirectional LSTM (BiLSTM). Jan Neerbeky et al. [5,6] proposed converting text into syntactic trees and recursively generating semantic embeddings of phrase structures from the bottom up. These embeddings are then represented using Recursive Neural Networks (RvNNs) to achieve contextualized sensitive information recognition. Xu et al. [7] argued that the Recursive Neural Network (RvNN) approach is less efficient and proposed using the convolutional kernels and pooling layers of Text-CNN to capture text features for document classification and sensitive document recognition. Yan Li et al. [8] combined Convolutional Neural Networks (CNNs) and Bidirectional Long Short-Term Memory networks (BiLSTMs) to extract both local and global document features, followed by feature fusion for improved representation. Antreas Pogiatzis et al. [9] combined Bidirectional Long Short-Term Memory networks (BiLSTMs) with Conditional Random Fields (CRFs) to achieve context-aware recognition of sensitive information in dialogue data.

Yongyan Guo et al. [10] proposed the ExSense framework, which employs BERT to encode text and integrates BiLSTM with attention mechanisms to recognize sensitive information entities within documents. Meng Ding et al. [11] crawled news articles and comments from Chinese internet platforms to construct a Chinese sensitive information corpus and fine-tuned the BERT model for sensitive information recognition. However, this corpus has temporal limitations and is not publicly available; therefore, it was not used for experiments in this paper. Kai Cong et al. [12] proposed the KGDetector framework, which leverages pre-trained Chinese knowledge graphs (CN-Dbpedia) and BERT to obtain entity-level embedded representations and fuse them to enhance sensitive information classification. Anand S et al. [13] utilized the pre-trained BERT model and its variant CharacterBERT to obtain word and character embeddings of input text, respectively, and combined them with Conditional Random Fields (CRF) to recognize sensitive information entities. Chenglong Ren et al. [14] employed the ELECTRA model to encode characters, combined with BiLSTM and Conditional Random Fields (CRF), to recognize sensitive information in Chinese text.

However, methods based on rules, statistical learning, and traditional machine learning suffer from limitations such as restricted rule coverage, dependence on statistical features, and the requirement for large amounts of manually labeled data. Consequently, these methods tend to be less efficient and less accurate. In recent years, the emergence and rapid development of Large Language Models (LLMs) have opened new avenues for unstructured sensitive data recognition.

Shuhe Wang et al. [15] designed a prompt framework for NER tasks and first proposed the idea of optimizing prompt samples learned from fewer samples using a sample search strategy. This work also designed an output format that standardizes the large language model’s output by preserving the original text and marking recognized entities with special symbols. Dhananjay Ashok et al. [16] were the first to introduce chain-of-thought prompting in large-model NER tasks. Their approach includes providing the entity definition and a complete example in the prompt, followed by the corresponding answer and explanation as a thought chain. Although NER tasks generally do not require complex reasoning like question-answering or arithmetic, incorporating thought chains still enhances recognition performance to some extent. Mingchen Li et al. [17] combined the approaches mentioned above and applied them to the NER task in the biomedical domain. Unlabeled training texts were first annotated using GPT-4. Then, BERT was employed to obtain encoded representations of the sample sentences, while K-nearest neighbors (k-NN) was used for sample retrieval. A chain-of-thought was constructed using a Prompt-NER-like approach. Their findings indicate that thought chains containing both positive and negative samples outperform those consisting solely of positive samples. Anand [18] leveraged ChatGPT to recognize sensitive attribute columns in structured data under a zero-shot learning setting. Ruoyu Wen et al. [19] explored the effectiveness of zero-shot learning using models such as GPT-4 and GPT-4o for detecting sensitive information in the mental health domain.

In summary, the development of unstructured sensitive data recognition technology has evolved from rule-based matching to statistical learning and deep learning methods, and more recently has advanced to approaches that leverage large language models. Meanwhile, the task of recognizing unstructured sensitive data shows a high degree of similarity to named entity recognition (NER). As a result, cutting-edge techniques from the NER domain can be effectively transferred to unstructured sensitive data recognition tasks to enable more accurate recognition of specific sensitive entity types.

However, most existing methods rely on complex prompt engineering or chain-of-thought strategies, which increase computational costs and model overhead. Furthermore, they are often insufficiently adapted to Chinese sensitive data, as the majority focus on English corpora or structured data. Moreover, few approaches integrate prompt optimization, few-shot learning, efficient fine-tuning, and multi-instance integration, resulting in limited effectiveness across domains and in high-accuracy scenarios. To address these challenges, we propose the CSIER-FM framework, a Chinese sensitive information entity recognition method that combines prompt optimization, few-shot learning, parameter-efficient fine-tuning, and multi-instance integration to enhance recognition accuracy.

3. Methods

To address the shortcomings of previous methods for unstructured Chinese sensitive data recognition, such as low accuracy and limited contextual understanding, we propose an unstructured Chinese sensitive data recognition framework based on NER, integrating few-shot learning and multi-instance approaches.

3.1. Methodological Framework for Chinese Sensitive Information Recognition

Figure 1 illustrates the overall framework of Chinese sensitive information entity recognition based on large language models. CSIER-FM leverages the strong few-shot learning and transfer learning capabilities of large language models. Specifically, the framework consists of 2 stages: the construction of prompts and the optimization of large language model recognition performance. In the prompt construction stage, we drew inspiration from NER tasks, transforming Chinese sensitive information recognition into a sensitive information entity recognition task and building the prompts accordingly. Additionally, to better integrate few-shot learning into prompt construction, we employed a k-NN strategy for sample selection. In the optimization of the LLM stage, we applied LoRA fine-tuning to the Qwen2.5-7B model. Furthermore, we adopted a model multi-instance integration approach to further improve F1-score.

Figure 1. Framework of Chinese sensitive data recognition (CSIER-FM).

• Prompt Construction. To enable LLM to extract sensitive entity information from target text, we first need to design appropriate prompts for the sensitive information recognition task. Drawing on the paradigm of using large models for NER, we developed a prompt framework tailored for Chinese sensitive information recognition.
• Sample Selection Strategy. We incorporated few-shot learning techniques into the prompt templates to enhance recognition performance. To effectively leverage the few-shot learning capabilities of large models, we designed a k-NN sample selection strategy to choose samples most similar to the target text. This approach aims to provide the model with more relevant task examples, thereby improving its effectiveness in recognizing sensitive information within the target text.
• Model Fine-tuning. We locally deployed the Qwen2.5-7B model for LoRA fine-tuning and compared its fine-tuned performance with the DeepSeek-V3 online model.
• Model Multi-Instance Integration. To further enhance recognition performance, we propose an LLM-based multi-instance ensemble method. Building on the fine-tuning approach, we generate multiple model instances, each specialized in recognizing a specific type of sensitive information. The outputs from these instances are then integrated to form the final recognition result.

3.2. Prompt Construction

In the context of large language models, a prompt refers to a piece of instructional text that guides the model toward producing the desired output. Constructing well-designed prompts for specific tasks can effectively enhance an LLM’s understanding of the task and improve the accuracy of its output.

In addressing sensitive entity recognition tasks on unstructured Chinese text, we propose a prompt framework consisting of 4 key elements: (1) role and task description, (2) entity information to be recognized, (3) output description of the model, and (4) few-shot learning. Details on the prompt format are provided in Appendix A.

• Prompt = {
•   Roles and Task Descriptions;
•   Entity Information to Be Recognized;
•   Output Description of the Model;
•   Few-shot learning.
• }

(1)

Roles and Task Descriptions

For the recognition of sensitive data in unstructured Chinese text, it is essential to first clarify the role of the large language model and the specific task to be performed. This helps constrain the model’s identity, maintain key point on the task, and enhance recognition consistency, thereby improving overall recognition accuracy.

It can be formalized as follows:

• Roles and Task Descriptions = {
•   You are an expert in Chinese named entity recognition. Your task is to
•   identify entities of the type {entity_label} ({entity_type}, {entity_
•   description}) from the given text. Please output the results strictly
•   in JSON format.
• }

Here, “entity_label (entity_type, entity_description)” represents the entity information to be recognized in subsequent steps.

(2)

Entity Information to Be Recognized

Since the text to be analyzed may contain multiple types of entity information, with an uncertain number of entities within each type, we adopt a multi-round dialogue approach to design prompts that recognize each type of sensitive entity separately. In each dialogue round, the model focuses on recognizing only one specific entity type.

Therefore, the prompts must include detailed information about the entity type currently targeted for recognition, such as the entity label, entity type, and entity description. This information aids the model in accurately understanding and recognizing the sensitive information entities.

It can be formalized as follows:

• Entity Information = {
•   {entity_label} ({entity_type}, {entity_description})
• }

(3)

Output Description of the Model

To enable the model to better understand and perform its tasks, as well as to facilitate the subsequent processing of its outputs, an output description section is necessary to inform the model user of the expected output format and content.

For the task of sensitive information entity recognition, the key outputs typically include the recognized entity text, the entity’s location within the text, the entity type, and other relevant attributes. To recognize the most effective output format, we designed 2 MRC-based output description formats: MRCSIM and QWEN. And we conducted experiments to compare their performance with the MRC [20] and LABEL [15] formats.

Format 1: MRC Format [20]

MRC format requires the model to output key information, including “context”, “entity_text”, “span_position” and “entity_label”. In this context, “context” refers to the complete original text. “entity_text” denotes the specific entity identified within the text. “span_position” indicates the start and end positions of this entity in the original text. Finally, “entity_label” represents the category assigned to the entity by the large language model. All content is organized into a single string that conforms to the JSON format, thereby simplifying the subsequent extraction and processing of sensitive information.

It can be formalized as follows:

• {
•   “context”: “original text to be recognized”,
•   “entity_text”: [entity_text],
•   “span_position”: [entity_start_position, entity_end_position],
•   “entity_label”: entity_type_label
• }

Format 2: MRCSIM Format

Since the MRC format outputs more detailed information, it can pose challenges for smaller-scale models with limited ability to follow instructions. These challenges may include difficulties in producing the required output format or reduced output accuracy. To simplify the output description, the MRCSIM format is designed to exclude the “context” in the MRC format.

It can be formalized as follows:

• {
•   “entity_text”: [entity_text],
•   “span_position”: [entity_start_position, entity_end_position],
•   “entity_label”: entity_type_label
• }

Format 3: QWEN Format

To further simplify model outputs and enhance the model’s instruction-following ability and output stability, the QWEN format is designed to require the model to output only the recognized entity text and its corresponding entity type.

It can be formalized as follows:

• {
•   “entity_text”: [entity_text],
•   “entity_label”: entity_type_label
• }

Format 4: LABEL Format [15]

The previous 3 formats require the model to output key sensitive information integrated into a JSON format. In contrast, the LABEL format does not impose JSON formatting requirements. Instead, it instructs the model to output the raw text with recognized entities marked by special symbols. Specifically, the symbol “@@” is inserted before the start position of each recognized entity, and “##” is appended after its end position. The model outputs the text with these markers, and entity extraction is performed during subsequent processing.

It can be formalized as follows:

• {
•   “Input”: “raw text entity raw text”,
•   “Output”: “raw text @@entity ## raw text”
• }

(4)

Few-shot Learning

After defining the role and task descriptions, specifying the entity information to be recognized, and selecting the optimal output format, we incorporate the few-shot learning technique into the prompt design. Few-shot learning techniques attach a small number of labeled examples to the prompt, providing the model with additional task-relevant information through validated input–output pairs. This enables the model to better understand and perform tasks involving the recognition of sensitive information in unstructured Chinese text, thereby improving overall accuracy. Additionally, to enhance sample selection, we adopted a k-NN strategy based on cosine similarity.

3.3. Sample Selection Strategy

When combined with few-shot learning, most existing approaches rely on random sampling strategies. However, such methods often result in the selection of low-quality samples. To better leverage few-shot learning and provide more relevant samples for large language models, we replaced random selection with a k-NN–based sample selection strategy.

As shown in Figure 2, our approach begins by employing the RoBERTa model [21] as an encoder to transform the sample dataset into dense vector representations. These vectors capture the semantic information of the samples in a high-dimensional space, enabling more accurate comparisons between texts. Simultaneously, we use RoBERTa to encode the text to be identified into vectors. After obtaining the embedding vector, we calculate the cosine similarity between the target sample and the sample dataset. This similarity measure allows us to identify samples that are semantically closer to the target text. Finally, we select the top k most similar samples based on their similarity scores and use them as few-shot learning examples.

Figure 2. k-NN sample selection strategy.

In this manner, the sample information embedded within our constructed prompts maintains a high degree of semantic similarity to the carefully selected examples. By leveraging these representative samples, the large language model can more effectively capture the contextual patterns and entity characteristics relevant to the target task. This not only improves the model’s ability to interpret and follow task-specific instructions but also provides more accurate guidance in distinguishing subtle differences between entity types.

3.4. Model Fine-Tuning and Multi-Instance Integration

We initially considered using DeepSeek online for unstructured Chinese sensitive data recognition; however, the results did not meet our expectations. Consequently, we opted to deploy the LLM locally and fine-tune it to achieve higher recognition accuracy.

Large language models deployed locally are constrained by limited computational resources, making it challenging to utilize models with a large number of parameters. Smaller-scale models, however, often exhibit suboptimal performance and accuracy in understanding tasks and following prompted instructions. Parameter-efficient fine-tuning techniques can inject domain-specific knowledge into these models, optimizing them for particular tasks while preserving their foundational capabilities. That effectively balances deployment resource requirements with task performance. Specifically, we employ the Low-Rank Adaptation (LoRA) technique to achieve efficient parameter fine-tuning for our models.

However, during this process, some unresolved issues remain in the field of LLM recognition. For instance, the model may assign the same entity to multiple categories across different rounds of Q&A, which negatively impacts the accuracy of sensitive data recognition. To address this issue, we propose a model multi-instance integration method. It constructs multiple model instances, each dedicated to recognizing a specific type of sensitive information entity, and then integrates the outputs from all instances to produce the final recognition result.

As shown in Figure 3, we first classify the training dataset according to entity types. Based on these tailored train datasets, the base model is fine-tuned separately for each entity type, resulting in multiple specialized recognition models.

Figure 3. Framework for Model Multi-Instance Integration Methodology.

During the recognition stage, these fine-tuned recognition models are individually applied to their corresponding entity categories. In this way, each model is responsible only for recognizing a single type of sensitive entity, which reduces the likelihood of cross-category interference and confusion. By narrowing the scope of recognition tasks, the models can better capture the semantic and contextual characteristics unique to their assigned entity type. This specialization not only improves recognition precision but also enhances the robustness of the overall system. Ultimately, the integration of these multiple fine-tuned models allows for comprehensive coverage of sensitive information entities, leading to a more accurate and reliable recognition framework.

4. Experiments

4.1. Experimental Dataset and Model Selection

(1)

Experimental Dataset

To evaluate the accuracy and effectiveness of CSIER-FM, our method primarily focuses on selecting datasets for the task of Chinese sensitive information entity recognition. Given the similarity between named entity recognition tasks and sensitive information entity recognition tasks, and the current lack of publicly available datasets for sensitive information recognition, this paper utilizes the ResumeNER dataset to evaluate the effectiveness of CSIER-FM.

The ResumeNER dataset, published by Zhang et al. [22] in 2018, is designed for the task of Chinese named entity recognition and comprises 1027 executive resumes from Chinese listed companies. The ResumeNER dataset is a widely used benchmark for NER tasks. Since resumes typically contain sensitive information such as phone numbers and locations, we selected this dataset to evaluate the effectiveness of the proposed CSIER-FM framework. We cleaned and segmented the text contained in the ResumeNER dataset and annotated it using the BIO annotation scheme, where “B” denotes the beginning of an entity, “I” denotes the inside or continuation of an entity, and “O” denotes non-entity tokens. Subsequently, by combining the BIO tags with entity type labels, we annotated the entity types and their locations separately.

The ResumeNER dataset contains eight types of entities, as shown in Table 2. The ResumeNER dataset includes entity types that can serve as sensitive information entities, so we used it to train and evaluate our model. The dataset comprises 3821 samples in the training set, 463 samples in the validation set, and 477 samples in the test set.

Table 2. Entity Type Included in the ResumeNER Dataset.

Entity Type	Label	Definition
Country	CONT	Country of Citizenship
Educational Background	EDU	Academic Background (Degree or University)
Location	LOC	Geographically Classified Countries, Cities,
		Towns and Continents
Name	NAME	Names of Real or Fictional Persons
Organization	ORG	Organization Names (e.g., Companies, Political
		Parties, Educational Institutions, News Outlets)
Profession	PRO	The Field of Study or Profession One Pursues
RACE	RACE	An Individual’s Ethnic Group
Title	TITLE	Professional Titles, Achievements, and Roles

(2)

Model Selection

Both the Qwen2.5-7B model and the Deepseek model are well suited to the Chinese semantic environment. Since our approach is primarily designed for processing sensitive information in unstructured Chinese text, we selected Qwen2.5-7B as the base model for fine-tuning, and used Deepseek-V3 as an online model. We initially experimented with the DeepSeek-V3 online model, but its $F_1$ fell below 86%, failing to meet expectations. Consequently, we deployed the Qwen2.5-7B model locally and applied LoRA fine-tuning to achieve a higher $F_1$.

4.2. Experimental Environment and Evaluation Metrics

(1)

Experimental Environment

This experiment was conducted in a Linux environment, with the related experimental setup detailed in Table 3.

Table 3. Experimental Environment Configuration.

Environment	Configuration
CPU	Intel(R) Xeon(R) Gold 6248R Processor
GPU	2 × NVIDIA RTX A6000
Memory	512 GB
OS	Ubuntu 22.04.3 LTS (64 bit)
Python	Python 3.12.9
CUDA	12.7
Pytorch	2.7.0
PEFT	0.13.2

(2)

Evaluation Metrics

In this experiment, we use $precision$, $recall$, and $F_1$ as evaluation metrics to measure the effectiveness of sensitive information entity recognition. The $precision$ is the proportion of samples predicted by the model as positive that are actually positive. The $recall$ is the proportion of actual positive samples that are correctly identified as positive by the model. The $F_1$ is the harmonic mean of $precision$ and $recall$, combining the $F_1$ and $recall$ into a single measure that provides a more comprehensive evaluation of the model’s performance.

CSIER-FM is essentially a classification task, so we focus primarily on the $F_1$ as the metric for evaluating experimental results, with $precision$ and $recall$ serving as complementary indicators. The formulas for $precision$, $recall$, and $F_1$ are given in Equations (1)–(3). TP refers to positive samples correctly predicted as positive by the model, FP refers to negative samples incorrectly predicted as positive, and FN refers to positive samples incorrectly predicted as negative. We adopt entity-level exact matching and compute the $F_1$ using the micro-averaging method.

$$precision={\displaystyle \frac{TP}{TP+FP}}$$

(1)

$$recall={\displaystyle \frac{TP}{TP+FN}}$$

(2)

$$F_1={\displaystyle \frac{2\times precision\times recall}{precision+recall}}$$

(3)

4.3. Experimental Design

To evaluate the effectiveness of the CSIER-FM, we design experiments using the ResumeNER dataset. These experiments focus on 4 key technical aspects: prompt construction, sample selection strategy, model fine-tuning, and model multi-instance integration, aiming to comprehensively verify the CSIER-FM presented in this paper.

4.3.1. Experiment on Prompt Design

As described in Section 4.1, we evaluated the effectiveness of prompt construction by conducting experiments using the online Deepseek-V3 and Qwen2.5-7B models. This experiment confirm the effectiveness of prompts in 4 different output formats for Chinese sensitive entity recognition. For sample selection in this experiment, a one-shot strategy was employed, with a carefully chosen fixed sample for each entity type used to construct the prompt.

The specific experimental results are shown in Table 4. On the locally deployed Qwen2.5-7B model, considering $F_1$ alone, the QWEN prompt format achieved the best performance, reaching an $F_1$ of 0.7120. The MRCSIM output format ranked second, while the LABEL format performed the worst. In addition, from MRC to MRCSIM to QWEN, the requirements for output formats gradually decrease, while the $F_1$ correspondingly increases. This indicates that for Qwen2.5-7B, fewer output format constraints lead to better performance.

Table 4. Experimental Results of Prompt Construction Strategies.

Model	Prompt Format	$\mathit{precision}$	$\mathit{recall}$	${\mathit{F}}_1$
Qwen2.5-7B	QWEN	0.7158	0.7083	0.7120
	MRCSIM	0.6875	0.7253	0.7059
	MRC	0.7903	0.5104	0.6203
	LABEL	0.4828	0.4375	0.4590
Deepseek-V3	QWEN	0.6668	0.8246	0.7549
	MRCSIM	0.6453	0.8265	0.7247
	MRC	0.7834	0.8026	0.7929
	LABEL	0.4464	0.7768	0.5670

For the online Deepseek-V3 model, prompts using the MRC output format perform best, followed by the QWEN format, while prompts in the LABEL format perform the worst. When using DeepSeek-V3, although MRC’s score is only about 3.80% higher than QWEN’s $F_1$, the MRC output format is considerably more complex compared to QWEN’s. However, Qwen2.5-7B achieved a 9.17% performance improvement with the QWEN output format compared to the MRC output format. Further experiments showed that even with DeepSeek-V3, the QWEN output format consistently delivered the best results. Consequently, we adopted the QWEN output format as the primary prompt design for model fine-tuning and subsequent experiments.

To sum up, without fine-tuning and under the one-shot setting, the QWEN prompt format yields comparatively better results than the other three prompt formats discussed in Section 3.2. This may be because the model is not required to output the original text and outputs less entity-related information. On the one hand, this allows the model to focus more on key details such as entity text and location; on the other hand, it reduces the chances of errors caused by excessive output content.

At the same time, the performance of the LABEL format lags behind the other three formats, possibly because the model’s ability to understand and follow the special markup is not as strong as its ability to generate the standardized JSON output required by the other prompt formats.

4.3.2. Experiment on Sample Selection Strategies

We conducted 2 experiments on Deepseek-V3 and Qwen2.5-7B to evaluate the effectiveness of the k-NN sample selection strategy.

(1)

Evaluation of Sample Selection Strategies on Deepseek-V3

In the sample selection strategy evaluation experiments, we employed the online Deepseek-V3 and Qwen2.5-7B models. Based on the findings in Section 4.3.1, the QWEN prompt format demonstrated superior and more stable performance in prompt construction. Therefore, the QWEN format was primarily used throughout this experiment to evaluate the effectiveness of different sample selection strategies.

We first employ the Deepseek-V3 model to evaluate the recognition performance under different sample selection strategies. As outlined in Section 3.3, two strategies are considered in this experiment: k-nearest neighbor (k-NN) selection and random selection. To account for both the diminishing marginal returns in recognition performance when using more samples and the input length limitations of large language models, we cap the number of selected samples at 20.

The experimental results are presented in Figure 4. When using the $F_1$ as the evaluation metric, the k-NN selection strategy consistently outperforms the random selection strategy. For the k-NN selection strategy, the $F_1$ reaches its maximum value of 0.8555 when $k=17$. This is because the k-NN-based sample selection strategy, which selects samples based on textual similarity, ensures that the examples provided are closely related to the target task. This more targeted support helps the model generalize more effectively in few-shot learning, thereby significantly enhancing its ability to recognize sensitive information entities. However, when k becomes excessively large, the inclusion of too many example samples in the prompt may reduce the model’s focus on the specific task, thereby lowering the $F_1$.

Figure 4. $F_1$ Comparison of k-NN and Random Selection Strategies on Deepseek-v3 (mean ± 95% CI).

Overall, the effectiveness of the random selection strategy is inferior to that of the k-nearest neighbor (k-NN) selection approach.

(2)

Evaluation of Sample Selection Strategies on Qwen2.5-7B

In addition to the experiments conducted with the online Deepseek-V3, we further evaluated the recognition performance of the locally deployed, non-fine-tuning Qwen2.5-7B model. In this experiment, prompts were constructed using the QWEN format, and a few-shot learning strategy based on k-nearest neighbor (k-NN) selection was applied. The experimental results are presented in Figure 5. Consistent with the results on Deepseek-V3, the k-NN selection strategy outperformed the random selection strategy on Qwen2.5-7B. However, it achieves its maximum value of 0.7106 when $k=18$.

Figure 5. $F_1$ Comparison of k-NN and Random Selection Strategies on Qwen2.5-7B (mean ± 95% CI).

Although the k-NN selection strategy achieved an $F_1$ of 0.8555 on Deepseek-V3, the result still fell short of expectations. The base Qwen2.5-7B model performed even lower, with an $F_1$ of only 0.7106. Therefore, we proceeded to fine-tune Qwen2.5-7B to further enhance its performance.

4.3.3. Experiment on Model Fine-Tuning and Multi-Instance Integration

(1)

Model fine-tuning

To evaluate the impact of introducing a parameter-efficient fine-tuning technique, we used the Qwen2.5-7B model as the base model for fine-tuning. Considering the experimental environment and resources outlined in Table 3, we applied the LoRA method for efficient parameter fine-tuning and evaluated the fine-tuned model’s ability to recognize sensitive entities in unstructured Chinese text. The model fine-tuning utilizes a k-NN sample selection strategy, with k set to 17.

We constructed fine-tuning instruction sets based on the prompt words from each of the 4 formats and applied them to fine-tuning the base model accordingly. The relevant parameter settings for the LoRA fine-tuning process are summarized in Table 5.

Table 5. LoRA Fine-Tuning Parameter Configuration.

Parameter	Value
Rank	8
Alpha	32
Dropout	0.1
Learning Rate	$1\times {10}^{-4}$
Batch Size	4
Epochs	2
Target Modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Optimizer	AdamW
Max Sequence Length	2048
Embedding Dimension	3584
Task Type	CAUSAL-LM
Learning Rate Scheduler	LINEAR
Merge Rules	Dynamic Merge
Warmup Steps	0
Logging Steps	10
Save Steps	100
Gradient Checkpointing	True

The experimental results of model fine-tuning using the 4 prompt formats are presented in Table 6. After fine-tuning Qwen2.5-7B with LoRA, the $F_1$ across all 4 output formats showed significant improvement. Specifically, the QWEN output format achieved the highest performance, with its $F_1$ increasing from 0.7106 to 0.9518. Compared with the other output formats, the QWEN format consistently delivered the best results. Additionally, the fine-tuned Qwen2.5-7B achieved an $F_1$ that significantly surpassed Deepseek-V3’s 0.8555 on the experimental dataset, demonstrating the effectiveness of our fine-tuning approach in improving accuracy.

Table 6. Experimental results of fine-tuned models with different prompt formats. (mean ± 95% CI).

Prompt Format	$\mathit{precision}$	$\mathit{recall}$	${\mathit{F}}_1$
MRC	0.9449 ± 0.0008	0.9321 ± 0.0012	0.9385 ± 0.0007
LABEL	0.9749 ± 0.0005	0.9187 ± 0.0016	0.9460 ± 0.0010
MRCSIM	0.9391 ± 0.0007	0.9528 ± 0.0013	0.9459 ± 0.0010
QWEN	0.9476 ± 0.0016	0.9561 ± 0.0012	0.9518 ± 0.0010

(2)

Model multi-instance integration

Although the fine-tuned Qwen2.5-7B reached an $F_1$ of 0.9518, it still does not fully meet our expectations. When recognizing multiple entity types, an LLM may assign the same entity to different categories across separate rounds of question-answering, thereby reducing the accuracy of sensitive data recognition. Therefore, to further enhance recognition accuracy, we employed a multi-instance integration approach.

To evaluate the effectiveness of the model’s multi-instance integration approach, we use the QWEN prompt format along with the training set of the ResumeNER dataset to construct fine-tuning instruction sets for different entity types. Afterwards, we perform LoRA fine-tuning of Qwen2.5-7B using multiple fine-tuning instruction sets, resulting in several recognition model instances tailored to specific entity types. The final recognition result is determined by integrating the recognition results of each entity type. The recognition performance of each instance, as well as the results of the model multi-instance integration, is shown in Table 7.

Table 7. Experimental Results of Model Multi-Instance Integration. (mean ± 95% CI).

Entity Type	$\mathit{precision}$	$\mathit{recall}$	${\mathit{F}}_1$
CONT	1.0000 ± 0.0000	1.0000 ± 0.0000	1.0000 ± 0.0000
EDU	0.9657 ± 0.0067	0.9554 ± 0.0096	0.9604 ± 0.0076
LOC	1.0000 ± 0.0000	1.0000 ± 0.0000	1.0000 ± 0.0000
NAME	0.9749 ± 0.0005	0.9187 ± 0.0016	0.9460 ± 0.0010
ORG	0.9286 ± 0.0045	0.9555 ± 0.0048	0.9419 ± 0.0045
PRO	0.8919 ± 0.0000	1.0000 ± 0.0000	0.9429 ± 0.0000
RACE	0.9217 ± 0.0229	1.0000 ± 0.0000	0.9591 ± 0.0126
TITLE	0.9560 ± 0.0029	0.9604 ± 0.0026	0.9582 ± 0.0024
Integration	0.9488 ± 0.0023	0.9619 ± 0.0031	0.9553 ± 0.0027

By comparing the experimental results in Table 6 and Table 7, we observed that the multi-instance fine-tuning outperforms the fine-tuning approaches, with the $F_1$ increasing from 0.9518 to 0.9553. At this stage, the $F_1$ of the CSIER-FM surpassed 0.35%, successfully meeting our requirements. This suggests that, compared with fine-tuning large models for multi-entity type recognition, employing model multi-instance integration can further enhance the discovery and coverage of sensitive information entities in the target data, reduce the number of missed entities, and thereby improve the overall recognition performance.

Table 8 presents the error analysis for the NER dataset. As shown, Missed Entity, Boundary Error, and Spurious Entity constitute the majority of errors, together accounting for 92%, while the remaining three types make up only 8%. This distribution indicates that most recognition errors arise from incomplete boundary detection or missed entity recognition, rather than from incorrect entity classification. These findings suggest that enhancing boundary recognition and improving the model’s ability to distinguish overlapping or ambiguous entities could further improve the overall performance of sensitive entity recognition tasks.

Table 8. Error Analysis Table.

Error Type	Count	Percentage	Description
Missed Entity	63	42.00%	The model fails to detect an existing entity.
Boundary Error	51	34.00%	Incorrect span detected.
Spurious Entity	24	16.00%	The model predicts a non-existent entity.
Type Error	7	4.67%	Wrong entity type assigned.
Nested Entity Error	3	2.00%	The model fails to detect nested entities.
Hallucination	2	1.33%	Model generates an entity not in the text.

Table 9 presents a comparative evaluation of our method and several baseline approaches on the ResumeNER dataset. Since the source code for ELECTRA-BiLSTM-CRF was unavailable, its results were adopted from Paper [14], while the results for all other methods were obtained through our own experiments.

Table 9. Comparison of model effects for the ResmumeNER dataset. (mean ± 95% CI).

Model	$\mathit{precision}$	$\mathit{recall}$	${\mathit{F}}_1$
RoBERTa-CRF	0.9239 ± 0.0040	0.9298 ± 0.0030	0.9268 ± 0.0021
SoftLexicon	0.9476 ± 0.0018	0.9456 ± 0.0014	0.9466 ± 0.0010
BERT-MRC-NER	0.9029 ± 0.0027	0.9626 ± 0.0030	0.9318 ± 0.0023
ELECTRA-BiLSTM-CRF	0.9531	0.9251	0.9389
CSIER-FM	0.9488 ± 0.0023	0.9619 ± 0.0031	0.9553 ± 0.0027

Experimental results show that CSIER-FM achieves superior $F_1$ compared with all baseline methods. In addition, CSIER-FM achieves a $recall$ score that is only 0.07% lower than BERT-MRC-NER. In terms of $precision$, CSIER-FM attains a result only 0.43% lower than ELECTRA-BiLSTM-CRF. Overall, the results indicate that CSIER-FM exhibits robust performance and is capable of addressing sensitive entity recognition tasks effectively.

4.3.4. Experimental Results on the Synthetic Sensitive Information Dataset

To facilitate transfer learning from general NER tasks to sensitive entity recognition, two sensitive data fields, ID and ADDR, were manually created using the ResumeNER dataset and randomly embedded into both the training and testing sets. The attribute information for ID and ADDR is summarized in Table 10. Through random sampling, a total of 1103 ID and 1154 ADDR data points were added to the training set, while 285 ID and 275 ADDR data points were added to the test set. Although these data points were inserted randomly into the text, we ensured that they did not interfere with existing entities.

Table 10. Newly Added Entity Types in ResumeNER Dataset.

Entity Type	Label	Definition
ID Number	ID	A person’s unique ID number.
Address	ADDR	A person’s residential or contact address.

The experimental results of the model’s multi-instance integration on these two entities are presented in Table 11. The integrated $F_1$ reached 0.9879, indicating that substituting entity information in ResumeNER with sensitive entities enables highly effective recognition of sensitive information. These results further demonstrate the strong capability of CSIER-FM in sensitive entity recognition. For the ADDR entity, the model successfully recognized all 275 entities in the test set. This result may be attributed to the high structural consistency among the generated ADDR samples, which uniformly follow the format: province–city–county–district–street–neighborhood–unit. Despite employing random combinations of up to 10 provinces, 27 cities, 79 districts, 7 subdistricts, 14 neighborhoods, and 20 units, the structural similarity of the ADDR data still led to model overfitting during training. Future work may involve modifying the ADDR structure and removing redundant components to improve data reliability.

Table 11. Experimental Results of Multi-Instance Integration for ID and ADDR. (mean ± 95% CI).

Entity Type	$\mathit{precision}$	$\mathit{recall}$	${\mathit{F}}_1$
ID	0.9696 ± 0.0014	0.9832 ± 0.0014	0.9763 ± 0.0014
ADDR	1.0000 ± 0.0000	1.0000 ± 0.0000	1.0000 ± 0.0000
Integration	0.9844 ± 0.0007	0.9914 ± 0.0007	0.9879 ± 0.0007

4.3.5. Experimental Analysis of Model Overhead

Additionally, we conducted supplementary experiments to analyze the training and testing time costs, the inference latency across different input lengths, and the variations in model throughput as the K value increases.

Table 12 presents the memory usage, as well as the training and testing times, for both the single fine-tuned model and the multi-instance integration. Since the ResumeNER dataset contains 8 types of entities, multi-instance integration requires training 8 separate models. Consequently, the GPU memory usage is approximately 8 times that of fine-tuning a single model. Furthermore, during the training and testing of multi-instance integration models, the 8 models can be processed in parallel, resulting in a total training and testing time that is significantly shorter than that required for a single model.

Table 12. Training and Testing Time Comparison.

Model	GPU Memory Usage (GB)	Train (min)	Test (min)	${\mathit{F}}_1$
Fine-tuning	15	190.25	40	0.9518 ± 0.0010
Multi-instance Integration	119	26.75	6	0.9553 ± 0.0027

Table 13 compares the model’s inference latency across input lengths of 1 K, 2 K, 4 K, and 10 K Characters. The inference latency increases significantly as the input length grows. The model records an average latency of 2.0859 s for 1 K-character inputs, which is considered acceptable. In contrast, processing 10K-character inputs results in a substantial increase to 52.6173 s, indicating a 25.2-fold rise in average time cost per instruction. This level of latency leads to a poor user experience. Therefore, we recommend limiting input content to no more than 1000 characters.

Table 13. Comparison of Inference Latency for Different Input Lengths (mean ± 95% CI).

Model	Inference Latency(s)
1 K Character	2.0859 ± 0.9958
2 K Character	3.8902 ± 2.3707
4 K Character	23.7519 ± 13.9335
10 K Character	52.6173 ± 13.6986

Figure 6 presents the changes in the processing throughput (instructions per second) of the LLM in the few-shot learning setting as the value of K increases. As K increases, the throughput gradually decreases, with a maximum reduction of 28.74%. Nevertheless, the decline remains within an acceptable range.

Figure 6. Instruction throughput versus K value in few-shot learning.

5. Discussion

The CSIER-FM framework incorporates multiple components specifically designed to improve the performance of sensitive entity recognition within NER tasks. These components include prompt design, k-NN sample selection, model fine-tuning, and multi-instance integration. Importantly, these 4 components do not operate independently; rather, they function in a sequential and complementary manner, with each component building upon the previous one to progressively enhance recognition performance. As a result, it is not feasible to conduct ablation experiments that isolate the effect of each component individually, because removing any single component would disrupt the intended workflow and compromise the overall performance. Nevertheless, empirical observations indicate that as these components are incorporated step by step, the F1-score of the system consistently improves. This progressive improvement provides indirect but compelling evidence of the effectiveness of each individual component. In addition, the synergistic interaction among the components highlights the importance of considering the CSIER-FM framework as an integrated pipeline rather than a collection of independent modules, emphasizing that the performance gains are a result of the combined contributions of all components rather than any single element.

6. Conclusions

As a cornerstone of data security governance, the recognition of unstructured Chinese sensitive data in the era of big data poses challenges to the processing capacity, efficiency, and accuracy of recognition models. To address these challenges, this paper proposes the CSIER-FM framework.

The primary goal in recognizing unstructured Chinese data is to recognize specific types of sensitive information entities within text. On the one hand, we designed prompt templates for the QWEN and MRCSIM output formats, both of which are simplified versions of the MRC output format. We also introduced few-shot learning and employed a k-NN sample selection strategy to construct the best prompts, thereby enhancing the model’s understanding of task descriptions and entity types in the target domain. On the other hand, we use LoRA fine-tuning to adapt large language models to sensitive entity recognition tasks with domain-specific data, and combine it with multi-instance integration to further improve recognition accuracy.

Experimental results evaluate the effectiveness of the proposed methods. Notably, the performance of the fine-tuned models surpasses that of larger, non-fine-tuned models in recognizing sensitive entities. Furthermore, applying few-shot learning techniques to non-fine-tuned models significantly improves their recognition capabilities. Evaluation results on the ResumeNER dataset, which contains unstructured Chinese text data, show that the CSIER-FM framework achieves an F1 score of 0.9553 in recognizing specific sensitive entity types. Furthermore, on the ResumeNER dataset, our method achieves a higher F1-score than the competing baseline methods. This indicates that the proposed CSIER-FM framework can effectively perform Chinese sensitive information recognition tasks.

Future research can explore more efficient sample selection strategies to further optimize the performance of few-shot learning in sensitive information recognition. Additionally, to reduce the overhead of multi-instance integration, future work may explore multi-category integration approaches, enabling fine-tuned models to recognize multiple entities within the same category and lowering training costs.