Breaking the Speed–Accuracy Trade-Off: A Novel Embedding-Based Framework with Coarse Screening-Refined Verification for Zero-Shot Named Entity Recognition

Abstract

Although fine-tuning pretrained language models has brought remarkable progress to zero-shot named entity recognition (NER), current generative approaches still suffer from inherent limitations. Their autoregressive decoding mechanism requires token-by-token generation, resulting in low inference efficiency, while the massive parameter scale leads to high computational and deployment costs. In contrast, span-based methods avoid autoregressive decoding but often face large candidate spaces and severe noise redundancy, which hinder efficient entity localization in long-text scenarios. To overcome these challenges, we propose an efficient Embedding-based NER framework that achieves an optimal balance between performance and efficiency. Specifically, the framework first introduces a lightweight dynamic feature matching module for coarse-grained entity localization, enabling rapid filtering of potential entity regions. Then, a hierarchical progressive entity filtering mechanism is applied for fine-grained recognition and noise suppression. Experimental results demonstrate that the proposed model, which is trained on a single RTX 5090 GPU for only 24 h, attains approximately 90% of the performance of the SOTA GNER-T5 11B model while using only one-seventh of its parameters. Moreover, by eliminating the redundancy of autoregressive decoding, the proposed framework achieves a 17× faster inference speed compared to GNER-T5 11B and significantly surpasses traditional span-based approaches in efficiency.

1. Introduction

Named Entity Recognition (NER), as a fundamental task in Natural Language Processing (NLP), plays an indispensable role in advanced applications such as text information extraction, question-answering systems, sentiment analysis, and machine translation. Its core objective is to automatically identify and extract entities with specific semantic features from text, such as person names, locations, organization names, and domain-specific entities [1]. Traditional NER models face significant limitations in recognizing entities from new domains or out-of-distribution (OOD) data, as expanding to new domains or identifying unseen entity types requires retraining with large-scale high-quality annotated datasets. The construction of such datasets typically requires substantial human and time resources, particularly in highly specialized domains such as medicine and law [2,3]. Additionally, traditional methods are strictly constrained by predefined entity category frameworks, with their classification systems being rigid and lacking dynamic adjustment capabilities, making it difficult to flexibly adapt to the emergence of new entity types or changes in the semantic boundaries of existing categories in real-world scenarios [4]. To address the dual challenges of scarce annotated data and dynamic entity category expansion, Zero-Shot named entity recognition (Zero-Shot NER) technology has emerged. Its core objective is to directly recognize open-domain or previously unseen entity categories through semantic reasoning and knowledge transfer, without relying on annotated data in the target domain.

In recent years, the development of large language models (LLMs), such as GPT and LLaMa [5,6], has brought transformative breakthroughs to the field of natural language processing. Through techniques like prompt engineering [7] or fine-tuning [8], researchers have achieved remarkable results in Zero-Shot learning scenarios for NER tasks [9]. These approaches effectively address the challenges of limited annotation resources and difficulties in entity category expansion. However, they still face two critical challenges in practical applications: (1) The training process requires exorbitant computational costs, relying on large-scale GPU clusters for computational support [10], while the massive parameter scales substantially increase storage and computational resource consumption. Furthermore, the extreme parameter size elevates deployment barriers, hindering model adaptation to resource-constrained environments such as edge devices and embedded systems [11]. (2) The inherent autoregressive generation mechanism necessitates iterative token-by-token processing. In long-sequence generation scenarios, this sequential processing nature leads to exponential growth in cumulative latency effects [12,13]. Achieving the optimal balance between model performance and resource efficiency while improving inference speed remains a critical technical bottleneck to overcome.

Meanwhile, span-based NER methods have also drawn wide attention. Such methods enumerate all possible contiguous spans in text and classify them, thereby performing entity detection and recognition, and can naturally handle nested entities and open-category entities [14,15]. However, these methods remain inefficient in real-world scenarios, as the candidate span space grows rapidly with text length, leading to prohibitively high computational and memory costs when handling long documents or large span windows [16,17]. In addition, extreme class imbalance between positive and negative spans causes training to be susceptible to noise, and insufficient boundary precision may lead to entity truncation or redundancy, especially in domain-specific texts [15,18]. Finally, processing long texts and multi-layer nested entities further increases the complexity of training and inference, reducing efficiency and generalization [17,19]. Consequently, neither autoregressive LLMs nor enumeration-based span approaches are capable of simultaneously balancing performance, computational efficiency, and generalization ability in zero-shot NER settings [20].

In this paper, we propose CSRVNER,  an Embedding-based universal named entity recognition framework based on a non-autoregressive architecture. Inspired by cognitive process modeling, this framework simulates the semantic understanding mechanism of human annotators by decomposing the NER task into three stages: “Reading → Annotation → Review”.The main contributions of this work are summarized as follows:

• We propose an encoding-based NER framework that simulates human cognitive processes through a Hierarchical Progressive Entity Filtering (HPEF) mechanism for open-type entity recognition. Experimental results show that the framework significantly improves recognition accuracy while maintaining highly efficient inference performance.
• To address the challenge of effective entity localization, we design a Dynamic Feature-to-Entity Mapping (DFEM) module. DFEM integrates entity description semantics with contextual text features through cross-attention and adaptive feature fusion, enabling dynamic modeling of contextual dependencies and improving semantic alignment. The resulting representations offer high-quality contextual semantics for the subsequent entity filtering stage.
• Building upon DFEM, we further introduce a Dynamic Span Feature Encoder (DSFE) as a refinement stage to enhance candidate entity representations. DSFE utilizes a multi-layer cross-attention framework for fine-grained semantic verification, enabling it to suppress low-confidence predictions and strengthen entity boundary coherence.

2. Related Work

2.1. Traditional NER Approaches

2.1.1. StatisticalMachine Learning Methods

Sequence labeling models centered on Conditional Random Fields (CRF) [21,22] and Support Vector Machines (SVM) [23], constructed through manual feature engineering (e.g., part-of-speech tags, morphological features), achieved notable progress on benchmark datasets like CoNLL-2003 [24,25,26,27,28]. These approaches faced dual constraints: heavy reliance on large-scale annotated data and difficulties in transferring handcrafted features across domains.

2.1.2. Deep Learning Methods

Neural architectures based on BiLSTM-CRF [29,30] and Transformer [31,32] automatically learned contextual semantic features through distributed representations. Pre-trained language models like BERT [33,34,35,36] further achieved performance breakthroughs on complex corpora such as OntoNotes 5.0 [37]. A fundamental limitation persists: dependency on predefined entity type schemas impedes dynamic adaptation to emerging entity types in open-domain scenarios. Both paradigms share the core characteristic of modeling constrained entity type spaces through either manual feature engineering or neural architectures, remaining bound by closed entity-type systems.

2.2. Span-Based NER Approaches

Unlike traditional token-level sequence labeling frameworks, span-based methods regard Named Entity Recognition as a span classification problem, where all possible contiguous token spans are enumerated and classified into entity categories or non-entity types. This paradigm effectively handles nested and overlapping entities that are challenging for sequence labeling models. Early span-based models [38,39,40] introduced exhaustive span enumeration followed by feed-forward classification, yet suffered from high computational cost due to the quadratic number of candidate spans. To mitigate this, several studies proposed span pruning and boundary refinement mechanisms [41,42,43], achieving significant improvements in efficiency and precision.

Recent advances further integrated span-based modeling with pre-trained encoders such as BERT and RoBERTa [44,45,46], leveraging contextualized embeddings to enhance span boundary sensitivity. Moreover, adaptive span selection techniques [47,48] and multi-stage filtering strategies have been developed to reduce negative span imbalance and improve entity boundary detection. Despite these improvements, span-based NER approaches still face limitations: the combinatorial explosion of candidate spans in long texts, imbalance between positive and negative samples, and challenges in adapting to unseen entity categories in open-domain scenarios. These constraints motivate the exploration of more efficient and flexible zero-shot span frameworks capable of dynamically aligning entity semantics without relying on predefined schemas.

2.3. Zero-Shot NER

Large language models (LLMs) introduced paradigm-shifting advancements in NER [49,50]. Recent studies demonstrate that LLMs exhibit remarkable Zero-Shot transfer capabilities through instruction tuning and knowledge distillation. InstructUIE [51] validated model generalizability across 14 information extraction datasets via a multi-task instruction framework, while UniversalNER [8] enhanced cross-domain adaptability through conversational training paradigms. GoLLIE [9] innovatively integrated code-style instructions to improve structured output capabilities. Furthermore, GNER [52] introduced negative samples during training and proposed an efficient Longest Common Subsequence (LCS) matching algorithm to further optimize Zero-Shot performance. Despite these advancements, LLMs face three persistent challenges in NER applications: (1) high computational costs during fine-tuning, (2) inference latency impacting real-time deployment, and (3) model compression difficulties in low-resource scenarios.

3. Method

This chapter systematically presents the proposed CSRVNER framework and its core modules, together with the training strategy and loss functions. As illustrated in Figure 1, the CSRVNER framework is designed as an efficient embedding-based system that operates in three stages: Read → Annotation → Review. The model takes the target text and entity description as inputs. In the Read stage, we employ Qwen2.5-1.5B as the backbone for feature extraction.

Specifically, we utilize the model’s inherent causal masking mechanism to extract high-dimensional hidden states via a single forward pass, without performing autoregressive token generation.

Subsequently, in the Annotation stage, the Dynamic Feature Extraction Module (DFEM) performs cross-attention interactions between the text and entity description to produce candidate entities and dynamic feature representations. Finally, in the Review stage, the Dynamic Semantic Filtering Engine (DSFE) refines and verifies candidate entities through fine-grained semantic validation. Together, DFEM and DSFE form the Hierarchical Progressive Entity Filtering (HPEF) mechanism, enabling progressive entity recognition from coarse detection to fine verification.

3.1. Hierarchical Progressive Entity Filtering

To improve both efficiency and accuracy in entity recognition, we propose a hierarchical progressive entity filtering mechanism. This mechanism consists of two layers: the first layer (DFEM)  locates potential entity spans, and the second layer (DSFE) evaluates and filters these candidates to retain only high-quality entities.

Formally, given an input text sequence T, the mechanism can be expressed as a two-step process:

$${\mathcal{C}}^{\left(1\right)}=\mathrm{DFEM}\left(T\right)$$

(1)

$$\widehat{\mathcal{C}}=\mathrm{DSFE}\left({\mathcal{C}}^{\left(1\right)}\right)$$

(2)

where ${\mathcal{C}}^{\left(1\right)}$ denotes the candidate entity set produced by DFEM through entity span localization, and $\widehat{\mathcal{C}}$ represents the final refined candidate entity set obtained by DSFE through semantic-based filtering.

Overall, the proposed mechanism operates under a coarse-to-fine, hierarchical filtering framework. Specifically, DFEM performs coarse-grained span detection to generate potential entity regions, while DSFE conducts fine-grained filtering to remove noisy or redundant spans, leading to significant gains in both precision and computational efficiency.

3.2. Dynamic Feature-to-Entity Mapping

Dynamic Feature-to-Entity Mapping module dynamically aligns text representations with entity descriptions through an iterative cross-attention refinement mechanism. Given input text enbedding matrix $A\in {\mathbb{R}}^{n\times d_{\mathrm{hid}}}$ and entity description matrix $B\in {\mathbb{R}}^{m\times d_{\mathrm{hid}}}$, the module operates as follows:

3.2.1. Iterative Attention Refinement

The representation is refined through T layers of multi-head attention:

$$A^{\left(t\right)}=\mathrm{LayerNorm}\left(A^{(t-1)}+\mathrm{MHA}(A^{(t-1)},B)\right)$$

(3)

where $A^{\left(0\right)}$ is the initial text enbedding and B contains fixed entity patterns.

3.2.2. Attention Computation

Each attention head computes token-description alignment:

$${\mathrm{Attn}}_h=\mathrm{softmax}\left({\displaystyle \frac{A{W}_{Q_h}{\left(B{W}_{K_h}\right)}^{\top }}{\sqrt{d_h}}}\right)B{W}_{V_h}$$

(4)

with projection matrices $W_{Q_h},W_{K_h},W_{V_h}\in {\mathbb{R}}^{d_{\mathrm{hid}}\times d_h}$ where $d_h=d_{\mathrm{hid}}/n_{\mathrm{head}}$. The multi-head output combines all heads:

$$\mathrm{MHA}(A,B)=\left[{\mathrm{Attn}}_1\oplus \dots \oplus {\mathrm{Attn}}_{n_{\mathrm{head}}}\right]W_O$$

(5)

3.2.3. Classification

After t refinement iterations, the final representation $A^{\left(T\right)}$ (DynaEmbed) is fed into a token-wise classifier:

$$P\left(y_i\right)=\mathrm{softmax}\left(\mathrm{MLP}\left(a_i^{\left(t\right)}\right)\right)$$

(6)

where $a_i^{\left(t\right)}$ denotes the refined embedding of the i-th token in $A^{\left(T\right)}$, and the MLP projects the embedding to entity-type logits. The model predicts entity labels by matching token representations with descriptions in B, enabling cross-domain generalization through description conditioning.

3.3. Dynamic Span Feature Encoder

To address the issue of false positives generated in the first stage, we introduce the Dynamic Span Feature Encoder (DSFE) as a second-stage Review Module. This module operates at the span level rather than the token level. It takes the candidate spans generated by the Annotation module and performs a fine-grained classification to determine the final entity type.

3.3.1. Candidate Span Generation via Greedy Strategy

The input to the DSFE relies on the output of the first-stage Annotation module. We adopt a greedy strategy to form candidate spans based on the discrete classification results: continuous sequences of tokens classified as entities are concatenated to form the candidate span set $\mathcal{S}=\{S_1,S_2,\dots ,S_K\}$. Here, each span $S_k$ corresponds to a start–end interval $[s_k,e_k]$. This strategy ensures that the DSFE focuses solely on plausible entity segments suggested during the first stage.

3.3.2. Span Representation and Fusion

For each candidate span $S_k$, we extract its dynamic feature sequence $H[s_k:e_k]$ from the shared encoder. To capture the interaction between the span content and specific entity types, we employ an n-layer cross-attention network:

$$Z_k^{\left(l\right)}=\mathrm{CrossAttn}(Q=Z_k^{(l-1)},K=\mathcal{E},V=\mathcal{E}),l=1,\dots ,n,$$

(7)

where $Z_k^{\left(0\right)}=H[s_k:e_k]$, and $\mathcal{E}$ represents the learnable embeddings of entity types. This mechanism aligns span-specific signals with semantic category features.

3.3.3. Span-Level Classification

Unlike the first stage, which predicts per-token labels, the DSFE aggregates the fused features of the entire span into a single vector representation via average pooling:

$$v_k=\mathrm{AvgPool}\left(Z_k^{\left(n\right)}\right),p_k=\mathrm{softmax}(W_c{v}_k+b_c)$$

(8)

• $p_k\in {\mathbb{R}}^{C+1}$ represents the probability distribution of span $S_k$ over entity categories (including a “None” class for invalid spans). The final decision depends entirely on this review result.

3.4. Training Objectives

This paper employs a two-phase training strategy to sequentially optimize token-level recall and span-level precision; during this process, Qwen remains frozen as the feature extraction model, and only the parameters of the DFEM and DSFE modules as well as the classifier are optimized.

3.4.1. Phase 1: Annotation Module (Token-Level Learning)

In the first phase, we train the Annotation module to identify potential entity boundaries. This is a token-level task utilizing a weighted cross-entropy loss:

$${\mathcal{L}}_{\mathrm{token}}^{\left(1\right)}=-{\displaystyle \frac{1}{N}}\sum _{i=1}^N{w}_i·y_{i}log\left({\widehat{y}}_i\right)$$

(9)

where N is the total number of tokens in the sequence, $y_i$ is the ground truth label for token i, and weights $w_i$ are set higher (e.g., 25) for positive tokens to improve recall.

3.4.2. Phase 2: Review Module (Span-Level Learning)

In the second phase, we freeze the parameters of the Annotation module and train the DSFE Review module. Crucially, the optimization target shifts from tokens to spans. Based on the candidate spans $\mathcal{S}$ generated by the greedy strategy in Phase 1, we minimize the span classification loss as follows:

$${\mathcal{L}}_{\mathrm{span}}^{\left(2\right)}=-{\displaystyle \frac{1}{K}}\sum _{k=1}^K{w}_k^{\prime }·Y_{k}log\left(P_k\right)$$

(10)

where

• K is the number of candidate spans generated by the greedy strategy.
• $Y_k$ is the ground truth label for the k-th span. We assign a positive entity label to $S_k$ if its Intersection over Union (IoU) with a ground truth entity exceeds 0.5; otherwise, it is labeled as “None” (invalid).
• $P_k$ is the model’s predicted probability that span $S_k$ belongs to category $Y_k$.
• $w_k^{\prime }$ is a weight coefficient assigned to positive entity spans to address the imbalance between valid entities and false-positive spans.

This formulation explicitly treats the review process as a span classification problem, ensuring high-quality entity recognition.

4. Experiment Settings

4.1. Training Data

We employ the Pile-NER dataset released by [8], which comprises 44,889 high-quality text entries containing 240,000 annotated entity instances and covering 13,000 distinct entity types, ensuring the training data’s advantages in both domain coverage and entity diversity. To facilitate the zero-shot implementation of our framework, we utilized GPT-4o to generate structured descriptions for the diverse entity types. The process involved designing tailored prompts to capture core characteristics (e.g., specific attributes for the event entity). The generated descriptions were subsequently calibrated through a post-processing step that strictly focused on ensuring linguistic fluency and definition generality. Further details regarding the prompt designs are provided in the Appendix A.

4.2. Evaluation

4.2.1. Datasets

We primarily evaluate our model under a Zero-Shot setting following established protocols from prior studies [8,51]. Formally, we define this setting as Zero-Shot Cross-Dataset Transfer: while the model benefits from broad semantic supervision during training, the target datasets are strictly unseen, and no fine-tuning or few-shot examples are provided during inference. This setting evaluates the model’s capability to handle domain shifts and generalize to new data distributions. Our evaluation framework comprises three benchmarks: Cross-Domain NER Benchmark (

Table 1. Zero-Shot Scores on the Out-of-Domain NER Benchmark. We compared our model with various Open NER models. Results for Vicuna, ChatGPT, and UniNER are from Zhou et al. [8]; USM and InstructUIE are from Wang et al. [51]; GoLLIE is from Sainz et al. [9]; GLiNER is from [10]; and GNER-FlanT5 is from [52].

Model	Params	AI	Literature	Music	Politics	Science	Average
Vicuna-7B	7B	12.8	16.1	17.0	20.5	13.0	18.5
Vicuna-13B	13B	22.7	22.7	26.6	27.0	22.0	24.2
USM	0.3B	28.2	56.0	44.9	36.1	44.0	41.8
ChatGPT-3.5	–	52.4	39.8	66.6	68.5	67.0	58.9
InstructUIE	11B	49.0	47.2	53.2	48.1	49.2	49.3
UniNER-7B	7B	53.6	59.3	67.0	60.9	61.1	60.4
UniNER-13B	13B	54.2	60.9	64.5	61.4	63.5	60.9
GoLLIE	7B	63.0	62.7	67.8	57.2	55.5	61.2
GLiNER-L	0.3B	57.2	64.4	69.6	72.6	62.6	65.3
CSRVNER	1.5B	57.6	64.2	71.5	70.2	65.6	65.8

), which integrates 5 multi-domain datasets from CrossNER [4], specifically designed to assess out-of-domain generalization capabilities of NER models; and Multi-Domain NER Benchmark (

Table 2. Zero-shot performance on 15 NER datasets. Results of ChatGPT and UniNER are reported from [8] and GLiNER is form [10]. The best results are highlighted in bold.

Dataset	ChatGPT	UniNER	GLiNER	CSRVNER
Params	-	7B	0.3B	1.5B
AnatEM	30.7	25.1	33.3	32.1
bc2gm	40.2	46.2	47.9	35.2
bc4chemd	35.5	47.9	43.1	44.5
bc5cdr	52.4	68.0	66.4	68.6
CoNLL03	52.5	72.2	64.6	71.5
FindVehicle	10.5	22.2	41.9	21.2
GENIA	41.6	54.1	55.5	57.8
HarveyNER	11.6	18.2	22.7	34.6
MultiNERD	58.1	59.3	59.7	71.6
ncbi	42.1	60.4	61.9	60.4
OntoNotes	29.7	27.8	32.2	42.5
PolyglotNER	33.6	41.8	42.9	51.7
TweetNER7	40.1	42.7	41.4	41.5
WikiANN	52.0	55.4	58.9	65.6
WikiNeural	57.7	69.2	71.8	75.6
Average	39.2	47.3	49.6	51.6

), covering 15 classical datasets across diverse domains.

4.2.2. BaseLines

To validate the effectiveness of our proposed framework, we compare CSRVNER with recent leading methods in open-domain NER, establishing a comprehensive benchmark for performance evaluation. We first evaluate prompting-based chat models, including ChatGPT and Vicuna [53], which adopt the prompting strategy proposed by [54]; their performance metrics are presented as reported by [8]. Additionally, we compare with the following large language models (LLMs) specifically fine-tuned for NER:InstructUIE [51], built upon the FlanT5-11B architecture and fine-tuned across multiple NER datasets; UniNER [8], which leverages a LLaMa model fine-tuned on ChatGPT-generated synthetic data; GoLLIE [9], based on CodeLLama and enhanced through guideline-aware fine-tuning to improve generalization on unseen information extraction tasks. Finally, we include comparisons with USM [55] and GLiNER [10], both employing compact architectures with reduced parameter sizes but differing in structural design.

4.2.3. Metrics

Evaluation follows the standard exact-match protocol for NER, in which F1-scores are computed by requiring complete agreement between predicted and annotated entities with respect to both span boundaries and entity categories.

5. Results and Analysis

5.1. Zero-Shot Performance

5.1.1. OOD NER Benchmark

We first evaluate our model on the out-of-domain (OOD) benchmark, as summarized in Table 1. The comparative results against various baseline models demonstrate the superior performance of our approach. Specifically, CSRVNER surpasses general-purpose language models such as ChatGPT and Vicuna, and further outperforms the 11B InstructUIE model, which is instruction-tuned specifically for NER tasks.

5.1.2. 15 NER Benchmark

Table 2 presents the comparative results on 15 diverse NER datasets, evaluated against ChatGPT, UniNER, and GLiNER. Consistent with the OOD benchmark results, ChatGPT performs substantially worse than fine-tuned models, lagging behind UniNER. CSRVNER achieves state-of-the-art performance on 8 datasets, outperforming GLiNER by an average margin of 2 percentage points, which underscores its robust cross-domain generalization and adaptability.

5.2. Inference Speed

Figure 2 shows that CSRVNER (1.5B) achieves an outstanding balance between performance and efficiency in zero-shot NER tasks. Although its parameter size is only one-seventh that of GNER-T5-xxl (11B), CSRVNER attains approximately 90% of its performance while achieving a 17× faster inference speed. This clearly demonstrates that CSRVNER greatly enhances computational and deployment efficiency while maintaining high accuracy.

To further evaluate the model’s efficiency in long-text scenarios, we conducted a comparative experiment against the span-based baseline, W2NER. Note that GLiNER was excluded from this comparison as it does not support the extraction of excessively long texts. We measured Latency, Throughput, and GPU Memory usage (VRAM) across input sequence lengths of 1000, 2000, and 3000 tokens. As shown in

Table 3. Comparison of inference speed and memory usage between W2NER and our proposed CSRVNER across different input lengths. Thr. denotes Throughput. The best results are highlighted in bold. ↑ indicates higher is better; ↓ indicates lower is better.

Length	W2NER			CSRVNER (Ours)
	Latency	Thr.	VRAM	Latency	Thr.	VRAM
	(ms)↓	(token/s)↑	(GB)↓	(ms)↓	(token/s)↑	(GB)↓
1000	85.61	11,680.96	5.75	273.98	3649.91	8.11
2000	2384.00	838.93	14.10	520.34	3843.65	8.53
3000	5287.26	567.40	27.94	971.28	3088.72	9.15

, while W2NER exhibits lower latency on shorter sequences (1000 tokens), its computational cost increases drastically as the sequence length grows. Specifically, at a length of 3000 tokens, W2NER’s VRAM consumption surges to 27.94 GB with a significant drop in throughput to 567.40 tok/s, likely due to the quadratic complexity inherent in its grid-tagging architecture. In contrast, our CSRVNER demonstrates superior scalability. Even at 3000 tokens, CSRVNER maintains a stable throughput of over 3000 tok/s and consumes only 9.15 GB of VRAM. These results confirm that CSRVNER is significantly more efficient and suitable for processing long documents compared to traditional span-based approaches.

5.3. Ablation Studies

This study systematically validates the effectiveness of core model components through ablation experiments, with experimental designs detailed as follows.

5.3.1. Effectiveness Evaluation of DFEM

To further demonstrate the contribution of DFEM, we design a controlled ablation study comparing two configurations of the model. (1) Direct concatenation of entity descriptions with input texts followed by traditional self-attention for entity localization (SA-Concat). (2) The dual-stream interactive architecture adopted in DFEM incorporates a cross-attention mechanism (CA-Cross). All experiments are conducted under strictly controlled conditions, with identical training parameters and a consistent phased training strategy,

Table 4. Cross-attention (CA-Cross) vs. concatenation-based self-attention (SA-Concat) for entity localization, showing an average F1 improvement of $\Delta$ = 30.7 across five benchmarks. The best results are highlighted in bold.

Dataset	SA-Concat	CA-Cross
AI	31.1	57.6
Literature	28.9	64.2
Music	34.0	71.5
Politics	39.5	70.2
Science	41.9	65.6
Avg.	35.1	65.8

demonstrates that CA-Cross achieves significantly superior F1-scores across all seven benchmark datasets compared to SA-Concat (average improvement $\Delta$ = 30.7), confirming the critical role of cross-attention in facilitating feature interaction with entity descriptions.

5.3.2. Impact of Model Capacity Analysis

To address the concern that the performance improvements of our proposed method might stem solely from an increase in trainable parameters, we conducted a rigorous ablation study focusing on model capacity. We designed a variant named DFEM-Deep, which extends the standard DFEM module by increasing its layer depth. This ensures that DFEM-Deep possesses a parameter count comparable to our complete HPEF model (specifically aligning with the parameter scale of the DSFE module). As presented in

Table 5. Ablation study on model capacity. DFEM denotes the standard module, while DFEM-Deep represents the variant with increased layers to match the parameter count of the full HPEF model. The best results are highlighted in bold.

Dataset	DFEM	DFEM-Deep
AI	43.6	44.3
Literature	45.6	44.0
Music	51.5	49.8
Politics	55.7	55.6
Science	50.3	49.9
Avg.	49.3	48.7

, we compared the standard DFEM against the capacity-enhanced DFEM-Deep across five datasets. The results indicate that simply scaling up the model size does not lead to performance gains; in fact, DFEM-Deep exhibited a slight performance degradation in most domains (Avg. 48.7) compared to the standard DFEM (Avg. 49.3). This observation suggests that the entity recognition task in these domains is not bottlenecked by model capacity but rather requires effective feature decoupling and refinement mechanisms. Consequently, this validates that the superiority of our proposed framework is attributed to its hierarchical progressive architecture rather than a mere increase in parameters.

5.3.3. Contribution Analysis of HPEF

We conducted ablation studies from the temporal perspective of the decision-making process to evaluate the proposed HPEF mechanism, including two settings: (1) single-phase decision using only the first-stage output (D1-Only), and (2) the complete Hierarchical Progressive Entity Filtering (HPEF) mechanism. As shown in

Table 6. Ablation of the HPEF framework: HPEF achieves $\Delta$ = 16.5 F1 gains over single-phase baselines. The best results are highlighted in bold.

Dataset	D1-Only	HPEF
AI	43.6	57.6
Literature	45.6	64.2
Music	51.5	71.5
Politics	55.7	70.2
Science	50.3	65.6
Avg.	49.3	65.8

, HPEF consistently outperforms D1-Only across all datasets (average improvement $\Delta$ = 16.5), demonstrating that the hierarchical progressive filtering mechanism effectively decouples boundary detection and type classification subtasks, and significantly enhances complex entity recognition through iterative, layer-wise refinement.

5.4. Qualitative Analysis

To investigate the limitations of the proposed framework, we conducted a qualitative analysis on the prediction mismatches. As detailed in

Table 7. Distribution of error types based on the qualitative analysis of sampled instances. The analysis focuses on the 22 mismatch cases found in the sample set.

Error Type	Count	Percentage	Primary Cause
Span Boundary Error	9	40.9%	Inclusion of titles/modifiers (e.g., Job Titles)
False Positives	8	36.4%	Adjectival entities & Ground truth omissions
False Negatives	4	18.2%	Missed capitalized datelines & Ambiguity
Tokenization Artifacts	1	4.5%	Possessive suffix handling (’s)

, Span Boundary Errors constitute the primary bottleneck (40.9%). This phenomenon indicates that while the model successfully locates the core entity, it tends to capture longer semantic dependencies, frequently including professional titles (e.g., “Prime Minister”) or modifiers within the entity span. False Positives (36.4%) rank second, which can be attributed to the model’s sensitivity to adjectival demonyms (e.g., “British”) that are often excluded in specific annotation schemas, as well as occasional inconsistencies in the ground truth labels themselves. False Negatives (18.2%) and minor tokenization artifacts account for the remainder, primarily occurring in non-standard sentence structures such as capitalized datelines.

The analysis demonstrates that CSRVNER possesses robust semantic understanding and feature matching capabilities based on entity descriptions, as the majority of errors stem from precision issues in boundary delineation rather than detection failures. The model effectively identifies the semantic focus, further validating the efficacy of our methodology; however, it remains challenged in strictly separating entities from their immediate syntactic modifiers. Future improvements could focus on integrating boundary-aware constraints or employing data augmentation techniques specifically targeting complex noun phrases and non-standard text formats to refine the granularity of entity extraction.

6. Conclusions

This study proposes a non-autoregressive zero-shot NER framework that leverages dynamic entity description features to effectively alleviate the high resource consumption and token-by-token autoregressive latency encountered by generative models in zero-shot NER. To address the limitations of span-based methods—large candidate space, severe class imbalance, and insufficient boundary accuracy—we design a Hierarchical Progressive Entity Filtering (HPEF) mechanism, which determines candidate regions in a single pass, significantly reducing computational and memory overhead. Trained on a single 32 GB GPU for 24 h, the framework achieves approximately 90% of SOTA performance across multiple domain datasets, with an inference speed 17 times faster than that of SOTA methods. The framework maintains high recognition accuracy while greatly improving computational efficiency, providing a practical solution for the efficient deployment of zero-shot NER.