A Noise-Weighted Unsupervised Denoising Approach for Distant Supervision Relation Extraction

Abstract

Distant supervision relation extraction (DS-RE) provides an efficient way to construct large-scale training data by automatically aligning knowledge base relations with unstructured texts. However, this process inevitably introduces erroneous labels because sentences containing the same entity pair do not always express the corresponding knowledge base relations. To address this problem, this paper proposes a noise-weighted unsupervised denoising framework that integrates sentence-level prior confidence estimation, multi-factor representation learning, fine-grained noise detection, and clustering-based label generation. The framework first estimates noise-aware prior weights by matching sentence instances with semantically similar relation triples. It then incorporates lexical, positional, and entity-type factors to enhance sentence representations. For detected noisy instances, an unsupervised clustering-based label generation module is used to regenerate relation labels rather than directly discarding them. Experimental results on the DSRED dataset show that the proposed method achieves 89.7% Precision, 90.6% Recall, 90.1% F1-score, and a PR-AUC of $0.942\pm 0.004$, outperforming the strongest baseline EFEAPN by 1.7 percentage points in F1-score. Statistical analysis further shows that the PR-AUC improvement remains significant after Bonferroni correction ($p_{\mathrm{adj}}=0.0094$). Module-level ablation experiments, sensitivity analysis, and clustering quality evaluation further verify the effectiveness of the noise weighting and clustering-based label generation modules. Supplementary experiments with Transformer-based encoders and cross-dataset evaluation further show that the main performance gain comes from the proposed denoising framework rather than from a specific sentence encoder.

1. Introduction

Relation extraction is a fundamental task in information extraction and knowledge graph construction [1,2], aiming to identify semantic relations between entities from unstructured text. Distant supervision relation extraction (DS-RE) has been widely adopted [3] as a cost-effective solution for large-scale relation extraction by automatically aligning structured relations from knowledge bases with raw textual data. By avoiding expensive manual annotation, DS-RE enables the rapid construction of training corpora and has shown promising performance in many general-domain applications [4,5].

However, the effectiveness of distant supervision critically depends on its underlying assumptions, which are often violated in real-world scenarios. Early DS-RE approaches are typically built upon the vanilla distant supervision assumption [3], which assumes that any sentence containing a given entity pair expresses the corresponding relation in the knowledge base. This assumption is overly strict and rarely holds in natural language texts, because entity co-occurrence does not necessarily imply relation expression. Consequently, a large number of incorrectly labeled sentences are inevitably introduced, leading to severe noise in training data and degrading model performance [6].

To alleviate this issue, subsequent studies proposed relaxed assumptions, such as the At-Least-One assumption [4,7], which posits that at least one sentence in each entity-pair bag correctly expresses the target relation. Based on this assumption, various multi-instance learning and attention-based methods have been developed to select or weight sentences with higher confidence [5,8,9,10]. While these approaches partially mitigate noise, they still rely on the premise that each sentence bag contains at least one correctly labeled instance. In practice, especially in application-oriented corpora, this assumption does not always hold, and some sentence bags may consist entirely of noisy samples.

The noise problem becomes even more pronounced in complex real-world scenarios. Texts in such settings often exhibit frequent entity co-occurrences and highly diverse relation expressions, which significantly increase the difficulty of accurate entity–relation alignment under distant supervision. Moreover, semantic patterns are often complex and context-dependent, making it challenging to distinguish correct and incorrect labels using a single semantic feature or heuristic rule. Therefore, merely relaxing the distant supervision assumption is insufficient, and more refined modeling strategies are required to explicitly identify and handle mislabeled instances at the model level.

Existing DS-RE denoising studies can be broadly grouped into several categories. The first category is based on multi-instance learning and attention mechanisms, which select or assign larger weights to high-confidence sentences within an entity-pair bag [5,8,10]. Although these methods can reduce the influence of mislabeled instances, they still largely rely on the At-Least-One assumption and therefore struggle when an entire sentence bag contains no correctly labeled instances.

The second category explores noise-aware learning and instance weighting, where suspicious instances are down-weighted during optimization [9]. However, most of these methods mainly rely on learned attention scores or limited semantic signals, while explicit knowledge base consistency, entity-type constraints, and relation-expression matching are not sufficiently exploited as prior confidence information.

The third category adopts clustering-based denoising or label refinement. For example, DCRE [11] reuses noisy sentences by reassigning relation labels according to latent semantic structures, while RECON [12] introduces knowledge graph context to enhance relation representations. Nevertheless, clustering-based relabeling may still propagate unreliable pseudo-labels when noisy samples are clustered without sentence-level prior confidence. In addition, knowledge-enhanced methods improve representation learning, but they do not explicitly model how prior noise confidence should guide clustering-based label regeneration.

Recent Transformer-based encoders, such as BERT and its Chinese variants, have further improved contextual representation learning for many NLP tasks [13,14,15]. However, stronger sentence encoders alone do not directly solve the erroneous labeling problem caused by distant supervision. A sentence may be represented accurately but still be assigned an incorrect distant supervision label. Therefore, representation enhancement should be combined with explicit denoising mechanisms that can estimate label reliability and regenerate more reliable labels for noisy instances.

Based on the above analysis, three research gaps remain. First, existing MIL and attention-based DS-RE methods insufficiently handle sentence bags that are entirely noisy. Second, clustering-based label refinement methods lack explicit sentence-level noise priors to guide the relabeling process, which may lead to unreliable pseudo-label propagation. Third, existing noise-aware methods often rely on limited semantic signals and rarely integrate lexical, positional, entity-type, and knowledge base consistency constraints for fine-grained noise detection. These gaps motivate the proposed noise-prior-guided denoising framework.

Motivated by these limitations, this paper focuses on the problem of erroneous labeling noise in distant supervision relation extraction. We propose a noise-prior-guided denoising framework that combines sentence-level noise weighting, multi-factor representation learning, fine-grained noise detection, and unsupervised clustering-based label generation. Different from attention-based methods that mainly select high-confidence sentences within a bag, the proposed framework explicitly estimates sentence-level prior confidence using knowledge base matching. Different from clustering-based denoising methods that directly relabel noisy instances, our method uses noise priors and multi-factor representations to guide noise detection and confidence-weighted label regeneration.

Extensive experiments on DSRED and additional relation extraction datasets demonstrate that the proposed framework effectively mitigates erroneous labeling noise and consistently outperforms representative baseline approaches across multiple evaluation metrics. In addition to the main comparison experiments, we further conduct Transformer-based supplementary comparison, encoder replacement analysis, cross-dataset evaluation, statistical significance analysis, module-level ablation, multi-factor ablation, sensitivity analysis, clustering quality evaluation, qualitative error analysis, and computational complexity analysis. These results validate the effectiveness, robustness, and scalability of the proposed framework for relation extraction under noisy distant supervision.

The main contributions of this work are summarized as follows:

• To address the limitation of MIL and attention-based methods under entirely noisy sentence bags, we propose a noise-prior-guided denoising framework that performs fine-grained sentence-level noise detection rather than blindly relying on the At-Least-One assumption.
• To reduce unreliable pseudo-label propagation in clustering-based relabeling, we design a noise weighting mechanism that combines entity-pair consistency, relation-expression similarity, and entity-type constraints to provide sentence-level prior confidence before label regeneration.
• To overcome the limitation of single-signal noise estimation, we incorporate multi-factor representation learning by jointly modeling POS information, relative position, and entity type information, thereby enhancing lexical and structural constraints for noise detection.
• We introduce a confidence-weighted clustering-based label generation strategy to reuse noisy instances rather than directly discarding them. The regenerated labels are weighted by clustering confidence to improve training stability and reduce the influence of secondary noise.
• We conduct extensive experiments on DSRED and additional relation extraction datasets, including baseline comparison, Transformer-based supplementary comparison, encoder replacement analysis, cross-dataset evaluation, statistical significance analysis, module-level ablation, multi-factor ablation, sensitivity analysis, clustering quality evaluation, qualitative error analysis, and computational complexity analysis, to verify the effectiveness and robustness of the proposed framework.

2. Methods

2.1. Overall Framework

To address the erroneous labeling problem in distant supervision relation extraction, this paper proposes a noise-prior-guided denoising framework that integrates sentence-level noise weighting, multi-factor representation learning, fine-grained noise detection, and clustering-based label generation. Following the multi-instance learning (MIL) paradigm, sentences containing the same entity pair are grouped into sentence bags, while entity names, entity types, and relation triples are used as weak supervision signals. The overall framework is illustrated in Figure 1.

The proposed framework contains five functional modules.

• Noise Weighting Module. This module estimates sentence-level prior confidence by matching sentence instances with relation triples in the knowledge base. It considers entity-pair consistency, relation-expression similarity, and head–tail entity type constraints. For each sentence instance, it outputs a prior relation label $l_{\mathrm{prior}}$ and a confidence score $s_i$, which provide coarse-grained guidance for subsequent noise detection.
• Multi-factor Encoding Module. This module models lexical and structural factors that may influence relation expression. Specifically, POS information, relative position, and entity type information are transformed into continuous influence weights. The output of this module is the multi-factor influence matrix $\alpha$, which is used to enhance sentence representations.
• Sentence Encoding Module. This module encodes sentence bags using a Piecewise Convolutional Neural Network (PCNN). The encoded sentence representations are further integrated with the multi-factor influence matrix to obtain the final sentence-bag representation H.
• Noise Detection Module. This module evaluates the relevance between each sentence and its distantly supervised relation label. It jointly uses the sentence representation H, the prior confidence score $s_i$, the prior relation label $l_{\mathrm{prior}}$, and relation embeddings to identify valid, noisy, and uncertain samples. Sentences with confidence scores lower than the threshold ${\varphi }_1$ are treated as noisy samples and passed to the label generation module.
• Label Generation Module. This module reuses detected noisy samples through unsupervised clustering-based label refinement. Instead of discarding noisy instances, it projects them into a relation-aware feature space, estimates clustering confidence $q_{ij}$, and regenerates relation labels $\widehat{y}$ for high-confidence noisy samples.

To clarify the role of each component and reduce the perceived complexity of the framework,

Table 1. Functional roles of the main modules in the proposed framework.

Module	Input	Output	Role
Noise Weighting Module	Sentence instances, entity pairs, entity types, and knowledge base triples	Prior relation label $l_{\mathrm{prior}}$ and confidence score $s_i$	Provides sentence-level prior confidence for coarse-grained noise estimation and guides subsequent noise detection.
Multi-factor Encoding Module	POS information, relative position, and entity type information	Multi-factor influence matrix $\alpha$	Enhances sentence representations with lexical, positional, and entity-type cues.
Sentence Encoding Module	Sentence bags and position-enhanced word representations	Sentence-bag representation H	Extracts contextual features from sentence bags and integrates multi-factor influence weights.
Noise Detection Module	H, $s_i$, $l_{\mathrm{prior}}$, relation embeddings, and distant supervision labels	Valid, noisy, and uncertain sample sets	Performs fine-grained noise identification at the sentence-bag level and selects noisy samples for label regeneration.
Label Generation Module	Detected noisy samples and relation-aware representations	Regenerated labels $\widehat{y}$ with clustering confidence $q_{ij}$	Reuses noisy instances through clustering-based label regeneration instead of directly discarding them.

summarizes the input, output, and function of the main modules.

In summary, the proposed framework establishes a closed-loop denoising process. The noise weighting module first provides coarse-grained prior confidence, the multi-factor encoding and sentence encoding modules then construct enhanced sentence representations, the noise detection module separates valid and noisy samples, and the label generation module regenerates labels for recoverable noisy instances. By combining these components, the framework mitigates the impact of noisy distant supervision while preserving useful training information without requiring additional large-scale manual annotation.

2.2. Noise Weighting Module

The noise weighting module is designed to provide a coarse-grained sentence-level prior confidence before neural noise detection and clustering-based label regeneration. In distant supervision, not every sentence containing the same entity pair truly expresses the bag-level relation. If such noisy sentences are directly used for downstream denoising or label regeneration, unreliable pseudo-labels may be propagated. Therefore, this module estimates whether a sentence is likely to support its distantly supervised relation by jointly considering entity-pair consistency, relation-expression similarity, and entity-type constraints.

More specifically, the module matches sentence instances with candidate triples in the knowledge base and outputs two quantities for each sentence instance: a prior relation label $l_{\mathrm{prior}}$ and a sentence-level confidence score $s_i$. This prior confidence is used as an auxiliary signal for subsequent fine-grained noise detection rather than as a direct replacement for the final relation prediction.

2.2.1. Entity-Pair Similarity Calculation

Given a sentence containing an entity pair $(e_1,e_2)$, we first constrain the candidate relation space using head–tail entity type constraints. This step removes semantically incompatible triples and reduces unnecessary comparisons. To bridge the gap between surface mentions in text and canonical entity names in the knowledge base, we represent entity names using TF-IDF weighted vectors.

The TF-IDF score of a word w in an entity-name text segment s is defined as

$$\mathrm{TFIDF}(w,s)={\displaystyle \frac{n_{w,s}}{{\sum }_{k\in s}{n}_{k,s}}}\times log{\displaystyle \frac{\left|S\right|}{1+\left|\right\{s^{\prime }\in S:w\in s^{\prime }\left\}\right|}},$$

(1)

where $n_{w,s}$ denotes the frequency of word w in entity-name segment s, and S denotes the collection of entity names in the knowledge base.

Based on the TF-IDF representations, the semantic similarity between a sentence mention and a candidate knowledge base entity is computed by cosine similarity

$$s_e={\displaystyle \frac{{\sum }_{i=1}^n{x}_i{y}_i}{\sqrt{{\sum }_{i=1}^n{x}_i^2}\sqrt{{\sum }_{i=1}^n{y}_i^2}}},$$

(2)

where $X=(x_1,\dots ,x_n)$ and $Y=(y_1,\dots ,y_n)$ are the TF-IDF vectors of the textual mention and the candidate knowledge base entity, respectively.

For an entity pair $(e_1,e_2)$, the joint entity-pair similarity is computed by the harmonic mean of the two individual entity similarities

$$s_1={\displaystyle \frac{2s_{e_1}{s}_{e_2}}{s_{e_1}+s_{e_2}}}.$$

(3)

If $s_1<{\tau }_e$, the candidate triple is regarded as weakly aligned and is filtered out. Here, ${\tau }_e$ denotes the entity-pair similarity threshold. According to the sensitivity analysis in Section 3.6, ${\tau }_e$ is set to 0.50 in our experiments.

2.2.2. Relation-Word Similarity Calculation

Although entity alignment is necessary, correct entity alignment alone does not guarantee that the sentence explicitly expresses the target relation. Therefore, we further estimate relation-expression similarity. Relations are typically triggered by verbs, verbal phrases, or nominalized expressions. For each relation type, we construct a synonym set based on an extended thesaurus resource [16].

Let $P_i$ denote the sequence of relation-indicative words extracted from sentence i, and let $G_r=\{g_1,g_2,\dots ,g_m\}$ denote the synonym set of relation r. For each synonym phrase $g_j\in G_r$, we compute a normalized longest common subsequence (LCS) score

$$\text{LCS-Sim}(P_i,g_j)={\displaystyle \frac{\mathrm{LCS}(P_i,g_j)}{max\left(|P_i|,|g_j|\right)}},$$

(4)

where $\mathrm{LCS}(P_i,g_j)$ denotes the length of the longest common subsequence between the sentence-side relation phrase and the synonym phrase.

The relation-word similarity of sentence i with respect to relation r is then defined as

$$s_2=\underset{g_j\in G_r}{max}\text{LCS-Sim}(P_i,g_j).$$

(5)

If $s_2<{\tau }_r$, the sentence is considered weakly matched to the candidate relation expression. Here ${\tau }_r$ denotes the relation-word similarity threshold. In the experiments, ${\tau }_r$ is set to 0.65. The sensitivity analysis in Section 3.6 shows that $({\tau }_e,{\tau }_r)=(0.50,0.65)$ provides the best trade-off between relation extraction performance and clustering quality.

2.2.3. Comprehensive Noise Weighting

Finally, we combine the entity-pair similarity $s_1$ and the relation-word similarity $s_2$ to obtain a sentence-level comprehensive confidence score. To encourage both semantic alignment and relation-expression consistency, we again use the harmonic mean

$$s_i={\displaystyle \frac{2s_1{s}_2}{s_1+s_2}}.$$

(6)

Here, $s_i\in [0,1]$ denotes the prior confidence that sentence i correctly supports the candidate relation. For each sentence, the candidate relation with the highest confidence score is selected as the prior relation label $l_{\mathrm{prior}}$, and the corresponding score $s_i$ is used as the prior noise weight. In this way, the noise weighting module provides a coarse-grained reliability estimate for subsequent fine-grained noise detection and label regeneration.

The overall workflow of the noise weighting module is illustrated in Figure 2.

2.3. Multi-Factor Encoding Module

To capture complex semantic patterns in distantly supervised texts, we propose a multi-factor encoding module that transforms discrete linguistic and structural features into continuous relational influence weights. Specifically, three categories of factors are considered: part-of-speech (POS) information, relative position, and entity type information. These factors provide complementary cues for relation expression. POS information helps identify relation-indicative words, relative position captures entity-centered contextual relevance, and entity type information constrains semantically plausible relation patterns.

2.3.1. Feature Quantification and Gaussian Smoothing

The raw values of these factors are discrete or scale-inconsistent. Directly concatenating them may introduce abrupt changes and make it difficult for the model to learn smooth feature interactions. Therefore, we map each factor into a continuous influence score through a Gaussian kernel. The Gaussian mapping serves as a soft-masking mechanism: features close to the expected relation-indicative region receive stronger influence, while less relevant features are smoothly down-weighted.

For a factor value $x_q$, the smoothed influence score is defined as

$$f_q\left(x_q\right)={\displaystyle \frac{1}{{\sigma }_q\sqrt{2\pi }}}exp\left(-{\displaystyle \frac{{(x_q-{\mu }_q)}^2}{2{\sigma }_q^2}}\right),q\in \{\mathrm{pos},\mathrm{dist},\mathrm{type}\},$$

(7)

where q denotes the factor type, ${\mu }_q$ is the expected central value of the corresponding factor, and ${\sigma }_q$ controls the smoothness of the influence distribution.

The Gaussian parameters ${\mu }_q$ and ${\sigma }_q$ are fixed hyperparameters determined according to prior linguistic intuition and validation settings. For relative position, ${\mu }_q=0$ emphasizes words close to the target entities, reflecting the observation that relation triggers often appear near entity mentions. For POS information, relation-indicative POS categories such as verbs and nouns are assigned higher initial importance. Although the Gaussian parameters are fixed to maintain structural stability, the subsequent transformation matrices in the integration network are learnable, allowing the model to adaptively project these smoothed priors into the final feature space.

2.3.2. Factor Integration

For POS information, we assign a binary indicator $x_{\mathrm{pos}}\in \{0,1\}$ to each token. Verbs and nouns, which often carry relational semantics, are assigned 1, while other tokens are assigned 0. Gaussian smoothing then converts this binary signal into a continuous influence score $f_{\mathrm{pos}}$.

For relative position, let $x_{\mathrm{dist}}$ denote the distance between a token and the nearest target entity. The Gaussian kernel implements a distance-decay effect, where tokens closer to the target entities receive larger influence scores. This helps the model focus on local contexts that are more likely to express relations.

For entity type information, we transform head–tail entity type pairs into type-based importance scores. Entity type constraints are useful because many relations are only valid for specific combinations of head and tail entity types. The smoothed entity type factor $f_{\mathrm{type}}$ provides a structural prior for relation classification and noise detection.

The three smoothed factors are concatenated into a factor vector

$$\mathbf{F}={\left[f_{\mathrm{pos}},f_{\mathrm{dist}},f_{\mathrm{type}}\right]}^{\top }.$$

(8)

To capture nonlinear interactions among the three factors, we feed $\mathbf{F}$ into a deep transformation network

$$\mathbf{f}=PReLU\left({\mathbf{W}}_{3}PReLU\left({\mathbf{W}}_{2}PReLU\left({\mathbf{W}}_1\mathbf{F}+{\mathbf{b}}_1\right)+{\mathbf{b}}_2\right)+{\mathbf{b}}_3\right),$$

(9)

where ${\mathbf{W}}_1$, ${\mathbf{W}}_2$, and ${\mathbf{W}}_3$ are learnable transformation matrices, and ${\mathbf{b}}_1$, ${\mathbf{b}}_2$, and ${\mathbf{b}}_3$ are bias vectors.

Finally, a Softmax layer is applied to generate the multi-factor influence matrix

$$\alpha =Softmax\left(\mathbf{f}\right).$$

(10)

The influence matrix $\alpha$ is used to weight sentence representations in the subsequent sentence encoding module.

2.4. Sentence Encoding Module

To capture contextual information within sentence bags, this work adopts the Piecewise Convolutional Neural Network (PCNN) [4] as the default sentence encoder. PCNN consists of an input layer, a convolutional layer, a piecewise max-pooling layer, and an output layer.

Although Transformer-based encoders such as BERT have achieved strong performance in many relation extraction tasks, PCNN is adopted as the default sentence encoder in this work for two reasons. First, PCNN is a classical and computationally efficient sentence encoder widely used in distant supervision relation extraction, which enables fair comparison with representative DS-RE baselines. Second, the main focus of this work is not to design a new encoder architecture, but to evaluate the effectiveness of the proposed noise-prior-guided denoising framework.

To further examine the influence of encoder choice, we additionally conduct supplementary experiments with Transformer-based baselines and encoder replacement variants in Section 3.4.2. The results show that Transformer encoders slightly improve representation quality under the full denoising framework, but the improvement is not statistically significant after Bonferroni correction, while the training cost increases substantially. Therefore, PCNN is retained as the default encoder for computational efficiency, fair comparison, and clearer evaluation of the proposed denoising modules.

2.4.1. Input Layer

All baseline models were reimplemented and retrained on DSRED under the same train/validation/test split and sentence-bag-level evaluation protocol. For EFEAPN, only the input representation and output classification layer were adapted to the DSRED relation label space, while the core model structure was kept unchanged.

For a sentence $x_i=[w_1,w_2,\dots ,w_m]$ in a sentence bag, we first construct the token-level input representation. Each token $w_t$ is represented by a word embedding vector ${\mathbf{v}}_t\in {\mathbb{R}}^{d_w}$. To incorporate structural information, we compute the relative positions of token $w_t$ with respect to the head entity $e_h$ and the tail entity $e_t$. The final token representation is obtained by concatenating the word embedding and the two position embeddings

$${\mathbf{x}}_t=\left[{\mathbf{v}}_t;{\mathbf{p}}_t^{e_h};{\mathbf{p}}_t^{e_t}\right],$$

(11)

where ${\mathbf{p}}_t^{e_h}$ and ${\mathbf{p}}_t^{e_t}$ denote the relative position embeddings with respect to the head and tail entities, respectively. The dimension of ${\mathbf{x}}_t$ is $d=d_w+2d_p$. Therefore, the input sentence matrix is represented as

$${\mathbf{X}}_i=\left[{\mathbf{x}}_1,{\mathbf{x}}_2,\dots ,{\mathbf{x}}_m\right].$$

(12)

2.4.2. Convolutional and Piecewise Max-Pooling Layers

In the convolutional layer, $d_c$ convolutional filters with window size c are applied to the sentence matrix ${\mathbf{X}}_i$ to extract local contextual features. Each filter produces a feature map. According to the positions of the two target entities, each feature map is divided into three segments: the left context, the middle context between the entities, and the right context. Piecewise max-pooling is then performed on each segment separately. The three pooled vectors are concatenated to obtain the sentence representation

$${\mathbf{h}}_i\in {\mathbb{R}}^{3d_c}.$$

(13)

2.4.3. Output Layer and Feature Integration

For a sentence bag containing b sentences, the sentence representations are organized as

$$\mathbf{S}={\left[{\mathbf{h}}_1,{\mathbf{h}}_2,\dots ,{\mathbf{h}}_b\right]}^{\top }.$$

(14)

The multi-factor influence matrix $\alpha$ generated in Section 2.3 is used to weight the sentence representations. After dimension alignment, the final feature-weighted sentence-bag representation is computed as

$$\mathbf{H}={\displaystyle \sum _{i=1}^b}{\alpha }_i\odot {\mathbf{h}}_i,$$

(15)

where ⊙ denotes element-wise multiplication, and ${\alpha }_i$ represents the multi-factor influence weight corresponding to sentence representation ${\mathbf{h}}_i$.

2.5. Noise Detection Module

The objective of the noise detection module is to partition sentences within a bag into high-confidence valid samples, potential noisy samples, and uncertain samples. This fine-grained identification allows the model to retain reliable supervision while sending suspicious instances to the label generation module for further refinement.

2.5.1. Relevance Estimation with Prior Guidance

Let ${\mathbf{H}}_b=\{{\mathbf{h}}_1,{\mathbf{h}}_2,\dots ,{\mathbf{h}}_b\}$ denote the sentence representations in a bag, and let $\mathbf{L}\in {\mathbb{R}}^{k\times d_s}$ denote the relation embedding matrix for k relation types. Under the guidance of the noise weighting module in Section 2.2, each sentence is associated with a prior relation label $l_{\mathrm{prior}}$ and a prior confidence score $s_i$.

Different from standard MIL methods that only rely on the distantly supervised bag-level label $l_j$, we combine the distantly supervised relation embedding and the prior relation embedding to calibrate sentence relevance. The relevance score of sentence ${\mathbf{h}}_i$ is computed as

$$e_i=(1-s_i)\left({\mathbf{h}}_i·{\mathbf{l}}_j^{\top }\right)+s_i\left({\mathbf{h}}_i·{\mathbf{l}}_{\mathrm{prior}}^{\top }\right),$$

(16)

where ${\mathbf{l}}_j$ is the relation embedding assigned by distant supervision, and ${\mathbf{l}}_{\mathrm{prior}}$ is the candidate relation embedding identified by the noise weighting module.

The relevance scores are normalized through a Softmax function across the sentence bag

$$a_i={\displaystyle \frac{exp\left(e_i\right)}{{\sum }_{m=1}^{b}exp\left(e_m\right)}},$$

(17)

where $a_i$ represents the probability that sentence ${\mathbf{h}}_i$ correctly expresses the target relation.

2.5.2. Fine-Grained Noise Identification Strategy

To improve the robustness of training data, we introduce a three-way decision strategy based on the confidence probability $a_i$. For a sentence bag, the sentence with the highest confidence is first selected as the representative valid sample

$$i^{*}=arg\underset{i}{max}{a}_i,V=\left\{{\mathbf{h}}_{i^{*}}\right\}.$$

(18)

The remaining sentences are then categorized according to the noise identification threshold ${\varphi }_1$. Sentences with confidence scores lower than ${\varphi }_1$ are identified as noisy samples

$$N=\left\{{\mathbf{h}}_i\mid i\ne i^{*},a_i<{\varphi }_1\right\}.$$

(19)

Sentences that are neither selected as valid samples nor flagged as noisy samples are treated as uncertain samples

$$U={\mathbf{H}}_b\setminus (V\cup N).$$

(20)

The threshold ${\varphi }_1$ controls the strictness of noisy-sample detection. A smaller ${\varphi }_1$ may allow more low-confidence instances to remain outside the label generation module, while a larger ${\varphi }_1$ may send more ambiguous samples into clustering-based label regeneration. In this study, ${\varphi }_1$ is set to 0.50 according to the sensitivity analysis in Section 3.6.

This strategy allows the model to keep a representative valid instance for each bag while still identifying clearly unreliable sentences for label regeneration. As a result, the supervision signal is refined without directly discarding potentially useful noisy samples.

2.6. Label Generation Module

To recover informative signals from identified noisy samples rather than simply discarding them, we propose a clustering-based label generation module. This module is inspired by the Deep Embedded Clustering (DEC) framework [17], but it is adapted to distant supervision relation extraction by incorporating relation-aware projection and confidence-weighted joint training.

2.6.1. Relation-Aware Feature Projection

Let ${\mathbf{H}}_N=[{\mathbf{h}}_1,{\mathbf{h}}_2,\dots ,{\mathbf{h}}_n]\in {\mathbb{R}}^{n\times d_s}$ denote the representations of detected noisy samples, where n is the number of noisy samples. Let $\mathbf{L}\in {\mathbb{R}}^{k\times d_s}$ denote the embeddings of all k relation types. To align sentence representations with the relation label space, we project noisy sample representations into a relation-aware feature space

$$\mathbf{C}={\mathbf{H}}_N{\mathbf{L}}^{\top }+{\mathbf{b}}_r,$$

(21)

where ${\mathbf{b}}_r$ is the bias term. This operation can be viewed as a relation-aware projection in which relation embeddings serve as semantic anchors for measuring the relevance between noisy samples and candidate relations.

2.6.2. Unsupervised Label Refinement

The projected representations $\mathbf{C}$ are fed into a clustering layer. Following DEC [17], we treat the cluster centers ${\left\{{\mu }_j\right\}}_{j=1}^k$ as trainable parameters. The Student’s t-distribution is used to measure the similarity between the projected representation ${\mathbf{c}}_i$ and cluster center ${\mu }_j$

$$q_{ij}={\displaystyle \frac{{\left(1+\parallel {\mathbf{c}}_i-{\mathit{\mu }}_j{\parallel }^2\right)}^{-1}}{{\sum }_{j^{\prime }=1}^k{\left(1+\parallel {\mathbf{c}}_i-{\mathit{\mu }}_{j^{\prime }}{\parallel }^2\right)}^{-1}}},$$

(22)

where $q_{ij}$ represents the soft assignment probability, which is also used as clustering confidence.

To iteratively refine the clusters, we minimize the Kullback–Leibler divergence between the soft assignment distribution $\mathbf{Q}$ and an auxiliary target distribution $\mathbf{P}$

$${\mathcal{L}}_{\mathrm{clu}}=\mathrm{KL}(\mathbf{P}\parallel \mathbf{Q})=\sum _i\sum _j{p}_{ij}log{\displaystyle \frac{p_{ij}}{q_{ij}}}.$$

(23)

The target distribution is designed to sharpen high-confidence assignments and normalize the contribution of each cluster. Let $f_j={\sum }_i{q}_{ij}$ denote the soft frequency of cluster j. The target distribution is defined as  

$$p_{ij}={\displaystyle \frac{q_{ij}^2/f_j}{{\sum }_{j^{\prime }=1}^k{q}_{i{j}^{\prime }}^2/f_{j^{\prime }}}}.$$

(24)

For each noisy sample, the regenerated label is determined by the cluster with the highest assignment probability

$${\widehat{y}}_i=arg\underset{j}{max}{q}_{ij}.$$

(25)

2.6.3. Confidence-Weighted Joint Training

Our improvement over the standard DEC framework lies in the integration of clustering results into the supervised training process. For samples identified as noisy, we use clustering-derived labels to regenerate relations. To mitigate the risk of introducing secondary noise, we further apply a secondary confidence threshold ${\varphi }_2$ to the clustering assignment probability.

Let $q_i^{*}={max}_j{q}_{ij}$ denote the maximum clustering confidence of noisy sample i. If $q_i^{*}<{\varphi }_2$, the regenerated label is not used for supervised update in the current iteration. Therefore, ${\varphi }_2$ serves as a conservative filter for low-confidence clustering assignments. Let $N_{{\varphi }_2}$ denote the subset of noisy samples whose maximum clustering confidence satisfies $q_i^{*}\ge {\varphi }_2$

$$N_{{\varphi }_2}=\left\{x_i\in N\mid q_i^{*}\ge {\varphi }_2\right\}.$$

(26)

The confidence-weighted supervised objective is defined as

$$J(\Theta )=-\sum _{(x_i,y_i)\in V}logp\left(y_i\right|x_i;\Theta )-\lambda \sum _{x_i\in N_{{\varphi }_2}}{q}_i^{*}logp\left({\widehat{y}}_i\right|x_i;\Theta ),$$

(27)

where V denotes the high-confidence valid sample set, $N_{{\varphi }_2}$ denotes the filtered noisy sample set, ${\widehat{y}}_i$ is the regenerated label, and $q_i^{*}$ is the corresponding clustering confidence. The coefficient $\lambda$ controls the contribution of regenerated noisy samples in the joint objective. A larger $\lambda$ increases the influence of clustering-based labels, while a smaller $\lambda$ makes the model rely more on high-confidence valid samples. In this study, ${\varphi }_2$ and $\lambda$ are selected according to the sensitivity analysis in Section 3.6.

This confidence-weighted mechanism enables the model to reuse mislabeled data with high clustering confidence while maintaining training stability.

3. Experiments

3.1. Dataset

To support distant supervision-based relation extraction, we construct a dedicated dataset named Distant Supervision Relation Extraction Dataset (DSRED). The motivation stems from the scarcity of publicly available Chinese datasets for relation extraction under distant supervision, as well as the limitations of general-domain resources in capturing diverse entity relations and linguistic patterns.

The construction of DSRED follows a knowledge-guided distant supervision paradigm. A task-oriented knowledge base is first established to provide relational supervision, after which large-scale textual data are collected and aligned with the knowledge base under the multi-instance learning assumption. To balance coverage and data quality, the knowledge base is built by integrating a large-scale open-domain knowledge graph with a curated structured knowledge source. Specifically, CN-DBpedia provides broad encyclopedic coverage, while the curated structured knowledge source contributes more fine-grained and reliable relational information. Triples are extracted, aligned, and filtered through entity alignment, semantic similarity checking, consistency verification, and redundant triple removal to form a unified supervision source.

Textual data are collected from open-source encyclopedic platforms and online text corpora. To improve corpus diversity, the collected texts cover both encyclopedic descriptions and naturally written online textual resources, which helps include different writing styles and relation expression patterns. After data collection, a standard preprocessing pipeline is applied, including content extraction, deduplication, text normalization, sentence segmentation, part-of-speech tagging, named entity recognition, and entity-pair alignment. Sentences containing the same entity pair are grouped into sentence bags. Relation labels are then automatically assigned to each bag by heuristically aligning entity pairs with relations in the knowledge base, following the At-Least-One assumption.

To improve data quality, we conduct filtering at both the knowledge base level and the textual level. At the knowledge base level, triples with inconsistent entity representations or low semantic consistency are removed during the alignment process. At the textual level, duplicate sentences, incomplete sentence fragments, and sentences without recognizable entity pairs are filtered out. This preprocessing procedure reduces obvious low-quality samples before distant supervision labeling. Nevertheless, because the labels are automatically generated through knowledge base alignment, DSRED remains a weakly supervised dataset and inevitably contains annotation noise.

The annotation noise in DSRED mainly comes from three sources. First, a sentence may mention an entity pair without actually expressing the relation assigned by the knowledge base. Second, an entity pair may participate in multiple possible relations, while only one relation is assigned during heuristic alignment. Third, entity recognition or entity linking errors may lead to incorrect sentence–triple matching. These noise sources are consistent with the typical challenges of distant supervision relation extraction and motivate the denoising framework proposed in this work.

As a result, DSRED consists of 24,982 sentence bags, including 24,855 bags labeled with specific relations and 127 bags labeled as no relation. Each bag contains up to 15 sentences, with a maximum sentence length of 512 characters. The dataset is divided into training, validation, and test sets, containing 17,488, 3747, and 3747 sentence bags, respectively. The overall statistics, coarse label distribution, fine-grained relation-level distribution, train/validation/test split, and estimated annotation noise ratio of DSRED are summarized in

Table 2. Overall statistics of the DSRED dataset.

Statistic	Value
Number of sentence bags	24,982
Bags with specific relation labels	24,855
Bags labeled as no relation	127
Total number of sentences	298,006
Average sentences per bag	11.93
Maximum sentences per bag	15
Maximum sentence length	512 characters
Training bags	17,488
Validation bags	3747
Test bags	3747

, 

Table 3. Coarse label distribution of the DSRED dataset.

Label Type	# Sentence Bags	Percentage
Specific relation	24,855	99.49%
No relation	127	0.51%
Total	24,982	100.00%

, 

Table 4. Relation-level label distribution of the DSRED dataset. Relation names are anonymized for data-usage restrictions. Percentages are calculated over all 24,982 sentence bags.

Relation Type	Train	Validation	Test	Total Bags	Percentage
${\mathrm{Relation}}_1$	2240	480	480	3200	12.81%
${\mathrm{Relation}}_2$	1960	420	420	2800	11.21%
${\mathrm{Relation}}_3$	1680	360	360	2400	9.61%
${\mathrm{Relation}}_4$	1470	315	315	2100	8.41%
${\mathrm{Relation}}_5$	1260	270	270	1800	7.21%
${\mathrm{Relation}}_6$	1120	240	240	1600	6.40%
${\mathrm{Relation}}_7$	1014	218	218	1450	5.80%
${\mathrm{Relation}}_8$	910	195	195	1300	5.20%
${\mathrm{Relation}}_9$	806	172	172	1150	4.60%
${\mathrm{Relation}}_{10}$	734	158	158	1050	4.20%
${\mathrm{Relation}}_{11}$	666	142	142	950	3.80%
${\mathrm{Relation}}_{12}$	594	128	128	850	3.40%
${\mathrm{Relation}}_{13}$	526	112	112	750	3.00%
${\mathrm{Relation}}_{14}$	454	98	98	650	2.60%
${\mathrm{Relation}}_{15}$	420	90	90	600	2.40%
${\mathrm{Relation}}_{16}$	386	82	82	550	2.20%
${\mathrm{Relation}}_{17}$	350	75	75	500	2.00%
${\mathrm{Relation}}_{18}$	302	64	64	430	1.72%
${\mathrm{Relation}}_{19}$	266	57	57	380	1.52%
${\mathrm{Relation}}_{20}$	241	52	52	345	1.38%
No relation	89	19	19	127	0.51%
Total	17,488	3747	3747	24,982	100.00%

, 

Table 5. Train/validation/test split of the DSRED dataset.

Dataset Split	# Sentence Bags	# Sentences
Train	17,488	209,856
Validation	3747	43,078
Test	3747	45,072
Total	24,982	298,006

, and 

Table 6. Estimated annotation noise ratio based on manual inspection of sampled DSRED bags. The confidence interval is computed using the Wilson score interval.

Manual Inspection Result	# Bags	Percentage	95% CI
Correct or acceptable distant-supervision label	409	81.80%	–
Noisy label: entity co-occurrence without target relation	55	11.00%	–
Noisy label: wrong relation due to multi-relation entity pair	27	5.40%	–
Noisy label: entity recognition or linking error	9	1.80%	–
Estimated noisy labels	91	18.20%	[15.06%, 21.82%]

, respectively.

Table 4 further reports the fine-grained relation-level distribution of DSRED. The dataset exhibits a long-tailed distribution, where the most frequent relation type contains 3200 bags, while several low-frequency relation types contain fewer than 500 bags. This imbalance reflects the natural skewness of relation distributions in distant supervision settings and motivates the use of PR-AUC in addition to Precision, Recall, and F1-score.

Since DSRED is automatically labeled under the distant supervision assumption, an exact corpus-level gold-standard noise ratio would require exhaustive manual verification of all sentence bags. To provide an approximate estimate of the annotation noise level, we manually inspected 500 randomly sampled bags from DSRED. The sampled bags were categorized into correct or acceptable distant-supervision labels, entity co-occurrence without the assigned target relation, wrong relation caused by multi-relation entity pairs, and entity recognition or entity linking errors. As shown in Table 6, 91 out of 500 sampled bags were judged to contain noisy distant-supervision labels, resulting in an estimated noisy-label ratio of 18.20%, with a 95% Wilson confidence interval of [15.06%, 21.82%]. This manually estimated noise ratio is used only for dataset characterization and is not used during model training or evaluation.

3.2. Evaluation Metrics

The objective of distant supervision-based relation extraction is to accurately identify relations between entity pairs from weakly labeled sentence bags. Following the multi-instance learning setting, all evaluation metrics are computed at the sentence-bag level. We adopt Precision, Recall, and F1-score as the primary evaluation metrics, which are widely used in relation extraction studies.

Precision measures the proportion of correctly predicted positive samples among all predicted positive samples, reflecting the reliability of extracted relations. Recall measures the proportion of correctly predicted positive samples among all actual positive samples, reflecting the model’s ability to identify relevant relations. F1-score is the harmonic mean of Precision and Recall, providing a balanced evaluation of overall extraction performance. The definitions of Precision, Recall, and F1-score are given as follows:

$$P={\displaystyle \frac{TP}{TP+FP}},$$

(28)

$$R={\displaystyle \frac{TP}{TP+FN}},$$

(29)

$$F1={\displaystyle \frac{2\times P\times R}{P+R}},$$

(30)

where $TP$ denotes true positives, $FP$ denotes false positives, and $FN$ denotes false negatives.

In addition to Precision, Recall, and F1-score, we further report the area under the Precision–Recall curve (PR-AUC). PR-AUC summarizes the trade-off between precision and recall under different decision thresholds and is particularly useful for evaluating relation extraction models on imbalanced datasets. In this study, PR-AUC is computed at the sentence-bag level under the same evaluation protocol as the other metrics.

To assess the stability of the reported results, each experiment is repeated five times with different random seeds. PR-AUC is reported as mean ± standard deviation. The 95% confidence interval is computed as

$$95\%\mathrm{CI}=\overline{x}\pm t_{0.975,n-1}{\displaystyle \frac{s}{\sqrt{n}}},$$

(31)

where $\overline{x}$ is the mean PR-AUC, s is the standard deviation, and n is the number of repeated runs. In our experiments, $n=5$ and $t_{0.975,4}=2.776$.

Statistical significance is evaluated using Welch’s two-sample t-test by comparing the PR-AUC scores of each baseline or ablated variant with those of the corresponding full model. Bonferroni correction is applied to account for multiple comparisons. The significance levels are denoted as follows: * indicates $p_{\mathrm{adj}}<0.05$, ** indicates $p_{\mathrm{adj}}<0.01$, and *** indicates $p_{\mathrm{adj}}<0.001$.

3.3. Experimental Setup

Experiments are conducted to evaluate the proposed noise-weighted unsupervised denoising framework on DSRED. Following the multi-instance learning paradigm, sentences containing the same entity pair are grouped into sentence bags, and relation labels are automatically assigned through knowledge base alignment. The dataset split described in Table 5 is kept fixed for all experiments to ensure fair comparison.

All comparative methods are evaluated under the same train/validation/test split and the same sentence-bag-level evaluation protocol. For baseline models, we follow the main settings reported in the corresponding original papers and adapt only the input representation and output classification layer when necessary to make them compatible with the DSRED label space. Unless otherwise specified, the final results are obtained on the test set after model selection on the validation set.

The training procedure of the proposed framework consists of three main stages. First, sentence representations and knowledge base information are used by the noise-weighting module to estimate sentence-level prior confidence and identify semantically similar relational triples. Second, lexical, positional, and entity-type factors are embedded through the multi-factor encoding module and integrated with sentence representations. Third, the combined representations are fed into the PCNN encoder and the noise detection module. Noisy samples detected during training are further refined by the unsupervised clustering-based label generation module. K-means is used to initialize cluster centers for faster convergence, and Dropout [18] is applied to alleviate overfitting.

For optimization, all neural components are trained using stochastic gradient descent (SGD), implemented with TensorFlow’s GradientDescentOptimizer. The learning rates are set separately for different stages: 0.4 for representation pre-training, 0.004 for clustering refinement, and 0.1 for the final denoising model training. The maximum number of epochs is set to 30 for pre-training, 50 for clustering refinement, and 100 for the final joint training stage. Early stopping is applied during final training: training is stopped if the validation-set F1-score does not improve for 10 consecutive epochs. The validation-set F1-score is used as the model selection metric, and the model checkpoint with the best validation F1-score is used for final test-set evaluation. For clustering refinement, training is also stopped when the relative change of cluster assignments is smaller than ${10}^{-3}$ between two consecutive evaluation intervals or when the maximum clustering epoch is reached.

To assess the stability of the results, each experiment is repeated with five fixed random seeds: 13, 21, 42, 87, and 100. These seeds are used for parameter initialization, mini-batch shuffling, K-means initialization, and TensorFlow random operations. The reported PR-AUC values are averaged over these five runs and presented as mean ± standard deviation. Statistical significance is then evaluated based on the five repeated runs, as described in Section 3.2.

The main hyperparameter settings are listed in

Table 7. Hyperparameter settings used in the proposed framework.

Parameter	Symbol	Value
Word vector dimension	$d_w$	50
Position feature dimension	$d_p$	5
Number of convolution kernels	$d_c$	230
Convolutional kernel size	c	3
Batch training size	batch_size	160
Optimizer	–	SGD
Pre-training learning rate	pre-train_learning_rate	0.4
Clustering learning rate	clustering_learning_rate	0.004
Model learning rate	model_learning_rate	0.1
Maximum pre-training epochs	–	30
Maximum clustering epochs	–	50
Maximum joint-training epochs	–	100
Early stopping patience	–	10 epochs
Validation metric	–	F1-score
Random seeds	–	13, 21, 42, 87, 100
Noise identification threshold	${\varphi }_1$	0.5
Secondary confidence threshold	${\varphi }_2$	0.1
Entity-pair similarity threshold	${\tau }_e$	0.50
Relation-word similarity threshold	${\tau }_r$	0.65
Dropout probability	Dropout	0.5
Sliding window size	$w_s$	3
Balance coefficient	$\lambda$	0.6

, the hyperparameter search space is summarized in

Table 8. Search ranges of key denoising-related hyperparameters.

Hyperparameter	Symbol	Search Range
Noise identification threshold	${\varphi }_1$	$\{0.30,0.40,0.50,0.60,0.70\}$
Secondary confidence threshold	${\varphi }_2$	$\{0.05,0.10,0.15,0.20\}$
Clustering loss coefficient	$\lambda$	$\{0.20,0.40,0.60,0.80,1.00\}$
Entity-pair and relation-word thresholds	$({\tau }_e,{\tau }_r)$	$\left\{\right(0.40,0.55),(0.45,0.60),(0.50,0.65)$, $(0.55,0.70),(0.60,0.75)\}$

, and the experimental environment is summarized in

Table 9. Experimental environment.

Configuration	Specification
CPU	Intel(R) Xeon(R) Platinum 8358P
GPU	NVIDIA GeForce RTX 3090
RAM	512 GB
Operating system	Ubuntu 18.04.6
Python version	3.6.13
TensorFlow version	1.14.0

. Hyperparameter selection is performed only on the validation set. The test set is not used for hyperparameter tuning and is used only for final performance reporting.

To improve transparency, we report the implementation environment, fixed dataset split, evaluation settings, random seeds, optimizer, early stopping criterion, final hyperparameter configuration, and hyperparameter search ranges. Although the processed dataset and source code are not publicly released at this stage, these additional details are provided to make the experimental setting clearer and more reproducible.

The hyperparameters in Table 7 are the final configurations used in the main experiments. Parameters related to the PCNN encoder, such as word vector dimension, position feature dimension, convolution kernel number, and kernel size, are set following commonly used DS-RE settings to ensure comparability with representative PCNN-based baselines. Parameters related to the proposed denoising process, including ${\varphi }_1$, ${\varphi }_2$, $\lambda$, and $({\tau }_e,{\tau }_r)$, control noise identification, prior confidence estimation, and confidence-weighted label regeneration. Their effects are further examined through the sensitivity analysis in Section 3.6.

The validation set is used to select the final values of the denoising-related hyperparameters. Specifically, ${\varphi }_1$ controls the strictness of noisy-sample identification, ${\varphi }_2$ controls the secondary filtering of low-confidence regenerated labels, $\lambda$ balances the contribution of regenerated noisy samples in the joint training objective, ${\tau }_e$ controls the entity-pair similarity filtering threshold, and ${\tau }_r$ controls the relation-word similarity filtering threshold in the noise-weighting module. The detailed search ranges are shown in Table 8. For each group of parameters, all other hyperparameters are kept fixed during sensitivity analysis.

For the supplementary Transformer-based experiments, BERT-base-Chinese, Chinese-RoBERTa-wwm, and Chinese-MacBERT are evaluated under the same DSRED split and sentence-bag-level evaluation protocol. We compare Transformer-based MIL or attention models with the proposed PCNN-based denoising framework. In addition, encoder replacement experiments are conducted by replacing the PCNN encoder in the full denoising framework with Transformer-based encoders. The relative training cost is normalized by the training time of the PCNN-based full framework.

For cross-dataset generalization evaluation, the proposed framework is further retrained and tested on additional relation extraction datasets under the same evaluation protocol. The core denoising modules and hyperparameter settings are kept unchanged unless dataset-specific input adaptation is required. This setting is used to evaluate whether the proposed noise-prior-guided denoising strategy remains effective beyond DSRED. The external datasets include NYT-10 [19], Wiki-KBP [20], and SciERC-RE [21], following their original public releases and standard relation extraction settings.

3.4. Comparative Experiments

3.4.1. Comparison with Representative Baselines

To evaluate the effectiveness of the proposed noise-weighted unsupervised denoising framework, we compare it with representative relation extraction baselines on the DSRED dataset. All methods are evaluated under the same train/validation/test split and the same sentence-bag-level evaluation protocol. Precision, Recall, F1-score, and PR-AUC are used as evaluation metrics. For methods originally designed under slightly different relation extraction settings, only the input representation and output classification layer are adapted to the DSRED label space, while their core modeling strategies are kept unchanged as much as possible.

To make the comparison clearer, the selected baselines are grouped into four categories. The first category includes early distant supervision and multi-instance learning methods. The second category includes neural PCNN-based and attention-based DS-RE models. The third category includes noise-aware or clustering-based denoising methods, which are most closely related to the proposed framework. The fourth category includes knowledge-enhanced or feature-enhancement relation extraction methods, which provide additional comparison with recent representation-enhancement strategies.

Specifically, the following baseline methods are considered:

• Mintz [3]: This method introduces the distant supervision assumption and automatically aligns entity pairs in text with relations in a knowledge base. It serves as an early rule-based distant supervision baseline.
• MIML [7]: This method formulates distant supervision relation extraction as a multi-instance multi-label learning problem and uses latent variables to model the relationship between sentence instances and relation labels.
• PCNN [4]: This model adopts piecewise convolutional neural networks to extract sentence-level features and uses bag-level training to reduce the influence of noisy labels under distant supervision.
• APCNN [5]: This method extends PCNN by introducing sentence-level selective attention, enabling the model to assign larger weights to informative sentences within a bag.
• PCNN + WN [9]: This model incorporates sentence correlation and relevance modeling within sentence bags to further reduce the effect of noisy instances.
• DCRE [11]: This method applies unsupervised deep clustering to noisy sentences and reassigns relation labels, making it one of the closest clustering-based denoising baselines to our work.
• RECON [12]: This method incorporates knowledge graph context into relation extraction through graph neural networks, providing a knowledge-enhanced baseline for distant supervision relation extraction.
• EFEAPN [22]: This method enhances entity features and uses an attention-based prototypical network for relation extraction. Since it was originally proposed for few-shot relation extraction, we include it as a feature-enhancement comparison model and adapt it to the DSRED relation classification setting.

The main comparison results are shown in

Table 10. Comparison with representative relation extraction baselines on DSRED under the same bag-level evaluation protocol.

Model	Precision	Recall	F1	PR-AUC
Mintz	31.8%	38.5%	34.8%	$0.421\pm 0.018$
MIML	44.7%	32.9%	37.9%	$0.457\pm 0.017$
PCNN	53.6%	50.1%	51.2%	$0.628\pm 0.015$
APCNN	69.2%	72.8%	71.0%	$0.781\pm 0.011$
PCNN + WN	73.5%	78.2%	75.8%	$0.827\pm 0.010$
DCRE	85.9%	87.9%	86.9%	$0.908\pm 0.007$
RECON	86.3%	88.5%	87.4%	$0.916\pm 0.006$
EFEAPN	87.6%	89.2%	88.4%	$0.925\pm 0.006$
Ours	89.7%	90.6%	90.1%	$\mathbf{0}.\mathbf{942}\pm \mathbf{0}.\mathbf{004}$

. The statistical analysis of PR-AUC results is reported in

Table 11. Statistical analysis of PR-AUC results for comparison experiments on DSRED.

Model	PR-AUC	95% CI	${\mathit{p}}_{\mathbf{adj}}$ vs. Ours	Sig.
Mintz	$0.421\pm 0.018$	[0.399, 0.443]	$9.09\times {10}^{-7}$	***
MIML	$0.457\pm 0.017$	[0.436, 0.478]	$8.51\times {10}^{-7}$	***
PCNN	$0.628\pm 0.015$	[0.609, 0.647]	$2.49\times {10}^{-6}$	***
APCNN	$0.781\pm 0.011$	[0.767, 0.795]	$5.00\times {10}^{-6}$	***
PCNN + WN	$0.827\pm 0.010$	[0.815, 0.839]	$1.18\times {10}^{-5}$	***
DCRE	$0.908\pm 0.007$	[0.899, 0.917]	$4.56\times {10}^{-4}$	***
RECON	$0.916\pm 0.006$	[0.909, 0.923]	$7.10\times {10}^{-4}$	***
EFEAPN	$0.925\pm 0.006$	[0.918, 0.932]	$0.0094$	**
Ours	$\mathbf{0}.\mathbf{942}\pm \mathbf{0}.\mathbf{004}$	[0.937, 0.947]	–	–

$p_{\mathrm{adj}}$ denotes Bonferroni-adjusted p-values compared with the proposed method. Significance levels: ** $p_{\mathrm{adj}}<0.01$, *** $p_{\mathrm{adj}}<0.001$.

.

As shown in Table 10, early distant supervision methods such as Mintz and MIML obtain relatively low performance. This is mainly because they rely heavily on heuristic alignment or handcrafted features and have limited ability to handle sentence-level label noise. PCNN improves performance by automatically learning sentence representations through piecewise convolution and by conducting bag-level training under the multi-instance learning assumption. APCNN further improves performance by assigning attention weights to sentences within a bag, which helps the model focus on more informative instances.

Compared with PCNN-based attention models, PCNN+WN further models sentence correlation within bags and achieves stronger denoising performance. DCRE and RECON obtain higher F1-scores because they explicitly exploit noisy data or external knowledge. DCRE reuses noisy instances through clustering-based label reassignment, while RECON introduces structured knowledge graph context to enrich relation representations. These methods are therefore more closely related to the denoising objective of this work.

The proposed default PCNN-based framework achieves the best performance among the representative baseline methods, with a Precision of 89.7%, Recall of 90.6%, F1-score of 90.1%, and PR-AUC of $0.942\pm 0.004$. Compared with the strongest baseline EFEAPN, the proposed method improves F1-score from 88.4% to 90.1% and PR-AUC from $0.925\pm 0.006$ to $0.942\pm 0.004$. Table 11 further reports the 95% confidence intervals and Bonferroni-adjusted significance test results for PR-AUC. Compared with EFEAPN, the improvement remains statistically significant after Bonferroni correction ($p_{\mathrm{adj}}=0.0094$), indicating that the observed PR-AUC improvement is unlikely to be caused by random variation.

In addition, a one-way ANOVA is conducted to examine the overall difference among all compared methods in terms of PR-AUC. The results show a significant difference among the nine methods, $F(8,36)=1578.09$, $p=1.95\times {10}^{-43}$, with a large effect size of ${\eta }^2=0.997$. To further examine the differences among competitive methods, we also conduct an additional one-way ANOVA on DCRE, RECON, EFEAPN, and the proposed method. The results also show a significant difference, $F(3,16)=31.08$, $p=6.59\times {10}^{-7}$, with a large effect size of ${\eta }^2=0.854$.

Overall, these results suggest that introducing sentence-level noise prior to weights before clustering-based label regeneration helps reduce unreliable pseudo-label propagation. In addition, the multi-factor representation module further strengthens noise detection by incorporating lexical, positional, and entity-type information.

3.4.2. Supplementary Comparison with Transformer-Based Encoders

To further address the concern that PCNN may be less expressive than modern Transformer-based encoders, we conduct supplementary experiments with several Transformer-based baselines on DSRED. These baselines replace the PCNN-style sentence representation with BERT-family encoders and use common bag-level denoising strategies such as MIL pooling or selective attention. The purpose of this comparison is to examine whether representation enhancement alone can sufficiently address noisy distant supervision.

As shown in

Table 12. Supplementary comparison with Transformer-based baselines on DSRED.

Model	Encoder	Denoising Strategy	Prec.	Rec.	F1	PR-AUC
BERT-MIL	BERT-base-Chinese	MIL pooling	82.6%	83.9%	83.2%	$0.881\pm 0.008$
BERT-ATT	BERT-base-Chinese	Selective attention	84.8%	85.7%	85.2%	$0.899\pm 0.007$
RoBERTa-ATT	Chinese-RoBERTa-wwm	Selective attention	85.5%	86.5%	86.0%	$0.907\pm 0.007$
MacBERT-ATT	Chinese-MacBERT	Selective attention	86.0%	87.0%	86.5%	$0.912\pm 0.006$
Ours-PCNN	PCNN	Noise weighting + clustering	89.7%	90.6%	90.1%	$\mathbf{0}.\mathbf{942}\pm \mathbf{0}.\mathbf{004}$

, Transformer-based baselines achieve stronger performance than several traditional DS-RE models, indicating that contextualized sentence representations are beneficial for relation extraction. However, these Transformer-based baselines still underperform the proposed framework. This suggests that representation enhancement alone is insufficient for noisy distant supervision, and that explicit noise weighting and clustering-based label regeneration play a more important role in reducing erroneous supervision.

Table 13. Encoder replacement analysis under the full denoising framework on DSRED.

Encoder	Prec.	Rec.	F1	PR-AUC	95% CI	${\mathit{p}}_{\mathbf{adj}}$ vs. PCNN	Train Cost
PCNN (default)	89.7%	90.6%	90.1%	$0.942\pm 0.004$	[0.937, 0.947]	–	$1.0\times$
BERT-base-Chinese, frozen	89.6%	90.2%	89.9%	$0.941\pm 0.005$	[0.935, 0.947]	>0.99	$3.1\times$
BERT-base-Chinese, fine-tuned	90.2%	90.5%	90.4%	$0.946\pm 0.005$	[0.940, 0.952]	0.86	$8.2\times$
Chinese-RoBERTa-wwm	90.3%	90.7%	90.5%	$0.947\pm 0.005$	[0.941, 0.953]	0.52	$8.6\times$
Chinese-MacBERT	90.5%	90.8%	90.6%	$0.948\pm 0.004$	[0.943, 0.953]	0.18	$8.9\times$

further evaluates whether replacing PCNN with stronger Transformer-based encoders improves the full denoising framework. The results show that Transformer encoders slightly improve sentence representation quality. For example, replacing PCNN with Chinese-MacBERT increases F1-score from 90.1% to 90.6% and PR-AUC from $0.942\pm 0.004$ to $0.948\pm 0.004$. However, this improvement is not statistically significant after Bonferroni correction ($p_{\mathrm{adj}}=0.18$), while the relative training cost increases to 8.9×.

A one-way ANOVA over the full-framework variants with different encoders shows no statistically significant difference in PR-AUC, $F(4,20)=2.27$, $p=0.098$. This result suggests that the main performance gain comes from the proposed noise-prior-guided denoising framework rather than from a specific sentence encoder. Therefore, PCNN is adopted as the default encoder in this work to ensure computational efficiency, fair comparison with representative DS-RE baselines, and clearer evaluation of the proposed denoising modules.

3.5. Ablation Experiments

To further evaluate the contribution of different components in the proposed framework, we conduct two groups of ablation experiments. The first group investigates the module-level contribution of the noise weighting module and the clustering-based label generation module. The second group analyzes the effect of different impact factors in the multi-factor encoding module, including POS information, relative position, and entity type information.

3.5.1. Module-Level Ablation Study

To verify the effectiveness of the core components, we conduct module-level ablation experiments on the noise weighting module and the clustering-based label generation module. Four model variants are evaluated: the core framework without both modules, the full framework without the clustering module, the full framework without the noise weighting module, and the complete framework.

As shown in

Table 14. Module-level ablation study of the noise weighting module and clustering-based label generation module on DSRED.

Model Variant	NW	CL	Precision	Recall	F1	PR-AUC	95% CI
Core framework	No	No	86.0%	85.6%	85.8%	$0.896\pm 0.008$	[0.886, 0.906]
Full w/o clustering	Yes	No	88.5%	86.9%	87.7%	$0.912\pm 0.007$	[0.903, 0.921]
Full w/o noise weighting	No	Yes	87.4%	88.8%	88.1%	$0.918\pm 0.006$	[0.911, 0.925]
Full framework	Yes	Yes	89.7%	90.6%	90.1%	$\mathbf{0}.\mathbf{942}\pm \mathbf{0}.\mathbf{004}$	[0.937, 0.947]

NW denotes the noise weighting module, and CL denotes the clustering-based label generation module.

, removing either the noise weighting module or the clustering-based label generation module leads to clear performance degradation. When the clustering module is removed, the F1-score decreases from 90.1% to 87.7%, and the PR-AUC decreases from $0.942\pm 0.004$ to $0.912\pm 0.007$. This indicates that clustering-based label regeneration is effective for reusing noisy instances and improving the reliability of supervision.

When the noise weighting module is removed, the F1-score decreases to 88.1%, and the PR-AUC decreases to $0.918\pm 0.006$. This result suggests that noise-aware prior weighting helps improve fine-grained noise identification before label regeneration. When both modules are removed, the F1-score further decreases to 85.8%, and the PR-AUC decreases to $0.896\pm 0.008$, confirming the complementary contributions of the two modules.

The statistical results in

Table 15. Statistical analysis of module-level ablation results on DSRED.

Comparison	PR-AUC Difference	${\mathit{p}}_{\mathbf{adj}}$	Sig.
Full vs. core framework	+0.046	$8.94\times {10}^{-5}$	***
Full vs. full w/o clustering	+0.030	$3.60\times {10}^{-4}$	***
Full vs. full w/o noise weighting	+0.024	$4.42\times {10}^{-4}$	***

$p_{\mathrm{adj}}$ denotes Bonferroni-adjusted p-values. Significance levels: *** $p_{\mathrm{adj}}<0.001$.

further demonstrate the robustness of these improvements. Compared with the full framework, all ablated variants show statistically significant decreases in PR-AUC after Bonferroni correction. Specifically, the full framework improves PR-AUC by 0.030 over the variant without clustering and by 0.024 over the variant without noise weighting, both with $p_{\mathrm{adj}}<0.001$.

In addition, the two-way ANOVA results in

Table 16. Two-way ANOVA results for the effects of noise weighting and clustering modules on PR-AUC.

Source	df	F-Value	p-Value	Partial ${\mathit{\eta }}_{\mathit{p}}^2$
Noise weighting	1	48.48	$3.20\times {10}^{-6}$	0.752
Clustering	1	81.94	$1.08\times {10}^{-7}$	0.837
Noise weighting × clustering	1	1.94	0.183	0.108
Error	16	–	–	–

show that both the noise weighting module and the clustering module have significant main effects on PR-AUC. The noise weighting module significantly improves PR-AUC, $F(1,16)=48.48$, $p=3.20\times {10}^{-6}$, with a large effect size of ${\eta }_p^2=0.752$. The clustering module also shows a significant effect, $F(1,16)=81.94$, $p=1.08\times {10}^{-7}$, with a large effect size of ${\eta }_p^2=0.837$. The interaction between the two modules is not statistically significant, $F(1,16)=1.94$, $p=0.183$, suggesting that the two modules provide complementary but mainly additive improvements.

3.5.2. Factor-Level Contribution Analysis of Multi-Factor Encoding

To further analyze the role of the multi-factor encoding module, we conduct an additional factor-level ablation study. This experiment investigates the incremental effects of POS information, relative position, and entity type information. The results are shown in

Table 17. Ablation study on multi-factor encoding.

Impact Factors	Precision	Recall	F1	PR-AUC
None	67.1%	60.8%	63.8%	$0.691\pm 0.014$
+ POS	73.7%	64.2%	68.2%	$0.739\pm 0.012$
+ POS + relative position	76.3%	70.1%	73.1%	$0.791\pm 0.010$
+ POS + Relative Position + Entity Type	79.5%	72.7%	75.9%	$\mathbf{0}.\mathbf{823}\pm \mathbf{0}.\mathbf{008}$

, and the corresponding statistical analysis is shown in

Table 18. Statistical analysis of PR-AUC results for multi-factor ablation experiments on DSRED.

Impact Factors	PR-AUC	95% CI	${\mathit{p}}_{\mathbf{adj}}$ vs. Full	Sig.
None	$0.691\pm 0.014$	[0.674, 0.708]	$2.90\times {10}^{-6}$	***
+ POS	$0.739\pm 0.012$	[0.724, 0.754]	$1.14\times {10}^{-5}$	***
+ POS + relative position	$0.791\pm 0.010$	[0.779, 0.803]	$0.0018$	**
+ POS + Relative Position + Entity Type	$\mathbf{0}.\mathbf{823}\pm \mathbf{0}.\mathbf{008}$	[0.813, 0.833]	–	–

$p_{\mathrm{adj}}$ denotes Bonferroni-adjusted p-values compared with the full impact-factor model. Significance levels: ** $p_{\mathrm{adj}}<0.01$, *** $p_{\mathrm{adj}}<0.001$.

.

This factor-level ablation is conducted to isolate the internal contribution of the multi-factor encoding module, and therefore its absolute values are not directly compared with the full denoising framework in Table 14.

Table 17 analyzes the contribution of different impact factors in the multi-factor encoding module. Without any impact factors, the model obtains an F1-score of 63.8% and a PR-AUC of $0.691\pm 0.014$. After adding POS information, the F1-score increases to 68.2%, indicating that syntactic cues help identify relation-indicative words. When relative position information is further incorporated, the F1-score reaches 73.1%, showing that entity-centered positional information helps the model focus on relation-relevant contexts. Finally, after adding entity type information, the model achieves the best performance in this factor-level setting, with an F1-score of 75.9% and a PR-AUC of $0.823\pm 0.008$.

Table 18 reports the 95% confidence intervals and Bonferroni-adjusted significance test results. Compared with the full impact-factor model, removing any group of impact factors leads to a statistically significant decrease in PR-AUC. In addition, a one-way ANOVA shows a significant effect of impact-factor configuration, $F(3,16)=133.97$, $p=1.51\times {10}^{-11}$, with a large effect size of ${\eta }^2=0.962$. These results support the effectiveness of jointly modeling POS information, relative position information, and entity type information.

3.6. Sensitivity Analysis

To examine the robustness of the proposed framework with respect to key hyperparameters, we conduct sensitivity experiments on the noise identification threshold ${\varphi }_1$, the secondary confidence threshold ${\varphi }_2$, the clustering loss coefficient $\lambda$, and the noise-weighting thresholds $({\tau }_e,{\tau }_r)$. Here, ${\tau }_e$ denotes the entity-pair similarity threshold, and ${\tau }_r$ denotes the relation-word similarity threshold in the noise weighting module. When one group of hyperparameters is varied, all other settings are kept unchanged.

As shown in

Table 19. Sensitivity analysis of key hyperparameters on DSRED.

Setting	F1	PR-AUC	NMI	ARI	Silhouette	Purity
Noise identification threshold ${\varphi }_1$
${\varphi }_1=0.30$	88.9%	$0.928\pm 0.006$	$0.663\pm 0.014$	$0.586\pm 0.016$	$0.355\pm 0.011$	$0.789\pm 0.013$
${\varphi }_1=0.40$	89.7%	$0.936\pm 0.005$	$0.688\pm 0.012$	$0.607\pm 0.014$	$0.373\pm 0.010$	$0.807\pm 0.011$
${\mathit{\varphi }}_{\mathbf{1}}=\mathbf{0.50}$	90.1%	$\mathbf{0}.\mathbf{942}\pm \mathbf{0}.\mathbf{004}$	$\mathbf{0.701}\pm \mathbf{0}.\mathbf{011}$	$\mathbf{0}.\mathbf{623}\pm \mathbf{0}.\mathbf{012}$	$\mathbf{0}.\mathbf{382}\pm \mathbf{0}.\mathbf{009}$	$\mathbf{0}.\mathbf{814}\pm \mathbf{0}.\mathbf{010}$
${\varphi }_1=0.60$	89.8%	$0.938\pm 0.005$	$0.690\pm 0.012$	$0.610\pm 0.014$	$0.371\pm 0.010$	$0.806\pm 0.011$
${\varphi }_1=0.70$	89.2%	$0.929\pm 0.006$	$0.671\pm 0.015$	$0.589\pm 0.017$	$0.353\pm 0.012$	$0.792\pm 0.014$
Secondary confidence threshold ${\varphi }_2$
${\varphi }_2=0.05$	89.5%	$0.933\pm 0.006$	$0.676\pm 0.014$	$0.596\pm 0.016$	$0.359\pm 0.012$	$0.797\pm 0.013$
${\mathit{\varphi }}_{\mathbf{2}}=\mathbf{0}.\mathbf{10}$	90.1%	$\mathbf{0}.\mathbf{942}\pm \mathbf{0}.\mathbf{004}$	$\mathbf{0}.\mathbf{701}\pm \mathbf{0}.\mathbf{011}$	$\mathbf{0}.\mathbf{623}\pm \mathbf{0}.\mathbf{012}$	$\mathbf{0}.\mathbf{382}\pm \mathbf{0}.\mathbf{009}$	$\mathbf{0}.\mathbf{814}\pm \mathbf{0}.\mathbf{010}$
${\varphi }_2=0.15$	89.7%	$0.936\pm 0.005$	$0.689\pm 0.013$	$0.608\pm 0.015$	$0.371\pm 0.011$	$0.805\pm 0.012$
${\varphi }_2=0.20$	89.0%	$0.928\pm 0.006$	$0.664\pm 0.015$	$0.584\pm 0.017$	$0.350\pm 0.013$	$0.787\pm 0.014$
Clustering loss coefficient λ
$\lambda =0.20$	89.1%	$0.930\pm 0.006$	$0.666\pm 0.015$	$0.588\pm 0.017$	$0.356\pm 0.012$	$0.791\pm 0.014$
$\lambda =0.40$	89.8%	$0.938\pm 0.005$	$0.690\pm 0.013$	$0.609\pm 0.014$	$0.372\pm 0.010$	$0.807\pm 0.011$
$\mathit{\lambda }=\mathbf{0}.\mathbf{60}$	90.1%	$\mathbf{0}.\mathbf{942}\pm \mathbf{0}.\mathbf{004}$	$\mathbf{0}.\mathbf{701}\pm \mathbf{0}.\mathbf{011}$	$\mathbf{0}.\mathbf{623}\pm \mathbf{0}.\mathbf{012}$	$\mathbf{0}.\mathbf{382}\pm \mathbf{0}.\mathbf{009}$	$\mathbf{0}.\mathbf{814}\pm \mathbf{0}.\mathbf{010}$
$\lambda =0.80$	89.8%	$0.936\pm 0.005$	$0.687\pm 0.013$	$0.606\pm 0.015$	$0.369\pm 0.011$	$0.804\pm 0.012$
$\lambda =1.00$	89.4%	$0.931\pm 0.006$	$0.672\pm 0.015$	$0.591\pm 0.017$	$0.357\pm 0.012$	$0.793\pm 0.014$
Noise-weighting thresholds $({\tau }_e,{\tau }_r)$
$(0.40,0.55)$	89.6%	$0.934\pm 0.006$	$0.682\pm 0.014$	$0.600\pm 0.016$	$0.362\pm 0.012$	$0.802\pm 0.013$
$(0.45,0.60)$	90.0%	$0.939\pm 0.005$	$0.694\pm 0.012$	$0.615\pm 0.014$	$0.375\pm 0.010$	$0.810\pm 0.011$
$(\mathbf{0}.\mathbf{50},\mathbf{0}.\mathbf{65})$	90.1%	$\mathbf{0}.\mathbf{942}\pm \mathbf{0}.\mathbf{004}$	$\mathbf{0}.\mathbf{701}\pm \mathbf{0}.\mathbf{011}$	$\mathbf{0}.\mathbf{623}\pm \mathbf{0}.\mathbf{012}$	$\mathbf{0}.\mathbf{382}\pm \mathbf{0}.\mathbf{009}$	$\mathbf{0}.\mathbf{814}\pm \mathbf{0}.\mathbf{010}$
$(0.55,0.70)$	89.7%	$0.936\pm 0.005$	$0.687\pm 0.013$	$0.608\pm 0.015$	$0.369\pm 0.011$	$0.803\pm 0.012$
$(0.60,0.75)$	89.2%	$0.928\pm 0.006$	$0.669\pm 0.015$	$0.587\pm 0.017$	$0.351\pm 0.013$	$0.789\pm 0.014$

, the proposed method achieves the best or near-best performance when ${\varphi }_1=0.50$, ${\varphi }_2=0.10$, $\lambda =0.60$, and $({\tau }_e,{\tau }_r)=(0.50,0.65)$. For ${\varphi }_1$, a value that is too small may fail to identify enough noisy instances for label regeneration, while a value that is too large may send too many ambiguous samples into the clustering module. For ${\varphi }_2$, a lower threshold may accept unreliable regenerated labels, whereas an overly high threshold may discard recoverable noisy samples. For $\lambda$, a small value underuses regenerated noisy instances, while a large value may overemphasize clustering-derived pseudo-labels. For the noise-weighting thresholds $({\tau }_e,{\tau }_r)$, overly permissive thresholds may introduce weakly matched triples, whereas overly strict thresholds may filter out useful relation evidence. Overall, the results indicate that the proposed framework is relatively stable within a reasonable hyperparameter range, and the default configuration provides the best trade-off between relation extraction performance and clustering quality.

3.7. Clustering Quality Evaluation

To further verify the effectiveness of the clustering-based label generation module, we evaluate the clustering quality on detected noisy instances using NMI, ARI, silhouette score, and cluster purity. NMI, ARI, and purity measure the consistency between cluster assignments and reference relation labels, while the silhouette score evaluates the compactness and separation of clusters in the relation-aware feature space.

As shown in

Table 20. Clustering quality evaluation on detected noisy instances of DSRED.

Method	NMI	ARI	Sil.	Purity
K-means on PCNN features	$0.512\pm 0.018$	$0.421\pm 0.020$	$0.237\pm 0.013$	$0.648\pm 0.016$
DEC without relation-aware projection	$0.603\pm 0.015$	$0.520\pm 0.018$	$0.309\pm 0.011$	$0.721\pm 0.014$
Ours w/o noise weighting	$0.646\pm 0.013$	$0.565\pm 0.014$	$0.337\pm 0.010$	$0.759\pm 0.012$
Full framework	$\mathbf{0}.\mathbf{701}\pm \mathbf{0}.\mathbf{011}$	$\mathbf{0}.\mathbf{623}\pm \mathbf{0}.\mathbf{012}$	$\mathbf{0}.\mathbf{382}\pm \mathbf{0}.\mathbf{009}$	$\mathbf{0}.\mathbf{814}\pm \mathbf{0}.\mathbf{010}$

, the full framework achieves the highest values across all clustering quality metrics, with an NMI of $0.701\pm 0.011$, an ARI of $0.623\pm 0.012$, a silhouette score of $0.382\pm 0.009$, and a purity of $0.814\pm 0.010$. Compared with K-means on PCNN features, the proposed method substantially improves the consistency and compactness of the clusters. Compared with DEC without relation-aware projection, the full framework also achieves better clustering quality, indicating that relation-aware projection is beneficial for aligning noisy sentence representations with relation semantics.

The comparison with the variant without noise weighting further shows that sentence-level prior confidence helps improve clustering reliability. These results demonstrate that the proposed noise weighting and relation-aware clustering strategy can generate more coherent and relation-consistent clusters, thereby improving the reliability of regenerated labels for noisy instances.

3.8. Cross-Dataset Generalization

To further evaluate the generalizability of the proposed framework, we conduct cross-dataset experiments on multiple relation extraction datasets, including DSRED, NYT-10, Wiki-KBP, and SciERC-RE. For each dataset, the model is retrained and evaluated under the same sentence-bag-level evaluation protocol. The core denoising modules are kept unchanged, while dataset-specific input and label spaces are adapted accordingly. The strongest baseline is selected according to the best PR-AUC among the compared baseline methods on each dataset.

As shown in

Table 21. Cross-dataset generalization results under the same evaluation protocol.

Dataset	Strongest Baseline	Baseline F1	Baseline PR-AUC	Ours F1	Ours PR-AUC	${\mathit{p}}_{\mathbf{adj}}$
DSRED	EFEAPN	88.4%	$0.925\pm 0.006$	90.1%	$\mathbf{0}.\mathbf{942}\pm \mathbf{0}.\mathbf{004}$	0.0094
NYT-10	EFEAPN	85.4%	$0.897\pm 0.007$	86.7%	$\mathbf{0}.\mathbf{918}\pm \mathbf{0}.\mathbf{006}$	0.018
Wiki-KBP	EFEAPN	83.7%	$0.881\pm 0.008$	85.3%	$\mathbf{0}.\mathbf{904}\pm \mathbf{0}.\mathbf{007}$	0.014
SciERC-RE	EFEAPN	81.5%	$0.863\pm 0.010$	83.1%	$\mathbf{0}.\mathbf{887}\pm \mathbf{0}.\mathbf{008}$	0.026

, the proposed framework consistently outperforms the strongest baseline across all evaluated datasets in terms of F1-score and PR-AUC. Although the absolute performance decreases slightly on external datasets due to different relation distributions and corpus characteristics, the proposed method still maintains stable improvements. These results indicate that the noise-prior-guided denoising strategy is not limited to DSRED and can be transferred to other relation extraction datasets.

Table 22. Clustering quality of the proposed framework across datasets.

Dataset	NMI	ARI	Silhouette	Purity
DSRED	$\mathbf{0}.\mathbf{701}\pm \mathbf{0}.\mathbf{011}$	$\mathbf{0}.\mathbf{623}\pm \mathbf{0}.\mathbf{012}$	$\mathbf{0}.\mathbf{382}\pm \mathbf{0}.\mathbf{009}$	$\mathbf{0}.\mathbf{814}\pm \mathbf{0}.\mathbf{010}$
NYT-10	$0.668\pm 0.013$	$0.589\pm 0.015$	$0.351\pm 0.011$	$0.782\pm 0.013$
Wiki-KBP	$0.641\pm 0.014$	$0.556\pm 0.016$	$0.332\pm 0.012$	$0.761\pm 0.014$
SciERC-RE	$0.618\pm 0.015$	$0.531\pm 0.018$	$0.318\pm 0.013$	$0.742\pm 0.016$

further reports the clustering quality of the proposed framework across datasets. The full framework achieves the strongest clustering quality on DSRED and maintains reasonable clustering consistency on NYT-10, Wiki-KBP, and SciERC-RE. The results suggest that the clustering-based label generation module can produce meaningful relation-aware clusters across different relation distributions. This further supports the generalizability of the proposed denoising framework.

3.9. Error Analysis

To better understand the advantages and limitations of the proposed framework, we conduct a qualitative error analysis on representative examples from DSRED. The examples are anonymized and translated from Chinese for readability. The reference labels in

Table 23. Qualitative examples of correct predictions on DSRED. All examples are anonymized and translated from Chinese for readability.

Case	Sentence/Bag Evidence	Labels	Analysis
C1	“The [Entity-A] was developed by [Organization-B] and completed several field tests in 2022.”	DS: Developed_by Ref.: Developed_by Pred.: Developed_by	Correct. The trigger word “developed” is close to the target entity pair, and the entity types are compatible with the relation. The instance receives a high noise weight and is selected as a valid sample.
C2	“The [System-C] has been deployed on the [Platform-D] for long-range monitoring tasks.”	DS: Manufactured_by Ref.: Deployed_on Pred.: Deployed_on	Correct after label regeneration. The distant supervision label is noisy, but the phrase “deployed on” better matches another relation. The clustering module regenerates a more reliable label.
C3	“At the exhibition, [Organization-E] introduced several products, including [Object-F], [Object-G], and communication devices.”	DS: Produced_by Ref.: No_relation Pred.: No_relation	Correct noise suppression. The target entity pair co-occurs in the sentence, but no explicit production relation is expressed. The model avoids forcing a positive relation.

and 

Table 24. Qualitative examples of incorrect predictions on DSRED. All examples are anonymized and translated from Chinese for readability.

Case	Sentence/Bag Evidence	Labels	Analysis
E1	“After several rounds of testing, the [System-J] entered service with the [Unit-K].”	DS: Deployed_in Ref.: Deployed_in Pred.: No_relation	False negative. The relation is expressed implicitly by “entered service with”, which is not strongly matched by the relation-word similarity module.
E2	“[Organization-L] acquired [Organization-M], which later developed the [Object-N].”	DS: Developed_by Ref.: No_relation Pred.: Developed_by	False positive. The trigger word “developed” refers to Organization-M and Object-N, but the target entity pair is Organization-L and Object-N. The model is misled by multiple entity pairs.
E3	“[Institute-O] completed the prototype design of [Device-P], and [Factory-Q] began mass production.”	DS: Manufactured_by Ref.: Developed_by Pred.: Manufactured_by	Relation confusion. “Design” and “mass production” appear together, making Developed_by and Manufactured_by difficult to distinguish. The regenerated label is biased toward a semantically close cluster.

are manually checked for qualitative analysis only and are not used to estimate the overall annotation noise ratio of DSRED.

As shown in Table 23 and Table 24, the proposed framework performs well when the relation expression is explicit and close to the target entity pair. Case C1 is a typical example. The relation-indicative word “developed” provides a clear lexical cue, while the entity-type constraint and relative-position information help the model focus on the relation-relevant context. This indicates that the multi-factor encoding module is effective in capturing lexical and structural evidence for relation extraction.

Case C2 demonstrates the benefit of the clustering-based label generation module. Although the distant supervision label is incorrect, the sentence contains a clear relation expression that is inconsistent with the original label. By incorporating sentence-level prior confidence and relation-aware clustering, the model regenerates a more appropriate relation label instead of directly discarding the noisy instance. This suggests that the proposed framework can reuse informative noisy samples and reduce information loss caused by distant supervision noise.

Case C3 shows another advantage of the proposed method. The target entity pair co-occurs in the same sentence, but the sentence does not explicitly express the distantly supervised relation. Conventional distant supervision methods may incorrectly treat such a sentence as a positive instance. In contrast, the proposed method assigns low confidence to the instance and avoids forcing a positive relation prediction, which helps reduce false positives caused by simple entity co-occurrence.

Despite these advantages, several limitations remain. Case E1 is a false negative caused by an implicit relation expression. The sentence expresses the relation through the phrase “entered service with”, but the expression is indirect and does not strongly match the relation-word synonym set. As a result, the model assigns low confidence and predicts No_relation. This indicates that the current noise weighting strategy may be insufficient for highly implicit or context-dependent relation expressions.

Case E2 illustrates the challenge of multiple entity pairs. The sentence contains a clear relation trigger, but the trigger corresponds to another entity pair rather than the target pair. Since distant supervision operates at the entity-pair bag level, such cases can easily lead to incorrect sentence–relation alignment. This suggests that more fine-grained entity-pair-aware relation localization is needed.

Case E3 shows that semantically similar relations may still be confused. In technical descriptions, relations such as Developed_by and Manufactured_by may share overlapping contexts and relation expressions. Although relation-aware clustering improves the consistency of regenerated labels, it may still assign noisy instances to an incorrect but semantically close cluster when the textual evidence is ambiguous.

Overall, the qualitative analysis shows that the proposed framework is effective for explicit relation expressions, noisy-label correction, and false-positive reduction caused by simple entity co-occurrence. However, the model still faces challenges in implicit relation reasoning, multi-entity-pair disambiguation, entity alignment errors, and fine-grained distinction between semantically similar relations. These limitations motivate future work on context-aware relation disambiguation, stronger entity linking, and more adaptive relation-expression modeling.

3.10. Computational Complexity and Scalability

The proposed framework contains several components, including noise weighting, sentence encoding, noise detection, and clustering-based label generation. Compared with a standard PCNN-based distant supervision relation extraction model, the additional computational cost mainly comes from the noise weighting module and the clustering-based label generation module.

The noise weighting module is designed as a lightweight prior estimation process. For each sentence instance, entity-pair similarity and relation-word similarity are computed against candidate relation triples after entity-type filtering. Let M denote the number of sentence instances, and let r denote the average number of candidate relation triples after entity-type filtering. The overall cost of the noise weighting module can be approximately expressed as $O\left(Mr\right)$ when the cost of vector similarity and relation-word matching is treated as a fixed operation. Since entity-type filtering reduces the candidate relation space, the practical cost is much lower than comparing each sentence instance with all triples in the knowledge base. In addition, the noise-prior computation can be implemented as an offline preprocessing step, thereby reducing the online training overhead.

For sentence encoding, PCNN has a linear cost with respect to sentence length and the number of convolution kernels. Given a sentence length m, convolution window size c, and $d_c$ convolution kernels, the sentence encoding cost is approximately $O(m·c·d_c)$ for each sentence. This makes PCNN computationally efficient for sentence-bag-level distant supervision relation extraction. In contrast, Transformer-based encoders usually introduce self-attention costs that grow quadratically with sentence length, approximately $O(L·m^2·d_h)$ for L Transformer layers and hidden dimension $d_h$. Therefore, although Transformer encoders can enhance sentence representations, they also substantially increase computational cost.

The encoder replacement results in Table 13 further support this observation. Replacing PCNN with BERT-family encoders slightly improves sentence representation quality under the full denoising framework, but the improvement is not statistically significant after Bonferroni correction. Meanwhile, the relative training cost increases from $1.0\times$ for PCNN to $8.9\times$ for Chinese-MacBERT. Therefore, PCNN is adopted as the default encoder in this work to maintain computational efficiency, fair comparison with representative DS-RE baselines, and clearer evaluation of the proposed denoising modules.

The clustering-based label generation module operates mainly on detected noisy instances rather than the entire training set. Let n denote the number of detected noisy instances, k denote the number of relation clusters, d denote the dimension of the relation-aware feature space, and T denote the number of clustering refinement iterations. The clustering refinement cost is approximately $O(T·n·k·d)$. Since d is fixed in the implementation, this cost can be simplified as $O\left(Tnk\right)$. More importantly, because only noisy samples identified by the noise detection module are passed to the clustering module, the practical overhead is controlled by the noise detection process.

Overall, the proposed framework introduces additional denoising and clustering costs compared with a standard PCNN-based DS-RE model. However, these costs are partly offset by offline noise-prior computation, efficient PCNN sentence encoding, and applying clustering only to detected noisy samples. Therefore, the framework remains scalable for sentence-bag-level distant supervision relation extraction while providing more reliable supervision through noise weighting and label regeneration.

4. Conclusions

In this paper, we address the erroneous labeling problem inherent in distant supervision relation extraction and propose a noise-weighted unsupervised denoising framework. The proposed framework integrates sentence-level noise weighting, multi-factor representation learning, fine-grained noise detection, and clustering-based label generation. Instead of directly discarding noisy instances, the framework attempts to reuse them by regenerating more reliable relation labels, thereby reducing information loss caused by distant supervision noise.

Specifically, the noise weighting module estimates prior confidence by measuring the relevance between sentence instances and knowledge-base triples. This prior confidence provides guidance for subsequent noise detection. The multi-factor encoding module further incorporates POS information, relative position, and entity type information to enhance sentence representations from lexical and structural perspectives. Based on these representations, the noise detection module identifies valid, noisy, and uncertain samples at the sentence-bag level. For detected noisy samples, the clustering-based label generation module reassigns relation labels according to latent semantic structures, allowing potentially useful noisy instances to be reused during training.

Experimental results on DSRED show that the proposed default PCNN-based framework achieves a Precision of 89.7%, Recall of 90.6%, F1-score of 90.1%, and PR-AUC of $0.942\pm 0.004$. Compared with the strongest representative baseline EFEAPN, the proposed method improves F1-score from 88.4% to 90.1% and PR-AUC from $0.925\pm 0.006$ to $0.942\pm 0.004$. The PR-AUC improvement remains statistically significant after Bonferroni correction ($p_{\mathrm{adj}}=0.0094$). The module-level ablation experiments further show that removing the clustering-based label generation module decreases the F1-score from 90.1% to 87.7%, while removing the noise weighting module decreases the F1-score to 88.1%. These results indicate that both modules contribute positively to the proposed denoising framework.

The multi-factor ablation experiments demonstrate that POS information, relative position, and entity type information are all beneficial for relation extraction under noisy distant supervision. The sensitivity analysis shows that the selected hyperparameters, including ${\varphi }_1$, ${\varphi }_2$, $\lambda$, and $({\tau }_e,{\tau }_r)$, provide the best or near-best performance on DSRED and that the proposed framework remains relatively stable within a reasonable parameter range. In addition, the clustering quality evaluation shows that the full framework produces more coherent and relation-consistent clusters than K-means, DEC without relation-aware projection, and the variant without noise weighting.

We further conduct supplementary experiments with Transformer-based encoders. The results show that BERT-family encoders can slightly improve sentence representation quality under the full denoising framework. However, the improvement over PCNN is not statistically significant after Bonferroni correction, while the training cost increases substantially. This suggests that the main performance gain comes from the proposed noise-prior-guided denoising framework rather than from a specific sentence encoder. Cross-dataset experiments on NYT-10, Wiki-KBP, and SciERC-RE also show that the proposed framework maintains stable improvements beyond DSRED, indicating its potential generalizability under different relation distributions and corpus characteristics.

Although the experiments provide PR-AUC-based statistical analysis, module-level ablation, sensitivity analysis, clustering quality evaluation, Transformer-based supplementary comparison, cross-dataset evaluation, qualitative error analysis, and computational complexity discussion, this study still has several limitations. First, the processed DSRED dataset and source code are not publicly released at this stage due to data usage and project restrictions, which may limit direct reproducibility. To improve transparency, we have provided detailed dataset construction procedures, preprocessing steps, data split statistics, evaluation settings, experimental environment, and hyperparameter configurations.

Second, although the sensitivity analysis verifies the robustness of the selected thresholds on DSRED, the current noise weighting strategy still relies on heuristic similarity measures and fixed thresholds. Adaptive threshold learning and more flexible noise estimation strategies may further improve robustness. Third, although the supplementary Transformer experiments show that encoder replacement can slightly improve performance, the increased training cost suggests that more efficient integration of Transformer-based encoders with noise-prior-guided denoising remains worth exploring. Finally, the qualitative error analysis shows that implicit relation expressions, multiple entity pairs in one sentence, entity alignment errors, and semantically similar relation types remain challenging cases. Future work will investigate context-aware relation disambiguation, stronger entity linking, and more fine-grained clustering constraints.

Overall, the proposed framework provides a promising approach for reducing noisy supervision in distant supervision relation extraction. By combining sentence-level prior confidence estimation, multi-factor representation learning, and clustering-based label regeneration, this work offers useful insights for improving relation extraction under weak supervision.