TERN: Type-Aware Evidence Reasoning for Multimodal Fake News Detection ^†

Abstract

Multimodal fake news detection remains challenging because deceptive posts exhibit heterogeneous manipulation patterns, while most existing methods still rely on a unified fusion strategy. This mismatch limits their ability to adapt to different evidence preferences across samples, encourages entanglement between deception cues and topical semantics, and weakens decision making when textual, visual, and cross-modal signals conflict. To address these issues, we propose TERN, a type-aware evidence reasoning network for multimodal fake news detection. TERN induces latent deception types from image-side multimodal features through prototype-based clustering, uses the induced assignments as structural priors for downstream veracity prediction, disentangles type-discriminative factors from semantic content, and performs type-conditioned hierarchical reasoning over text semantics, image authenticity, and cross-modal consistency. Experiments on MR2-Chinese, MR2-English, Weibo, and PHEME show that TERN achieves an average accuracy of 93.21% and an average F1 score of 91.09% while also improving Matthews correlation coefficient over representative multimodal baselines.

1. Introduction

Online social media platforms have become a primary channel for news consumption, but they have also accelerated the dissemination of false and misleading content. Empirical studies show that false information spreads faster and farther than truthful content on social networks [1,2]. In multimodal fake news, textual claims and visual content can reinforce each other in ways that increase perceived credibility and complicate verification. Compared with text-only falsehoods, such multimodal posts are therefore more difficult to detect and debunk.

Multimodal fake news detection has evolved from early feature-concatenation pipelines into more complex architectures that explicitly model cross-modal interaction, cross-modal consistency, and discriminative representations. Representative studies have explored co-attention [3], hierarchical contextual modeling [4], and cross-modal ambiguity learning [5] to improve cross-modal alignment and discriminative representation learning. This line of research has been extended through contrastive weighting in COOLANT [6], multifaceted reasoning in MRE-FND [7], and inconsistency-aware reasoning in MFIR [8]. Related developments have further examined LLM-augmented fake news detection [9], large vision–language model (LVLM) paradigms for multimodal misinformation analysis [10], retrieval-augmented multimodal verification [11], and structured multimodal interaction in SR-CIBN [12] and MVML [13]. Survey and dataset-oriented studies [14,15] further suggest that marginal performance gains are no longer driven primarily by stronger encoders, but increasingly by how heterogeneous evidence is organized, reasoned about, and fused across diverse deception patterns. Despite this progress, most existing methods are still developed around a unified post-level fusion paradigm. As a result, three fundamental limitations remain largely unaddressed in the literature.

These limitations motivate our work. A key challenge is that multimodal fake news exhibits heterogeneous deception patterns. Some posts are dominated by visual tampering, others by text–image mismatch, and still others by exaggerated textual framing around authentic images. Since these cases depend on different evidence sources, their decision boundaries are not governed by a single evidence preference. Consequently, a model with a fixed aggregation strategy for all cues struggles to adapt its focus to the most informative evidence for each sample.

A second limitation concerns the entanglement between topical semantics and deception patterns in current multimodal representations. The same latent space is frequently expected to encode both what the post is about and how the post becomes deceptive, which increases the risk of spurious topic–veracity associations and weakens generalization under topic, event, and platform shifts. Although representation learning research has long emphasized the value of factorized structure for robust generalization [16], multimodal detection studies also indicate the importance of separating consistency-related cues [17] from reasoning-related evidence patterns [18]. However, this principle has not yet been effectively connected to sample-level deception heterogeneity in existing detection frameworks.

A third challenge arises from the resolution of conflicting evidence. Many existing methods still rely on shallow or single-stage fusion, which makes it difficult to explicitly reason about the relative reliability of different evidence views. When conflicting cues arise from text, image authenticity, and cross-modal consistency views, such models cannot flexibly reweight evidence according to the underlying deception pattern, and the final prediction may be dominated by less reliable or less relevant signals.

To address these limitations, we propose TERN, a type-aware evidence reasoning network for multimodal fake news detection. TERN adopts a three-stage hierarchical architecture consisting of latent deception type induction, type–semantic disentanglement, and type-conditioned evidence reasoning. Specifically, TERN performs prototype-based clustering in the image-side multimodal feature space to induce latent deception types, and then incorporates the induced assignments into the separation of type-discriminative and semantic factors as well as the hierarchical fusion of textual, visual, and cross-modal consistency evidence.

A core distinguishing feature of TERN is its task-coupled latent deception type induction mechanism. Unlike general self-supervised clustering frameworks such as DeepCluster [19] and SwAV [20], which treat prototype assignment as an independent representation learning objective, TERN integrates the induced latent types into downstream veracity prediction as a structural prior for latent type representation learning, type–semantic disentanglement, and evidence fusion. Under this formulation, latent type induction in TERN is not pursued as an independent goal, but as a mechanism for organizing heterogeneous evidence for sample-adaptive reasoning, thereby enabling adaptive evidence weighting across different deception patterns.

The main contributions of this work are threefold:

• We propose a task-coupled latent deception type induction mechanism that introduces sample-level structural priors to guide representation learning and evidence fusion, enabling the model to account for heterogeneous deception patterns without requiring fine-grained type annotations.
• We develop a type-aware evidence reasoning framework that integrates type–semantic disentanglement and hierarchical fusion over textual semantics, image authenticity, and cross-modal consistency into a unified end-to-end framework, thereby supporting adaptive evidence weighting across different deception patterns.
• We conduct comprehensive experiments on four public benchmarks and complement the main detection results with ablation analysis, sensitivity analysis, complexity evaluation, and latent type quality analysis, providing a systematic empirical assessment of the proposed framework.

2. Related Work

The most relevant literature falls into four interconnected research streams: foundational multimodal fake news detection, fine-grained interaction and consistency modeling, contrastive learning and foundation-model adaptation, and evidence-aware reasoning and knowledge-enhanced modeling. Work on LVLM-based detection, retrieval-augmented verification, multilingual misinformation modeling, and generative-AI-era misinformation further broadens this research landscape.

2.1. Foundational Multimodal Fake News Detection

Early multimodal fake news detection studies established the importance of jointly modeling textual and visual information for veracity prediction. EANN [21] introduced event-level bias mitigation, while SpotFake [22] represented an early feature-concatenation pipeline combining BERT and visual cues. Survey evidence further confirms that such foundational systems remain key baselines in multimodal fake news detection research [14,23]. Multilingual misinformation studies further show that detector behavior can vary substantially across languages, platforms, and event contexts [24]. Nevertheless, their decision processes are largely built around global, sample-invariant fusion strategies, preventing them from dynamically adapting the relative contribution of different evidence sources when deception patterns vary across posts.

2.2. Fine-Grained Interaction and Consistency Modeling

To improve upon early fusion pipelines, later studies explored finer-grained cross-modal interaction and consistency modeling. SAFE [25] introduced similarity-aware weighting, MCAN [3] modeled bidirectional co-attention, and HMCAN [4] performed hierarchical contextual interaction. Other models further strengthened local alignment in MMTN [26], relationship-aware attention in MRAN [27], consistency-aware balancing in SR-CIBN [12], and multi-view interaction in MVML [13]. Although these methods improve instance-level matching and cross-modal correlation modeling, they still predominantly estimate evidence interaction within each sample and do not explicitly organize fusion around shared latent deception structures across samples.

2.3. Contrastive Learning and Foundation-Model Adaptation

Another active research stream focuses on contrastive learning, multimodal pre-training, and foundation-model adaptation. CAFE [5] measures cross-modal ambiguity, COOLANT [6] combines contrastive learning with adaptive multimodal weighting, and MCOT [28] improves alignment through optimal transport. Related work has also examined cross-modal knowledge distillation [29], multi-granularity fusion [30], and cross-modal content correlation modeling [31]. This direction also includes synthetic-data-assisted multimodal detection [32], LLM-enhanced multimodal detection [9], and LLM-guided label propagation for multimodal fake news detection [33]. LVLM-oriented surveys and retrieval-augmented models further highlight the importance of explicitly connecting multimodal evidence retrieval, visual forensics, and reasoning with model decisions [10,11,34]. These studies improve representation quality, cross-modal alignment, and transferability. However, their optimization objectives are still largely centered on global classification or feature refinement, and they do not explicitly connect latent structure induction to downstream evidence reasoning under heterogeneous deception patterns.

2.4. Evidence-Aware Reasoning and Knowledge-Enhanced Modeling

Evidence-aware methods emphasize evidence decomposition, consistency reasoning, and knowledge-enhanced decision making in multimodal fake news detection. BTIC [35] enriches multimodal detection with entity-enhanced clues, KAN [36] incorporates external knowledge, and ERIC-FND [37] leverages external reliable information. MRE-FND [7], MFIR [8], and HCCIN [17] further model evidence interactions, inconsistency reasoning, and consistency inference across multiple views. Related reasoning-oriented designs, including multi-reading habit fusion reasoning [18] and TRUST-VL [38], provide additional support for multi-view evidence modeling. Beyond fake news detection, TFGIN [39] shows that structured inference over evidence units can also benefit fact verification tasks. Rationale-generation and cross-verification frameworks further suggest that interpretable evidence organization is becoming central to multimodal misinformation detection [40]. These studies strongly motivate evidence-aware multimodal decision making. However, they generally do not jointly address sample-level deception heterogeneity, type–semantic disentanglement, and adaptive evidence weighting within a unified framework.

3. Methodology

This section presents the proposed TERN framework in detail. Multimodal fake news detection is first formalized under latent deception heterogeneity, followed by the four core components of TERN: multimodal feature extraction and adaptation, latent deception type induction, type–semantic disentanglement, and type-conditioned hierarchical evidence reasoning. The training objective and overall optimization procedure are then introduced.

Figure 1 illustrates the overall architecture of TERN and its correspondence to the following subsections. Starting from multimodal image and text features (Section 3.2), the framework induces latent deception types as structural priors (Section 3.3), separates type-related and semantic information (Section 3.4), and performs hierarchical reasoning over textual semantics, image authenticity, and cross-modal consistency to predict post veracity (Section 3.6).

3.1. Problem Formulation

Let each news post be represented as a multimodal pair $x=(I,T)$, where I denotes the associated image and T denotes the accompanying text. Given a labeled dataset $\mathcal{D}={\left\{(x_i,y_i)\right\}}_{i=1}^N$, where $y_i\in \{0,1\}$ indicates whether the i-th post is fake $\left(1\right)$ or real $\left(0\right)$, the goal is to learn a mapping $f:\mathcal{X}\to \{0,1\}$ that predicts the veracity of unseen multimodal posts.

A key challenge is that multimodal fake news is generated through heterogeneous deception patterns, such as visual tampering, heavily edited or synthetic imagery, misleading textual framing, and text–image mismatch. Since current benchmark datasets typically provide only binary veracity labels rather than fine-grained type annotations, these deception patterns cannot be directly supervised during training. TERN addresses this setting by inducing latent deception types from the multimodal feature space and treating them as internal structural variables that guide representation learning and downstream evidence reasoning.

3.2. Multimodal Feature Extraction and Adaptation

TERN adopts a multi-encoder strategy to capture complementary information from images and text.

3.2.1. Image Features

For each news image I, we extract four types of global and local visual features:

• Global semantic feature ${\mathbf{f}}_{mae}\in {\mathbb{R}}^{768}$: obtained from a MAE-Base encoder [41] pre-trained with a masked image modeling objective, capturing high-level scene and object semantics.
• Cross-modal alignment feature ${\mathbf{f}}_{clip}^{img}\in {\mathbb{R}}^{512}$: extracted by the CLIP ViT-B/32 visual encoder [42], mapping images into a joint vision–language embedding space aligned with text.
• Tampering feature ${\mathbf{f}}_{tamp}\in {\mathbb{R}}^{576}$: obtained from a MobileNetV3-Large network [43] fine-tuned on image manipulation detection datasets, focusing on pixel-level artifacts such as splicing and copy-move.
• Style feature ${\mathbf{f}}_{style}\in {\mathbb{R}}^{512}$: extracted from the conv4_1 layer of VGG19 [44], sensitive to texture and color patterns, which helps capture anomalies in visually manipulated or heavily edited images.

3.2.2. Text Features

For each text T, we extract four types of textual features:

• Global semantic feature ${\mathbf{f}}_{bert}\in {\mathbb{R}}^{768}$: the special classification-token representation, denoted as [CLS], from a BERT-base encoder, capturing topic, narrative style, and linguistic patterns.
• Cross-modal alignment feature ${\mathbf{f}}_{clip}^{txt}\in {\mathbb{R}}^{512}$: obtained from the CLIP text encoder [42], aligned with ${\mathbf{f}}_{clip}^{img}$ in a shared embedding space.
• Entity feature ${\mathbf{f}}_{ent}\in {\mathbb{R}}^{512}$: constructed from named entities and salient tokens identified by SpaCy [45], encoded via CLIP to facilitate fine-grained text–image entity matching.
• Emotion feature ${\mathbf{f}}_{emo}\in {\mathbb{R}}^{128}$: derived from sentiment-related tokens encoded by BERT and projected through a small MLP, capturing emotional intensity and polarity that are often exploited in fake news.

3.2.3. Feature Adaptation

Since the above features have different dimensions and statistical properties, we project them into a unified $d=512$-dimensional space. This shared dimensionality is used consistently by the clustering module, the disentanglement module, and all three evidence generators. For any feature ${\mathbf{f}}_i\in {\mathbb{R}}^{d_i}$ with index $i\in \{1,\dots ,M\}$ over the M extracted feature types, we apply a linear layer followed by layer normalization [46]:

$${\tilde{\mathbf{f}}}_i=\mathrm{LayerNorm}({\mathbf{W}}_i{\mathbf{f}}_i+{\mathbf{b}}_i),$$

(1)

where ${\mathbf{W}}_i\in {\mathbb{R}}^{512\times d_i}$ and ${\mathbf{b}}_i\in {\mathbb{R}}^{512}$ are learnable parameters, and ${\tilde{\mathbf{f}}}_i$ denotes the adapted version of ${\mathbf{f}}_i$ used in all subsequent computations.

To build a cross-modal fused feature, we leverage CLIP’s alignment capability. We first compute the cosine similarity between adapted CLIP text and image features:

$$s_{\mathrm{cor}}={\displaystyle \frac{{\tilde{\mathbf{f}}}_{clip}^{txt}·{\tilde{\mathbf{f}}}_{clip}^{img}}{{\parallel {\tilde{\mathbf{f}}}_{clip}^{txt}\parallel }_2{\parallel {\tilde{\mathbf{f}}}_{clip}^{img}\parallel }_2}}.$$

(2)

Here, $s_{\mathrm{cor}}$ is the global CLIP-based text–image correlation score; the numerator is the Euclidean inner product between the two adapted CLIP embeddings, and ${\parallel ·\parallel }_2$ denotes the Euclidean norm. We then concatenate ${\tilde{\mathbf{f}}}_{clip}^{img}$ and ${\tilde{\mathbf{f}}}_{clip}^{txt}$ and transform them with a multilayer perceptron (MLP), scaled by $s_{\mathrm{cor}}$:

$${\mathbf{f}}_{fused}=s_{\mathrm{cor}}·\mathrm{MLP}\left([{\tilde{\mathbf{f}}}_{clip}^{img};{\tilde{\mathbf{f}}}_{clip}^{txt}]\right).$$

(3)

The notation $[·;·]$ denotes feature concatenation along the channel dimension; unlike the inner product in Equation (2), this operation stacks two vectors before the MLP transformation. This weighting mechanism strengthens the fused representation when text and image are highly consistent, and suppresses it when they are semantically mismatched.

Finally, we aggregate the adapted image features $\{{\tilde{\mathbf{f}}}_{mae},{\tilde{\mathbf{f}}}_{clip}^{img},{\tilde{\mathbf{f}}}_{tamp},{\tilde{\mathbf{f}}}_{style}\}$ with an attention-based fusion:

$${\mathbf{f}}_{img}=\sum _{i\in \mathcal{S}}{\alpha }_i{\tilde{\mathbf{f}}}_i,{\alpha }_i={\displaystyle \frac{exp\left({\mathbf{w}}_i^{\top }{\tilde{\mathbf{f}}}_i\right)}{{\sum }_{j\in \mathcal{S}}exp\left({\mathbf{w}}_j^{\top }{\tilde{\mathbf{f}}}_j\right)}},$$

(4)

where $\mathcal{S}=\{\mathrm{mae},\mathrm{clip},\mathrm{tamp},\mathrm{style}\}$ and ${\mathbf{w}}_i\in {\mathbb{R}}^{512}$ is a learnable attention vector for feature type i; the superscript ⊤ denotes vector transposition used to produce a scalar attention logit. The resulting ${\mathbf{f}}_{img}\in {\mathbb{R}}^{512}$ serves as the input to the type-clustering module.

3.3. Type Clustering and Adaptive Detection

3.3.1. Feature Projection and Normalization

The clustering module takes ${\mathbf{f}}_{img}$ as input and projects it into a lower-dimensional clustering space via a two-layer MLP, followed by $L_2$ normalization:

$$\mathbf{z}={\displaystyle \frac{\mathrm{MLP}\left({\mathbf{f}}_{img}\right)}{{\parallel \mathrm{MLP}\left({\mathbf{f}}_{img}\right)\parallel }_2}}.$$

(5)

This places all samples on the unit hypersphere, making clustering depend primarily on feature directions.

3.3.2. Prototype-Based Clustering

We maintain K learnable prototype vectors ${\left\{{\mathbf{p}}_k\right\}}_{k=1}^K\in {\mathbb{R}}^{256}$, each representing a cluster center in the clustering space. The probability that sample i belongs to type k is computed via a temperature-scaled softmax:

$$q_{ik}={\displaystyle \frac{exp({\mathbf{z}}_i^{\top }{\mathbf{p}}_k/\tau )}{{\sum }_{j=1}^{K}exp({\mathbf{z}}_i^{\top }{\mathbf{p}}_j/\tau )}},$$

(6)

where $\tau >0$ is a temperature hyperparameter controlling the sharpness of assignments: smaller values make the softmax distribution more peaked, whereas larger values produce smoother type probabilities for both image and text-conditioned downstream reasoning.

To learn meaningful clusters, we use two complementary losses. In Equations (6) and (7), N denotes the number of training samples in the current optimization set or mini-batch estimate. The sample entropy loss encourages confident assignments for each sample:

$${\mathcal{L}}_{\mathrm{sample}}=-{\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{\displaystyle \sum _{k=1}^K}{q}_{ik}log{q}_{ik}.$$

(7)

The marginal entropy loss prevents all samples from collapsing into a single cluster:

$${\mathcal{L}}_{\mathrm{marginal}}={\displaystyle \sum _{k=1}^K}{\overline{q}}_{k}log{\overline{q}}_k,{\overline{q}}_k={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{q}_{ik}.$$

(8)

The total clustering loss is:

$${\mathcal{L}}_{\mathrm{cluster}}={\mathcal{L}}_{\mathrm{sample}}+{\mathcal{L}}_{\mathrm{marginal}}.$$

(9)

During training, prototype vectors are updated with momentum using batch-wise statistics, which stabilizes the evolution of cluster centers. Prototype parameters are initialized randomly and refined jointly with the fake news detection objective; clustering is therefore coupled to the downstream task rather than performed as a disconnected preprocessing step. The operating number of clusters K for each dataset is determined through cluster validity analysis on the training-set feature space [47], and the resulting operating setting is then used for downstream training and evaluation.

The soft assignments $\left\{q_{ik}\right\}$ provide type information that will later guide disentanglement and evidence fusion, enabling TERN to adopt differentiated detection strategies for different latent types.

3.4. Type–Semantic Disentanglement

Directly using the fused feature ${\mathbf{f}}_{fused}$ risks mixing deception-type information with content semantics. To address this issue, TERN introduces a type–semantic disentanglement module that decomposes ${\mathbf{f}}_{fused}$ into a type-specific subspace and a semantic subspace.

3.4.1. Mask-Based Decomposition

Given ${\mathbf{f}}_{fused}\in {\mathbb{R}}^{512}$, we use two MLPs followed by sigmoid activations to estimate element-wise soft masks ${\mathbf{m}}_{\mathrm{type}}$ and ${\mathbf{m}}_{\mathrm{sem}}$:

$${\mathbf{f}}_{\mathrm{type}}^{\prime }={\mathbf{f}}_{fused}\odot {\mathbf{m}}_{\mathrm{type}},{\mathbf{f}}_{\mathrm{sem}}^{\prime }={\mathbf{f}}_{fused}\odot {\mathbf{m}}_{\mathrm{sem}},$$

(10)

where ⊙ denotes element-wise multiplication. The two masked features are then passed through separate projection networks to obtain the final type feature ${\mathbf{f}}_{\mathrm{type}}\in {\mathbb{R}}^{128}$ and semantic feature ${\mathbf{f}}_{\mathrm{sem}}\in {\mathbb{R}}^{384}$. The lower dimensionality of ${\mathbf{f}}_{\mathrm{type}}$ reflects the relatively small number of latent types, whereas ${\mathbf{f}}_{\mathrm{sem}}$ requires higher capacity to encode diverse content semantics.

3.4.2. Disentanglement Losses

We employ three complementary losses to encourage type–semantic separation.

1.

Type classification loss: using the soft assignments $q_{ik}$ from the clustering module as pseudo labels, we train a classifier on ${\mathbf{f}}_{\mathrm{type}}$ to predict the type distribution ${\widehat{\mathbf{p}}}_{\mathrm{type},i}$:

$${\mathcal{L}}_{\mathrm{dom}}=-{\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}{\displaystyle \sum _{k=1}^K}{q}_{ik}log{\widehat{p}}_{\mathrm{type},ik}.$$

(11)

This encourages ${\mathbf{f}}_{\mathrm{type}}$ to be predictive of the latent type.

2.

Semantic entropy loss: to prevent the semantic branch from encoding type information, we enforce its predicted type distribution ${\widehat{\mathbf{p}}}_{\mathrm{sem},i}$ to be close to a uniform distribution ${\mathcal{U}}_K$:

$${\mathcal{L}}_{\mathrm{se}}=\mathrm{KL}({\widehat{\mathbf{p}}}_{\mathrm{sem}}\parallel {\mathcal{U}}_K).$$

(12)

3.

Orthogonality loss: we regularize the cosine similarity between ${\mathbf{f}}_{\mathrm{type}}$ and ${\mathbf{f}}_{\mathrm{sem}}$ to encourage geometric orthogonality:

$${\mathcal{L}}_{\mathrm{ortho}}={\parallel {\mathbf{f}}_{\mathrm{type}}^{\top }{\mathbf{W}}_o{\mathbf{f}}_{\mathrm{sem}}\parallel }_2.$$

(13)

The total disentanglement loss is a weighted combination:

$${\mathcal{L}}_{\mathrm{disentangle}}={\mathcal{L}}_{\mathrm{dom}}+{\mathcal{L}}_{\mathrm{se}}+{\lambda }_{\mathrm{ortho}}{\mathcal{L}}_{\mathrm{ortho}},$$

(14)

where ${\mathbf{W}}_o\in {\mathbb{R}}^{128\times 384}$ projects the semantic feature to the type-feature dimension before the bilinear orthogonality penalty, ${\parallel ·\parallel }_2$ denotes the Euclidean norm, and ${\lambda }_{\mathrm{ortho}}$ controls the strength of the orthogonality constraint.

3.5. Three-View Evidence Generation

Building on the disentangled features and modality-specific encodings, TERN instantiates three neural evidence generators corresponding to textual semantics, image authenticity, and cross-modal consistency. For clarity, we briefly summarize the main computations of each view.

3.5.1. Text Semantic View

Let ${\tilde{\mathbf{f}}}_{bert}\in {\mathbb{R}}^{512}$ denote the adapted BERT feature and ${\mathbf{f}}_{emo}\in {\mathbb{R}}^{128}$ the emotion feature. We first expand ${\tilde{\mathbf{f}}}_{bert}$ into a length-1 sequence and apply three parallel 1D convolutions with kernel sizes 1, 3, and 5 and 512 channels:

$${\mathbf{h}}^{\left(r\right)}={\mathrm{Conv}1\mathrm{d}}_r\left({\tilde{\mathbf{f}}}_{bert}\right),r\in \{1,3,5\},$$

(15)

each followed by batch normalization, GELU, and dropout. The resulting feature vectors are stacked into a $3\times 512$ tensor and processed by an 8-head self-attention layer to obtain an aggregated text representation ${\mathbf{h}}_{\mathrm{text}}\in {\mathbb{R}}^{512}$. The 128-dimensional emotion feature is then projected to a 512-dimensional gate:

$${\mathbf{g}}_{\mathrm{emo}}=\sigma \left({\mathbf{W}}_{\mathrm{emo}}{\mathbf{f}}_{emo}\right)\in {(0,1)}^{512},$$

(16)

which performs element-wise modulation:

$${\mathbf{h}}_{\mathrm{text}}^{\prime }={\mathbf{g}}_{\mathrm{emo}}\odot {\mathbf{h}}_{\mathrm{text}}.$$

(17)

Finally, a Transformer encoder layer [48] refines ${\mathbf{h}}_{\mathrm{text}}^{\prime }$ and an MLP maps the result to a two-dimensional evidence logit vector $e_{\mathrm{text}}\in {\mathbb{R}}^2$ capturing textual semantics, writing style, and emotional manipulation cues.

3.5.2. Image Authenticity View

The image view takes the adapted MAE feature ${\tilde{\mathbf{f}}}_{mae}$, CLIP image feature ${\tilde{\mathbf{f}}}_{clip}^{img}$, tampering feature ${\tilde{\mathbf{f}}}_{tamp}$, and style feature ${\tilde{\mathbf{f}}}_{style}$, all in ${\mathbb{R}}^{512}$, together with the type feature ${\mathbf{f}}_{\mathrm{type}}\in {\mathbb{R}}^{128}$. Each branch is first transformed by an MLP to produce four 512-dimensional vectors ${\mathbf{h}}_{\mathrm{recon}}$, ${\mathbf{h}}_{\mathrm{sem}}$, ${\mathbf{h}}_{\mathrm{tamp}}$, and ${\mathbf{h}}_{\mathrm{style}}$. The type feature is passed through a small network to predict branch importance scores:

$$\alpha =\mathrm{softmax}\left({\mathbf{W}}_{\mathrm{type}}{\mathbf{f}}_{\mathrm{type}}\right)\in {\mathbb{R}}^4,$$

(18)

which modulate the four branches, ${\tilde{\mathbf{h}}}_b={\alpha }_b{\mathbf{h}}_b$ for $b\in \{\mathrm{recon},\mathrm{sem},\mathrm{tamp},\mathrm{style}\}$. The modulated features are concatenated and fused by another MLP, followed by a Transformer encoder layer, yielding an image representation ${\mathbf{h}}_{\mathrm{img}}\in {\mathbb{R}}^{512}$ that focuses on type-dependent visual manipulation cues. An MLP then produces the two-dimensional image evidence vector $e_{\mathrm{img}}\in {\mathbb{R}}^2$.

3.5.3. Cross-Modal Consistency View

The consistency view operates on global and entity-level CLIP features. Given global text/image embeddings ${\tilde{\mathbf{f}}}_{clip}^{txt},{\tilde{\mathbf{f}}}_{clip}^{img}\in {\mathbb{R}}^{512}$ and their entity sets ${\mathbf{E}}^{txt}\in {\mathbb{R}}^{K_t\times 512}$ and ${\mathbf{E}}^{img}\in {\mathbb{R}}^{K_i\times 512}$ with masks, where $K_t$ and $K_i$ denote the numbers of extracted textual and visual entities, respectively, we first compute their cosine similarity:

$$s_{\mathrm{clip}}=cos({\tilde{\mathbf{f}}}_{clip}^{txt},{\tilde{\mathbf{f}}}_{clip}^{img})\in [-1,1],$$

(19)

where $cos(\mathbf{a},\mathbf{b})={\mathbf{a}}^{\top }\mathbf{b}/\left({\parallel \mathbf{a}\parallel }_2{\parallel \mathbf{b}\parallel }_2\right)$ for two non-zero vectors. This score is later rescaled to $[0,1]$ as an interpretable global consistency score. Entity-level alignment is modeled by an eight-head attention module that matches text entities to image entities and averages attended representations into an entity-consistency vector ${\mathbf{h}}_{\mathrm{ent}}\in {\mathbb{R}}^{512}$. We then form

$$\mathbf{u}=[{\tilde{\mathbf{f}}}_{clip}^{txt};{\tilde{\mathbf{f}}}_{clip}^{img};s_{\mathrm{clip}}]\in {\mathbb{R}}^{1025},$$

(20)

project it to ${\mathbb{R}}^{512}$, add ${\mathbf{h}}_{\mathrm{ent}}$, and apply another Transformer encoder layer [48] to obtain ${\mathbf{h}}_{\mathrm{cons}}\in {\mathbb{R}}^{512}$. Concatenating ${\mathbf{h}}_{\mathrm{cons}}$ with the rescaled similarity $(s_{\mathrm{clip}}+1)/2$ and feeding them into a small MLP yields the two-dimensional consistency evidence $e_{\mathrm{cons}}\in {\mathbb{R}}^2$. When entity extraction is sparse or unreliable, the consistency branch can still fall back on global CLIP alignment, so entity matching acts as an auxiliary refinement rather than a mandatory dependency.

For notational simplicity, we still denote the three evidence vectors as follows:

$$E=[e_{\mathrm{text}};e_{\mathrm{img}};e_{\mathrm{cons}}].$$

(21)

3.6. Type-Guided Hierarchical Evidence Reasoning

Given the three evidence vectors and the disentangled type and semantic features, TERN performs type-guided hierarchical evidence reasoning in two steps.

3.6.1. Evidence Self-Attention

We first apply a self-attention layer over E to model interactions among the three views and obtain refined evidence representations $E^{\prime }$. This step allows information from one view (e.g., strong inconsistency evidence) to influence the representation of others.

3.6.2. Type-Guided Fusion

Next, we combine $E^{\prime }$ with ${\mathbf{f}}_{\mathrm{type}}$ and ${\mathbf{f}}_{\mathrm{sem}}$ using a type-guided fusion module, which outputs view-level weights ${\mathbf{w}}_{\mathrm{final}}$:

$${\mathbf{w}}_{\mathrm{final}}=\mathrm{TypeGuidedFusion}(E^{\prime },{\mathbf{f}}_{\mathrm{type}},{\mathbf{f}}_{\mathrm{sem}}).$$

(22)

These weights reflect the reliability and relevance of each evidence view for the current sample, conditioned on its latent type and semantic content. The final aggregated evidence vector is computed as follows:

$$e_{\mathrm{final}}=\sum _i{w}_{\mathrm{final},i}{E}_i^{\prime },$$

(23)

which is then fed into a classifier to produce the prediction:

$$\widehat{y}=\mathrm{Softmax}\left(\mathrm{MLP}\left(e_{\mathrm{final}}\right)\right).$$

(24)

This hierarchical, type-guided evidence reasoning enables TERN to emphasize different evidence sources for different deception types (e.g., focusing more on cross-modal consistency for text–image mismatch cases and more on visual authenticity for manipulation-heavy cases).

3.7. Training Objective and Algorithm Summary

TERN is trained end-to-end with a combination of classification, clustering, and disentanglement losses. The binary classification loss is the cross-entropy between predictions and ground-truth labels:

$${\mathcal{L}}_{\mathrm{cls}}=-{\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}\left[(1-y_i)log{\widehat{y}}_{i,\mathrm{real}}+y_{i}log{\widehat{y}}_{i,\mathrm{fake}}\right].$$

(25)

The total loss is:

$${\mathcal{L}}_{\mathrm{total}}={\mathcal{L}}_{\mathrm{cls}}+{\lambda }_c{\mathcal{L}}_{\mathrm{cluster}}+{\lambda }_d{\mathcal{L}}_{\mathrm{disentangle}},$$

(26)

where ${\lambda }_c$ and ${\lambda }_d$ control the contributions of the clustering and disentanglement objectives.

The objective in Equation (24) couples veracity prediction with latent type induction and type–semantic disentanglement. As a result, the learned type assignments are optimized not as independent clusters, but as structural variables that influence the downstream evidence-reasoning process.

Algorithm Summary

Algorithm 1 summarizes the overall training procedure.

Algorithm 1 TERN training algorithm

1: Input: Training set $\mathcal{D}={\left\{(x_i,y_i)\right\}}_{i=1}^N$ and hyperparameters $(\eta ,B,T,K,\tau ,\mu ,{\lambda }_c,{\lambda }_d)$ 2: Output: Trained TERN model parameters $\theta$ 3: Initialize network parameters $\theta$ and K prototype vectors ${\left\{{\mathbf{p}}_k\right\}}_{k=1}^K$ 4: $best\_f1\leftarrow 0$, $patience\leftarrow 0$ 5: for $epoch=1$ to T do 6:    for each mini-batch $(x_b,y_b)\in \mathcal{D}$ do 7:      Extract multi-view image and text features, adapt them to 512-d and compute fused feature ${\mathbf{f}}_{fused}$ 8:      Compute aggregated image feature ${\mathbf{f}}_{img}$ and type assignments $q_{ik}$, obtain ${\mathcal{L}}_{\mathrm{cluster}}$ and update prototypes with momentum 9:      Perform type–semantic disentanglement on ${\mathbf{f}}_{fused}$ to obtain ${\mathbf{f}}_{\mathrm{type}}$ and ${\mathbf{f}}_{\mathrm{sem}}$, compute ${\mathcal{L}}_{\mathrm{disentangle}}$ 10:      Generate three evidence views $e_{\mathrm{text}},e_{\mathrm{img}},e_{\mathrm{cons}}$ 11:      Apply evidence self-attention and type-guided fusion to obtain $e_{\mathrm{final}}$ and prediction $\widehat{y}$ 12:      Compute ${\mathcal{L}}_{\mathrm{cls}}$ and ${\mathcal{L}}_{\mathrm{total}}$, update $\theta$ via back-propagation 13:    end for 14:    Evaluate on validation set, update $best\_f1$ and $patience$ with early stopping and learning rate scheduling 15:    if $patience\ge 10$ then 16:      break 17:    end if 18: end for 19: Load the best checkpoint and return $\theta$

4. Experiments

This section presents a comprehensive empirical evaluation of TERN. The evaluation first introduces the datasets and experimental setup, and then compares TERN with strong multimodal baselines, followed by ablation studies, hyperparameter sensitivity analysis, statistical stability analysis, visualization, and case studies.

4.1. Datasets

We evaluate TERN on four public multimodal fake news or rumor detection benchmarks covering Chinese microblogs and English Twitter. MR2-Chinese and MR2-English are taken from MR2 [49]. Weibo follows the commonly used Sina Weibo rumor benchmark [50]. PHEME follows the standard rumor benchmark setting introduced in prior work [51]. In our experiments, we adopt the binary veracity mapping commonly used in subsequent multimodal detection studies [17]. For all datasets, we use the original or widely adopted train/validation/test splits from the corresponding benchmark settings.

Table 1. Dataset statistics.

Dataset	Train	Val	Test	Total	Language	K
MR2-Chinese	3289	337	357	3983	Chinese	3
MR2-English	2369	419	492	3280	English	4
Weibo	6247	777	781	7805	Chinese	3
PHEME	1412	202	404	2018	English	4

reports only the statistics that directly affect reproducibility: split sizes, language, and the operating number of latent deception types K used in TERN.

4.2. Experimental Setup

The experimental implementation follows the representation design introduced in Section 3. The visual branch uses complementary semantic, alignment, tampering, and style cues derived from MAE-Base, CLIP ViT-B/32, MobileNetV3-Large, and VGG19, respectively. The textual branch uses BERT-base semantic representations, CLIP text representations, and entity-level evidence. Image-side entities are obtained from salient regions proposed by Faster R-CNN [52] and are then used in the CLIP-based entity matching view. All feature encoders are kept fixed so that the empirical comparison focuses on the proposed type induction, disentanglement, evidence reasoning, and classification modules.

The main trainable hyperparameters follow the architecture in Section 3: the unified feature dimension is 512, the type branch dimension is 128, the semantic branch dimension is 384, and the emotion feature dimension is 128. We use AdamW [53] with weight decay ${10}^{-4}$, gradient clipping with max norm 1.0, and validation-F1-based model selection. The learning rate is searched from $\{{10}^{-5},5\times {10}^{-5},{10}^{-4},3\times {10}^{-4},5\times {10}^{-4},{10}^{-3},3\times {10}^{-3},5\times {10}^{-3}\}$, batch size from $\{16,32,64,128,256\}$, dropout from $\{0.1,0.2,0.3\}$, and disentanglement weight from $\{0.3,0.5\}$. The clustering weight is set to 0.1 unless otherwise specified, and training runs for at most 30 epochs with early stopping patience of 10.

We report accuracy (Acc), precision (Pre), recall (Rec), F1 score (F1), and the Matthews correlation coefficient (MCC). In the main comparison, ablation study, and stability analysis, precision, recall, and F1 score are computed for the fake-news class. Macro-F1 score is used only in the hyperparameter sensitivity analysis to summarize class-balanced selection trends. The main comparison, ablation, and hyperparameter-search experiments were conducted on a workstation with an NVIDIA GeForce RTX 5080 GPU (16 GB). The repeated-training stability analysis and inference-time profiling were conducted on a workstation with an NVIDIA GeForce RTX 5070 GPU (12 GB), and the corresponding hardware is reported with those measurements. Hyperparameter search and model selection use only the training and validation splits, and the test split is reserved for final evaluation. Feature extraction is conducted without using veracity labels.

4.3. Complexity Analysis

TERN combines multiple evidence sources and therefore has a higher representation cost than single-encoder detectors. Its feature construction involves BERT-base textual encoding, CLIP-based text–image alignment, MAE-Base visual semantics, MobileNetV3-Large tampering cues, VGG19 style cues, Faster R-CNN region proposals, and entity extraction. The subsequent trainable reasoning module contains approximately 20.08 M parameters and requires $0.442\pm 0.007$ ms per sample for forward inference on the RTX 5070 workstation with batch size 16. Across the four benchmarks, one training epoch takes approximately 2.5–12.8 s on the same profiling workstation, depending on dataset scale. These measurements indicate that the main computational burden lies in constructing richer multimodal evidence, while the type-aware reasoning component remains relatively compact.

4.4. Baseline Methods

To ensure representative comparisons, we select eight multimodal methods that cover early fusion, similarity-aware attention, entity-enhanced reasoning, contrastive learning, and multi-granularity fusion. Together, these baselines span the technical routes most relevant to TERN’s design.

• SpotFake [22]: concatenates text features from BERT and visual features from VGG-19 to form a joint multimodal representation for classification, representing early feature concatenation strategies.
• SAFE [25]: uses CNNs to extract text and image features and exploits similarity between modalities to adjust fusion weights, introducing a similarity-based dynamic fusion mechanism.
• BTIC [35]: employs BERT and ResNet to extract text and image-block features, and proposes an entity-enhanced multimodal fusion framework to improve detection performance.
• CAFE [5]: introduces cross-modal ambiguity learning to quantify semantic ambiguity between modalities and adaptively aggregates unimodal features and cross-modal relations to enhance detection.
• COOLANT [6]: adopts a cross-modal contrastive learning framework with attention-guided modules to adaptively weight unimodal and cross-modal features.
• TTEC [54]: combines data augmentation strategies with contrastive learning to construct diverse training samples and improve generalization.
• MMFN [30]: uses a multi-granularity fusion strategy to integrate cross-modal features from different layers for fake news detection.
• SR-CIBN [12]: proposes a semantic relationship-based consistency and inconsistency balancing network to enhance multimodal fake news detection by jointly modeling cross-modal consistency and inconsistency signals.

4.5. Main Results and Analysis

Table 2. Performance comparison with baseline methods. Precision, recall, and F1 score are computed for the fake-news class. Qwen2.5-7B-Instruct denotes a text-only LLM baseline. Bold numbers indicate the best performance on each dataset.

Method	MR2-Chinese					MR2-English
	Acc	Pre	Rec	F1	MCC	Acc	Pre	Rec	F1	MCC
SpotFake	82.24	81.02	71.17	75.78	62.83	82.09	80.02	74.17	76.98	62.13
SAFE	83.49	82.13	73.24	77.43	65.29	83.42	78.57	72.49	75.41	64.71
BTIC	87.39	85.51	83.96	84.73	73.73	84.49	80.49	76.82	78.61	66.78
CAFE	84.40	78.12	84.14	81.02	68.02	86.63	85.21	79.93	82.49	71.34
COOLANT	86.93	83.57	84.76	84.16	72.88	86.10	83.41	78.62	80.94	70.23
TTEC	87.61	87.41	85.26	86.32	74.23	86.90	83.34	80.81	82.06	72.02
MMFN	86.24	85.61	79.85	82.63	71.17	84.76	83.44	75.91	79.50	67.25
SR-CIBN	89.08	92.17	84.88	88.38	78.63	88.53	90.77	81.32	85.78	76.35
Qwen2.5-7B (LLM)	72.83	66.97	85.38	75.06	47.81	69.92	65.71	47.92	55.42	34.51
TERN	95.52	95.32	95.32	95.32	91.02	96.34	96.28	94.27	95.26	92.30
Method	Weibo					PHEME
	Acc	Pre	Rec	F1	MCC	Acc	Pre	Rec	F1	MCC
SpotFake	85.69	84.62	75.12	79.59	68.92	79.03	72.51	45.76	56.11	46.22
SAFE	83.82	85.72	93.73	89.51	67.41	79.57	70.22	37.96	49.28	47.90
BTIC	88.22	85.71	81.95	83.79	74.60	82.26	75.49	60.86	67.24	55.97
CAFE	88.04	86.77	80.01	83.25	74.12	81.72	72.68	66.41	69.40	54.74
COOLANT	89.86	85.65	87.32	86.47	78.37	81.63	70.59	67.25	68.88	55.74
TTEC	90.04	85.71	87.80	86.75	78.78	83.87	74.34	69.81	71.92	60.96
MMFN	89.42	79.94	89.15	84.27	77.28	82.80	72.54	65.83	69.02	58.72
SR-CIBN	91.61	93.92	89.79	91.85	83.63	85.04	80.34	62.46	70.30	61.19
Qwen2.5-7B (LLM)	69.17	67.89	92.52	78.31	33.66	72.03	61.90	11.02	18.71	16.84
TERN	93.60	95.20	92.97	94.08	87.14	87.38	75.19	84.75	79.68	70.84

reports the performance of TERN and baseline methods on the four datasets. In addition to conventional multimodal detectors, Qwen2.5-7B-Instruct is included as a text-only instruction-following LLM baseline, providing a language-model reference within the evaluated benchmarks.

Overall, TERN achieves the best or most balanced results across all four datasets, especially in F1 score and MCC. The LLM baseline performs competitively on Weibo in recall but remains weaker in F1 score and MCC, suggesting that language-only reasoning captures part of the veracity signal while leaving visual authenticity and cross-modal consistency underused. This pattern is particularly evident on MR2-English and PHEME, where event-specific visual context and imbalanced rumor distributions make text-only judgment less reliable. These results highlight the continued value of explicit multimodal evidence organization.

Compared with early fusion baselines such as SpotFake and similarity-aware attention models such as SAFE, TERN shows that stronger performance does not come only from richer encoders. The main gain comes from organizing the decision process around type-aware evidence fusion. By inducing latent deception types and using them to modulate evidence weights, TERN avoids treating text, image authenticity, and consistency cues as uniformly reliable across samples.

Compared with contrastive or multi-granularity baselines such as CAFE, COOLANT, TTEC, and MMFN, TERN further improves stability by separating type-related signals from semantic content. This helps the model avoid overfitting topic-specific shortcuts and strengthens performance on datasets with more diverse manipulation strategies.

The PHEME results further clarify the behavior of TERN under small-scale and imbalanced conditions. Several baselines achieve reasonable precision but weaker recall, implying conservative predictions for fake-rumor samples. TERN attains a higher F1 score and MCC on this dataset, highlighting the value of type-conditioned evidence reasoning when one evidence source is noisy, sparse, or event-specific.

4.6. Ablation Studies

To evaluate the contribution of each core component, we design four ablation settings by removing: (i) the type learning module (unsupervised clustering and disentanglement), (ii) the three-view evidence generation module, (iii) the type-guided hierarchical evidence reasoning module, and (iv) the entity feature module. All ablated models are trained with the same optimal hyperparameters as the full model to ensure fair comparison.

Table 3. Ablation results (F1 score, %).

Configuration	MR2-Chinese	MR2-English	Weibo	PHEME
Full model	95.32	95.26	94.08	79.68
w/o type learning	94.56	57.88	93.51	74.01
w/o three-view	91.62	93.87	92.34	72.81
w/o evidence fusion	88.77	86.53	93.61	77.69
w/o entity features	93.68	92.31	93.56	76.19

and

Table 4. Ablation results (accuracy, %).

Configuration	MR2-Chinese	MR2-English	Weibo	PHEME
Full model	95.52	96.34	93.60	87.38
w/o type learning	94.68	73.37	92.96	85.40
w/o three-view	91.60	95.33	91.68	84.65
w/o evidence fusion	88.24	90.45	92.96	86.63
w/o entity features	93.84	94.31	93.09	86.39

report ablation results in terms of F1 score and accuracy, respectively.

Table 3 and Table 4 reveal several observations. First, removing the type learning module leads to the largest average F1 score drop (11.10 percentage points), including a decrease of 37.38 percentage points on MR2-English. This demonstrates that for datasets with highly heterogeneous feature distributions, type discovery and type–semantic disentanglement make a substantial contribution to performance. Without this module, the model reverts to using a unified fusion strategy for all samples and cannot adapt its decisions to different deception patterns. Second, removing the hierarchical evidence reasoning module causes an average F1 score drop of 4.44 percentage points, showing that shallow, single-stage fusion is insufficient to handle conflicting evidence from textual, visual and consistency views. Multi-level, type-guided evidence reasoning helps suppress unreliable views and emphasize critical ones when evidence is inconsistent. Third, removing the three-view evidence generation module results in an average F1 score drop of 3.43 percentage points, with a larger decrease of 6.87 percentage points on PHEME. This highlights the importance of jointly modeling text semantics, image authenticity, and cross-modal consistency, especially for datasets with diverse deception strategies and complex context. Finally, removing the entity feature module leads to a smaller but non-negligible average F1 score drop of 2.15 percentage points, suggesting that entity features provide fine-grained clues for detecting mismatches at the entity level.

Overall, the contributions of different modules are consistent with their design focus: type learning primarily enhances modeling of heterogeneous samples, hierarchical evidence reasoning improves evidence integration under conflict, and three-view evidence plus entity features strengthen the model’s ability to capture complex semantics and entity-level inconsistencies. Together, they support the overall performance of TERN.

The PHEME result also helps clarify the role of entity cues. Removing entity features causes a drop from 79.68 to 76.19 in F1 score and from 87.38 to 86.39 in accuracy, showing that entity alignment is helpful but not indispensable. This is consistent with the design in Section 3: on entity-sparse or entity-noisy samples, TERN can still rely on textual semantics and global cross-modal consistency when explicit entity matching is weak.

4.7. Hyperparameter Sensitivity Analysis

We conduct a grid search with 240 configurations per dataset to systematically analyze the sensitivity of TERN to key hyperparameters. Validation performance is used only to select hyperparameters and checkpoints on the validation split; after this selection is fixed, the held-out test split is evaluated once to obtain the main results in Table 2. The following sensitivity tables report macro-F1 score, which is used here to compare hyperparameter trends without favoring either class.

Table 5. Learning rate sensitivity (macro-F1 score). Bold values indicate the best result in each column.

Learning Rate	MR2-Chinese	MR2-English	Weibo	PHEME
$1\times {10}^{-5}$	0.7335	0.9427	0.9292	0.8263
$5\times {10}^{-5}$	0.7934	0.9397	0.9356	0.8350
$1\times {10}^{-4}$	0.8907	0.9461	0.9342	0.8526
$3\times {10}^{-4}$	0.8935	0.9532	0.9178	0.7861
$5\times {10}^{-4}$	0.9048	0.9436	0.9106	0.7940
$1\times {10}^{-3}$	0.9551	0.9527	0.9007	0.7859
$3\times {10}^{-3}$	0.9412	0.9304	0.8872	0.7823
$5\times {10}^{-3}$	0.9299	0.9614	0.8953	0.7564

,

Table 6. Batch size sensitivity (macro-F1 score). Bold values indicate the best result in each column.

Batch Size	MR2-Chinese	MR2-English	Weibo	PHEME
16	0.9327	0.9461	0.9303	0.7895
32	0.9128	0.9614	0.9252	0.8264
64	0.9159	0.9465	0.9356	0.8526
128	0.9154	0.9311	0.9288	0.8332
256	0.9551	0.9331	0.9277	0.8411

,

Table 7. Dropout sensitivity (macro-F1 score). Bold values indicate the best result in each column.

Dropout	MR2-Chinese	MR2-English	Weibo	PHEME
0.1	0.9383	0.9614	0.9356	0.8332
0.2	0.9551	0.9510	0.9237	0.8526
0.3	0.9467	0.9307	0.9251	0.8298

and

Table 8. Disentanglement weight sensitivity (macro-F1 score). Bold values indicate the best result in each column.

Disentangle wt.	MR2-Chinese	MR2-English	Weibo	PHEME
0.3	0.8905	0.9305	0.9302	0.8526
0.5	0.9551	0.9614	0.9356	0.8337

reveal several trends. Learning rate is the most sensitive hyperparameter, with optimal values ranging from $5\times {10}^{-5}$ to $1\times {10}^{-3}$. Large-scale datasets such as Weibo prefer smaller learning rates ($5\times {10}^{-5}$) for stable convergence; overly large learning rates lead to instability. Datasets with clearer feature distributions (MR2-Chinese, MR2-English) can tolerate relatively larger learning rates (around $3\times {10}^{-4}$ to $1\times {10}^{-3}$), which help quickly converge to good optima. The optimal batch size varies from 32 to 256. Larger batches benefit the stability of prototype updates in the clustering module, as the type distribution in each batch is closer to the global distribution. Smaller batches, on the other hand, introduce gradient noise that helps escape local optima and discover more diverse type patterns. Dropout values in the 0.1–0.2 range work best: smaller datasets (MR2-Chinese, PHEME) benefit from stronger regularization (Dropout = 0.2) to avoid overfitting, while the larger Weibo dataset prefers lighter regularization (Dropout = 0.1). Regarding the disentanglement weight, most datasets perform better with a higher weight (0.5), confirming the importance of type–semantic disentanglement for model performance, whereas the smaller PHEME dataset prefers a lower weight (0.3), where overly strong regularization may cause underfitting.

In summary, hyperparameter selection should be adapted to dataset characteristics: large-scale datasets require smaller learning rates and lighter regularization, small-scale datasets require moderate learning rates and stronger regularization, and datasets with highly heterogeneous feature spaces benefit from clustering-aware tuning instead of one fixed configuration.

4.8. Statistical Stability Analysis

We further evaluate statistical stability through five independent retraining runs under fixed validation-selected settings on the RTX 5070 workstation. This analysis serves a different purpose from the main comparison: Table 2 reports the final test results of the checkpoint selected by validation performance for comparison with baselines, whereas

Table 9. Repeated-training stability of TERN under fixed validation-selected settings. Values are mean ± standard deviation over five independent runs, with 95% confidence intervals in parentheses.

Dataset	Acc	Pre	Rec	F1	MCC
MR2-Chinese	92.61 ± 1.64 (2.04)	90.99 ± 2.63 (3.26)	93.92 ± 0.98 (1.21)	92.42 ± 1.59 (1.98)	85.27 ± 3.19 (3.96)
MR2-English	94.76 ± 0.63 (0.79)	95.16 ± 1.67 (2.07)	91.25 ± 2.37 (2.94)	93.14 ± 0.89 (1.11)	88.98 ± 1.34 (1.67)
Weibo	93.19 ± 0.37 (0.46)	93.78 ± 0.73 (0.91)	93.77 ± 1.39 (1.73)	93.77 ± 0.39 (0.48)	86.28 ± 0.74 (0.91)
PHEME	83.96 ± 0.85 (1.05)	70.42 ± 2.77 (3.44)	78.31 ± 5.18 (6.43)	74.00 ± 1.50 (1.86)	62.80 ± 1.94 (2.41)

reports the average behavior of independently trained TERN models. The resulting mean, standard deviation, and confidence interval quantify the robustness of TERN to initialization and stochastic optimization.

Table 9 shows that the model remains stable on Weibo and reasonably stable on MR2-Chinese and MR2-English. The repeated-training means are not intended to replace the validation-selected results in Table 2; instead, they measure how much performance changes across independent optimizations of the same model. On MR2-English, TERN obtains a mean F1 of 93.14% with a 95% confidence interval of 1.11 percentage points. PHEME exhibits larger variance, especially for recall, which is consistent with its smaller scale and imbalanced event-driven label distribution. These results indicate that the performance gains are not solely driven by a single favorable run, while smaller rumor benchmarks remain more sensitive to optimization variance.

4.9. Latent Type Quality and Semantic Interpretability Analysis

The induced latent types are evaluated through both cluster-quality statistics and qualitative semantic interpretation. These types function as task-coupled structural priors rather than supervised deception categories. The operating K values are determined through cluster-validity inspection on training-set features, and the resulting groups are examined for internal structure and recurring evidence-use tendencies.

Table 10. Cluster-quality statistics for the operating latent type setting of each dataset.

Dataset	K	Silhouette	Calinski–Harabasz	Davies–Bouldin
MR2-Chinese	3	0.1199	348.47	2.5824
MR2-English	4	0.0491	95.09	4.0576
Weibo	3	0.0520	354.92	3.7149
PHEME	4	0.0563	67.93	3.5392

presents the silhouette score, Calinski–Harabasz score, and Davies–Bouldin score computed on the training-set features. The absolute values are moderate, which is consistent with the fact that fake news manipulation patterns do not form sharply separated semantic classes. Even so, the feature space shows non-trivial internal structure that can support downstream evidence fusion.

Table 11. Qualitative interpretation of induced latent deception types. The labels are post hoc semantic summaries rather than supervised ground-truth categories.

Dataset	Type	Semantic Interpretation	Dominant Evidence
MR2-Chinese	0	Event-consistent crisis reports	Consistency + image authenticity
MR2-Chinese	1	Emotionally framed textual manipulation	Text semantics
MR2-Chinese	2	Context-mismatched visual evidence	Cross-modal consistency
MR2-English	0	Authentic image with misleading narrative	Text semantics + consistency
MR2-English	1	Visual manipulation or synthetic-looking imagery	Image authenticity
MR2-English	2	Entity-level text-image mismatch	Entity consistency
MR2-English	3	Highly consistent real-news pattern	Consistency
Weibo	0	Platform-specific visual rumor template	Image authenticity + text
Weibo	1	Text-image mismatch	Consistency
Weibo	2	Authentic multimodal reports	Consistency
PHEME	0	Event-sparse rumor with weak visual evidence	Text semantics
PHEME	1	Entity-anchored factual update	Entity consistency
PHEME	2	Ambiguous breaking-news claim	Text + consistency
PHEME	3	Visually grounded misinformation	Image authenticity + consistency

summarizes representative post hoc interpretations obtained by inspecting high-confidence samples from each type together with their evidence-view weights. These interpretations provide semantic anchors for the clusters and show that the induced groups correspond to recurring evidence-use patterns rather than externally validated manipulation taxonomies. TERN uses these latent types to improve evidence allocation, not to replace human-defined misinformation categories.

4.10. Visualization Analysis

Figure 2 indicates that the predictions on MR2-Chinese and MR2-English are concentrated near the main diagonal, with relatively low false-positive and false-negative rates, indicating clear class separation on these two datasets. A similarly stable pattern is observed on Weibo, where both real and fake news achieve recognition rates above 93%. On PHEME, the recall of fake news is comparatively lower, which is consistent with the pronounced class imbalance in that dataset, yet the overall decision pattern remains stable.

Figure 3 further shows that real (blue) and fake (red) samples occupy distinguishable regions in the learned feature space across all datasets, indicating that TERN acquires discriminative multimodal representations. Several localized substructures are also visible, particularly on MR2-English, which is consistent with the latent-type modeling hypothesis. The Weibo representations appear comparatively compact, whereas PHEME exhibits a higher degree of class overlap, in line with its smaller scale and event-driven sparsity.

4.11. Case Studies

To examine the decision behavior of TERN at the instance level, two case studies are presented below. One real- and one fake-news sample assigned to different latent types are selected to illustrate the effect of type-aware fusion. Figure 4 compares these two cases.

4.11.1. Case 1 (Real News)

This is a real-news sample whose image shows an explosion site in Shenyang, China, with the text describing casualties. TERN correctly classifies it as real with 99.99% confidence and assigns it to Type 0 with probability 82.6%. The three evidence views support the real label with 45.0% (text), 51.7% (image), and 95.0% (consistency), and the fusion weights are 19.2% (text), 35.3% (image), and 45.5% (consistency). The consistency view detects strong alignment between text and image (consistency score 95.0%), and the type-guided fusion strategy assigns the largest weight to this view, reflecting a learned pattern for Type 0: cross-modal consistency is the most reliable evidence.

4.11.2. Case 2 (Fake News)

This is a fake-news sample whose image shows Indian banknotes and coins, while the text claims that the Reserve Bank of India has issued new currency. TERN correctly classifies it as fake with 99.80% confidence and assigns it to Type 2 with probability 83.1%. The three views support the fake label with 41.8% (text), 51.7% (image), and 63.0% (consistency), while the fusion weights are 45.0% (text), 0.5% (image), and 57.7% (consistency). The sample is assigned to Type 2, and the consistency view detects a semantic mismatch between text and image (consistency score 63.0%, much lower than 95.0% for the real case). The corresponding fusion profile places negligible weight on the image view (0.5%) and relies primarily on consistency (57.7%) and text (45.0%).

Comparing the fusion weights of the two cases reveals distinct evidence allocation patterns. For Type 0, the weights are 19.2% (text), 35.3% (image), and 45.5% (consistency), yielding a relatively balanced configuration. For Type 2, the weights are 45.0% (text), 0.5% (image), and 57.7% (consistency), yielding a markedly concentrated configuration with minimal reliance on visual evidence. This contrast illustrates how different latent types are associated with different evidence configurations under the type-guided fusion mechanism.

5. Conclusions

In this paper, we revisited multimodal fake news detection from the perspective of heterogeneous deception patterns and evidence reliability. We proposed TERN, a unified framework that combines latent deception type induction, type–semantic disentanglement, and type-conditioned hierarchical evidence reasoning. Instead of applying a single global fusion rule to all posts, TERN uses latent type information to organize how textual semantics, image authenticity, and cross-modal consistency should contribute to the final decision.

Experiments on MR2-Chinese, MR2-English, Weibo, and PHEME show that TERN improves upon the compared multimodal baselines, particularly in F1 score and MCC. The empirical analyses indicate that latent type modeling contributes substantially to the overall gains, that the prediction module remains computationally manageable after feature extraction, and that the induced types correspond to distinguishable evidence-use patterns.

TERN can serve as an evidence-ranking and decision-support component in human-in-the-loop moderation rather than as an autonomous arbiter of truth. From a theoretical perspective, this study shows that latent structural priors can help organize heterogeneous evidence when fine-grained deception-type annotations are unavailable. Several limitations remain. The discovered latent deception types do not have ground-truth semantic annotations; so, their semantic labels are post hoc interpretations. Cross-platform and cross-lingual transfer also remains challenging because datasets differ in language, event distribution, platform style, and entity availability. In addition, although the TERN prediction module is efficient, the overall framework is heavier than lightweight single-encoder baselines because it depends on multiple feature extractors. Practical deployment should further consider false positives, topic bias, uneven performance across languages and communities, uncertainty reporting, appeal mechanisms, and human review. Generative AI introduces additional threats such as deepfakes and synthetic images that may require specialized forensic signals beyond the current feature set. Future work will focus on stronger semantic grounding of latent types, broader transfer evaluation, lighter model variants, bias-aware auditing, and robustness against generative-AI-produced misinformation.