Attention-Level Causal Intervention Framework for Multimodal Fake News Detection

Abstract

Multimodal fake news detectors may learn biased dependencies from imbalanced event distributions and incidental text–image associations, causing attention to capture dataset-specific patterns rather than reliable discriminative evidence. To address this problem, we propose an Attention-level Causal Intervention Framework (ACIM), which performs causal adjustment directly within the attention learning process. Unlike prior causal debiasing methods operating at the feature-representation level, ACIM intervenes in attention distributions where cross-modal bias emerges. By treating attention representations as mediators, ACIM applies front-door causal intervention to mitigate confounding effects and estimate attention-level intervention effects without requiring fully observed confounders. This principle is implemented through a Causal Attention Layer Module (CALM), integrated into BERT-based textual and Swin Transformer-based visual encoders to jointly model in-sample and cross-sample attention. A causal-aware fusion layer further reconstructs cross-modal attention to suppress misleading text–image co-occurrence patterns. Experiments on Twitter and PHEME achieve accuracies of 0.906 and 0.909, improving upon the strongest reported accuracy baselines by 0.9 and 0.6 percentage points, respectively, while maintaining competitive precision, recall, and F1 performance. Ablation and sensitivity analyses further support the contribution and stability of the proposed approach.

1. Introduction

The rapid proliferation of social media platforms has fundamentally transformed how news is produced and consumed, enabling users to share information instantaneously through multimodal formats that combine text, images, and videos [1,2]. While this democratization of information has improved communication efficiency and accessibility, social media has also enabled misleading information to circulate rapidly and reach large audiences. The resulting consequences extend beyond individual misinformation, potentially affecting public sentiment, social trust, political discourse, and, in severe cases, broader societal stability [3]. In military and geopolitical contexts, multimodal misinformation has emerged as a strategic instrument of cognitive warfare: the deliberate fabrication and amplification of false narratives through social media has been observed in conflicts such as the Russia–Ukraine war, where misleading images and text were deployed to shape public perception and undermine decision-making [4]. Compared with traditional text-only misinformation, multimodal fake news combines richer semantic cues across modalities, making it substantially harder to detect and more effective at deceiving audiences [5]. These characteristics make reliable automated identification of multimodal fake news increasingly important.

Automated fake news detection has therefore become an increasingly critical research challenge. Existing multimodal detection methods have advanced significantly by integrating textual and visual representations, demonstrating that cross-modal evidence substantially improves detection accuracy over single-modal approaches [6]. However, despite these advances, two fundamental limitations remain unresolved.

The first limitation concerns spurious correlations induced by dataset bias. In real-world social media datasets, fake news tends to cluster around a small number of high-frequency events, producing long-tail distributions in which models are incentivized to overfit dominant co-occurrence patterns [7]. As a result, attention mechanisms learn to associate certain visual contexts or linguistic expressions with fake labels based on statistical regularity rather than genuine semantic evidence—a phenomenon known as shortcut learning [8]. For instance, if protest-related imagery is repeatedly paired with inflammatory descriptions in the training data, a model may learn to associate “crowd scenes” with “violent conflict” even when the depicted event is entirely peaceful. Such shortcut behaviors significantly degrade robustness under distribution shift and on unseen events.

The second limitation concerns partially observed and unobserved confounding factors in multimodal representation learning. Most existing fusion methods implicitly assume that statistical correlations between modalities correspond to meaningful semantic relationships [9]. In practice, however, these correlations may be influenced by confounding factors—such as event-specific background patterns, source-platform styles, or template-based compositions—that simultaneously affect both the input features and the classification labels. Although some of these factors may have observable proxies, such as event identifiers, platform information, or stylistic attributes, these proxies cannot be guaranteed to constitute a complete and sufficient adjustment set. Therefore, directly applying back-door adjustment may leave residual confounding effects. When such confounding effects are not adequately controlled, attention weights may be drawn toward superficial co-occurrence patterns rather than causally relevant evidence. Meanwhile, recent attention-based multimodal methods improve fine-grained text–image interaction through co-attention, cross-attention, and cross-modal alignment [10,11,12], while causal debiasing methods attempt to suppress confounding or modality-specific biases through intervention, counterfactual reasoning, or invariant representation learning [13,14,15]. However, these two research directions have largely evolved separately: attention-based methods primarily optimize cross-modal interactions, whereas existing causal methods mainly intervene on learned feature representations rather than the attention distributions through which these interactions are formed. Existing debiasing strategies, including adversarial learning, invariant risk minimization, and contrastive learning, partially address this problem, but they typically rely on predefined assumptions about bias distributions or require explicit domain annotations, limiting their applicability in noisy and open-ended social media environments [13,16]. More importantly, none of these approaches directly intervenes at the attention level, despite attention distributions being the core mechanism through which cross-modal semantic interactions are mediated and aggregated.

From a causal perspective, these limitations highlight the need to estimate the effect of multimodal evidence on model predictions while mitigating the influence of partially observed and unobserved confounding factors.

In attention-based multimodal learning, attention distributions serve as intermediate representations through which cross-modal evidence is selected and aggregated, making them suitable mediators for front-door intervention within attention-dependent prediction pathways. Unlike correlation-driven fusion, causal intervention explicitly separates genuine semantic dependencies from spurious associations, providing a theoretically grounded basis for debiasing.

Accordingly, this study develops an Attention-level Causal Intervention Framework for multimodal fake news detection, aiming to reduce the effects of confounding factors in textual, visual, and cross-modal attention pathways and limit the model’s dependence on spurious patterns arising from biased and long-tailed data distributions. To achieve this objective, this paper proposes an Attention-level Causal Intervention Framework (ACIM) for multimodal fake news detection. ACIM introduces front-door causal intervention directly into the attention learning process to mitigate partially observed and unobserved confounding effects and suppress spurious cross-modal correlations. A novel Causal Attention Layer Module (CALM) is developed to jointly model in-sample and cross-sample attention distributions under the front-door adjustment framework: the in-sample branch preserves local semantic dependencies, while the cross-sample branch introduces global dictionary representations to approximate intervention distributions and counteract biased co-occurrence patterns. CALM is embedded into both BERT-based textual encoders and Swin Transformer-based visual encoders to perform causal debiasing at the feature extraction stage. A causal-aware multimodal fusion layer further reconstructs cross-modal attention distributions to enhance robustness under long-tail and noisy social media conditions. Section 3 describes the causal formulation and implementation of ACIM in detail. Evaluation on the Twitter and PHEME benchmarks demonstrates consistent performance gains in accuracy, together with competitive precision, recall, and F1 scores. The contributions of the proposed attention-level intervention components are further examined through ablation and sensitivity experiments. The main contributions of this paper are as follows:

• We formulate multimodal fake news detection from an attention-level causal perspective, treating attention representations as mediators within attention-dependent prediction pathways. This formulation provides a basis for applying front-door intervention to mitigate the influence of partially observed and unobserved confounding factors;
• We propose an Attention-level Causal Intervention Framework (ACIM) with a Causal Attention Layer Module (CALM). CALM jointly models in-sample and cross-sample attention through ISA and CSA and is integrated into textual encoding, visual encoding, and cross-modal fusion to reduce sample-specific spurious correlations throughout multimodal representation learning;
• We instantiate ACIM using BERT and Swin Transformer encoders and conduct extensive experiments on the Twitter and PHEME datasets, including baseline comparisons, ablation studies, sensitivity analyses, and case studies. The experimental findings provide empirical support for the performance and stability of the proposed framework.

2. Related Works

Early fake news detection methods primarily operated on a single modality. In the text domain, initial approaches relied on handcrafted linguistic features—including writing style, emotional tone, punctuation patterns, and keyword distributions—combined with shallow classifiers such as Support Vector Machines and logistic regression [17]. Although effective in constrained scenarios, these methods depended heavily on manual feature engineering and could not generalize across topics or platforms. As deep learning advanced, researchers shifted toward end-to-end representation learning. Ma et al. [18] employed recurrent neural networks (RNNs) to model the temporal evolution of news text, capturing sequential dependencies that static features could not represent. Yu et al. [19] further introduced convolutional neural networks (CNNs) to capture local n-gram patterns, while Kaliyar et al. [20] proposed FakeBERT, which combined BERT’s contextual language modeling with CNN-based local feature extraction, achieving strong robustness on short social media text. In the visual domain, researchers exploited frequency-domain artifacts and pixel-level inconsistencies to detect manipulated images [21]. Qi et al. [22] developed a multi-view neural network integrating frequency features with pixel-level cues, significantly improving image forgery detection accuracy. Despite these advances, unimodal methods are fundamentally limited by their inability to exploit cross-modal complementarity, particularly for fake news that achieves its deceptive effect through inconsistencies between text and image.

The recognition that multimodal content offers richer semantic cues motivated a shift toward joint text–image modeling. Jin et al. [23] were among the first to exploit news images for fake news detection, designing a visual-attention mechanism that selectively weighted image regions based on textual context. Building on this, they subsequently proposed an RNN-based multimodal fusion framework that integrated visual features with user social signals [24]. Khattar et al. [6] introduced MVAE, a multimodal variational autoencoder that encoded text and images into a shared latent space, leveraging cross-modal reconstruction to enforce semantic consistency. Wang et al. [25] proposed EANN, which used event adversarial training to extract event-invariant representations, improving generalization to unseen events. Zhou et al. [26] introduced SAFE, which measured the semantic similarity between news text and paired images via cosine distance, operating on the hypothesis that authentic news exhibits higher text–image agreement than fabricated content.

More recent work has shifted from global feature concatenation toward fine-grained cross-modal interaction. Hu et al. [10] proposed MMCAN, a co-attention network that explicitly models word-region correspondences between text tokens and image patches, combined with mutual knowledge distillation to enhance inter-modal alignment. Jing et al. [27] proposed a progressive fusion network (MPFN) that integrates shallow and deep multimodal features through an MLP-Mixer fusion module combined with a Swin Transformer visual encoder, mitigating information loss caused by purely deep-feature fusion. Zhang et al. [28] integrated a cross-modal alignment module into the Transformer architecture, aligning textual and visual semantic spaces to improve detection accuracy and robustness. Monti et al. [29] embedded multimodal features into social relationship graphs and applied graph convolutional networks (GCNs) to exploit both content semantics and propagation structure simultaneously. Comprehensive surveys by Alam et al. [30] and Comito et al. [31] have systematically reviewed these developments, identifying modal information redundancy and bias propagation as key unresolved challenges in current multimodal detection pipelines. A more recent systematic review by Nasser et al. [32] further summarizes deep-learning-based multimodal fake news detection involving textual, visual, audio, video, and social-context information, and identifies robustness, multilinguality, explainability, and real-time detection as continuing research challenges.

More recently, Kou et al. [33] proposed a Potential Features Fusion Network (PFFN) that simultaneously models explicit features such as text and image alongside potential features such as textual emotion and image category through a mixture-of-experts structure, achieving competitive performance on Weibo and Twitter benchmarks. Shen et al. [34] proposed GAMED, a knowledge-adaptive multi-experts decoupling framework presented at WSDM 2025, which generates discriminative features through modal decoupling and integrates external knowledge to enhance cross-modal synergies, demonstrating strong interpretability and performance on the Fakeddit dataset. At the event level, Ma et al. [35] proposed Event-Radar, which integrates visual manipulation, textual emotion, and multimodal inconsistency through event-driven multi-view learning. Zhang et al. [36] developed MCAN to jointly capture semantic and physical consistency features and enhance multimodal interactions through a cross-aware fusion mechanism. Recent attention-based studies have further focused on fine-grained cross-modal interaction. Saioni and Giannone [11] employed cross-attention to explicitly model intermodal relationships for multimodal fake news classification. Wu et al. [12] proposed a cross-modal prompt learning framework to enhance early-stage modality interaction and domain robustness, while Qian et al. [37] introduced a dual contrastive learning framework with gated cross-attention to strengthen cross-modal alignment and fine-grained multimodal fusion.

The advent of large vision–language pretrained models has opened new directions for multimodal fake news detection. CLIP-based methods [38] exploit contrastive image–text pretraining to align visual and linguistic representations in a shared embedding space, reducing modality gaps. BLIP [39] further extends this by incorporating generative bootstrapping to improve cross-modal grounding. Despite the strong representational capabilities of these foundation models, recent studies have shown that they remain vulnerable to spurious cross-modal correlations and shortcut learning in misinformation detection scenarios [29], motivating the need for explicitly causal debiasing mechanisms. Recent studies have further explored task-specific multimodal pre-training and large-model-assisted reasoning. Zhang et al. [40] proposed KAMP, a knowledge-aware multimodal pre-training framework that jointly models unimodal semantics, cross-modal alignment, and background knowledge. Zheng et al. [41] introduced adversarial arguments generated by multimodal large language models to enhance cross-modal reasoning, while Hu et al. [42] combined LLM-generated pseudo-labels with global label propagation for multimodal fake news detection.

Causal inference has emerged as a principled framework for addressing the limitations of correlation-driven models. Pearl’s structural causal model (SCM) and do-calculus provide formal tools for estimating causal effects, distinguishing genuine semantic dependencies from spurious associations induced by confounding variables [9]. In machine learning, Peters et al. [43] formalized invariant prediction as a criterion for identifying causal features stable across environments, while Arjovsky et al. [44] translated this into the invariant risk minimization (IRM) objective, enabling models to learn representations that generalize beyond training distributions.

In natural language processing, Kaushik et al. [45] proposed counterfactually augmented data (CAD), which generates minimally edited contrastive samples to suppress non-causal lexical shortcuts. In multimodal tasks, Niu et al. [46] applied counterfactual reasoning to visual question answering, removing language priors by contrasting factual and counterfactual question representations. Tang et al. [47] introduced total direct effect (TDE) estimation for scene graph generation, mitigating long-tail prediction biases by explicitly separating direct causal effects from confounded pathways. Yang et al. [48] applied deconfounding techniques to image captioning, improving cross-domain generation robustness.

In the fake news detection domain, causal approaches have gained increasing traction. Hu et al. [13] specifically addressed image–text matching bias using causal inference, separating genuine semantic consistency from spurious co-occurrence correlations in multimodal fusion. Chen et al. [14] proposed the CCD framework, utilizing front-door causal intervention to remove psycholinguistic bias in text features while applying counterfactual reasoning to eliminate image-only bias, achieving consistent improvements on two benchmark datasets. Cheng et al. [49] modeled the causal mechanisms of fake news propagation across social networks, constructing structural priors from network topology, agent behavior, and content signals. Liu et al. [16] proposed a dual adversarial learning framework for out-of-distribution fake news detection, improving robustness under domain shift. Sun et al. [50] designed a knowledge-guided dual-consistency network for multimodal rumor detection, jointly modeling cross-modal semantic inconsistency and background knowledge inconsistency. Related causal debiasing strategies have also been explored in neighboring multimodal misinformation tasks. Yu et al. [15] proposed a causal representation inference approach for multimodal clickbait detection, disentangling invariant, causal, and non-causal factors to reduce spurious co-occurrence bias.

These methods intervene at different stages of the misinformation detection pipeline: Hu et al. [13] focus on deconfounding image–text matching representations, Chen et al. [14] perform causal intervention on psycholinguistic text features together with counterfactual correction of image-only bias, and Yu et al. [15] disentangle causal and non-causal factors at the representation level.

Despite these efforts, existing causal debiasing methods for multimodal fake news detection mainly operate at the feature level, leaving attention-level confounding insufficiently addressed. Since attention governs cross-modal interactions in Transformer-based architectures, biased attention weights may amplify spurious co-occurrence patterns. To address this gap, ACIM introduces front-door causal intervention directly into attention learning to mitigate confounded cross-modal dependencies.

3. Method

Attention mechanisms are indispensable components of modern visual and language models [9,51,52,53,54,55]. Following the general Query–Key–Value (Q-K-V) formulation [56], a query $q$ is associated with each value $v_i$ through normalized attention weights ${\alpha }_i=\mathrm{softmax}(q^T{k}_i)$, where $k_i$ and $v_i$ denote the key and value vectors, respectively; the output is their weighted sum:

$$\mathrm{Attention}(q,K,V)={\displaystyle \sum _i{\alpha }_i}{v}_i$$

(1)

Because attention is the bridge through which cross-modal evidence is selected and aggregated, biased attention weights propagate directly into predictions, motivating intervention at the attention level rather than the feature level.

From a causal perspective, the fundamental challenge in multimodal fake news detection is estimating the attention-level intervention effect between multimodal evidence and prediction outcomes while eliminating the influence of hidden confounders [9]. As shown in Figure 1a, the basic causal graph assumes a direct effect from multimodal input $X$ to prediction $Y$. However, in real-world social media data, a hidden confounder $Z$—such as event-specific background patterns or platform-level stylistic conventions—simultaneously influences both $X$ and $Y$, as illustrated in Figure 1b. This backdoor path $Z\to X\to Y$ introduces spurious correlations that corrupt standard attention learning, causing models to overfit co-occurrence patterns rather than genuine semantic dependencies. Front-door causal intervention theory offers a principled solution: by introducing an observable mediating variable $M$ that blocks the confounded pathway, it enables causal effect estimation from purely observational data without requiring $Z$ to be directly observed [57]. As shown in Figure 1c, causal intervention blocks the backdoor path $Z\to X$—represented by the red cross—and estimates the attention-level intervention effect of $X$ on $Y$ through the front-door path $X\to M\to Y$.

However, the distribution of attention weights is frequently distorted by biases in the training data [58]. In multimodal fake news detection—for instance, during joint modeling of images and text—models may amplify certain modalities or regions due to skewed data distributions. For example, frequent pairing of “protest crowds” with “police conflict” in training data may cause the model to spuriously associate protest crowds with conflict and misclassify a “protest crowd—peaceful assembly” instance as text–image mismatched fake news. Building on the confounded causal graph in Figure 1b, we further derive the attention-level causal graph for multimodal fake news detection, as illustrated in Figure 2.

Increasing the dataset size alone is insufficient to remove such biases, because many biases reflect the intrinsic distributional properties of the data—such as Zipf’s Law [59] and social conventions [60]. For example, in news data, “government announcement—policy adjustment” pairings genuinely outnumber “government announcement—holiday celebration” pairings (as illustrated in Figure 1b). Consequently, relying solely on large-scale pretraining may even exacerbate attention bias. Figure 3 presents three real news items that were incorrectly classified by the model as text–image mismatched fake news, demonstrating how pretrained attention mechanisms can encode and amplify distributional biases.

At the implementation level, when both $q$ and $V$ originate from the same input $X$ (e.g., image region features), the mechanism is referred to as self-attention; when Q originates from a different modality (e.g., textual context), it is referred to as top-down attention. Intuitively, self-attention functions as a non-local [61] or graph convolutional network [62] that enriches each local representation using global relational features, whereas top-down attention captures cross-modal semantic interactions. The two can be combined and stacked into deep structures to form powerful multimodal encoder–decoder Transformer networks [53,60,63,64].

As the bridge connecting input features $X$ and output label $Y$, the quality of attention weights $\alpha$ directly governs downstream task performance. However, in the absence of explicit supervision, attention learning is susceptible to data bias, leading to erroneous decisions in fake news detection. We therefore introduce causal intervention to mitigate attention bias through structured modeling and improve model robustness.

Dataset bias can arise from confounding variables that simultaneously affect X and Y, thereby creating statistical dependence between them even when no direct causal link exists, as illustrated in Figure 1b. To better understand how this confounding effect manifests in the attention mechanism, we extend the causal graph in Figure 2 to the attention level, as shown in Figure 4. Let the confounder $Z$ represent the common sense prior “coffee cups are typically found on tables”. Then $Z\to X$ indicates that this prior leads dataset curators to collect a large number of images containing coffee cups on tables. Meanwhile, $Z\to Y$ indicates that the same prior may directly bias the model toward predictions associated with “a coffee cup on the table.” Here, the abstract mediator $M$ introduced above is instantiated as a concrete detected object set $H$. $X\to H$ indicates that an object set $H=\mathrm{coffee},\mathrm{cup},\mathrm{table}$ can be detected from the image (e.g., using Faster R-CNN); and $H\to Y$ indicates that the model generates descriptions such as “a coffee cup on the table” based on the detected objects.

Beyond the legitimate causal path $X\to H\to Y$, however, there also exists a backdoor path $X\leftarrow Z\to Y$ that additionally influences the distribution of $Y$. Consequently, learning predictions directly from the observational distribution $P(Y|X)$ may capture associations between $X$ and $Y$ that are partly induced by confounders, rather than recovering the attention-dependent causal effect of $X$ on $Y$ regardless of the training data volume [65,66]. More critically, if the distribution of the confounder differs between training and testing—for instance, “coffee cup on a table” scenarios are far more frequent than “coffee cup on the floor” in training, while the reverse holds at test time—$P(Y|X)$ the model learned during training will fail at inference [67].

ACIM is built on the theoretical foundation of the front-door adjustment [9], which does not require a fully observed confounder set but relies on the corresponding front-door identification assumptions. This formulation allows front-door intervention to be adapted to attention-dependent prediction pathways. This stands in sharp contrast to deconfounding methods based on back-door adjustment [68,69], which typically rely on domain-specific prior knowledge and suffer from limited applicability. ACIM further combines In-Sample Attention (ISA) and Cross-Sample Attention (CSA), both of which follow the standard Q-K-V operational form; in certain network architectures, the parameters of these operations can be shared between ISA and CSA to further improve computational efficiency.

3.1. Overview

ACIM takes news text and its accompanying image as input and performs fake news detection through unified representation learning and causal intervention modeling. The model first encodes textual and visual inputs separately to obtain their respective semantic representations, and then introduces attention mechanisms during cross-modal interaction to model text–image dependencies. Building on this, ACIM adopts a causal inference perspective by treating attention distributions as key mediating variables. Front-door causal intervention is introduced into the multimodal attention learning process. ISA computes in-sample attention from the current input, whereas CSA computes cross-sample attention using a global key–value dictionary of K-means training-set prototypes. The two branch outputs are integrated and passed to subsequent layers, incorporating cross-sample information to reduce sample-specific spurious correlations. The resulting multimodal representation is then fed into a classifier to produce the fake news detection result. The overall framework is illustrated in Figure 5.

3.2. Causal Attention Layer Module (CALM)

Based on the front-door causal intervention framework derived in the preliminary section, we design the Causal Attention Layer Module (CALM) for attention-level causal debiasing.

In multimodal fake news detection, we place the attention mechanism within the front-door causal graph framework. As shown in Figure 6, causal effects are transmitted from the input set $X$ to the target $Y$ through mediating variables $M$. Under this causal graph, the attention mechanism can be decomposed into two components:

• Selector: selects appropriate mediating variables $M$ from $X$;
• Classifier: utilizes $M$ to predict $Y$.

In this formulation, $M$ denotes the intermediate attention representation within the attention-dependent pathway from $X$ to $Y$. Other components, such as residual connections, positional encodings, and feed-forward layers, may form additional computational pathways. Therefore, the causal graph is regarded as a local abstraction of the attention mechanism rather than a complete causal description of the entire network.

In the multimodal setting, $X$ typically consists of news images and corresponding text. The attention mechanism selects relevant image regions as mediating variables $M$ based on the textual context, which are then used for subsequent prediction.

Conventional methods directly use the observed correlation $P(Y|X)$ as the training objective:

$$P(Y|X)=\underset{\text{IS-Sampling}}{\underbrace{{\displaystyle \sum _{M}P}(M|X)}}P(Y|M)$$

(2)

where $P(M|X)$ characterizes the in-sample selection process, with $M$ obtained from the representation of the current input $X$.

However, because the selection process is unsupervised, the classifier may be influenced by dataset bias. From a causal perspective, the backdoor path $M\leftarrow X\leftarrow Z\to Y$ may cause the model to exploit confounder-induced associations instead of capturing the intervention effect of $M$ on $Y$ at the attention level.

To reduce spurious correlations caused by the hidden confounder $Z$, it is necessary to remove the influence of the backdoor path between $Z$ and $X$ and evaluate the attention-level intervention effect of $X$ on $Y$, denoted as $P(Y|do(M))$, where $do(\cdot )$ represents the causal intervention operator. This is achieved by stratifying the input variable $X$, breaking the link $X\to M$, and calculating the average causal effect of $M$ on $Y$ via the following formula:

$$P(Y|do(M))=\underset{\text{CS-Sampling}}{\underbrace{{\displaystyle \sum _{x}P}(X=x)}}P(Y|X=x,M)$$

(3)

Here, $P\left(X=x\right)$ represents the marginal distribution of the input variable X in the theoretical front-door formulation. In the original formulation, this term requires marginalization over possible input realizations. In a neural implementation, however, explicitly enumerating all possible values of X is computationally infeasible. Therefore, the subsequent cross-sample attention mechanism is introduced as a tractable approximation to this marginalization rather than as an exact representation of $P\left(X=x\right)$.

Since $x$ here is drawn from other samples, this process is referred to as cross-sample sampling. Intuitively, it provides a computational approximation to the intervention by weakening the sample-specific association between the input and the mediating variable, thereby encouraging the model to rely on evidence that is less dependent on spurious co-occurrence patterns. For example, in news data, if a large number of training instances contain both “flood scenes” and “casualty descriptions”, the model may learn the spurious pattern “flood image → casualties”. Cross-sample sampling recombines “flood images” with other textual descriptions, reducing the model’s dependence on this spurious association and encouraging the predictor to focus on causally relevant factors.

Finally, by replacing $P(Y|M)$ in the original formula with $P(Y|do(M))$, we obtain the attention-level intervention effect:

$$P(Y|do(X))=\underset{\text{IS-Sampling}}{\underbrace{{\displaystyle \sum _{M}P}(M|X)}}\underset{\text{CS-Sampling}}{\underbrace{{\displaystyle \sum _{x}P}(X=x)}}P(Y|M,X=x)$$

(4)

Equation (4) provides the theoretical front-door formulation for the attention-level intervention. In the proposed neural implementation, however, the two summations are not evaluated explicitly over all possible mediator states and input realizations. Instead, as described below, they are approximated by the ISA and CSA branches, respectively. To implement front-door causal intervention within a deep model, we model the prediction distribution $P(Y|M,X)$ using a neural network $g(\cdot )$ followed by a Softmax layer, as multimodal detection is typically formulated as a classification task [6,25,70,71,72,73,74]:

$$P(Y|M,X)=\mathrm{Softmax}(g(M,X))$$

(5)

In Equation (5), we need to sample both $X$ and $M$ and feed them into the network to compute $P(Y|do(X))$. However, performing full forward passes over all samples is computationally prohibitive. We therefore adopt the Normalized Weighted Geometric Mean (NWGM) [75] approximation to absorb the outer sampling into the feature layer, so that the network only needs to perform a single forward computation on the absorbed input. Specifically, under the NWGM approximation, the IS-Sampling and CS-Sampling terms in Equation (4) are approximated as follows:

$$P(Y|do(X))\approx \mathrm{Softmax}(g(\widehat{M},\widehat{X}))$$

(6)

where

$$\widehat{M}={\displaystyle \sum _{M}P}(M|h(X))M,\hspace{1em}\widehat{X}={\displaystyle \sum _{x}P}(X=x|f(X))x$$

(7)

It is important to distinguish the theoretical marginal distribution $P\left(X=x\right)$ in Equation (4) from its neural approximation in Equation (7). Following CATT [9], cluster frequencies are not explicitly used to estimate P(X = x). Instead, CSA employs the query-conditioned weighting $P(X=x|f(X))$, because directly using the marginal prior would yield the same aggregated representation ${\displaystyle \sum _{x}P}(x)x$ for different inputs. The query-conditioned weighting therefore provides an input-dependent approximation to the cross-sample marginalization while preserving the representation capacity of the attention mechanism.

Here $h(\cdot )$ and $f(\cdot )$ are query embedding functions that map the input $X$ into different query sets, both of which can be parameterized by neural networks. In the model, $X$ and $M$ are both represented as embedding vectors—for instance, image regions are converted into Region of Interest (RoI) representations—and are therefore denoted in bold.

It should be noted that NWGM provides a computational approximation rather than an exact equality. Its accuracy depends on how well the weighted feature representations preserve the expectation of the prediction function after nonlinear transformation. When the sampled representations are highly heterogeneous, exhibit a strongly multi-peaked distribution, or are associated with overly concentrated attention weights, the approximation error may increase. Therefore, Equations (6) and (7) should be interpreted as providing a tractable approximation of the attention-level intervention effect rather than an exact causal estimate.

Under this framework, $\widehat{M}$ corresponds to the estimate from In-Sample Sampling (IS-Sampling), which is exactly the computation performed in classical attention networks and can be expressed via Q-K-V operations as:

$$\begin{array}{l}\mathrm{Input}: Q_I,K_I,V_I\hspace{1em}\\ \mathrm{Prob}: A_I=\mathrm{Softmax}(Q_I^{\top }{K}_I)\hspace{1em}\\ \mathrm{Output}: \widehat{M}=V_I{A}_I\end{array}$$

(8)

We term this process In-Sample Attention (ISA), with the subscript I indicating its role in estimating IS-Sampling. In this case, $K_I$ and $V_I$ are both obtained from the current input, such as RoI feature sets, while $Q_I$ is derived from $h(X)$—for instance, in top-down attention, $q_I$ represents the textual-context embedding; in self-attention, $Q_I$ is also the RoI feature set.

Analogously, we design Cross-Sample Attention (CSA) to estimate $\widehat{X}$, which takes the form:

$$\begin{array}{l}\mathrm{Input}: Q_C,K_C,V_C\hspace{1em}\\ \mathrm{Prob}: A_C=\mathrm{Softmax}(Q_C^{\top }{K}_C)\hspace{1em}\\ \mathrm{Output}: \widehat{X}=V_C{A}_C\end{array}$$

(9)

where $K_C$ and $V_C$ are obtained from different training samples, and $Q_C$ is derived from $f(X)$. In practice, $K_C$ and $V_C$ are represented by a global dictionary summarized from the training set, since explicitly attending over all samples is computationally infeasible. Each prototype in the dictionary represents a cluster-level summary of multiple training representations rather than an individual realization of the input variable $X$. Thus, the prototype dictionary provides a compact representation of the empirical support of $X$ for cross-sample aggregation. Only training-set embeddings are used for dictionary construction to avoid information leakage from the validation and test sets. This dictionary is constructed by applying K-means clustering to the embeddings of the training samples, ensuring that $V_C$ and $V_I$ share a common representation space, so that $\widehat{M}$ and $\widehat{X}$ are distributionally consistent. The dictionary size was empirically set to $K=64$ based on sensitivity analysis. The global dictionary was initialized before training and remained fixed throughout model optimization. This fixed-dictionary design may introduce representation-space mismatch as the BERT and Swin encoders are updated during training, which constitutes a limitation of the current implementation. Because the global dictionary is constructed from training data, it may still inherit systematic biases from the training distribution. Therefore, the dictionary-based intervention is intended to mitigate, rather than completely eliminate, dataset-level bias. The outputs of the ISA and CSA branches are then concatenated and jointly used to approximate the attention-level intervention effect $P(Y|do(X))$.

It should be emphasized that the causal role of CSA is derived from its role in the front-door adjustment rather than from the cross-sample attention operation alone. Specifically, CSA is designed to approximate the cross-sample marginalization over $P(X=x)$ required by the front-door formulation. By introducing representations beyond the current sample through the global dictionary, CSA provides a tractable implementation of the corresponding cross-sample sampling process. Therefore, although its computational form resembles prototype-based cross-sample attention, its role in CALM is explicitly motivated by the front-door formulation rather than being introduced merely as a regularization mechanism.

3.3. Causal Intervention in Transformer Encoders

3.3.1. Textual Feature Extraction

Extracting informative textual representations is essential for multimodal fake news detection. We integrate the proposed CALM module into the first attention layer of BERT to introduce causal constraints at the feature extraction stage, reducing spurious correlations caused by dataset bias. The architecture is illustrated in Figure 7.

Specifically, BERT receives the tokenized news text $T$ as input. After word embedding and positional encoding, the sequence is fed into multi-layer Transformer attention modules. We insert ISA and CSA sub-modules into the first attention layer to estimate the in-sample causal effect $\widehat{M}$ and the cross-sample causal effect $\widehat{X}$ respectively. The computational complexity of CALM is $O(K\times n)$ at the first attention layer and $O(n\times n)$ at subsequent layers, where $K$ is the global dictionary size and $n$ is the sequence length of text or image tokens. By absorbing the sampling operation into the feature representation, only a single forward pass is required to obtain a tractable approximation of the attention-level intervention effect.

The two branch outputs are concatenated and then passed through the remaining Transformer layers for further representation learning, ensuring that the resulting text features carry not only semantic information but also causal robustness. Unlike the original BERT architecture, CALM shares parameters between the ISA and CSA branches at this stage to maintain consistent feature distributions across in-sample and cross-sample paths, avoiding representation space inconsistencies caused by independent optimization. The formulation is as follows:

$$H_l=\mathrm{BERT}\_\mathrm{ACIM}\left(T\right)$$

(10)

3.3.2. Visual Feature Extraction

For visual feature extraction, conventional CNN-based architectures such as VGG and ResNet have inherent limitations in global modeling capacity and cannot adequately capture long-range dependencies within images. We therefore adopt Swin Transformer as the visual encoder. Swin Transformer initially divides each input image into fixed-size patches, which are projected into patch embeddings. These embeddings are subsequently processed by stacked Shifted Window Multi-Head Self-Attention (SW-MSA) and feed-forward network (FFN) blocks to obtain hierarchical visual representations. We insert ISA and CSA sub-modules into the first SW-MSA layer to estimate the in-sample causal effect $\widehat{M}$ and the cross-sample causal effect $\widehat{X}$ of the image features, as illustrated in Figure 8.

The resulting $\widehat{M}$ and $\widehat{X}$ are combined and subsequently processed by the remaining Swin Transformer layers to learn deeper visual representations. Consistent with the CALM integration in the text branch, ISA and CSA parameters are shared within each CALM module to ensure that the feature distributions across different sampling paths remain consistent, thereby stabilizing causal effect estimation. The formulation is as follows:

$$H_v=\mathrm{Swin}\_\mathrm{ACIM}\left(V\right)$$

(11)

3.3.3. Causal-Aware Multimodal Fusion

After completing unimodal feature extraction, we introduce the CALM module again in the cross-modal attention fusion layer. The inputs to this layer consist of two components:

• Text feature sequence $H_l\in {ℝ}^{B\times N_l\times d}$;
• Image feature sequence $H_v\in {ℝ}^{B\times N_v\times d}$.

• where B is the batch size, $N_l$ and $N_v$ denote the number of text tokens and image patches respectively, and $d$ is the hidden dimension.

The Fusion-CALM module simultaneously receives image features $H_v$ from the visual encoder and text features $H_l$ from the language encoder. Both feature types consist of current-sample embeddings and global dictionary representations, used for ISA and CSA modeling respectively. Within the fusion layer, visual features serve as query $Q$ and text features serve as key–value pairs ($K$,$V$). The two attention mechanisms model in-sample and cross-sample dependencies separately. IS-ATT captures salient semantic relationships between text and image within the current sample:

$$P(Y|M,X)=\mathrm{Softmax}(g(M,X))$$

(12)

while CS-ATT introduces cross-sample information through the global dictionary:

$$P(Y|do(M))={\displaystyle \sum _{x}P}(X=x)P(Y|M,X=x)$$

(13)

breaking spurious correlations caused by uneven data distribution.

Finally, the outputs $\widehat{M}$ and $\widehat{X}$ produced by ISA and CSA are combined into a unified multimodal representation. This representation is subsequently used by the classifier to predict the authenticity of the news. This design not only explicitly models causal structure at the feature interaction level, but also effectively improves the model’s robustness under long-tail scenarios and noisy data by incorporating cross-sample information.

3.4. Loss Function

The training objective combines three terms: the primary classification objective, the ACIM causal intervention loss, and a consistency loss for the fusion layer. For the final prediction, the classification term is computed using cross-entropy:

$${\mathcal{L}}_{\mathrm{cls}}=\mathrm{CE}(\mathrm{softmax}(z),y)$$

(14)

ACIM Causal Intervention Loss: CALM modules are inserted into the first attention layers of BERT and Swin Transformer encoders to constrain attention toward the mediating variable $M$ and improve model robustness. This loss consists of three components:

(i) Mask loss, ensuring that attention focuses on causally relevant mediating positions. Let $p_{\mathrm{med}}$ denote the attention probability assigned to the causal mediator M by the CALM module after Softmax normalization. This represents confidence that current attention focuses on causally relevant mediator positions. Maximizing $p_{\mathrm{med}}$ encourages the model to attend to causally informative regions while suppressing spurious correlations. The mask loss is defined as:

$${\mathcal{L}}_{\mathrm{mask}}^{\mathrm{bert}}=-\log p_{\mathrm{med}}^{\mathrm{bert}},\hspace{1em}{\mathcal{L}}_{\mathrm{mask}}^{\mathrm{swin}}=-\log p_{\mathrm{med}}^{\mathrm{swin}}$$

(15)

Here, $M$ is not identified from externally annotated causal positions or a manually predefined binary mask. Rather, it denotes the attention-mediated representation learned by CALM under the front-door causal formulation described in Section 3.2. Accordingly, $p_{med}$ is obtained from the Softmax-normalized attention scores associated with this learned mediator representation. Thus, mediator identification is performed implicitly through the attention learning process rather than through explicit causal-position labeling, and the mask loss encourages greater attention to mediator-associated information while suppressing spurious attention patterns.

(ii) Counterfactual Invariance, requiring that the CALM branch output remains stable under counterfactual perturbations. Specifically, $p^{\mathrm{bert},\mathrm{cf}}$ and $p^{\mathrm{swin},\mathrm{cf}}$ denote the attention distributions obtained from the counterfactual branch, and the invariance loss is defined as:

$${\mathcal{L}}_{\mathrm{inv}}^{\mathrm{bert}}=D_{KL}(p^{\mathrm{bert}}\Vert p^{\mathrm{bert},\mathrm{cf}}),\hspace{1em}{\mathcal{L}}_{\mathrm{inv}}^{\mathrm{swin}}=D_{KL}(p^{\mathrm{swin}}\Vert p^{\mathrm{swin},\mathrm{cf}})$$

(16)

Inspired by the intervention-based counterfactual reasoning paradigm adopted in CCD [14], the counterfactual branch does not explicitly synthesize new text–image samples. Instead, the counterfactual state is constructed by replacing the original attention state with its intervention-enhanced counterpart produced by CALM, thereby forming an alternative computational pathway under intervention. The resulting attention distributions are denoted as $p^{bert,cf}$ and $p^{swin,cf}$, respectively. The KL-divergence constraint in Equation (16) then encourages the original and counterfactual attention distributions to preserve task-relevant information while reducing sensitivity to spurious attention patterns.

(iii) Sparsity and Diversity Regularization, encouraging CALM attention to remain sparse while maintaining sufficient divergence from the original attention $A_0$:

$${\mathcal{L}}_{\mathrm{sparse}}=\Vert A_c{\Vert }_1,\hspace{1em}{\mathcal{L}}_{\mathrm{margin}}=\max (0,m-\Vert A_c-A_0{\Vert }_1)$$

(17)

Fusion Layer Loss: In the fusion layer, cross-modal outputs are required to remain consistent by minimizing the discrepancy between textual and visual predictions:

$${\mathcal{L}}_{\mathrm{align}}=D_{KL}(p^{\mathrm{bert}}\Vert p^{\mathrm{swin}})+D_{KL}(p^{\mathrm{swin}}\Vert p^{\mathrm{bert}})$$

(18)

Combining all components, the total loss is defined as:

$$\begin{array}{l}\mathcal{L}={\mathcal{L}}_{\mathrm{cls}}+{\lambda }_{\mathrm{m}}({\mathcal{L}}_{\mathrm{mask}}^{\mathrm{bert}}+{\mathcal{L}}_{\mathrm{mask}}^{\mathrm{swin}})+{\lambda }_{\mathrm{i}}({\mathcal{L}}_{\mathrm{inv}}^{\mathrm{bert}}+{\mathcal{L}}_{\mathrm{inv}}^{\mathrm{swin}})\\ +{\lambda }_{\mathrm{s}}{\mathcal{L}}_{\mathrm{sparse}}+{\lambda }_{\mathrm{g}}{\mathcal{L}}_{\mathrm{margin}}+{\lambda }_{\mathrm{a}}{\mathcal{L}}_{\mathrm{align}}\end{array}$$

(19)

where ${\lambda }_m,{\lambda }_i,{\lambda }_s,{\lambda }_g,{\lambda }_a$ denote the weighting coefficients of the corresponding loss terms. Unless otherwise specified, they are empirically set to 1.0, 0.5, 0.01, 0.1, and 1.0, respectively, in all experiments.

4. Experiments and Results

4.1. Experimental Setup

4.1.1. Datasets

Our experiments are evaluated using two benchmark datasets, Twitter [76] and PHEME [77], which differ from those adopted in our preliminary study. These datasets were selected primarily because they exhibit pronounced long-tail distribution characteristics: fake news categories are highly concentrated around a small number of breaking events, causing conventional models to easily overfit head samples. Unlike our preliminary study which focused on multimodal feature fusion bias, ACIM aims to alleviate representational bias induced by long-tail distributions through attention-level causal intervention, making it particularly challenging and practically valuable in real-world social media scenarios such as PHEME. Both datasets contain news text along with their associated images. For binary classification, we assign label “1” to fake news and label “0” to real news. The dataset statistics are reported in

Table 1. Statistics of the Twitter and PHEME datasets.

Dataset	# Fake News	# Real News	# Images
Twitter	6020	7898	514
PHEME	1972	3830	3670

# denotes the number of samples or images, as applicable.

.

Twitter dataset: Originally released for the MediaEval Multimedia Verification task, it contains textual content, image information, and associated social context metadata. In this work, we adopt a content-based approach and use only text and image information for fake news detection. The 514 images reported in Table 1 refer to distinct images rather than the total number of image occurrences. Since the same image may be reposted with different textual descriptions, multiple Twitter posts can correspond to one image, and each post–image pair is treated as an individual sample. The number of images is substantially smaller than the number of posts because multiple posts may share the same image while containing different textual descriptions. Each post–image pair is still treated as an independent multimodal sample.

PHEME dataset: Originally derived from the PHEME project, which aimed to explore automated detection and verification of rumors circulating on social media. The dataset is constructed around five breaking news events, each containing a large number of posts from social platforms. All posts are annotated as either “rumor” or “non-rumor” based on their content properties. In our setting, the former are treated as fake news samples and the latter as real news samples, providing a standardized data foundation for training and evaluating multimodal fake news detection models.

4.1.2. Experimental Settings

All models were implemented using PyTorch 2.8.0, and the experiments were performed on a workstation with an NVIDIA GeForce RTX 3080 Laptop GPU (16 GB memory; NVIDIA Corporation, Santa Clara, CA, USA) and 32 GB RAM. Adam was adopted for optimization, with the learning rate initialized to $2\times {10}^{-5}$, the batch size fixed at 64, and the training duration set to 200 epochs. For ACIM and each of its variants, five independent runs were carried out with different random seeds. Unless stated otherwise, the results presented for the main experiments, ablation studies, and sensitivity analyses correspond to the mean ± standard deviation calculated across these five runs.

For Twitter, we retained the benchmark’s predefined train–test partition and randomly held out 10% of its training samples for validation and hyperparameter selection. For PHEME, an 80/20 random partition was used to construct the training and test sets. We adopted the publicly available multimodal version that has been widely used in previous multimodal fake news detection studies. This benchmark associates source tweets with their corresponding images and follows the commonly adopted preprocessing protocol in previous work.

The baseline results reported in this study were obtained directly from their respective original publications under the corresponding benchmark datasets. Therefore, these results are used for reference comparison and should be interpreted with consideration of potential differences in experimental settings. Throughout this paper, label 1 denotes the positive (fake news) class, while label 0 denotes the negative (real news) class when defining TP, FP, FN, and TN.

4.1.3. Evaluation Metrics

We formulate fake news detection as a two-class prediction task and evaluate the classification outcomes using the confusion matrix presented in

Table 2. Confusion matrix for fake news detection.

	Predicted Positive	Predicted Negative
Actual Positive	TP	FN
Actual Negative	FP	TN

.

Each symbol is defined as follows:

• TP (True Positive): the number of samples whose true label is positive and are correctly predicted as positive.
• FP (False Positive): the number of samples whose true label is negative but are incorrectly predicted as positive.
• FN (False Negative): the number of samples whose true label is positive but are incorrectly predicted as negative.
• TN (True Negative): the number of samples whose true label is negative and are correctly predicted as negative.

Using the confusion matrix, we primarily assess detection performance by accuracy. Precision, recall, and F1 score are also reported to provide a more comprehensive evaluation under class imbalance, consistent with prior studies [25,78,79]:

Accuracy: the proportion of correctly classified samples:

$$\mathrm{Accuracy}={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$

(20)

Precision: the proportion of predicted fake news samples that are actually fake:

$$\mathrm{Precision}={\displaystyle \frac{TP}{TP+FP}}$$

(21)

Recall: the proportion of actual fake news samples that are correctly identified:

$$\mathrm{Recall}={\displaystyle \frac{TP}{TP+FN}}$$

(22)

F1 score: the harmonic mean of precision and recall:

$$F_1={\displaystyle \frac{2\times \mathrm{Precision}\times \mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}}$$

(23)

Macro-F1: the unweighted mean of the F1 scores of the fake news and real news classes, providing a class-balanced assessment of performance:

$$\mathrm{Macro}\text{-}\mathrm{F}1={\displaystyle \frac{F{1}_{\mathrm{fake}}+F{1}_{\mathrm{real}}}{2}}$$

(24)

Balanced accuracy: the average of the recall values for the positive and negative classes, reducing the influence of class imbalance:

$$\mathrm{Balanced}\mathrm{Accuracy}={\displaystyle \frac{1}{2}}\left({\displaystyle \frac{TP}{TP+FN}}+{\displaystyle \frac{TN}{TN+FP}}\right)$$

(25)

Matthews correlation coefficient (MCC): a correlation-based metric that considers all four entries of the confusion matrix and provides a balanced evaluation even when the class distribution is uneven:

$$\mathrm{MCC}={\displaystyle \frac{TP\times TN-FP\times FN}{\sqrt{(TP+FP)(TP+FN)(TN+FP)(TN+FN)}}}$$

(26)

4.2. Comparison with Baselines

ACIM is evaluated against a set of representative fake news detection baselines using the Twitter and PHEME benchmarks. For each baseline, we use the performance values reported in its original publication for the corresponding dataset. Accordingly, methods without published results on a particular benchmark are not included in the comparison for that dataset.

• GRU [18]: An RNN-based method that uses a multi-layer GRU network to model posts as variable-length time series for hidden representation learning in fake news detection.
• CNN [19]: Extracts feature representations via convolutional neural networks, converting related posts into fixed-length sequences for fake news identification and early detection.
• SAFE [26]: A similarity-aware fusion model that extracts independent text and image embeddings, computes cross-modal similarity features, and concatenates them for fake news prediction.
• EANN [25]: A multi-task adversarial network that learns shared text and image representations through an auxiliary event-adversarial objective.
• TextGCN [80]: Models the entire corpus as a heterogeneous graph using graph convolutional networks to jointly learn richer word and document embeddings for fake news detection.
• MVAE [6]: A multimodal variational autoencoder that learns a shared latent space and simultaneously reconstructs both text and image features for downstream classification tasks.
• SpotFake [81]: Extracts text features using a pretrained language model (BERT) and image features using VGG-19 pretrained on ImageNet for multimodal fake news detection.
• HMCAN [79]: Jointly models multimodal contextual information and hierarchical textual semantics using BERT and ResNet, and fuses them through a multimodal contextual attention network for fake news detection.
• CSFND [82]: A context-sensitive framework that incorporates contextual semantic information to resolve inconsistencies between semantic and decision spaces, enabling clearer boundary delineation under different news contexts.
• BMR [83]: A multi-view representation bootstrapping method that improves fake news detection performance through multi-view feature extraction, an enhanced fusion mechanism, and an adaptive weighting strategy.
• NLIN [84]: Unifies multimodal inputs into a text space and employs an encoder–decoder with a prompt-based reasoning architecture to address the modality disparity issue in multimodal fake news detection.
• MVML [85]: Simultaneously models fine-grained semantic relationships between text–scene and text–object pairs, leverages graph attention for intra-modal reasoning, applies cross-view attention for multi-perspective fusion, and utilizes a mutual learning mechanism to enhance classifier performance.
• CCD [14]: A causal debiasing framework that employs causal intervention to mitigate psycholinguistic confounding and uses counterfactual reasoning to reduce the image-only bias in multimodal fake news detection.

Table 3. Comparison results of ACIM against baseline methods.

Dataset	Method	Accuracy	Precision	Recall	F1
Twitter	GRU [18]	0.634	0.581	0.812	0.677
	CNN [19]	0.549	0.508	0.597	0.549
	SAFE [26]	0.766	0.777	0.795	0.786
	EANN [25]	0.648	0.810	0.498	0.617
	TextGCN [80]	0.703	0.808	0.365	0.503
	MVAE [6]	0.745	0.801	0.719	0.758
	SpotFake [81]	0.771	0.784	0.744	0.764
	HMCAN [79]	0.897	0.971	0.801	0.878
	HMCAN + CCD [14]	0.874	0.820	0.792	0.806
	CSFND [82]	0.873	0.899	0.799	0.846
	MVML [85]	0.882	0.915	0.801	0.893
	ACIM (ours)	0.906 ± 0.003	0.881 ± 0.001	0.844 ± 0.002	0.860 ± 0.001
PHEME	GRU [18]	0.832	0.782	0.712	0.745
	CNN [19]	0.779	0.732	0.606	0.663
	SAFE [26]	0.811	0.827	0.559	0.667
	EANN [25]	0.681	0.685	0.664	0.694
	MVAE [6]	0.852	0.806	0.719	0.760
	SpotFake [81]	0.823	0.743	0.745	0.744
	HMCAN [79]	0.881	0.830	0.838	0.834
	HMCAN + CCD [14]	0.859	0.764	0.689	0.724
	BMR [83]	0.884	0.872	0.840	0.855
	NLIN [84]	0.903	0.875	0.883	0.879
	MVML [85]	0.893	0.872	0.846	0.855
	ACIM (ours)	0.909± 0.001	0.905 ± 0.003	0.875 ± 0.003	0.890 ± 0.001

The best results are highlighted in bold, and the second-best results are highlighted in underline.

presents the comparison between ACIM and the baseline methods on the Twitter and PHEME datasets. Overall, ACIM achieves the highest accuracy on both datasets while maintaining competitive precision, recall, and F1 performance, demonstrating the effectiveness of attention-level causal intervention for multimodal fake news detection.

On the Twitter dataset, early unimodal and simple fusion methods exhibit relatively limited performance. CNN and GRU obtain accuracies of 0.549 and 0.634, respectively, while TextGCN achieves a recall of only 0.365, indicating limited capability in identifying fake news instances. Multimodal methods such as SAFE, MVAE, and SpotFake improve accuracy to between 0.745 and 0.771, confirming the benefit of combining textual and visual information. Among the stronger baselines, HMCAN achieves the highest precision of 0.971 and an accuracy of 0.897, whereas MVML obtains the highest fake news F1 score of 0.893. ACIM reaches an overall accuracy of 0.906 ± 0.003, which is 0.9 percentage points higher than HMCAN, the next strongest baseline in terms of accuracy. ACIM also achieves the highest fake news recall of 0.844 ± 0.002. Although its fake news F1 score of 0.860 ± 0.001 does not exceed that of MVML, the improved accuracy and recall indicate that ACIM provides a competitive balance between overall classification performance and the identification of fake news instances.

On the PHEME dataset, ACIM demonstrates a clearer overall advantage. Early methods such as EANN and CNN obtain accuracies of 0.681 and 0.779, respectively, while stronger multimodal baselines, including HMCAN, BMR, and MVML, achieve accuracies ranging from 0.881 to 0.893. NLIN provides the strongest baseline performance, with an accuracy of 0.903, a recall of 0.883, and an F1 score of 0.879. In comparison, ACIM obtains 0.909 ± 0.001 accuracy, 0.905 ± 0.003 precision, 0.875 ± 0.003 recall, and an F1 score of 0.890 ± 0.001. Compared with NLIN, ACIM improves accuracy, precision, and F1 by 0.6, 3.0, and 1.1 percentage points, respectively, while obtaining the second-best recall. These results suggest that attention-level causal intervention improves overall classification reliability and fake news discrimination without excessively sacrificing recall.

The externally reported HMCAN + CCD results achieve accuracies of 0.874 and 0.859 on Twitter and PHEME, respectively. ACIM improves upon these values by 3.2 and 5.0 percentage points. Nevertheless, this comparison is provided for reference because the official implementation of CCD is unavailable and the reported results may have been obtained under different experimental settings.

Although ACIM achieves the highest accuracy on both datasets, its advantage is not uniform across all evaluation metrics. On Twitter, ACIM improves accuracy over HMCAN from 0.897 to 0.906, whereas its fake news F1 score (0.860) remains lower than that of MVML (0.893). Similarly, on PHEME, ACIM achieves higher accuracy than NLIN (0.909 vs. 0.903), while its fake news recall is slightly lower (0.875 vs. 0.883). These results indicate a trade-off between overall classification accuracy and individual class-sensitive metrics. Therefore, the advantage of ACIM should be interpreted in terms of its overall and class-balanced predictive performance rather than superiority on every individual metric.

Table 4. Additional evaluation metrics of ACIM under class imbalance.

Dataset	Macro-F1	Balanced Accuracy	MCC
Twitter	0.895 ± 0.001	0.892 ± 0.004	0.791 ± 0.002
PHEME	0.907 ± 0.001	0.905 ± 0.002	0.813 ± 0.003

further reports macro-F1, balanced accuracy, and Matthews correlation coefficients (MCCs) to provide a more comprehensive evaluation under class imbalance. On Twitter, ACIM achieves a macro-F1 of 0.895, a balanced accuracy of 0.892, and an MCC of 0.791, while the corresponding values on PHEME are 0.907, 0.905, and 0.813, respectively. These results indicate that ACIM maintains relatively balanced predictive performance across the fake news and real news classes rather than obtaining its overall accuracy primarily from the majority class. The consistently positive MCC values further demonstrate strong agreement between the predicted and ground-truth labels when all four outcomes of the confusion matrix are considered. Together with the accuracy and class-wise precision, recall, and F1 scores reported above, these additional imbalance-aware metrics provide a more comprehensive assessment of ACIM on both datasets.

4.3. Ablation Study

We perform component-wise ablation studies on the Twitter and PHEME datasets to assess the contribution of each CALM module to multimodal fake news detection.

Table 5. Effect of Attention-Level Causal Intervention.

Dataset	Method	Accuracy	Fake News			Real News
			Precision	Recall	F1	Precision	Recall	F1
Twitter	w/o causal	0.862 ± 0.004	0.803 ± 0.005	0.798 ± 0.009	0.801 ± 0.006	0.893 ± 0.004	0.896 ± 0.003	0.894 ± 0.004
	w/o ISA	0.898 ± 0.004	0.864 ± 0.006	0.838 ± 0.011	0.851 ± 0.007	0.915 ± 0.004	0.930 ± 0.005	0.923 ± 0.004
	w/o CSA	0.894 ± 0.006	0.858 ± 0.008	0.832 ± 0.017	0.845 ± 0.011	0.912 ± 0.005	0.927 ± 0.006	0.919 ± 0.005
	w/o Aux. Loss	0.901 ± 0.003	0.868 ± 0.004	0.843 ± 0.007	0.855 ± 0.005	0.918 ± 0.003	0.932 ± 0.004	0.925 ± 0.003
	w/o V	0.890 ± 0.003	0.848 ± 0.004	0.832 ± 0.012	0.840 ± 0.005	0.911 ± 0.003	0.921 ± 0.004	0.916 ± 0.003
	w/o L	0.884 ± 0.005	0.830 ± 0.008	0.838 ± 0.018	0.834 ± 0.013	0.913 ± 0.004	0.908 ± 0.005	0.911 ± 0.004
	w/o F	0.895 ± 0.003	0.852 ± 0.004	0.845 ± 0.007	0.848 ± 0.005	0.918 ± 0.003	0.922 ± 0.003	0.920 ± 0.002
	ACIM	0.906 ± 0.003	0.881 ± 0.001	0.844 ± 0.002	0.860 ± 0.001	0.919 ± 0.003	0.939 ± 0.002	0.929 ± 0.002
PHEME	w/o causal	0.850 ± 0.005	0.836 ± 0.004	0.800 ± 0.015	0.817 ± 0.008	0.860 ± 0.005	0.886 ± 0.004	0.873 ± 0.005
	w/o ISA	0.900 ± 0.004	0.894 ± 0.006	0.864 ± 0.010	0.879 ± 0.007	0.904 ± 0.004	0.926 ± 0.005	0.915 ± 0.004
	w/o CSA	0.896 ± 0.006	0.891 ± 0.008	0.856 ± 0.016	0.873 ± 0.010	0.899 ± 0.005	0.925 ± 0.007	0.912 ± 0.005
	w/o Aux. Loss	0.904 ± 0.003	0.898 ± 0.004	0.870 ± 0.007	0.884 ± 0.005	0.908 ± 0.003	0.929 ± 0.004	0.918 ± 0.003
	w/o V	0.862 ± 0.004	0.850 ± 0.005	0.815 ± 0.010	0.832 ± 0.006	0.870 ± 0.004	0.896 ± 0.003	0.883 ± 0.004
	w/o L	0.883 ± 0.004	0.880 ± 0.004	0.835 ± 0.008	0.857 ± 0.005	0.885 ± 0.003	0.918 ± 0.004	0.901 ± 0.003
	w/o F	0.885 ± 0.003	0.880 ± 0.004	0.840 ± 0.006	0.860 ± 0.004	0.888 ± 0.003	0.918 ± 0.003	0.903 ± 0.003
	ACIM	0.909 ± 0.001	0.905 ± 0.003	0.875 ± 0.003	0.890 ± 0.001	0.912 ± 0.002	0.934 ± 0.002	0.923 ± 0.002

The best results are highlighted in bold.

summarizes the corresponding results. The compared model variants are as follows:

• Base (w/o causal): The baseline model without any causal intervention, i.e., standard multimodal feature concatenation followed by a classifier.
• w/o V: Removes the CALM module from the visual branch, retaining causal modeling only in the text branch and fusion layer.
• w/o L: Removes the CALM module from the text branch, retaining causal modeling only in the visual branch and fusion layer.
• w/o F: Removes the CALM module from the fusion layer, retaining causal modeling only in the text and visual branches.
• ACIM: The full model with CALM modules integrated across all three stages: text, image, and fusion.

The additional component-level ablations further demonstrate the complementary roles of ISA, CSA, and the auxiliary loss terms. On the Twitter dataset, removing ISA reduces accuracy from 0.906 to 0.898 and the fake news F1 score from 0.860 to 0.851, while removing CSA decreases them to 0.894 and 0.845, respectively. Removing the auxiliary loss terms results in a smaller performance degradation, with an accuracy of 0.901 and a fake news F1 score of 0.855. Similar trends are observed on PHEME, where accuracy decreases from 0.909 to 0.900, 0.896, and 0.904 when ISA, CSA, and the auxiliary losses are removed, respectively. These results indicate that both attention branches contribute to the final performance, with CSA showing a relatively larger contribution, while the auxiliary loss terms provide additional regularization benefits during causal attention learning and optimization.

Among the CALM placement ablations, w/o F achieves the strongest overall performance and is therefore selected as the representative placement-ablation model for statistical comparison. On the Twitter dataset, ACIM improves accuracy from 0.895 to 0.906 and the fake news F1 score from 0.848 to 0.860 compared with w/o F, representing gains of 1.1 and 1.2 percentage points, respectively.

On the PHEME dataset, ACIM increases accuracy from 0.885 to 0.909 and the fake news F1 score from 0.860 to 0.890, yielding improvements of 2.4 and 3.0 percentage points, respectively. Statistical significance was assessed using two-sided paired t-tests over five matched random seeds $\alpha =0.05$. For accuracy, ACIM versus w/o F yielded ($p=0.109$, $d_z=0.92$) on Twitter and ($p=0.045$, $d_z=1.21$) on PHEME. For fake news F1, the corresponding results were $p=0.075$, $d_z=1.07$ on Twitter and $p=0.046$, $d_z=1.28$ on PHEME. Thus, neither accuracy nor fake news F1 improvement on Twitter reaches statistical significance, whereas both improvements on PHEME are statistically significant.

Table 6. Sensitivity analysis results of ACIM on the Twitter dataset.

Parameters	Value	Accuracy	Fake News			Real News
			Precision	Recall	F1	Precision	Recall	F1
num_heads	2	0.889 ± 0.005	0.839 ± 0.006	0.842 ± 0.012	0.841 ± 0.008	0.916 ± 0.004	0.914 ± 0.005	0.915 ± 0.004
	4	0.898 ± 0.004	0.854 ± 0.005	0.852 ± 0.007	0.853 ± 0.005	0.921 ± 0.003	0.922 ± 0.004	0.922 ± 0.003
	8	0.906 ± 0.003	0.881 ± 0.001	0.844 ± 0.002	0.860 ± 0.001	0.919 ± 0.003	0.939 ± 0.002	0.929 ± 0.002
	16	0.901 ± 0.004	0.857 ± 0.006	0.858 ± 0.009	0.858 ± 0.007	0.924 ± 0.004	0.924 ± 0.005	0.924 ± 0.004
dropout	0	0.896 ± 0.005	0.852 ± 0.006	0.848 ± 0.011	0.850 ± 0.007	0.919 ± 0.004	0.922 ± 0.005	0.920 ± 0.004
	0.1	0.906 ± 0.003	0.881 ± 0.001	0.844 ± 0.002	0.860 ± 0.001	0.919 ± 0.003	0.939 ± 0.002	0.929 ± 0.002
	0.2	0.904 ± 0.004	0.863 ± 0.005	0.860 ± 0.006	0.862 ± 0.004	0.926 ± 0.003	0.927 ± 0.004	0.927 ± 0.003
	0.4	0.898 ± 0.006	0.854 ± 0.008	0.852 ± 0.014	0.853 ± 0.009	0.921 ± 0.005	0.922 ± 0.006	0.922 ± 0.005
K	16	0.893 ± 0.006	0.853 ± 0.008	0.836 ± 0.016	0.844 ± 0.010	0.914 ± 0.005	0.923 ± 0.006	0.918 ± 0.005
	32	0.901 ± 0.004	0.863 ± 0.005	0.850 ± 0.008	0.856 ± 0.005	0.921 ± 0.003	0.928 ± 0.004	0.924 ± 0.003
	64	0.906 ± 0.003	0.881 ± 0.001	0.844 ± 0.002	0.860 ± 0.001	0.919 ± 0.003	0.939 ± 0.002	0.929 ± 0.002
	128	0.899 ± 0.005	0.858 ± 0.006	0.850 ± 0.010	0.854 ± 0.007	0.921 ± 0.004	0.925 ± 0.005	0.923 ± 0.004

The best results are highlighted in bold.

and

Table 7. Sensitivity analysis results of ACIM on the PHEME dataset.

Parameters	Value	Accuracy	Fake News			Real News
			Precision	Recall	F1	Precision	Recall	F1
num_heads	2	0.898 ± 0.005	0.889 ± 0.006	0.868 ± 0.011	0.878 ± 0.007	0.905 ± 0.004	0.920 ± 0.005	0.912 ± 0.004
	4	0.904 ± 0.003	0.894 ± 0.005	0.878 ± 0.006	0.886 ± 0.004	0.911 ± 0.003	0.923 ± 0.004	0.917 ± 0.003
	8	0.909 ± 0.001	0.905 ± 0.003	0.875 ± 0.003	0.890 ± 0.001	0.912 ± 0.002	0.934 ± 0.002	0.923 ± 0.002
	16	0.906 ± 0.004	0.895 ± 0.005	0.882 ± 0.008	0.888 ± 0.005	0.914 ± 0.004	0.924 ± 0.005	0.919 ± 0.004
dropout	0	0.901 ± 0.005	0.904 ± 0.006	0.858 ± 0.013	0.880 ± 0.008	0.899 ± 0.005	0.933 ± 0.006	0.916 ± 0.005
	0.1	0.909 ± 0.001	0.905 ± 0.003	0.875 ± 0.003	0.890 ± 0.001	0.912 ± 0.002	0.934 ± 0.002	0.923 ± 0.002
	0.2	0.905 ± 0.004	0.909 ± 0.005	0.862 ± 0.008	0.885 ± 0.005	0.902 ± 0.004	0.937 ± 0.004	0.919 ± 0.003
	0.4	0.899 ± 0.006	0.902 ± 0.008	0.855 ± 0.015	0.878 ± 0.009	0.897 ± 0.005	0.931 ± 0.006	0.914 ± 0.005
K	16	0.897 ± 0.006	0.892 ± 0.008	0.860 ± 0.016	0.876 ± 0.010	0.901 ± 0.005	0.924 ± 0.006	0.912 ± 0.005
	32	0.905 ± 0.004	0.900 ± 0.005	0.870 ± 0.007	0.885 ± 0.005	0.908 ± 0.003	0.930 ± 0.004	0.919 ± 0.003
	64	0.909 ± 0.001	0.905 ± 0.003	0.875 ± 0.003	0.890 ± 0.001	0.912 ± 0.002	0.934 ± 0.002	0.923 ± 0.002
	128	0.903 ± 0.005	0.898 ± 0.006	0.868 ± 0.010	0.883 ± 0.007	0.906 ± 0.004	0.928 ± 0.005	0.917 ± 0.004

The best results are highlighted in bold.

and Figure 9 report the sensitivity of ACIM to three hyperparameters on Twitter and PHEME: attention-head number, dropout rate, and global dictionary size $K$.

Regarding the number of attention heads, model performance generally improves as num_heads increases from 2 to 8 on both datasets. On Twitter, $num\_heads=8$ achieves the highest accuracy of 0.906 and a fake news F1 score of 0.860. On PHEME, the same setting achieves the highest accuracy of 0.909 and a fake news F1 score of 0.890. Although some class-specific metrics reach slightly higher values when $num\_heads=16$, the setting of 8 provides the best overall accuracy on both datasets. This suggests that using a moderate number of attention heads facilitates information extraction from diverse semantic subspaces, while too many heads may produce repetitive representations and reduce the model’s discriminative ability.

Regarding the dropout ratio, the overall trend is consistent across both datasets. On Twitter, $dropout=0.1$ achieves the highest accuracy of 0.906 and a fake news F1 score of 0.860. On PHEME, the same setting achieves the highest accuracy of 0.909 and a fake news F1 score of 0.890. Although some individual precision, recall, or F1 values peak at $dropout=0.2$, $dropout=0.1$ provides the strongest overall classification accuracy. A moderate dropout ratio helps mitigate overfitting, whereas a larger value such as 0.4 may discard excessive feature information and impair representation quality.

Regarding the global dictionary size, we vary $K$ over $\left\{16,32,64,128\right\}$. On the Twitter dataset, $K=64$ achieves the highest accuracy of 0.906 and a fake news F1 score of 0.860. On the PHEME dataset, it also achieves the highest accuracy of 0.909 and a fake news F1 score of 0.890. A smaller dictionary may provide insufficient coverage of global semantic patterns, whereas an excessively large dictionary may introduce redundant or noisy prototypes. Therefore, $K=64$ provides the most suitable balance between dictionary coverage and representation redundancy.

Overall, ACIM delivers its strongest performance on both Twitter and PHEME with $num\_heads=8$, $dropout=0.1$, and $K=64$. These results demonstrate that appropriately controlling the number of attention heads, regularization strength, and global dictionary size is important for balancing discriminative capability and generalization performance in multimodal fake news detection.

4.4. Case Study

We further analyze the behavior of ACIM using six samples covering different scenarios, including three real news samples and three fake news samples. The first three samples were misclassified as fake news by the base model but correctly classified as real news by ACIM, while the remaining three fake news samples are also correctly classified by ACIM. Figure 10 presents the class probabilities assigned by ACIM to these six samples. For the first three cases, the predicted probabilities for the real news class are 62.41%, 59.64%, and 54.65%, respectively, indicating that ACIM can correct some erroneous predictions of the base model. For the remaining three cases, the predicted probabilities for the fake news class are 61.54%, 57.69%, and 55.21%, respectively. These cases provide a broader view of the behavior of the proposed intervention framework across different real and fake news scenarios.

The first case contains a large protest crowd, signs, and police-related expressions. Such features may activate spurious associations between crowd scenes, conflict-related language, and fake news labels learned from the training distribution. However, the accompanying text explicitly describes the event as peaceful. ACIM assigns a real news probability of 62.41% to this sample, which is consistent with reduced reliance on the superficial association between protest scenes and violent events.

The second and third cases similarly contain dense crowd scenes that may visually resemble protests, emergencies, or public disorder. By jointly considering the event descriptions and visual context, ACIM classifies both samples as real news, with probabilities of 59.64% and 54.65%, respectively. In particular, the relatively low confidence for the third case indicates that highly salient crowd-related visual patterns remain challenging. These cases suggest that ACIM can mitigate sample-specific co-occurrence bias, although ambiguity remains in difficult multimodal samples.

The fourth, fifth, and sixth cases are fake news samples involving an aircraft carrier, satellite imagery, and a smoke-related scene, respectively. ACIM correctly classifies these three samples as fake news, assigning fake news probabilities of 61.54%, 57.69%, and 55.21%, respectively. Compared with the first three real news cases, these samples contain different types of visually salient content and multimodal semantic patterns. The correct predictions across these diverse scenarios suggest that ACIM can make use of both textual and visual evidence rather than relying solely on superficial visual cues. Nevertheless, the relatively moderate confidence scores, particularly for the fifth and sixth cases, indicate that complex multimodal associations remain challenging for the model.

Overall, this case study provides qualitative evidence supporting the effectiveness of the proposed attention-level intervention mechanism across both real and fake news scenarios. However, these qualitative observations should not be regarded as direct proof that the model has learned causal representations. More rigorous quantitative intervention experiments would be required to establish such causal effects.

5. Conclusions

This paper proposes ACIM, an Attention-level Causal Intervention Framework, for multimodal fake news detection, which introduces front-door causal intervention into the attention learning process with the aim of mitigating hidden confounding effects and spurious cross-modal correlations. A Causal Attention Layer Module (CALM) is embedded into both BERT-based textual encoders and Swin Transformer-based visual encoders for causal debiasing during feature extraction, while a causal-aware fusion layer further suppresses misleading co-occurrence patterns during cross-modal interaction. Experimental results on Twitter and PHEME indicate that ACIM provides the strongest overall accuracy, together with competitive precision, recall, and F1 scores. The ablation results further demonstrate that causal intervention at the textual, visual, and fusion stages provides complementary benefits. These results demonstrate the effectiveness of ACIM in improving multimodal fake news detection performance and provide empirical support for the proposed attention-level causal intervention mechanism in mitigating the influence of spurious multimodal correlations.

Although the experiments on Twitter and PHEME demonstrate the effectiveness of ACIM in two representative multimodal settings, the current evaluation is limited to these two datasets. Therefore, the generalizability of the proposed framework across other languages, social platforms, and event domains remains to be further validated. Future work will extend ACIM to additional multilingual and cross-platform benchmarks and investigate its performance under cross-event and cross-domain settings. The framework will also be explored in richer multimodal scenarios involving audio, video, and temporal information.