Beyond Spurious Cues: Adaptive Multi-Modal Fusion via Mixture-of-Experts for Robust Sarcasm Detection

Abstract

Sarcasm is a complex emotional expression often marked by semantic contrast and incongruity between textual and visual modalities. In recent years, multi-modal sarcasm detection (MMSD) has emerged as a vital task in affective computing. However, existing models frequently rely on superficial spurious cues—such as emojis or hashtags—during training and inference, limiting their ability to capture deeper semantic inconsistencies and undermining generalization to real-world scenarios. To tackle these challenges, we propose Multi-Modal Mixture-of-Experts (MM-MoE), a novel framework that integrates diverse expert modules through a global dynamic gating mechanism for adaptive cross-modal interaction and selective semantic fusion. This architecture allows for the model to better capture modality-level incongruity. Furthermore, we introduce MMSD3.0 and MMSD4.0, two cross-dataset evaluation benchmarks derived from two open source benchmark datasets, MMSD and MMSD2.0, to assess model robustness under varying distributions of spurious cues. Extensive experiments demonstrate that MM-MoE achieves strong performance and generalization ability, consistently outperforming state-of-the-art baselines when encountering superficial spurious correlations.

1. Introduction

Sarcasm is a subtle and cognitively complex linguistic phenomenon characterized by a mismatch between surface expression and intended meaning [1,2]. With the proliferation of multi-modal content on social medias, users increasingly employ both text and images to express irony, sarcasm, or satire. Accurately detecting such multi-modal sarcasm is pivotal in applications such as sentiment analysis, misinformation detection, and public opinion monitoring [3,4,5,6,7].

The core challenge in multi-modal sarcasm detection (MMSD) lies in capturing the semantic incongruity and emotional tension between textual and visual components. This incongruity may occur at the semantic level (e.g., contradiction between textual description and image content) or the affective level (e.g., cheerful text paired with a gloomy image), making sarcasm expressions diverse, implicit, and highly context-dependent [8,9]. Therefore, effectively modeling cross-modal interactions remains a primary challenge for MMSD.

Despite advancements in cross-modal fusion methods, two key issues remain unresolved:

• Mainstream models often exploit superficial textual cues (e.g., emojis and hashtags) that are prevalent in noisy datasets. While prior works employ various fusion strategies, their architectures often fail to distinguish these cues from genuine semantic conflicts, inadvertently learning shortcuts that do not generalize to real-world scenarios. This fundamental weakness is exposed when such cues are removed, causing model performance to degrade significantly [10]. For example, as shown in Figure 1, traditional multi-modal models such as HFM [11], Att-BERT [12], CMGCN [13], HKE [14], DynRT [15], and G²SAM [16] perform significantly worse on the cleaned MMSD2.0 dataset [10], as the accuracy of the DynRT model drops by 23.22% from MMSD [11] to MMSD2.0.
• Lack of dynamic fusion mechanisms: Most current MMSD models adopt static fusion strategies, applying the same integration method to all inputs. This uniformity ignores the varying dominance or noise in different modalities across contexts, resulting in unstable performance.

To tackle these challenges, we propose MM-MoE, a multi-modal sarcasm detection framework based on the Mixture-of-Experts (MoE) paradigm. To provide a comprehensive analysis of multi-modal incongruity, MM-MoE is explicitly designed with four functionally distinct expert types: (1) Text Self-Attention Experts to capture internal textual context, (2) Image Self-Attention Experts to model local and global visual cues, (3) Text-Guided Cross-Modal Experts to identify how text semantics guide attention over visual features and (4) Image-Guided Cross-Modal Experts to perform the symmetric operation. This selection is justified by the need to model both intra-modal consistency and inter-modal incongruity, which are central to sarcasm. These experts are managed by a global gating network that dynamically adjusts their contributions based on the input context, facilitating fine-grained, adaptive fusion. Additionally, we introduce a data augmentation strategy using in-batch negative sampling, which serves as a regularization term to guide the model toward learning true semantic mismatches and reduce reliance on spurious correlations.

To thoroughly evaluate robustness and generalization, we design two cross-dataset setups:

• MMSD3.0: Trained on noisy MMSD, tested on clean MMSD2.0.
• MMSD4.0: Trained on clean MMSD2.0, tested on noisy MMSD.

These settings simulate realistic shifts in spurious cues distributions and enable rigorous testing of the model’s semantic grounding. Our contributions are summarized as follows:

• We propose a new systematic evaluation protocol to assess the robustness of multi-modal sarcasm detection models against distributional shifts of spurious cues. We instantiate this protocol with two challenging benchmarks, MMSD3.0 and MMSD4.0, which are designed to rigorously measure a model’s true generalization ability beyond memorizing superficial dataset artifacts.
• We propose MM-MoE, a Mixture-of-Experts framework that integrates diverse expert modules and a dynamic gating mechanism for adaptive semantic fusion. A regularization loss based on negative sampling augmentation is further introduced to enhance semantic alignment and reduce dependence on superficial cues.
• Extensive experiments show that MM-MoE performs well in the presence of spurious cues in multi-modal sarcasm detection, consistently outperforms state-of-the-art baselines, and exhibits strong generalization capabilities to spurious correlations.

2. Related Work

2.1. Multi-Modal Sarcasm Detection

Multi-modal sarcasm detection (MMSD) aims to identify sarcastic meanings expressed jointly through text, images, and other modalities. This task is inherently challenging due to three core characteristics: modal collaboration, implicit expression, and strong contextual dependency. As a result, sarcasm is often difficult to detect using uni-modal information, thus necessitating advanced cross-modal modeling techniques [8].

Early studies on sarcasm detection focused predominantly on textual content, exploring features such as sentiment reversal and semantic incongruity [17,18,19,20]. With the rise of image-rich expressions on social medias, researchers recognized the complementary semantic and emotional value of visual information in sarcastic communication. Schifanella et al. were among the first to introduce multi-modal sarcasm detection [21], followed by Cai et al., who constructed the MMSD dataset, paving the way for systematic research in this area [11].

Based on MMSD, various approaches have been proposed to model text–image sarcasm. For instance, a Graph Convolutional Network (GCN) can capture the ironic relationship between modalities [13,22]. Wen et al. designed a Dual Incongruity Perception (DIP) network to simultaneously detect emotional and factual mismatches [23]. Liu et al. designed a model framework that combines atomic-level consistency based on multi-head cross-attention with component-level consistency based on a graph neural network, which effectively improves the cross-modal sarcasm recognition ability [14]. Wei et al. further improved the model’s detection of cross-modal sarcasm by constructing global semantic consistency features [16].

While these methods achieve promising results on in-domain datasets, recent studies have revealed a significant limitation: many models rely heavily on explicit surface-level features such as emojis and hashtags, which do not generalize well to real-world sarcastic expressions. Qin et al. highlighted the issue of spurious correlations in the original MMSD dataset and proposed a cleaned version, MMSD2.0, by removing such misleading cues [10]. On MMSD2.0, most mainstream models suffered notable performance drops (for example, the DIP’s F1 score decreased by over 10% [24], indicating insufficient semantic incongruity modeling. While recent approaches like ESAM [25] improve robustness by applying static mechanisms like feature masking, our work introduces a different paradigm. We focus on making the fusion process itself dynamic and input-dependent through a Mixture-of-Experts architecture that adaptively selects the most suitable analysis pathway for each sample. Moreover, existing fusion architectures are typically static, applying the same integration strategy to all samples without considering the modality dominance or incongruity level. This rigidity is seen as a key reason for poor generalization.

2.2. Mixture-of-Experts (MoE) Architectures

Mixture-of-Experts (MoE) is a type of machine learning architecture that dynamically schedules and combines multiple sub-models (i.e., “experts”) to complete complex tasks. It was first proposed by Jacobs et al. [26]. Its core idea is to build a gating network that dynamically allocates the participation weight of each expert according to the characteristics of the input sample to achieve task-oriented adaptive modeling [27,28,29]. The MoE architecture divides complex problems into several subtasks, which are modeled by different experts, and only activates some experts in the inference stage, taking into account both model expressivity and computational efficiency. It is one of the key technologies that has promoted the development of large-scale deep learning models in recent years [30,31,32,33]. Since Google proposed the Switch Transformer [34], MoE has made significant progress in many frontier fields such as natural language processing (NLP), computer vision (CV), and multi-modal learning [35,36,37,38].

In multi-modal learning, MoE offers unique advantages by facilitating flexible expert routing tailored to the heterogeneous nature of modalities. Previous works have explored its application in various contexts: MIMoE-FND [39] employs semantic consistency for expert selection in fake news detection, MoPE leverages prompt-level expert fusion to enhance training efficiency [40], and MoBA integrates lightweight adapters for modeling modality conflict in sarcasm detection [41]. However, these methods still face significant limitations when applied to sarcasm detection, a task that inherently requires fine-grained modeling of semantic incongruity and pragmatic tension. Specifically, (1) many existing MoE-based approaches rely on static or locally scoped gating mechanisms, lacking a global view of modality dominance, and (2) they do not explicitly model cross-modal semantic misalignment, which is central to identifying sarcastic content.

To address the above challenges, we introduce a novel dynamic Mixture-of-Experts (MoE) modeling paradigm for multi-modal sarcasm detection. Specifically, we design a fine-grained expert system consisting of four structurally unique expert types: textual self-attention experts, visual self-attention experts, text-guided cross-modal interaction experts, and image-guided cross-modal interaction experts. We develop a global dynamic gating mechanism that adaptively routes input representations to the most appropriate combination of experts based on the fused multi-modal features; this goes beyond the traditional MoE architecture, which relies on a unified expert structure and static gating. In addition, we augment the training objective with a regularization term derived from mismatched pairs (negative samples), which enhances the model’s ability to capture inter-modal sarcasm. This combination effectively reduces the model’s reliance on surface cues such as emojis and hashtags, leading to more robust sarcasm understanding.

3. Methodology

This section presents the architecture of our proposed multi-modal sarcasm detection framework, MM-MoE, detailing the task formulation, feature extraction, expert module design, dynamic gating mechanism, multi-modal fusion strategy, and loss functions.

Our model is based on the pre-trained CLIP framework [42], which provides initial semantic representations for both text and image inputs. These representations are projected into a shared latent space and then processed through a Mixture-of-Experts module composed of four expert types:

• Text Self-Attention Experts;
• Image Self-Attention Experts;
• Text-Guided Cross-Modal Experts;
• Image-Guided Cross-Modal Experts.

A global dynamic gating network assigns context-dependent weights to the outputs of all experts, and a dynamically fused representation is generated for the final prediction. The model is optimized using a classification loss, which is enhanced by a data augmentation strategy that treats mismatched pairs as negative samples. Figure 2 illustrates the overall workflow.

3.1. Task Definition

Given an image-text pair $\langle v_i,t_i\rangle$, where $v_i$ denotes the visual input and $t_i$ the corresponding text, the task is to predict a sarcasm label $y_i\in \{0,1\}$, where 1 represents sarcasm and 0 denotes a non-sarcastic instance. To obtain high-quality initial features, we adopt the pre-trained CLIP model, which encodes both image and text using a contrastive training paradigm, yielding powerful cross-modal alignment capabilities. Specifically, we extract

$$T\in {\mathbb{R}}^{d_t},I\in {\mathbb{R}}^{d_i}$$

where $T\in {\mathbb{R}}^{d_t}$ and $I\in {\mathbb{R}}^{d_i}$ represent the text and image embeddings, respectively. These are projected into a shared hidden space through learnable linear projections:

$$T\prime =W_{t}T,I\prime =W_{i}I$$

with $W_t\in {\mathbb{R}}^{d_h\times d_t}$, and $W_i\in {\mathbb{R}}^{d_h\times d_i}$ being trainable matrices and $d_h$ being the hidden size. The vectors $T$ and $I$ are fed into the subsequent MoE module.

3.2. Mixture-of-Experts Module

To precisely capture the diverse intra-modal and inter-modal semantic information, we designed a Mixture-of-Experts (MoE) module comprising multiple types of experts. The design of these four expert types provides a comprehensive toolkit for sarcasm detection. It includes two specialized uni-modal experts (Text Self-Attention and Image Self-Attention) to independently model the unique context within each modality. Crucially, to capture the often asymmetric nature of sarcastic incongruity, it also features two-directional cross-modal experts (Text-Guided and Image-Guided), allowing for the model to analyze how one modality reframes the other. Our framework is composed of four functionally distinct expert types, each built upon the Transformer attention mechanism.

The fundamental structure of each expert module includes a multi-head attention layer, residual connections with Layer Normalization, and a Feed-Forward Network (FFN). For the j-th expert, its operation on an input feature vector $x_j$ can be formally expressed as

$$\begin{array}{cc}\hfill {\mathrm{AttnOut}}_j& \hfill =\mathrm{LayerNorm}(x_j+\mathrm{MultiHeadAttn}(x_j,x_j,x_j))\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill E_j& \hfill =\mathrm{LayerNorm}({\mathrm{AttnOut}}_j+\mathrm{FFN}\left({\mathrm{AttnOut}}_j\right))\hfill \end{array}$$

(2)

In these equations, j is the expert index, $x_j$ is the input feature vector, ${\mathrm{AttnOut}}_j$ is the intermediate output from the attention sub-layer, and $E_j$ is the final output representation of the expert module. The MultiHeadAttn function is defined by the standard scaled dot-product attention mechanism, $\mathrm{Attn}(Q,K,V)=\mathrm{softmax}\left({\displaystyle \frac{Q{K}^T}{\sqrt{d_k}}}\right)V$, where Q, K, and V represent the Query, Key, and Value matrices, and $d_k$ is the dimensionality of the key vectors. The FFN consists of two linear layers with a GELU activation function. The inputs to the attention mechanism vary by expert type. For the self-attention experts, the input $x_j$ corresponds to a single modality’s feature ($T$ or $I$), which is used to derive Q, K, and V. For the cross-modal experts, the inputs are adapted to model inter-modal relationships, where the Query is derived from one modality and the Key and Value are derived from the other (i.e., operations such as $\mathrm{Attn}(T\prime ,I\prime ,I\prime )$ or $\mathrm{Attn}(I\prime ,T\prime ,T\prime )$ replace the self-attention block).

The four specific types of experts we designed are as follows:

• Text Self-Attention Experts: These experts focus on capturing the internal contextual dependencies and semantic relationships within the text modality. Their input $x_j$ is the projected text feature T′.
• Image Self-Attention Experts: These experts are used to capture global and local visual cues within the image modality. Their input $x_j$ is the projected image feature I′.
• Text-Guided Cross-Modal Experts: These experts aim to model how textual semantics attend to visual features to identify cross-modal incongruities.
• Image-Guided Cross-Modal Experts: Symmetrical to the previous type, these experts model how visual semantics attend to textual features to enrich their representations.

3.3. Dynamic Fusion via Gating and Attention

To adaptively integrate the outputs of various experts based on the characteristics of different input samples, we introduce a dynamic fusion mechanism. This mechanism consists of a global Dynamic Gating Network and a final Fusion Attention Layer.

Dynamic Gating Network: We first concatenate the projected text and image features to serve as the input to the gating network.

$$g_{in}=[T\prime ;I\prime ]\in {\mathbb{R}}^{2d_h}$$

(3)

We then use a lightweight multi-head self-attention layer to capture the contextual information within this fused input, followed by a Multi-Layer Perceptron (MLP) to predict a weight $g_j$ for each of the N experts. Specifically, the MLP consists of two linear layers with a GELU activation function in between. It projects the input feature z from a dimension of $2d_h$ to a hidden dimension of $d_h$, and then to the final output dimension N, which corresponds to the total number of experts.

$$\begin{array}{cc}\hfill z& \hfill =\mathrm{Attn}(g_{in},g_{in},g_{in})\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill [g_1,\dots ,g_N]& \hfill =\mathrm{Softmax}\left(\mathrm{MLP}\right(z\left)\right)\hfill \end{array}$$

(5)

Here, ${\sum }_{j=1}^N{g}_j=1$. These weights, $g_j$, reflect the importance of each expert for the current sample.

Multi-Modal Fusion Strategy: Instead of simply using the weighted expert outputs for classification, we designed a more robust fusion strategy, as depicted in Figure 2. First, we form a combined representation, ${\mathbf{F}}_{\mathrm{comb}}$, by concatenating the weighted outputs of all experts $E_j\in {\mathbb{R}}^{d_h}$:

$${\mathbf{F}}_{\mathrm{comb}}=\mathrm{Concat}(g_1{E}_1,g_2{E}_2,\dots ,g_N{E}_N)$$

(6)

To further capture high-order interactions among the different expert perspectives, we then pass $F_{comb}$ through a final Fusion Attention Layer (a standard multi-head self-attention module) to obtain the final multi-modal representation, $F_{final}$, for classification.

$$F_{final}=\mathrm{LayerNorm}({\mathbf{F}}_{\mathrm{comb}}+\mathrm{MultiHeadAttn}({\mathbf{F}}_{\mathrm{comb}},{\mathbf{F}}_{\mathrm{comb}},{\mathbf{F}}_{\mathrm{comb}}))$$

(7)

This design allows for the model not only to dynamically select experts, but also to deeply integrate the selected information, enhancing the representation’s robustness.

3.4. Training Objective with Negative Sampling Augmentation

The model is optimized using the standard binary cross-entropy loss, ${\mathcal{L}}_{cls}$. To enhance robustness and prevent the model from relying on spurious uni-modal cues, we introduce a data augmentation strategy based on in-batch negative sampling.

The core idea is to treat randomly mismatched image-text pairs as synthetic “non-sarcastic” samples. For each batch of real training data $\left\{(T_i,I_i,y_i)\right\}$, we construct a corresponding batch of augmented samples. A synthetic sample $(T_i,I_j)$ is created by pairing a text feature from one sample with an image feature from a different sample within the same batch (where $j\ne i$). Drawing on the idea of contrastive learning, these augmented pairs are assumed to be inherently non-sarcastic and are thus assigned a ground-truth label of 0.

Both the original batch and the augmented batch are passed through the model to obtain classification predictions. The total loss, ${\mathcal{L}}_{total}$, is a weighted sum of the classification loss on the real data (${\mathcal{L}}_{real}$) and the classification loss on the augmented data (${\mathcal{L}}_{aug}$):

$${\mathcal{L}}_{total}={\mathcal{L}}_{real}+\lambda ·{\mathcal{L}}_{aug}$$

(8)

where $\lambda$ is a hyperparameter to balance the contribution of the augmented data. Each loss component is calculated using the binary cross-entropy formula:

$${\mathcal{L}}_{cls}(y,\widehat{y})=-\left[ylog\left(\widehat{y}\right)+(1-y)log(1-\widehat{y})\right]$$

(9)

This multi-task training approach compels the model not only to correctly classify authentic sarcastic expressions, but also to recognize the lack of semantic connection in mismatched pairs, thereby learning a more robust cross-modal understanding.

4. Experiments

4.1. Datasets

We evaluate our model by constructing datasets targeting spurious cues based on two open source benchmark datasets for multi-modal sarcasm detection, MMSD [11] and MMSD2.0 [10]. The details of the original two datasets are summarized in

Table 1. Dataset statistics for MMSD and MMSD2.0.

Dataset	Label	Train	Val	Test
MMSD	Sarcasm	8642	959	959
	N-sarcasm	11,174	1451	1450
	All	19,816	2410	2409
MMSD2.0	Sarcasm	9576	1042	1037
	N-sarcasm	10,240	1368	1372
	All	19,816	2410	2409

. Both datasets contain paired image-text posts and are widely used in multi-modal sarcasm detection research.

• MMSD is constructed from English tweets, where sarcasm is weakly labeled via hashtags such as #sarcasm. It contains numerous spurious cues, such as emojis and hashtags, which may bias the learning process. And the author officially divided the training set, the validation set, and the test set.
• MMSD2.0 [10] is a refined version of MMSD, where superficial cues (e.g., emojis, hashtags) are systematically removed, aiming to better expose genuine semantic incongruity between modalities. At the same time, MMSD2.0 does not change the original division of MMSD, but only processes the text and changes some inappropriate labels, so there is no data leakage problem at all.

Although MMSD2.0 significantly reduces spurious correlations, it still retains residual label and lexical biases, making generalization challenging. To facilitate a rigorous analysis of model robustness, we first establish clear definitions based on these datasets. We define “noisy” data as samples from the original MMSD dataset, which contains numerous explicit spurious cues (e.g., hashtags like #sarcasm, emojis) that are highly correlated with the labels. Conversely, we define “clean” data as samples from MMSD2.0, where these specific textual cues have been systematically removed, compelling models to rely on genuine semantic incongruity. To comprehensively evaluate model robustness in realistic scenarios, we propose two cross-dataset evaluation setups:

• MMSD3.0: Trained and validated on MMSD (with noise), tested on MMSD2.0 (clean).
• MMSD4.0: Trained and validated on MMSD2.0 (clean), tested on MMSD (noisy).

These setups simulate real-world distributional shifts in the presence of spurious cues and assess the true generalization ability of the model.

4.2. Evaluation Metrics

Following previous studies [10], we use four standard metrics to evaluate the performance of sarcasm detection: Accuracy, Precision, Recall, and F1 Score. Among them, the F1 score is the most valuable in the case of class imbalance and can better reflect the model’s ability to distinguish between sarcastic and non-sarcastic content.

4.3. Baselines

To validate the effectiveness of the proposed MM-MoE framework, we conduct comprehensive comparisons against a diverse set of strong baseline models, categorized into three groups: text-only, image-only, and multi-modal fusion methods.

• Text-Only Modality:

  -

  TextCNN [43]: A Convolutional Neural Network-based model for sentence classification. It utilizes convolutional filters of varying sizes to extract n-gram features, and has shown competitive performance across various text classification tasks.

  -

  Bi-LSTM [44]: A bidirectional long short-term memory network that captures both past and future dependencies in text sequences. It is widely used to model contextual semantics in sentiment and sarcasm detection.

  -

  SMSD [45]: The Self-Matching and Low-Rank Bilinear Pooling model, specifically designed for sarcasm detection. It captures intra-sentence incongruities by computing rich word-to-word interactions, and is effective in modeling subtle ironic cues.

  -

  BERT [46]: A Transformer-based language representation model pre-trained on large corpora. We fine-tune BERT for sarcasm classification tasks, leveraging its strong contextual modeling ability to capture implicit sentiment conflicts within text.

• Image-Only Modality:

  -

  ResNet [47]: A residual Convolutional Neural Network pre-trained on ImageNet. It is used as a visual backbone to extract deep hierarchical features from input images, enabling sarcasm detection based solely on visual signals.

  -

  ViT [48]: Vision Transformer, which splits images into patches and applies multi-head self-attention to model long-range dependencies. It provides global visual context, and has demonstrated strong performance in many vision tasks.

• Multi-Modal Fusion Modality:

  -

  HFM [11]: The Hierarchical Fusion Model, one of the earliest works in multi-modal sarcasm detection. It performs multi-level fusion of text, image, and visual attribute features to improve sarcasm recognition.

  -

  CMGCN [13]: The Cross-Modal Graph Convolutional Network constructs a graph across textual and visual modalities, capturing intricate sarcasm-related relationships via structured cross-modal interactions.

  -

  DIP [23]: The Dual Incongruity Perceiving Network introduces a two-branch architecture to simultaneously detect semantic and emotional incongruities between text and image, significantly enhancing multi-modal sarcasm understanding.

  -

  DynRT [15]: A dynamic routing transformer network that adapts fusion pathways based on input characteristics. It aims to handle semantic mismatches by dynamically routing features through different sub-networks.

  -

  Multi-View CLIP [10]: A multi-perspective framework based on CLIP that integrates textual, visual, and interactional views. It aggregates complementary sarcastic cues from various perspectives for robust sarcasm detection.

  -

  DMSD-CL [49]: This paper introduces a novel framework for debiasing multi-modal sarcasm detection using contrastive learning. It constructs positive samples with dissimilar word biases and negative samples with similar biases via counterfactual data augmentation, enhancing the model’s robustness in OOD settings.

  -

  G²SAM [16]: The paper introduces a novel inference paradigm leveraging graph-based global semantic awareness for multi-modal sarcasm detectio. It is the first work to combine global semantic congruity with label-aware contrastive learning for multi-modal classification.

  -

  ESAM [25]: The paper proposes a multi-modal sarcasm detection model called Enhancing Semantic Awareness Model (ESAM), which addresses feature shift issues through Sentiment Consistency Constraint (SCC) and Automatic Outlier Masking (AOM), thereby enhancing the model’s capability to identify sarcastic sentiments in multi-modal data.

  -

  GPT-4o [50]: The multi-modal large language model released by OpenAI has powerful multi-modal capabilities and has achieved advanced performance in many fields.

  -

  Qwen [51]: Qwen is a multi-modal large language model released by Alibaba. Its performance in all aspects is at the forefront of the world, especially its multi-modal processing capabilities.

4.4. Experimental Setup

To ensure a fair and reproducible comparison, all baseline models were reproduced using their official code repositories with default hyperparameters and pre-trained weights, without any modification. Both our method and the baselines were implemented using PyTorch 1.13.1 [52]. Visual and textual features were extracted from the frozen CLIP-ViT-L/14-336 encoder provided by Huggingface [42].

Our model was trained using the AdamW optimizer [53] with an initial learning rate of $3\times {10}^{-5}$, weight decay of $1\times {10}^{-5}$, cosine annealing scheduler, and a batch size of 64. The training was conducted for up to 100 epochs, with early stopping based on validation F1 (patience = 10). We optimized the total loss function, which is a weighted sum of the classification loss on real data and an augmentation loss on mismatched pairs, as defined in Equation (8).

All experiments were performed on a single NVIDIA RTX 4090 GPU. Each result is averaged over ten random seeds, and we report the mean.

To assess the statistical significance of performance improvements, we conducted a two-tailed paired t-test between our model and the baseline models, with a significance level (alpha) set at 0.05. We specifically tested if the improvements observed on the F1 score are statistically significant.

4.5. Hyperparameter Analysis

To validate our parameter choices, we performed a hyperparameter analysis, focusing on the most critical component of our training objective: the weighting coefficient $\lambda$ for the augmentation loss, as defined in Equation (8). This hyperparameter controls the influence of the regularization term derived from in-batch negative sampling. Our experiments, varying $\lambda$ from 0.1 to 2.0, revealed that the model achieves optimal performance with around $\lambda =0.3$, as shown in Figure 3. This setting strikes the best balance between the primary classification objective on authentic pairs (${\mathcal{L}}_{real}$) and the regularization task of identifying mismatched pairs (${\mathcal{L}}_{aug}$), effectively guiding the model to learn a robust cross-modal understanding without destabilizing the training process. This empirically validated setting was used for all of the main experiments.

4.6. Results and Analysis

To comprehensively evaluate the robustness and generalization capabilities of our proposed model, we conducted extensive experiments on two challenging cross-dataset benchmarks, MMSD3.0 and MMSD4.0. The detailed performance comparison against a wide range of baseline models is presented in

Table 2. Experimental results of different types of baseline models on MMSD3.0 and MMSD4.0 datasets. The best results are highlighted in bold. Performance is reported as mean ± std (%) over ten runs.

Model	MMSD3.0				MMSD4.0
	Acc (%)	P (%)	R (%)	F1 (%)	Acc (%)	P (%)	R (%)	F1 (%)
Text-modality Methods
TextCNN	70.69	71.98	52.27	60.56	73.52	62.72	90.05	73.94
Bi-LSTM	60.91	66.04	60.56	63.18	75.51	67.05	81.19	73.45
SMSD	69.49	66.38	59.02	62.48	73.06	63.01	85.77	72.65
BERT	76.21	80.05	59.59	68.33	79.99	70.87	84.46	77.07
Image-modality Methods
ResNet	68.83	60.58	72.34	65.94	66.21	58.61	73.19	65.09
ViT	75.63	68.43	77.21	72.56	71.56	63.71	78.88	70.49
Multi-modality Methods
HFM	69.90	70.10	52.46	60.01	72.97	65.44	78.69	71.45
CMGCN	71.94	65.74	72.71	69.05	74.71	68.17	77.24	72.42
DIP	76.31	76.72	74.31	75.50	84.83	84.27	85.13	84.70
DynRT	57.20	62.94	50.35	55.95	70.29	70.23	71.26	70.74
DMSD-CL	77.96	75.10	73.00	74.03	81.94	80.61	71.95	76.03
G2SAM	74.93	69.28	75.02	72.04	82.98	77.42	80.81	79.08
Multi-View CLIP	80.99	77.55	78.59	78.07	83.35	86.69	83.35	84.99
ESAM	75.97	70.56	75.80	73.08	80.95	76.15	75.91	76.03
GPT-4o	72.69	59.49	66.75	62.91	70.61	68.67	54.47	60.75
Qwen	70.78	67.74	56.64	61.70	71.86	58.42	65.55	61.78
MM-MoE	$\mathbf{81.61}\mathbf{\pm }\mathbf{0.22}$	$\mathbf{82.44}\mathbf{\pm }\mathbf{0.28}$	$\mathbf{80.11}\mathbf{\pm }\mathbf{0.25}$	$\mathbf{81.26}\mathbf{\pm }\mathbf{0.25}$	$\mathbf{87.21}\mathbf{\pm }\mathbf{0.28}$	$\mathbf{87.09}\mathbf{\pm }\mathbf{0.25}$	$\mathbf{88.13}\mathbf{\pm }\mathbf{0.31}$	$\mathbf{87.61}\mathbf{\pm }\mathbf{0.27}$

.

Overall, our proposed model, MM-MoE (ours), achieves state-of-the-art performance on both benchmarks across all evaluation metrics, including Accuracy, Precision, Recall, and F1 score. This consistently superior performance validates the effectiveness of our adaptive multi-modal fusion framework based on the Mixture-of-Experts architecture.

• Performance on MMSD3.0 (Train on Noisy, Test on Clean): This benchmark is particularly challenging, as it directly assesses a model’s ability to learn genuine semantic incongruity from a training set rife with spurious cues (e.g., hashtags, emojis) and generalize to a clean test set devoid of them. As shown in the table, our model obtains an accuracy of 81.61% and an F1 score of 81.26%. This result significantly surpasses the next best performing model, Multi-View CLIP, which scored an F1 of 78.07%. The improvement of 3.19 percentage points in F1 score is not only substantial in magnitude, but is also statistically significant (p < 0.05, paired t-test), which strongly highlights our model’s ability to mitigate the influence of spurious correlations. In contrast, several baseline models, such as DynRT, exhibit a dramatic performance collapse on this task with an F1 score of only 55.95%, indicating their heavy reliance on superficial cues and the failure to generalize.
• Performance on MMSD4.0 (Train on Clean, Test on Noisy): This setting evaluates the model’s robustness when faced with distracting spurious cues at test time, after having been trained on a clean dataset. Our model once again demonstrates its superiority, achieving a top accuracy of 87.21% and an F1 score of 87.61%. This performance represents a significant margin over other strong multi-modal methods like DIP (F1 of 84.70%) and Multi-View CLIP (F1 of 84.99%). Crucially, the performance gap between our model and the strongest baseline (Multi-View CLIP) is statistically robust, as confirmed by our significance test (p < 0.05). This success suggests that, by learning from clean data, our model builds a robust understanding of sarcasm that is not easily perturbed by the reintroduction of noisy signals. The dynamic gating mechanism in our framework likely plays a key role in adaptively focusing on meaningful cross-modal interactions while down-weighting potentially misleading uni-modal cues.
• Overall Observations: A key trend observed across nearly all models is that performance on MMSD4.0 is considerably higher than on MMSD3.0. This suggests that generalizing from a noisy training distribution is a fundamentally harder problem than being robust to noise in the test distribution. It underscores the importance of clean training data or, in its absence, models like ours that are explicitly designed to handle spurious correlations. Furthermore, the results consistently show that multi-modal models outperform their uni-modal counterparts (Text-only and Image-only methods), reaffirming the necessity of leveraging information from both modalities for reliable sarcasm detection. In conclusion, the comprehensive results strongly validate the effectiveness, generalization ability, and robustness of our proposed MM-MoE model.

4.7. Ablation Study

To investigate the individual contributions of the key components within our MM-MoE framework, we conducted a series of ablation studies on both the MMSD3.0 and MMSD4.0 datasets. We systematically removed one component at a time from our full model (termed BASELINE) and observed the impact on performance. The results are detailed in

Table 3. Ablation study results on MMSD3.0 and MMSD4.0. Performance is reported as mean ± std (%) over ten runs. BASELINE refers to our full proposed model without any components removed.

Model	MMSD3.0		MMSD4.0
	Acc (%)	F1 (%)	Acc (%)	F1 (%)
BASELINE	$\mathbf{81.61}\mathbf{\pm }\mathbf{0.22}$	$\mathbf{81.26}\mathbf{\pm }\mathbf{0.25}$	$\mathbf{87.21}\mathbf{\pm }\mathbf{0.28}$	$\mathbf{87.61}\mathbf{\pm }\mathbf{0.27}$
w/o Text_Self	$79.49\pm 0.45$	$78.24\pm 0.48$	$85.97\pm 0.39$	$85.28\pm 0.42$
w/o Image_Self	$80.90\pm 0.35$	$80.03\pm 0.38$	$86.84\pm 0.31$	$86.38\pm 0.35$
w/o Cross_Text	$79.37\pm 0.48$	$77.90\pm 0.51$	$86.63\pm 0.35$	$86.27\pm 0.38$
w/o Cross_Image	$80.86\pm 0.38$	$80.02\pm 0.41$	$86.55\pm 0.36$	$85.37\pm 0.40$
w/o All Cross	$76.38\pm 0.61$	$72.01\pm 0.65$	$83.27\pm 0.52$	$79.84\pm 0.55$
w/o Gating_network	$79.57\pm 0.55$	$77.66\pm 0.59$	$86.38\pm 0.41$	$84.98\pm 0.45$
w/o Aug	$78.04\pm 0.65$	$76.80\pm 0.68$	$85.47\pm 0.49$	$84.02\pm 0.53$

. To maintain conciseness in the table, we use the following abbreviations to denote the removed modules: w/o Text_Self and w/o Image_Self refer to the Text and Image Self-Attention Experts; w/o Cross_Text and w/o Cross_Image denote the Text-Guided and Image-Guided Cross-Modal Experts; w/o Gating_network signifies the Dynamic Gating Network; and w/o Aug indicates the removal of the augmentation loss component (${\mathcal{L}}_{Aug}$). w/o All_Cross means the all cross-modal experts.

The experimental results clearly demonstrate that all proposed components are integral to the model’s final performance, as their removal invariably leads to a decrease in both F1 score and accuracy. We can draw several key insights from these results:

• The Importance of Core Mechanisms: The two most critical components of our framework are the augmentation loss (Aug) and the dynamic gating network (Gating_network). Removing the augmentation loss (w/o Aug) results in the most substantial performance drop, with the F1 score decreasing by 4.46% on MMSD3.0 and 3.59% on MMSD4.0. This underscores the vital role of the negative sampling strategy in forcing the model to learn genuine cross-modal semantic alignment, which is essential for generalizing beyond spurious cues. Similarly, removing the gating network (w/o Gating_network) also causes a significant decline, proving that the ability to dynamically weight and select experts based on input context is crucial for robust fusion.
• The Necessity of Cross-Modal Interaction: The removal of either the text-guided (w/o Cross_Text) or image-guided (w/o Cross_Image) cross-modal experts leads to a notable drop in performance. This is expected, as sarcasm often arises from the incongruity between modalities. The degradation in performance confirms that explicitly modeling these cross-modal relationships is a cornerstone of effective sarcasm detection. To further emphasize their collective importance, removing all cross-modal experts at once (w/o All_Cross) triggers the most severe performance drop in all of the expert ablations, as detailed in Table 3. This powerfully confirms that the model’s ability to capture inter-modal incongruity is the single most vital element of the framework.
• The Role of Uni-Modal Understanding: The ablation of uni-modal self-attention experts (w/o Text_Self and w/o Image_Self) also negatively impacts the results. This indicates that a thorough understanding of each modality’s internal context is a necessary prerequisite for subsequent, more complex cross-modal reasoning. Interestingly, the performance drop from removing the image self-attention expert (w/o Image_Self) is the least pronounced on the MMSD3.0 task. This suggests that, when trained on the noisy MMSD dataset, where textual cues might be more dominant or misleading, the model learns to rely less heavily on intra-modal visual information for generalization.

In summary, the ablation study systematically validates our design choices. Each component, from the specific expert types to the overarching augmentation strategy and dynamic gating mechanism, contributes meaningfully to the model’s ability to robustly detect sarcasm, especially in challenging cross-dataset scenarios.

4.8. Performance on Standard Benchmarks

To offer a more holistic evaluation of our proposed MM-MoE model, beyond validating its robustness on our constructed cross-dataset benchmarks (MMSD3.0 and MMSD4.0), we also conducted an in-domain evaluation on the two widely-used standard benchmarks: MMSD and MMSD2.0. This experiment aims to directly compare MM-MoE against existing state-of-the-art (SOTA) models in a traditional setting. The detailed results are presented in

Table 4. Performance comparison on the original MMSD and MMSD2.0 test sets. The best results are highlighted in bold. Performance is reported as mean ± std (%) over ten runs.

Model	MMSD				MMSD2.0
	Acc (%)	P (%)	R (%)	F1 (%)	Acc (%)	P (%)	R (%)	F1 (%)
GPT-4o	72.69	59.49	66.75	62.91	70.61	68.67	54.47	60.75
Qwen	70.78	67.74	56.64	61.70	71.86	58.42	65.55	61.78
HFM	83.44	76.57	84.15	80.18	70.57	64.84	69.05	66.88
CMGCN	87.55	83.63	84.69	84.16	79.83	75.82	78.01	76.90
DIP	88.50	84.59	87.82	86.18	80.92	77.46	78.43	77.94
Multi-View CLIP	88.33	82.66	88.65	85.55	85.64	80.33	88.24	84.10
DMSD-CL	88.95	84.89	87.90	86.37	81.74	82.74	80.22	81.46
G2SAM	91.07	88.27	90.09	89.17	79.43	78.07	72.04	74.93
ESAM	90.11	86.87	89.54	88.19	85.87	83.12	86.05	84.56
MM-MoE	$\mathbf{91.12}\mathbf{\pm }\mathbf{0.18}$	$\mathbf{90.82}\mathbf{\pm }\mathbf{0.21}$	$\mathbf{90.44}\mathbf{\pm }\mathbf{0.24}$	$\mathbf{90.63}\mathbf{\pm }\mathbf{0.21}$	$\mathbf{85.89}\mathbf{\pm }\mathbf{0.31}$	$\mathbf{85.59}\mathbf{\pm }\mathbf{0.35}$	$\mathbf{86.16}\mathbf{\pm }\mathbf{0.33}$	$\mathbf{85.87}\mathbf{\pm }\mathbf{0.32}$

.

• Analysis of MMSD: The MMSD dataset, known to be “noisy” due to its inclusion of numerous spurious cues like hashtags (for example, #sarcasm) and emojis, poses a unique challenge. As observed in Table 4, our MM-MoE model achieves a comprehensive lead on this dataset, with an Accuracy of 91.12% and an F1 score of 90.63%. Both key metrics surpass all existing baselines, demonstrating that, even in an environment with superficial shortcuts, the Mixture-of-Experts architecture and dynamic fusion mechanism of MM-MoE can efficiently learn and integrate multi-modal information to achieve superior sarcasm detection.
• Analysis of MMSD2.0: The MMSD2.0 dataset, a refined version of MMSD with spurious cues removed, provides a more authentic test of a model’s ability to capture semantic incongruity. On this “clean” dataset, our MM-MoE model once again achieves SOTA performance, with an F1 score of 85.87% and an accuracy of 85.89%. It is noteworthy that MM-MoE maintains a clear performance advantage over strong competitors like G²SAM (74.93% F1 score) and ESAM (84.56% F1 score). This strongly indicates that our model’s success is not reliant on spurious cues, but rather on a deeper understanding of the semantic relationships between text and images.
• Observation on Performance Shift from MMSD to MMSD2.0: A critical observation is that nearly all models, including MM-MoE, experience a performance drop when transitioning from MMSD to MMSD2.0. This universally validates the core motivation of our study: many models inadvertently learn to depend on spurious correlations. For instance, the DIP model’s F1 score falls sharply from 86.18% to 77.94%, indicating a strong reliance on such cues. In contrast, while MM-MoE’s performance also naturally declines (F1 from 90.63% to 85.87%), its ability to remain the top-performing model after the removal of spurious cues highlights the superiority of its design. The dynamic gating network and the augmentation loss objective likely contribute to its robustness, enabling it to adaptively focus on genuine cross-modal conflicts when obvious cues are absent.
  In summary, these in-domain experimental results provide a powerful complement to our cross-domain findings. MM-MoE not only excels at generalization tasks involving distribution shifts (MMSD3.0 & MMSD4.0), but also sets a new performance benchmark in standard in-domain evaluations, comprehensively validating its effectiveness and robustness as an advanced framework for multi-modal sarcasm detection.

4.9. Impact of Expert Number

To determine the optimal number of experts for our MM-MoE framework, we experimented with varying the number of experts and evaluated the model’s performance on both MMSD3.0 and MMSD4.0 datasets. The results of this experiment are summarized in Figure 4.

As illustrated in the figure, the model’s performance is sensitive to the number of experts employed. We observe a clear trend: the F1 score and accuracy on both datasets initially increase as the number of experts grows from one to three. Specifically, on MMSD3.0, the F1 score increases from 78.48% (one expert) to a peak of 81.26% (three experts). A similar pattern is evident on MMSD4.0, where the F1 score improves from 85.78% to a maximum of 87.61% with three experts. This initial improvement suggests that increasing the number of experts enhances the model’s capacity and provides a richer set of specialized functions. With more experts, the gating network has more diverse options to choose from, enabling it to learn more fine-grained and effective routing strategies for different samples.

However, a further increase in the number of experts beyond three leads to a noticeable decline in performance. For instance, increasing the experts from three to four causes the F1 score to drop by 3.42% on MMSD3.0 and 2.81% on MMSD4.0. This degradation indicates that an overly complex model with too many experts may lead to overfitting on the training data, especially given the dataset’s size. A larger number of experts can also make the optimization of the gating network more challenging, potentially resulting in suboptimal expert combinations and harming the model’s generalization ability.

Based on these empirical results, we conclude that setting the number of experts to three provides the best trade-off between model capacity and generalization performance. Therefore, we adopted this configuration for all other experiments reported in this paper.

4.10. Computational Efficiency Analysis and Occam’s Razor

In machine learning, the principle of Occam’s Razor [54] suggests that among competing hypotheses that predict equally well, the one with the fewest assumptions or the simplest structure should be selected. In the context of deep learning, the complexity of the model is often correlated with the number of trainable parameters. A model that achieves high performance with fewer parameters is not only more computationally efficient, but is often considered more robust and better at generalization. This section analyzes our proposed MM-MoE model from this perspective.

As presented in Figure 5, our MM-MoE model, with approximately 105.5 million trainable parameters, embodies this principle of simplicity and efficiency. This marks a significant reduction in complexity compared to many other high-performing multi-modal methods. For instance, our model is considerably more lightweight than complex architectures like CMGCN (195.6 M) and DIP (172.5 M). It also maintains a smaller parameter footprint than other strong baselines like Multi-View CLIP (139.4 M) and Att-BERT (139.7 M).

This efficiency is a direct result of the Mixture-of-Experts architecture. While the model contains a large pool of specialized expert modules, the dynamic gating network ensures that only a sparse subset of these experts is activated for any given input. This sparse activation paradigm is key: it allows for the MM-MoE to maintain a high model capacity and flexibility, capable of handling diverse inputs, without incurring the full computational cost associated with a single, monolithic dense model of similar size. The model is large in capacity, but light in execution.

In conclusion, our MM-MoE framework provides a compelling case for the application of Occam’s Razor. It not only achieves state-of-the-art performance in robust sarcasm detection, but does so with a more parsimonious and efficient design. This strikes an excellent balance between high performance and computational feasibility, making MM-MoE a powerful, practical, and elegant solution for multi-modal sarcasm detection.

4.11. Qualitative Analysis and Visualization

To gain a deeper, more intuitive understanding of MM-MoE’s internal mechanisms, we visualized the average activation weights assigned by the dynamic gating network to each expert. This analysis allows us to “look inside the black box” and observe how the model adapts its strategy when confronted with different distributions of spurious cues. We compare two challenging settings: MMSD3.0 (training on a noisy dataset, testing on a clean one) and MMSD4.0 (training on a clean dataset, testing on a noisy one). The distinct behavioral patterns, shown in Figure 6, provide compelling evidence for our model’s adaptive reasoning capabilities.

• Analysis of MMSD3.0 (Train Noisy → Test Clean): When the model is trained on the noisy MMSD dataset, which is rife with spurious correlations (e.g., hashtags, emojis), it learns a “defensive” strategy. The dynamic gate assigns significantly higher weights to uni-modal experts, particularly Image_Self_3 (average weight 0.154), while suppressing the cross-modal experts. This behavior suggests that, during training, the model discovers that relying on the relationship between modalities can be misleading due to noise. Instead, it learns to distrust the cross-modal signals and latches onto more reliable, albeit superficial, patterns within a single modality as predictive shortcuts. This reliance on isolated features is a classic symptom of models exposed to spurious cues, and, while it shows adaptability during training, it also explains why generalizing from a noisy dataset is fundamentally challenging.
• Analysis of MMSD4.0 (Train Clean → Test Noisy): In stark contrast, when MM-MoE is trained on the clean MMSD2.0 dataset, it is forced to learn the genuine semantic and emotional incongruity that defines sarcasm. Having built this robust, “semantically-grounded” understanding, its behavior at test time on noisy data becomes highly insightful. As shown in Figure 6, the model dynamically and significantly increases the weights for the cross-modal experts (Cross_Text and Cross_Image). This strategic shift is crucial: when faced with distracting signals within the text or image, the model intelligently focuses its resources on analyzing the interaction between the modalities. It essentially learns to “cut through the noise” by prioritizing the cross-modal relationship, which it knows is the most reliable path to identifying true sarcastic incongruity. This is not merely a static fusion strategy; it is an active, input-dependent adaptation that serves as direct evidence of the model’s robustness.
• Case Study on Nuanced Sarcasm: To provide a more granular insight beyond average weights, we present a case study on the sarcastic example shown in Figure 7. The input consists of an image of a snowy owl, as shown in Figure 7a and the text: “snowy owl came too close to traffic camera and he is being ticketed for not reading the signs in #french”. Here, the sarcasm arises not from a direct visual-semantic conflict, but from a commonsense violation. Interestingly, the expert weight distribution in Figure 7b reveals that the gating network assigns the highest weights to uni-modal experts (both Image Self-attention and Text Self-attention) and relatively low weights to the cross-modal experts. This suggests a sophisticated reasoning process: the model first dedicates resources to deeply understand the image’s real-world context and the text’s internal absurdity independently. Only after forming strong uni-modal representations can the model detect the higher-level, commonsense incongruity in their combination. This case vividly illustrates our model’s flexibility in handling nuanced sarcasm where robust uni-modal understanding is a prerequisite to revealing the irony.

5. Conclusions and Future Work

This paper addresses the critical challenge of robust multi-modal sarcasm detection, where models often fail by relying on spurious cues rather than understanding true semantic incongruity. We introduced MM-MoE, a novel Mixture-of-Experts framework designed to overcome this limitation. At its core, MM-MoE employs a synergistic approach: a global dynamic gating mechanism adaptively routes multi-modal inputs to specialized experts (uni-modal and cross-modal), while a regularization loss based on negative sampling explicitly guides the model to learn genuine cross-modal alignment. To rigorously evaluate this capability, we proposed a new cross-dataset evaluation protocol, instantiated as the MMSD3.0 and MMSD4.0 benchmarks, which simulate real-world distributional shifts. Extensive experiments demonstrate that MM-MoE achieves state-of-the-art performance, significantly outperforming existing models in generalizing from and to noisy data, thereby proving its superior robustness.

Despite these advances, we acknowledge several limitations in our current work, which in turn paves the way for exciting future research directions.

First, the most significant limitation lies in the model’s reliance on self-contained information. Our framework, like most existing methods, operates as a closed system and lacks access to external commonsense knowledge. This restricts its ability to comprehend sarcasm that requires a deep understanding of real-world context or cultural nuances. For instance, identifying the irony in a statement like “Enjoying my spacious new mansion” paired with an image of a cramped studio apartment requires knowledge beyond the text–image pair itself. A promising future direction is to integrate external knowledge sources, such as knowledge graphs or Large Language Models (LLMs), to equip the model with the commonsense reasoning capabilities necessary for these more complex instances.

Second, the scope of generalization, while a core focus of this work, can be further expanded. Our evaluation is primarily conducted on the Twitter-originated MMSD dataset family. The model’s performance on other domains with distinct linguistic styles and content formats, such as product reviews, news forums, or other social media platforms, remains an open question. Furthermore, our analysis of robustness is centered on distributional shifts caused by spurious cues. Another critical dimension, adversarial robustness, has not been explored. Investigating the model’s resilience to subtle, maliciously crafted perturbations in text or images would be a crucial step towards developing more secure and reliable systems.

Finally, while our visualization of expert weights offers a valuable glimpse into the model’s decision-making process at a modular level, the internal workings of each expert remain a “black box”. Future work could pursue a more fine-grained interpretability, aiming to pinpoint which specific textual words and visual regions contribute most significantly to the detection of semantic conflict. This would not only enhance the transparency of the model, but also provide deeper insights into the nature of multi-modal sarcasm itself.

To facilitate these future endeavors, our code, data setups, and experimental configurations will be made publicly available.