Task-Adaptive and Multi-Level Contextual Understanding for Emotion Recognition in Conversations

Abstract

Emotion recognition in conversations (ERC) is a significant task in natural language processing, aimed at identifying the emotion of each utterance within a conversation. Current research predominantly relies on pre-trained language models, often incorporating sophisticated network architectures to capture complex contextual semantics in conversations. However, existing approaches have not successfully combined effective task-specific adaptation with adequate modeling of conversational context complexity. To address this, we propose a model named TAMC-ERC (Task-Adaptive and Multi-level Contextual Understanding for Emotion Recognition in Conversations). The model adopts a progressive recognition framework that sequentially builds on foundational utterance representations, integrates conversation-level contexts, and leads to a task-adaptive classification decision. First, the Task-Adaptive Representation Learning module produces highly discriminative utterance representations. It achieves this by integrating emotion space information into prompts and employing contrastive learning. Subsequently, the Multi-Level Contextual Understanding module performs in-depth modeling of the conversational context. It synergistically integrates both macroscopic narratives and microscopic interactions to construct a comprehensive emotional context. Finally, the classifier is directly parameterized by the emotion concept vectors from the task-adaptive stage. This creates a coherent task adaptation process, maintaining task-specific awareness from representation learning through to the final decision. Experiments on three benchmark datasets demonstrate that TAMC-ERC achieves highly competitive performance: it attains weighted average F1 scores of 71.04% on IEMOCAP, 66.95% on MELD, and 40.99% on EmoryNLP. These results set a new state of the art and demonstrate that the model outperforms most existing baselines. This work validates that integrating task adaptation with multi-level contextual modeling is key to addressing conversational complexity and improving recognition accuracy.

1. Introduction

Emotion recognition in conversations (ERC) has emerged as a significant task in natural language processing, aiming to automatically identify the emotional state of each utterance within a conversation. This technology finds broad applications in intelligent customer service, social media analysis, and human–computer interaction.

Early research primarily focused on modeling temporal dependencies using recurrent neural networks, as seen in DialogueRNN [1]. To more effectively capture complex inter-utterance interactions, graph-structured models were subsequently developed. For instance, DialogueGCN [2] represents utterances as graph nodes, utilizing various edge types to model speaker relationships. COGMEN [3] and DAG-ERC [4] employed graph Transformers and directed acyclic graphs, respectively, to handle global/local semantics and spatiotemporal structures. Concurrently, advanced learning objectives emerged to improve utterance representations. Representative works include Cog-BART [5], which integrates contrastive learning with BART [6], and SACL [7], which employs adversarial training for more robust representations. With pre-trained language models (PLMs) becoming the cornerstone technology, how to effectively adapt them to capture conversational emotional dynamics has emerged as a new focus.

Despite considerable progress, existing approaches face two fundamental challenges. First, pre-trained language models (PLMs) possess no intrinsic knowledge of the task-specific emotion space. This fundamental gap prevents them from performing accurate, task-aware semantic reasoning for emotion recognition. Second, existing approaches struggle to balance the macroscopic narrative flow and microscopic interactions in conversations, leading to an incomplete and superficial construction of the emotional context. Therefore, these two challenges define a progressive research problem. The first challenge concerns the acquisition of high-quality foundational utterance representations, while the second addresses the refinement of these representations within the complete conversational context.

Accordingly, this work aims to develop a new framework that addresses two progressive challenges. The first is the adaptation of PLMs to the task’s emotion space for obtaining high-quality foundational utterance representations. The second involves the design of a context encoder capable of comprehending and integrating both macroscopic narrative flow and microscopic interaction patterns to construct a complete and in-depth emotional context. Crucially, these two challenges are interlinked: high-quality task-adaptive representations provide a solid foundation for context modeling, while a comprehensively understood context delivers the necessary informational basis for final emotion classification. Therefore, a synergistic framework that addresses both in concert is required.

To this end, this paper proposes TAMC-ERC, advancing ERC research from both task adaptation and complex contextual understanding perspectives. Specifically, the model first processes utterances through a task-adaptive module that injects the dataset’s emotion space into the PLM, yielding high-quality representations. A multi-level contextual module then synergistically integrates macroscopic narratives and microscopic interactions to build a comprehensive emotional context. Finally, a classifier, parameterized by emotion concept vectors from the task-adaptive stage, ensures systematic task adaptation from feature learning to the final decision.

The main contributions of this work are as follows:

(1)

A task-adaptive representation learning mechanism that enhances PLM’s adaptability to specific emotion tasks through emotion space-constrained prompt learning and contrastive fine-tuning;

(2)

A multi-level contextual understanding architecture designed to systematically handle the macroscopic narrative flow and microscopic interactions in conversations;

(3)

Extensive experiments on benchmark datasets demonstrate that TAMC-ERC achieves highly competitive performance, with ablation studies confirming a synergistic effect between its core components.

2. Related Work

This section reviews previous research closely related to our work. We first summarize major approaches for context modeling in emotion recognition in conversations, including sequence-based, graph-based, and knowledge-fusion methods. We then review advances in contrastive learning, which is highly relevant to our proposed framework.

2.1. Emotion Recognition in Conversations

2.1.1. Sequence-Based Models

Early deep learning approaches for ERC predominantly leveraged sequential models. Majumder et al. [1] proposed DialogueRNN, which employs multiple RNNs to track dialogue states and speaker dynamics. Subsequent extensions include Cog-BART [5], which integrates a generative pre-trained model with contrastive learning, and DialogueEIN [8], which explicitly models emotional inertia and stimuli. To enhance robustness, Hu et al. [7] proposed an adversarial sample-learning framework (SACL), while Yang et al. [9] developed a modality-interaction framework (SCMM) to address semantic limitations. Furthermore, Yang et al. [10] explored a holistic curriculum-learning paradigm, and Lei et al. [11] and Zhang et al. [12] investigated instruction tuning for LLMs on the ERC task. Lee et al. [13] also studied learning from soft labels. However, the performance of these sequential models can be limited in scenarios requiring the modeling of complex, non-sequential dependencies between distant utterances and intricate speaker interactions [2,3]. This inherent challenge motivates the exploration of graph-structured representations.

2.1.2. Graph-Based Models

To explicitly model intra- and inter-speaker dependencies, graph neural networks have been widely adopted. Ghosal et al. [2] introduced DialogueGCN, constructing conversation graphs with distinct edge types. Joshi et al. [3] presented COGMEN, a hybrid architecture interleaving GNNs with graph-aware Transformers. Shen et al. [4] proposed DAG-ERC to model spatiotemporal dynamics via directed acyclic graphs, while Jia et al. [14] developed SIGAT, a unified heterogeneous graph fusing speaker and temporal information. Zhang et al. [15] introduced DualGATs to jointly exploit discourse structure and speaker context. For multimodal scenarios, Hu et al. [16] proposed MMGCN, and recently, Tu et al. [17] developed AdaIGN, which dynamically selects informative nodes and edges. A common characteristic of these graph-based methods is their reliance on predefined and relatively static graph architectures. While effective, this design may present a limitation in adapting to the unique and dynamically changing contextual demands of diverse conversational tasks.

2.1.3. Knowledge-Enhanced Models

Recognizing that human emotion is often grounded in common sense, another line of research incorporates external knowledge. Zhong et al. [18] introduced KET to augment representations with external common sense. Ghosal et al. [19] presented COSMIC, distilling speaker-specific common sense. Zhu et al. [20] devised TODKAT, fusing common sense with topic information, and Fu et al. [21] proposed CONSK-GCN for multimodal knowledge integration. Li et al. [22] introduced SKIER to model discourse relations with symbolic knowledge, Xie et al. [23] developed KI-Net with an auxiliary intensity prediction task, and Tu et al. [24] presented MKE-IGN for multi-knowledge enhancement. These methods demonstrate the benefits of incorporating external knowledge. However, this integration also introduces new considerations, such as computational overhead, potential noise, and the challenge of seamless fusion with the internal conversational context.

2.2. Contrastive Learning for ERC

Contrastive learning has recently gained traction for learning discriminative utterance representations by leveraging the relational nature of emotions [25,26]. Gunel et al. [27] proposed Supervised Contrastive Learning (SupCon), and Kang et al. [28] introduced KCL to address class imbalance. Li et al. [29] devised TSC to guide compact clustering, while Zhu et al. [30] and Zhang et al. [31] incorporated learnable prototypes and label embeddings into the contrastive loss, respectively. In conversational scenarios, Tu et al. [32] presented CKCL, a context-knowledge contrastive framework, and Yu et al. [33] introduced emotion-anchored contrastive learning (EACL). Notably, the contrastive objectives in these methods are often based on fixed heuristics for constructing positive and negative pairs. This fixed strategy struggles to capture the nuanced and task-specific relational structures in diverse ERC scenarios, highlighting the need for more adaptive contrastive mechanisms.

In summary, prior work has established robust paradigms for contextual modeling in ERC. A common observation, however, is the reliance on fixed architectural designs for capturing context, regardless of the specific conversation task. This reveals the need for a framework capable of synergistically and adaptively modeling both the macroscopic narrative flow and microscopic interaction patterns in conversations, and integrating this adaptability into its contextual modeling and learning objective.

2.3. Task-Adaptive Methods for ERC

Task-adaptive methods have emerged as a significant strategy in emotion recognition in conversations (ERC), with the goal of bridging the semantic gap between general-purpose pre-trained language models (PLMs) and the specific demands of the ERC task. These methods integrate task-specific knowledge (e.g., emotion category systems and conversational dynamics) during model training. By doing so, they guide PLMs to learn representations that are more discriminative for emotion recognition.

Existing task-adaptive approaches for ERC can be broadly categorized into three main strands. First, prompt-based methods, exemplified by Sentiprompt [34] and the framework of Cai et al. [35], utilize carefully designed, emotion-aware templates to reformat the input. This directs the model’s focus toward emotion-relevant aspects of the dialogue. However, the static nature of such prompts may limit their ability to fully model dynamic emotional transitions across conversation turns. Second, contrastive learning frameworks enhance adaptation by introducing contrastive objectives during representation learning. EACL [33], for instance, constructs emotion-anchored vectors to cluster representations of the same emotion class. A potential limitation is that its task adaptation is somewhat decoupled from the downstream classification objective. SCCL [36] employs a three-dimensional (speaker, content, and sequential) representation to construct contrastive pairs. Yet, its heuristic-based sample construction may not fully capture the rich semantic relationships within the emotion space. Although SupCon-ERC [27] effectively applies a supervised contrastive loss to ERC, its design lacks an explicit, built-in mechanism for task adaptation, which could impact generalization across diverse conversational settings. Third, knowledge-enhanced methods, such as TODKAT [20] and KI-Net [23], incorporate external common sense or topical knowledge to enrich the dialogue‘s semantic context. This integration, however, typically increases computational complexity and introduces the challenge of seamlessly combining external knowledge with the intrinsic conversational context.

In summary, a key limitation of existing methods lies in their compartmentalized adaptation mechanisms, which are often not coherently integrated throughout the learning pipeline. Furthermore, the structured semantics of emotion categories are rarely systematically utilized to inform the model’s learning objective. These gaps highlight the need for a unified and synergistic adaptive framework that can achieve coherent task adaptation while explicitly leveraging the structured emotion space. Addressing these gaps is therefore key to advancing towards more theoretically grounded and empirically robust solutions in ERC.

3. Problem Definition

We formalize a conversation as comprising an utterance sequence and a corresponding speaker sequence:

$$\begin{array}{l}U= \{u_1,u_2,\dots ,u_n\}\\ \mathrm{S}= \{s_1,s_2,\dots ,s_n\}\end{array}$$

(1)

where each $u_i$ is uttered by speaker $s_i$. Here, the conversation involves $\mathrm{m}$ distinct speakers ($\mathrm{m}$ ≤ $\mathrm{n}$), with each $s_i$ representing one unique speaker identity.

The emotion recognition in conversations (ERC) task requires predicting the emotion label ${\mathrm{L}}_{\mathrm{e}}^{\mathrm{i}}$ for every utterance $u_i\in U$. Each label ${\mathrm{L}}_{\mathrm{e}}^{\mathrm{i}}$ is drawn from the dataset’s emotion space E, which defines all admissible emotion categories—for instance, E = {neutral, happy, angry, sad, frustrated, excited} in the IEMOCAP benchmark.

For ease of explanation, relevant notations used in this article are summarized in

Table 1. Mathematical notations.

Symbol	Description
$\mathbf{U}$	A conversation, composed of multiple utterances.
$\mathbf{S}$	The set of speakers in a conversation.
$\mathbf{E}$	The emotion space of the dataset, for instance, E = {neutral, happy, angry, sad, frustrated, excited} in the IEMOCAP benchmark.
${\mathbf{e}}_{\mathbf{i}}$	The emotion concept vector corresponding to an emotion category in ${\mathrm{E}}_{\mathrm{r}}$.
${\mathbf{E}}_{\mathbf{r}}$	The set of all emotion concept vectors, {${\mathrm{e}}_{1,}{\mathrm{e}}_2$, …,${\mathrm{e}}_{\mathrm{n}}$}.
$\mathbf{p}$	A prompt template.
${\mathbf{M}\mathbf{L}\mathbf{P}}_{\mathbf{c}\mathbf{e}}$	The projection head for contrastive learning.
${\mathbf{d}}_{\mathbf{r}}$	The representation of an utterance.
${\mathcal{Z}}_{\mathbf{r}}$	The projected utterance representation for contrastive learning.
$\mathsf{\tau }$	Temperature hyperparameter.
$\mathbf{s}\mathbf{i}\mathbf{m}\left({\mathit{z}}_{\mathit{i}},{\mathit{z}}_{\mathit{j}}\right)$	Cosine similarity function.
${\mathbf{W}}_{\mathbf{t}}$	A local context window centered on the t-th utterance.
${\mathbf{C}}_{\mathbf{t}}^{\mathbf{s}\mathbf{e}\mathbf{l}\mathbf{f}}$	Historical utterances of the target speaker (up to time t).
${\mathbf{C}}_{\mathbf{t}}^{\mathbf{i}\mathbf{n}\mathbf{t}\mathbf{e}\mathbf{r}}$	Adjacent utterance pairs within the local context.

.

4. The TAMC-ERC Model

To tackle the core challenges in Emotion Recognition in Conversations defined in Section 3 (namely, the semantic gap between pre-trained language models and the task-specific emotion space, and the effective integration of global coherence and local interactions in conversations), this paper presents task-adaptive and multi-level contextual modeling (TAMC-ERC). The proposed framework is designed to achieve precise transformation from general semantic understanding to task-specific emotion discrimination through an end-to-end, progressive learning architecture. Specifically, the framework first injects emotion concepts into utterance representations via task-adaptive representation learning. It then constructs a comprehensive emotional context that synergistically integrates macroscopic narratives and microscopic interactions. Finally, a classifier parameterized by the emotion concept vectors transforms the refined representations into emotion predictions. The following sections detail the design rationale and technical specifications of each core component.

4.1. Model Framework Overview

To systematically address the progressive research problem identified in Section 1, this paper presents the TAMC-ERC model (illustrated in Figure 1). The model follows a “utterance–context–classification” architecture: it first learns foundational representations at the utterance level, then advances to contextual understanding at the conversation level, and finally achieves task-adaptive classification. Through this architecture, TAMC-ERC systematically tackles the two challenges outlined in the Introduction via three core components.

First, the Task-Adaptive Representation Learning module (Section 4.2) addresses the alignment between the PLM and the emotion space. Operating at the utterance level, this module leverages emotion space-constrained prompt encoding and contrastive fine-tuning to generate a high-quality foundational representation for each utterance that is highly discriminative within the emotion space. The output of this stage serves as a critical semantic foundation for the entire system and is fed into subsequent modules.

Subsequently, the Multi-Level Contextual Understanding module (Section 4.3) tackles the integration of global coherence and local interactions in conversations. Operating at the conversation level, this module processes information from two perspectives: the Global Contextual Understanding captures the macroscopic narrative structure and long-range dependencies via a bidirectional GRU, while the Local Contextual Understanding captures fine-grained interactive dynamics by modeling the local semantic field, speaker state trajectories, and direct interaction adjacency. The fusion of their outputs provides rich contextual information for each utterance, resulting in a contextually refined representation.

Finally, the Emotion Recognition and Optimization module (Section 4.4) performs the classification. A key design of this module is to share the emotion concept vectors with the task-adaptive stage and use them to parameterize the classifier. This ensures the task adaptation from representation learning to the final decision. The entire model is trained end-to-end by combining classification and contrastive learning objectives.

In conclusion, the TAMC-ERC model combines three core components: task-adaptive representation learning, multi-level contextual understanding, and a classifier parameterized by emotion concept vectors, thereby establishing a progressive reasoning paradigm for conversational emotion recognition.

4.2. Task-Adaptive Representation Learning

This module addresses the core challenge of adapting general-purpose pre-trained language models (PLMs) to the emotion recognition in conversations (ERC) task, where PLMs’ lack of intrinsic knowledge of the structured emotion space $\mathrm{E}$ hinders their discriminative capability. The proposed solution is a task-adaptive learning module that operates at the utterance level, explicitly leveraging the emotion space $\mathrm{E}$ as a semantic constraint to specialize PLM representations for ERC. The module operates in two cohesive stages: (1) task-adaptive prompt context encoding (Section 4.2.1) achieves an initial, static adaptation by incorporating $\mathrm{E}$ directly into the input; (2) task-adaptive contrastive fine-tuning (Section 4.2.2) builds upon this foundation to address the persistent ambiguity between semantically close emotion categories in the initially adapted representations, dynamically refining them using $\mathrm{E}$.

4.2.1. Task-Adaptive Prompt Context Encoding

Standard prompt-based methods often fall short in ERC due to their unrestricted output space, which is ill-suited for predicting a fixed set of emotions. To bridge this gap, this paper proposes a task-adaptive prompt that explicitly integrates the task dataset’s emotion space E as a semantic constraint, effectively closing the PLM’s output space.

Following the standard offline ERC benchmark, we construct the prompt using only the left context of the target utterance. This design is supported by empirical findings across prompt-based ERC studies, which show that emotional cues are predominantly triggered by preceding utterances, whereas subsequent utterances may contain sarcasm or emotional flip, acting as label noise (Li et al. [34], 2021; Cai et al. [35], 2025 findings). Accordingly, for a target utterance ${\mathrm{u}}_{\mathrm{t}}$, its historical context (comprising all preceding utterances and speakers) is utilized. This modeling approach aligns with conversational dynamics and ensures model practicality for real-time applications. Based on this historical context, the input sequence $x_t$ is constructed by appending a structured prompt $\mathrm{P}$:

$$x_t=\left[u_1,s_1, \dots ,u_t,s_t;P\right]$$

(2)

Here, $u_1,s_1, \dots ,u_t,s_t$ denote the sequence of utterances and speakers in the historical context, and $\mathrm{P}$ is the task-adaptive prompt. $\mathrm{P}$ is formulated as “Emotions in this conversation are confined to the set $\mathrm{E}$. For utterance $u_t$, speaker $s_t$ feels the emotion $\left[\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{k}\right]$”. In our experiments, $\mathrm{E}$ is instantiated with the emotion label set of the specific benchmark dataset, e.g., {happy, sad, angry, …} for IEMOCAP. This design provides the PLM with explicit knowledge of valid emotion categories, establishing task-adaptive semantic constraints that significantly reduce classification ambiguity for conversational emotion recognition.

The initial task-aware representation for the target utterance is derived from the hidden state of the $\left[\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{k}\right]$ token:

$${\mathcal{Z}}_{\left[mask\right]}= {PLM\left(x_t\right)}_{\left[mask\right]}$$

(3)

Here, ${\mathcal{Z}}_{\left[\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{k}\right]}\in {\mathbb{R}}^{\mathrm{d}}$ denotes the contextualized embedding for a single target utterance, serving as the initial task-aware representation for subsequent ERC steps. For a batch of $\mathrm{b}$ utterances, we obtain ${\mathcal{Z}}_{\left[\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{k}\right]}\in {\mathbb{R}}^{\mathrm{b}\times \mathrm{d}}$ where $\mathrm{b}$ is the batch size, and $\mathrm{d}$ is the hidden size of the PLM (i.e., 1024 for Sup-SimCSE-RoBERTa-large).

4.2.2. Task-Adaptive Contrastive Fine-Tuning

Building on the preliminary utterance-level representations from Section 4.2.1, this stage aims to enhance their discriminative power. Although the task-adaptive prompt encoding provides preliminary task awareness, its static nature results in representations that are often insufficient for fine-grained emotional distinctions (e.g., between ‘anger’ and ‘frustration’). To overcome this, we introduce a task-adaptive contrastive fine-tuning module. This component directly leverages the emotion space $\mathrm{E}$ of the task dataset through contrastive learning. The objective is to transform the initial task-aware representations into ones that are highly discriminative within the target emotion space.

To project the representations into a space optimized for similarity comparison, the representation ${\mathcal{Z}}_{\left[\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{k}\right]}$ is transformed into a lower-dimensional embedding space using a multi-layer perceptron (MLP):

$${\mathcal{Z}}_{\mathrm{r}}={\mathrm{M}\mathrm{L}\mathrm{P}}_{\mathrm{c}\mathrm{e}}\left({\mathcal{Z}}_{\left[mask\right]}\right)$$

(4)

where ${\mathrm{M}\mathrm{L}\mathrm{P}}_{\mathrm{c}\mathrm{e}}$ denotes the projection layer for contrastive learning, and ${\mathcal{Z}}_{\mathrm{r}}\in {\mathbb{R}}^{\mathrm{b}\times {\mathrm{d}}_{\mathrm{r}}}$ represents the refined utterance representations for the batch, with ${\mathrm{d}}_{\mathrm{r}}$ being the dimension of the contrastive embedding space.

To ground the contrastive-learning process, we first generate a concept vector for each emotion category in the task-specific set E. This vector is obtained by an independent MLP projection layer (${\mathrm{M}\mathrm{L}\mathrm{P}}_r$) that operates on the output of the PLM. Importantly, the PLM remains fully frozen throughout training:

$${\mathrm{E}}_{\mathrm{r}}={\mathrm{M}\mathrm{L}\mathrm{P}}_r\left(\mathrm{P}\mathrm{L}\mathrm{M}\left(E\right)\right)$$

(5)

where ${\mathrm{E}}_{\mathrm{r}}\in {\mathbb{R}}^{\left|\mathrm{E}\right|\times {\mathrm{d}}_{\mathrm{r}}}$ represents the emotion concept vectors, and $\left|\mathrm{E}\right|$ is the number of emotion categories. Subsequently, to construct a unified set for contrastive learning, we combine the utterance representations ${\mathcal{Z}}_{\mathrm{r}}$ with the emotion concept set ${\mathrm{E}}_{\mathrm{r}}$. Formally, this combined set is constructed as follows:

$${\mathcal{Z}}_a= torch.cat\left(\right[{\mathcal{Z}}_r,E_r],dim=0)$$

(6)

$$Y_a=torch.cat\left(\right[Y,Y_e],dim=0)$$

(7)

Here, ${\mathcal{Z}}_{\mathrm{a}}\in {\mathbb{R}}^{(\mathrm{b} + |\mathrm{E}\left|\right)\times {\mathrm{d}}_{\mathrm{r}}}$ denotes the concatenated representation matrix, where $\mathrm{b}$ is the batch size, $\left|\mathrm{E}\right|$ is the number of emotion categories, and $\mathrm{a}$ is equal to $\mathrm{b} + \left|\mathrm{E}\right|$. Y$\in {\mathbb{R}}^{\mathrm{b}}$ contains the ground-truth emotion labels for the $\mathrm{b}$ utterances in the batch, and ${\mathrm{Y}}_{\mathrm{e}}\in {\mathbb{R}}^{\left|\mathrm{E}\right|}$ contains the predefined labels for each vector in the emotion concept set ${\mathrm{E}}_{\mathrm{r}}$.

The supervised contrastive loss is then computed on this unified set ${\mathcal{Z}}_{\mathrm{a}}$. The pairwise similarity between two representations $z_i$ and $z_j$ in ${\mathcal{Z}}_{\mathrm{a}}$ is calculated as

$${\mathrm{c}}_{\mathrm{i}\mathrm{j}}={\displaystyle \frac{\mathrm{s}\mathrm{i}\mathrm{m}\left(z_i,z_j\right)}{\mathsf{\tau }}}$$

(8)

where $\mathrm{s}\mathrm{i}\mathrm{m}\left(z_i,z_j\right)$ denotes the cosine similarity function, and $\mathsf{\tau }$ is a temperature hyperparameter that controls the concentration level of the distribution.

The supervised contrastive loss ${\mathcal{L}}_{\mathrm{N}\mathrm{C}\mathrm{E}}$ on the unified set ${\mathcal{Z}}_{\mathrm{a}}$ is formulated as

$${\mathcal{L}}_{NCE}=-{\displaystyle \frac{1}{\left|Z_i^{+}\right|}}\sum _{i=1}^{a}log{\displaystyle \frac{{\sum }_{z_j\in Z_i^{+}}{ e}^{c_{ij}}}{{\sum }_{z_j\in {\mathcal{Z}}_a}{ e}^{c_{ij}}}}$$

(9)

where $Z_i^{+}=\{z_j\in {\mathcal{Z}}_a|{\mathrm{y}}_{\mathrm{j}}={\mathrm{y}}_{\mathrm{i}},\mathrm{j}\ne \mathrm{i}\}$ denotes the set of positive samples (i.e., those sharing the same emotion label), for instance $\mathrm{i}$, while the denominator sums over all samples in the unified set ${\mathcal{Z}}_{\mathrm{a}}$ except $\mathrm{i}$ itself. This loss enhances the model’s discriminative power by attracting representations of the same emotion class closer in the latent space, while repelling those from different classes.

To preserve the model’s direct classification capability and provide a complementary supervisory signal to contrastive learning, a standard cross-entropy classification loss is jointly optimized. The emotion prediction distribution and the loss are formulated as follows:

$$\widehat{y}=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\left({\mathrm{M}\mathrm{L}\mathrm{P}}_{\mathrm{c}\mathrm{s}}\left({\mathcal{Z}}_r\right)\right)$$

(10)

$${\mathcal{L}}_{CE}=-{\displaystyle \frac{1}{b}}\sum _{i=1}^b\sum _{j=1}^{\left|\mathrm{E}\right|}{\mathcal{Y}}_{ij }\mathrm{l}\mathrm{o}\mathrm{g}{\widehat{\mathcal{Y}}}_{ij}$$

(11)

where ${\mathrm{M}\mathrm{L}\mathrm{P}}_{\mathrm{c}\mathrm{s}}$ is the classification head, $\widehat{\mathrm{y}}\in {\mathbb{R}}^{\mathrm{b}\times \left|\mathrm{E}\right|}$ is the predicted probability distribution over the $\left|\mathrm{E}\right|$ emotion classes for the batch, and ${\mathcal{Y}}_{\mathrm{i}\mathrm{j}}$ is the ground-truth label (1 if sample $\mathrm{i}$ belongs to class $\mathrm{j}$ and 0 otherwise).

The final optimization objective combines both learning signals to achieve a balance between discriminative feature learning and accurate classification. To further sharpen the decision boundaries within the emotion space $\mathrm{E}$, a separation loss ${\mathcal{L}}_{\mathrm{E}}$ is applied to the emotion concept vectors, which minimizes the maximum pairwise cosine similarity among them. This loss promotes a more dispersed distribution of vectors in the feature space by minimizing the maximum pairwise cosine similarity. Thus, the overall task-adaptive loss is defined as the weighted sum:

$${\mathcal{L}}_E={\mathrm{m}\mathrm{a}\mathrm{x}}_{\begin{array}{c}i,j\in \left\{\mathrm{1,2},\dots ,\left|E\right|\right\}\\ i\ne j\end{array}}\mathrm{s}\mathrm{i}\mathrm{m}\left(e_i,e_j\right)$$

(12)

$${\mathcal{L}}_{\mathrm{T}\mathrm{A}}={\mathsf{\lambda }}_{\mathrm{C}}{\mathcal{L}}_{CE}+{\mathsf{\lambda }}_{\mathrm{N}}{\mathcal{L}}_{\mathrm{N}\mathrm{C}\mathrm{E}}+{\mathsf{\lambda }}_{\mathrm{E}}{\mathcal{L}}_{\mathrm{E}}$$

(13)

where ${\mathsf{\lambda }}_{\mathrm{C}},{\mathsf{\lambda }}_{\mathrm{N}},{\mathsf{\lambda }}_{\mathrm{E}}\in \left[\mathrm{0,1}\right]$ are the loss weighting coefficients that control the relative importance of the classification, contrastive learning, and emotion space separation objectives, satisfying ${\mathsf{\lambda }}_{\mathrm{C}}+{\mathsf{\lambda }}_{\mathrm{N}}+{\mathsf{\lambda }}_{\mathrm{E}}=1.$

In summary, this component performs task-adaptive refinement of the representations through fine-tuning. Building upon the prompt encoding, it generates final utterance representations ${\mathcal{Z}}_{\mathrm{r}}$ that are highly discriminative within the emotion space $\mathrm{E}$ of the task dataset. These refined representations are both context-aware and serve as a robust input for the subsequent ERC module.

4.3. Multi-Level Contextual Understanding

Accurate recognition of conversational emotion inherently depends on integrating both the global conversation flow and local inter-utterance interactions. To this end, this module employs a cascaded architecture designed to model these dual dynamics concurrently. The global module first encodes the entire conversation sequence to model the macro-level emotional flow. Subsequently, guided by this global context, the local module analyzes fine-grained interactions between adjacent utterances. This progressive, from-macro-to-micro comprehension ensures that the emotion of each utterance is derived from both its overall narrative position and its immediate conversational dynamics.

4.3.1. Global Contextual Understanding Module

Accurate emotion recognition in conversations requires understanding each utterance within its global conversational context, as emotions often evolve through long-range cross-utterance dependencies. To this end, this module leverages the strength of gated recurrent units (GRUs) in sequence modeling to capture such dependencies. It feeds the utterance representation sequence $\mathrm{X}=[{\mathrm{x}}_1,{\mathrm{x}}_2,\dots ,{\mathrm{x}}_{\mathrm{n}}]$ from the first stage into a bidirectional GRU:

$$\begin{array}{c}\overrightarrow{{\mathrm{h}}_t}=\overrightarrow{\mathrm{G}\mathrm{R}\mathrm{U}}({\mathrm{x}}_t,\overrightarrow{{\mathrm{h}}_{t-1}})\\ \overleftarrow{{\mathrm{h}}_t}=\overleftarrow{\mathrm{G}\mathrm{R}\mathrm{U}}({\mathrm{x}}_t,\overleftarrow{{\mathrm{h}}_{t+1}})\\ {\mathrm{h}}_t^g=\left[\overrightarrow{{\mathrm{h}}_t;}\overleftarrow{{\mathrm{h}}_t}\right],\mathrm{t}=1,2,\cdots , \mathrm{n}\end{array}$$

(14)

Here, $h_t^g$ denotes the global-contextualized representation of the t-th utterance, which encodes the macroscopic narrative structure of the conversation. This module outputs a sequence of global-contextualized representations ${\mathrm{H}}^{\mathrm{g}}$ = [${\mathrm{h}}_1^{\mathrm{g}}$, ${\mathrm{h}}_2^{\mathrm{g}}$, …, ${\mathrm{h}}_{\mathrm{n}}^{\mathrm{g}}$]. This sequence serves as the input to the Local Contextual Understanding module, providing a representation foundation enriched with global context for subsequent fine-grained analysis. Unlike the left-context prompt that generates an initial emotion-aware representation for each utterance, the bidirectional GRU refines these representations by exploiting long-range dependencies across the entire conversation.

4.3.2. Local Contextual Understanding Module

Building upon the macroscopic context from the global module, this module aims to capture the diverse factors driving local emotional dynamics. Its design is grounded in the principle that utterance emotion is systematically influenced by the local “topic atmosphere,” the speaker’s “own state,” and the immediate “conversation partner”. Accordingly, the module employs a hierarchical modeling strategy, constructing local contexts of varying scopes to quantify these three key factors, respectively.

We begin by constructing a local context window of radius r around the global representation $h_t^g$ of the t-th utterance. This window, extracted from the global representation sequence ${\mathrm{H}}^{\mathrm{g}}$, forms the basic unit for analysis.

$$W_t^g=\left[h_{t-r}^g,\dots ,h_t^g,\dots ,h_{t+r}^g\right]$$

(15)

Here, this window ${\mathrm{W}}_{\mathrm{t}}^{\mathrm{g}}$ contains the global-contextualized representations of the utterances most relevant to the current one, providing the foundational data for local context understanding.

(1)

Foundational Context: Local Semantic Field

The local window ${\mathrm{W}}_{\mathrm{t}}^{\mathrm{g}}$ provides the complete semantic environment nearest to the current utterance and establishes the local topical background for emotional expression.

(2)

Focused Context: Speaker State Trajectory

To capture a speaker’s emotional state continuity, we define a binary speaker mask ${\mathrm{M}}^{\mathrm{s}\mathrm{e}\mathrm{l}\mathrm{f}}$. Its elements ${\mathrm{M}}_{\mathrm{i}\mathrm{j}}^{\mathrm{s}\mathrm{e}\mathrm{l}\mathrm{f}}$ are set to 1 if speaker $s_j$ is the target speaker $s_t$ and 0 otherwise. Applying this mask to the local window ${\mathrm{W}}_{\mathrm{t}}^{\mathrm{g}}$ isolates the speaker’s state trajectory ${\mathrm{C}}_t^{\mathrm{s}\mathrm{e}\mathrm{l}\mathrm{f}}$, as formalized in Equation (16).

$$M_{ij}^{\mathrm{s}\mathrm{e}\mathrm{l}\mathrm{f}}=\left\{\begin{array}{c}1\\ 0\end{array}\right. {\displaystyle \genfrac{}{}{0pt}{}{\mathrm{i}\mathrm{f} s_j=s_t}{\mathrm{o}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{w}\mathrm{i}\mathrm{s}\mathrm{e}}}, {\mathrm{C}}_t^{\mathrm{s}\mathrm{e}\mathrm{l}\mathrm{f}}=M^{\mathrm{s}\mathrm{e}\mathrm{l}\mathrm{f}} ⨀{ W}_t^g$$

(16)

where $s_j$ and $s_t$ are the speaker identities of the j-th and t-th utterances, respectively. This operation yields a sequence ${\mathrm{C}}_{\mathrm{t}}^{\mathrm{s}\mathrm{e}\mathrm{l}\mathrm{f}}$ containing only the target speaker’s utterances, aimed at modeling individual emotional inertia.

(3)

Core Context: Direct Interaction Adjacency Pair

To capture the most direct emotional interaction in conversation, we apply an adjacency mask to extract utterance pairs that constitute direct responses to the current utterance within a local window:

$$M_{ij}^{\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r}}=\left\{\begin{array}{c}1\\ 0\end{array}\right.{\displaystyle \genfrac{}{}{0pt}{}{ if u_j is direct response to u_t}{otherwis\mathrm{e}}}, {\mathrm{C}}_t^{\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r}}=M^{\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r}} ⨀{ W}_t^g$$

(17)

This operation yields the crucial adjacency pair ${\mathrm{C}}_{\mathrm{t}}^{\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r}}$, which is designed to model the most direct emotional stimulus and response relationship.

To integrate the contextual features, the module processes three distinct sequences with separate LSTMs: the local window ${\mathrm{W}}_{\mathrm{t}}^{\mathrm{g}}$, the speaker-state sequence ${\mathrm{C}}_{\mathrm{t}}^{\mathrm{s}\mathrm{e}\mathrm{l}\mathrm{f}}$, and the adjacency-pair sequence ${\mathrm{C}}_{\mathrm{t}}^{\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r}}$. This is formalized in Equation (18):

$${\mathrm{h}}_t^{\mathrm{c}\mathrm{t}\mathrm{x}}=\mathrm{L}\mathrm{S}\mathrm{T}\mathrm{M}\left({ W}_t^g\right), {\mathrm{h}}_t^{\mathrm{s}\mathrm{e}\mathrm{l}\mathrm{f}}=\mathrm{L}\mathrm{S}\mathrm{T}\mathrm{M}\left({\mathrm{C}}_t^{\mathrm{s}\mathrm{e}\mathrm{l}\mathrm{f}}\right), {\mathrm{h}}_t^{\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r}}=\mathrm{L}\mathrm{S}\mathrm{T}\mathrm{M}\left({\mathrm{C}}_t^{\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r}}\right)$$

(18)

By processing the full local context, speaker-specific trajectory, and adjacency pairs through separate LSTMs, the module extracts their respective temporal features ${\mathrm{h}}_{\mathrm{t}}^{\mathrm{c}\mathrm{t}\mathrm{x}}, {\mathrm{h}}_{\mathrm{t}}^{\mathrm{s}\mathrm{e}\mathrm{l}\mathrm{f}}$, and ${\mathrm{h}}_{\mathrm{t}}^{\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r}}$.

Finally, the outputs of the LSTMs are concatenated to form the final local fusion representation for the i-th utterance:

$$h_t^l=\left[{\mathrm{h}}_t^{\mathrm{c}\mathrm{t}\mathrm{x}};{\mathrm{h}}_t^{\mathrm{s}\mathrm{e}\mathrm{l}\mathrm{f}};{\mathrm{h}}_t^{\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{e}\mathrm{r}}\right]\in {\mathbb{R}}^{3d}$$

(19)

where $\mathrm{d}$ denotes the dimensionality of a single utterance representation. This fused representation $h_t^l$ systematically integrates the three types of key contextual information: the local semantic field, the speaker state trajectory, and the direct interaction adjacency pair. The resulting representation thereby forms a fine-grained, multifaceted local context that provides crucial micro-level dynamic evidence, complementing the macro-level cues from the global context for final emotion determination.

4.4. Emotion Recognition and Optimization

This module performs the final emotion classification prediction through the systematic integration of the outputs of the preceding modules. It leverages both the context-enriched utterance representations from the multi-level context understanding module (Section 4.3) and the emotion concept vectors, which capture the core semantics of each emotion category, derived from the task-adaptive contrastive learning (Section 4.2).

To achieve direct alignment with the task-specific semantic space, we construct the final classifier by directly parameterizing it with the emotion concept vectors ${\mathrm{E}}_{\mathrm{r}}$. For a context-rich utterance representation ${\mathrm{h}}_i$, this classifier assigns the label of the most similar concept vector ${\mathrm{e}}_j\in {\mathrm{E}}_{\mathrm{r}}$. This design explicitly grounds the classification in the semantic space established during task adaptation. Formally, for a given utterance representation ${\mathrm{h}}_i$, the classification is performed by computing its similarity to all emotion concept vectors, as follows:

$$\widehat{y_i}=\mathrm{a}\mathrm{r}\mathrm{g}\underset{j\in \left\{1,\dots ,C\right\}}{\max }\sigma ({\mathrm{h}}_i,{\mathrm{e}}_j)$$

(20)

Here, ${\mathrm{e}}_j\in {\mathrm{E}}_{\mathrm{r}}$ denotes the emotion concept vector for the j-th emotion category, and $\mathsf{\sigma }({\mathrm{h}}_i,{\mathrm{e}}_j)$ is the cosine similarity function. This design ensures that the classification logic stems directly from the task-adaptive emotion concepts, rather than from traditional, data-agnostic model parameters.

The entire model is optimized end-to-end by minimizing the cross-entropy loss between the predictions and the ground-truth emotion labels:

$${\mathcal{L}}_{cls}=-{\displaystyle \frac{1}{N}}\sum _{i=1}^N\sum _{j=1}^C{1}_{\left[y_i=j\right]}\log \left(p_i^j\right)$$

(21)

where $\mathrm{N}$ is the batch size, $1_{\left[y_i=j\right]}$ is the indicator function, and $p_i^j$ is the probability distribution obtained by applying the Softmax function to the similarity scores.

5. Experiments

5.1. Datasets

In this work, we evaluate our model on three publicly available benchmarks for emotion recognition in conversations (ERC). The details of the datasets are as follows:

(1)

IEMOCAP Dataset [37]: This is a multimodal dataset. Each dialogue segment is performed by two actors based on scripts. Only the text modality is used in this experiment. The data samples in this dataset involve 6 emotion categories: happy, sad, angry, frustrated, excited, and neutral. Considering that there is no validation set in the IEMOCAP dataset, following the practice in DAG-ERC, we split the data into a training set and a validation set, with the last 20 dialogues held out for validation.

(2)

MELD Dataset [38]: This is a multi-party, multi-modal dialogue dataset derived from the TV show Friends. Different from IEMOCAP, this dataset encompasses seven emotion categories: anger, disgust, fear, happiness, neutral, sadness, and surprise.

(3)

EmoryNLP Dataset [39]: Similar to MELD, EmoryNLP is based on Friends, involving multi-party dialogues and containing 7 categories. However, EmoryNLP differs from MELD in scene selection, and the set of 7 emotion categories it defines differs from that of MELD, namely happy, powerful, peaceful, sad, crazy, fear, and neutral.

The statistical analysis of the above datasets is shown in

Table 2. Key statistics of the benchmark conversation datasets for ERC.

Dataset	Total Number of Conversations	Number of Emotion Categories	Average Number of Speakers per Conversation	Average Number of Clauses per Conversation
IEMOCAP	151	6	2	49.2
MELD	1433	7	9	9.6
EmoryNLP	897	7	9	14.1

. The variations in conversation structure and emotion categories across these datasets facilitate a comprehensive evaluation of model generalizability.

5.2. Experimental Setup

Hyperparameters were searched on the validation set of each dataset (

Table 3. Hyperparameters of TAMC-ERC for three benchmark datasets.

Hyperparameters	IEMOCAP	MELD	EmoryNLP
Learning rate	1 × 10−4	1 × 10−4	1 × 10−4
Dropout	0.2	0.2	0.2
Temperature	0.2	0.3	0.2
Maximum length	256	256	256
Batch size	32	32	32
Epochs	8	6	6

). Additionally, all experiments were conducted on a single RTX 4090 GPU manufactured by NVIDIA Corporation, based in Santa Clara, CA, USA, and the reported results are the average of five random runs on the test set.

5.3. Baseline Models

For a comprehensive evaluation, our approach is compared against the following baseline models:

(1)

Graph-based Models

DialogueGCN (Ghosal et al., 2019) [2] adopts GCN to capture contextual features for learning utterance representations. RGAT (Ishiwatari et al., 2020) [40] introduces relational position encoding to model speaker relationships and sequential information. DAG-ERC (Shen et al., 2021) [4] exploits an acyclic graph neural network to intuitively characterize the inherent structure of conversations without incorporating external information. DAG-ERC+HCL (Yang et al., 2022) [10] strengthens DAG-ERC by integrating a curriculum learning strategy, which trains the model on samples arranged from easy to difficult. SIGAT (Jia et al., 2023) [14] encodes speaker and sequence information into a unified graph to capture their mutual interactive effects. AdaIGN (Tu et al., 2024) [17] proposes an adaptive interaction graph network (IGN), which employs the Gumbel Softmax technique to adaptively select nodes and edges, thereby enhancing intra-modal and cross-modal interactions.

(2)

Sequence-based Models

Cog-BART (Li et al., 2022) [5] utilizes BART combined with contrastive learning for the joint tasks of emotion recognition and response generation. DialogueEIN (Liu et al., 2022) [8] includes emotion interaction and tendency modules to explicitly model emotional inertia and stimuli. CoMPM (Lee and Lee, 2021) [41] leverages pre-trained models to directly acquire contextual information and track conversation history. SupCon (Gunel et al., 2020) [27] is a supervised contrastive learning loss function. SCCL (Yang et al., 2023) [36] conducts contrastive learning using three-dimensional influence representations. DIEU (Zhao et al., 2023) [42] aims to resolve the issue of long-distance context propagation. CKCL (Tu et al., 2023) [32] filters out context- and knowledge-irrelevant noise during the training process. MPLP (Zhang et al., 2023) [43] models speakers’ historical experiences and employs paraphrasing to amplify the differences between labels. Emocaps (Li et al., 2022) [44] redesigns the Transformer into a novel architecture called the Bistable Transformer to extract the emotional tendency of utterances. SACL (Hu et al., 2023) [7] learns robust representations by fusing contrastive learning with adversarial training. SPCL+CL (Song et al., 2022) [45] integrates prototype contrastive learning and curriculum learning to address the problem of imbalanced emotion classes. ChatGPT-3.5 (Zhao et al., 2023) [46], whose 3-shot performance is reported as a reference. EACL (Yu et al., 2024) [33] is proposed using an emotion-anchored contrastive learning framework for discriminative utterance representations for similar emotions.

(3)

Knowledge-Enhanced Models

KET (Zhong et al., 2019) [18] is a knowledge-enhanced Transformer framework that dynamically utilizes external common sense knowledge and attention mechanisms to model and understand dialogue emotions. COSMIC (Ghosal et al., 2020) [19] incorporates various commonsense elements and leverages them to learn self-speaker dependencies. TODKAT (Zhu et al., 2021) [20] integrates commonsense knowledge with dialogue topic information to enhance adaptability across multiple scenarios. KI-Net (Xie et al., 2021) [23] combines multi-task learning for emotion polarity intensity prediction to strengthen knowledge interaction.

6. Experimental Results and Analysis

To comprehensively validate the design principles of the TAMC-ERC model, this section presents evaluations through three systematic experiments: We first establish its overall performance advantage by comparing with baseline models, then analyze the contribution of each core component through ablation studies, and finally, we investigate the synergistic working mechanism among key modules.

6.1. Overall Performance Comparison

Table 4. Performance comparison between TAMC-ERC and various baseline models.

Methods	IEMOCAP	MELD	EmoryNLP	Average
Graph-based models
DialogueGCN (Ghosal et al., 2019) [2]	64.91	63.02	38.10	55.34
RGAT (Ishiwatari et al., 2020) [40]	66.36	62.80	37.89	55.68
DAG-ERC (Shen et al., 2021) [4]	68.03	63.65	39.02	56.90
DAG-ERC+HCL (Yang et al., 2022) [10]	68.73	63.89	39.82	57.48
SIGAT (Jia et al., 2023) [14]	70.71	66.20	39.95	58.77
AdaIGN (Tu et al., 2024) [17]	70.74	66.79	-	-
Sequence-based models
CKCL (Tu et al., 2023) [32]	67.16	66.21	40.23	57.87
Cog-BART (Li et al., 2022) [5]	66.18	64.81	39.04	56.68
DialogueEIN (Liu et al., 2022) [8]	68.93	65.37	38.92	57.74
CoMPM (Lee and Lee., 2021) [41]	69.46	66.52	38.93	58.30
SupCon (Gunel et al., 2020) [27]	68.14	65.63	39.28	57.68
Emocaps (Li et al., 2022) [44]	69.49	63.51	-	-
SPCL+CL (Song et al., 2022) [45]	67.19	65.74	39.52	57.48
SACL (Hu et al., 2023) [7]	69.22	66.45	39.65	58.44
SCCL (Yang et al., 2023) [36]	69.88	65.70	38.75	58.11
DIEU (Zhao et al., 2023a) [42]	69.90	66.43	40.12	58.11
MPLP (Zhang et al., 2023) [43]	66.65	66.51	-	-
ChatGPT 3-shot (Zhao et al., 2023) [46]	48.58	58.35	35.92	47.62
EACL (Yu et al., 2024) [33]	70.41	67.12	40.24	59.26
Knowledge-Enhanced Models
KET (Zhong et al., 2019) [18]	59.56	58.18	33.95	50.56
COSMIC (Ghosal et al., 2020) [19]	65.25	65.21	38.11	56.19
TODKAT (Zhu et al., 2021) [20]	61.33	65.47	43.12	56.53
KI-Net (Xie et al., 2021) [23]	66.98	63.24	-	-
TAMC-ERC (ours)	71.04	66.95	40.99	59.66

presents the weighted average F1 scores of TAMC-ERC and baseline models on three benchmark datasets.

The experimental results demonstrate that TAMC-ERC achieves competitive performance across all datasets, obtaining the best results on IEMOCAP and performing comparably to advanced baselines on MELD and EmoryNLP. This robust performance highlights the efficacy of our architecture in handling conversational complexity and enabling multi-level contextual understanding. This is achieved through two key designs: (1) the Task-Adaptive Representation Learning module, which explicitly aligns the model with the task-specific semantic space via emotion concept vectors; and (2) the cascaded Global and Local Contextual Understanding modules, which systematically capture long-range dependencies and fine-grained interactions. The synergy between these components collectively contributes to accurate and robust emotion recognition.

The observed performance variations across datasets align with their inherent complexity levels. On IEMOCAP, a dyadic dataset with well-defined speaker roles, TAMC-ERC demonstrates clear advantages. This result confirms the effectiveness of the intra-speaker modeling mechanism in the Local Contextual Understanding (LCU) module for such scenarios. Although all models exhibit performance degradation on the more challenging multi-speaker datasets (MELD and EmoryNLP), TAMC-ERC maintains robust performance, comparable to state-of-the-art approaches like SIGAT [14] and EACL [33].

To statistically validate the comparison with the strongest baseline, paired t-tests were conducted between TAMC-ERC and EACL. The improvement across all three datasets was not statistically significant (p = 0.149). However, a focused test on IEMOCAP and EmoryNLP, where TAMC-ERC showed superior performance, yielded a significant result (p = 0.0037). This indicates that while EACL is a highly competitive benchmark, our model’s integrated architecture delivers robust and significant gains where its design is most effective.

Notably, the knowledge-enhanced model TODKAT [20] showed strong performance on EmoryNLP, likely benefiting from external commonsense knowledge integration. In contrast, TAMC-ERC achieved competitive results without external knowledge. This demonstrates that architectural innovations in task adaptation and multi-level contextual modeling are, by themselves, sufficient to substantially advance conversational emotion recognition. Furthermore, the results were consistent across multiple independent training runs, indicating robustness to random initialization. Regarding computational efficiency, TAMC-ERC introduced minimal overhead compared to standard PLM fine-tuning. The additional components—the contrastive loss ${\mathcal{L}}_{\mathrm{N}\mathrm{C}\mathrm{E}}$, the emotion separation loss ${\mathcal{L}}_{\mathrm{E}}$, and the lightweight context encoders—contributed to a negligible parameter increase (<2% of the PLM’s size). Consequently, the inference speed remained practically unchanged, ensuring the model’s feasibility for real-world applications.

6.2. Ablation Experiments

Ablation studies were conducted to validate the contribution of each core component in the TAMC-ERC framework. The experimental design corresponds to the two main innovations: the task-adaptive mechanism and multi-level contextual understanding.

Table 5. Ablation experimental results of the TAMC-ERC model on each dataset.

Dataset	IEMOCAP	MELD	EmoryNLP
Ours	71.04	66.95	40.99
w/o Task-Adaptive Contrastive Learning	68.77 (2.27 ↓)	66.2 (0.75 ↓)	40.7 (0.29 ↓)
w/o Prototype-Parameterized	69.45 (1.59 ↓)	63.24 (3.71 ↓)	36.61 (4.38 ↓)
w/o Global Contextual Understanding	70.84 (0.20 ↓)	66.79 (0.16 ↓)	40.81 (0.18 ↓)
w/o Local Contextual Understanding	70.8 (0.24 ↓)	66.82 (0.13 ↓)	40.77 (0.22 ↓)
w/o Global-Local Model	69.64 (1.4 ↓)	66.26 (0.69 ↓)	38.92 (2.07 ↓)

↓ indicates the amount of decrease.

shows the weighted average F1 scores on three benchmark datasets, where “↓” indicates performance degradation.

We first evaluated the task-adaptive mechanism. Disabling the contrastive learning component led to performance degradation of 2.27% on IEMOCAP and 0.75% on MELD. This decline underscores the module’s critical role in acquiring discriminative semantic representations. Furthermore, removing the emotion concept vectors from the classification layer caused more substantial performance drops (3.71% on MELD and 4.38% on EmoryNLP). Critically, this ablation severs the direct link between the adapted concept vectors and the decision layer. The consequent performance drop confirms that using ${\mathrm{E}}_{\mathrm{r}}$ as classifier parameters contributes to robustness. This advantage is likely achieved by preserving decision boundaries aligned with the contrastively structured embedding space. These findings collectively demonstrate that the task-adaptive integration of emotion concepts across both representation learning and classification is a cornerstone of the model’s effectiveness.

The ablation of either the global or local contextual modules led to modest but consistent performance degradation (averaging 0.18% and 0.20%, respectively). To thoroughly investigate their synergistic contribution and address potential concerns regarding the statistical significance of these improvements, we conducted an additional ablation where the global–local integration mechanism is removed. This ablation results in substantially larger performance drops: 1.40% on IEMOCAP, 0.69% on MELD, and 2.07% on EmoryNLP. Paired t-tests between the full model and this ablated variant confirm statistical significance (one-tailed p = 0.037, t = 3.475, df = 2), indicating over 95% confidence in the improvement. This stark contrast reveals that while individual modules have limited impact due to functional complementarity, their integrated modeling is essential for holistic conversation comprehension. The global module captures narrative-level flow and long-range dependencies, while the local module refines turn-level interaction dynamics. Their synergy provides the foundational context required by the task-adaptive mechanism, validating the multi-level contextual architecture as a core component of our framework.

6.3. Analysis of Synergistic Effects

To validate the synergistic effects between the global and local contextual modules in TAMC-ERC, we conducted a controlled experiment. We replaced the standard global-local interaction mechanism with a local-only variant, which processes adjacent utterances via LSTM without the global GRU-based context modeling. A performance comparison between these two architectures across three datasets is presented in Figure 2.

Experimental results indicate that the global–local interaction mechanism consistently outperforms the local-only baseline across all datasets. This performance gap occurs because modeling long-range emotional evolution across conversation turns is essential. However, the local-only interactions prove insufficient for this task, as they primarily capture surface-level semantic associations between adjacent utterances. The synergistic effect is achieved precisely through the distinct and complementary roles of these two modules. The Local Contextual Understanding module captures fine-grained details like speaker identity and adjacent relationships, providing crucial cues for refining the emotion concept vectors. In parallel, the Global Contextual Understanding module encodes the overarching conversation semantics, thereby setting the directional constraints that ensure all local adjustments remain coherent with the primary emotional trajectory.

This collaborative mechanism proves particularly important in semantically complex multi-party conversations. For instance, in a multi-party debate scenario, local interactions capture immediate rebuttals among participants, while the global context frames the broader emotional progression, such as a shift from “debate → banter → reconciliation”. This enables the dynamic adjustment of emotion representations based on local cues while ensuring their consistency with the global emotional narrative.

7. Conclusions

This work addressed two pivotal challenges in conversational emotion recognition: bridging the semantic gap between general-purpose pre-trained models and specialized emotion tasks and synergistically modeling global and local contextual information. Our proposed TAMC-ERC model tackles these through a synergistic integration of task-adaptive and multi-level contextual modeling. The model follows a progressive cognitive pipeline. First, the Task-Adaptive Representation Learning module injects the structure of the emotion space via prompts and contrastive learning, yielding highly discriminative utterance representations. Subsequently, the Multi-level Contextual Understanding module systematically integrates macro-level conversation flow and micro-level interactive dynamics through cascaded global and local modeling. Finally, the classifier is parameterized by the learned emotion concept vectors, ensuring semantic alignment with the task-specific space. Extensive experiments confirm TAMC-ERC’s competitive advantage, with ablation studies underscoring the indispensable role and superadditive synergy of both core components. Together, these findings establish the deep integration of task adaptation and multi-level contextual modeling as an effective paradigm for conversational emotion recognition.

However, experimental analysis shows that TAMC-ERC still faces challenges in comprehending complex scenarios within multi-party conversations (e.g., the EmoryNLP dataset). We will therefore develop effective multimodal fusion strategies to address this limitation. Beyond improving the specific model, the task-adaptive paradigm established by TAMC-ERC shows potential for extension to other conversational AI scenarios facing challenges of task-specificity and complex context. For example, the framework’s core principles are transferable to other sequential or textual analysis tasks requiring similar capabilities. Promising applications include fault diagnosis in building systems [47], where the method could model temporal sequences by adapting to fault concepts and capturing hierarchical contextual patterns, as well as automated classification of construction inspection reports [48], where task-adaptive prompts could encode defect types, and contextual modules could interpret technical descriptions within their narrative structure.