Transformer-Based Emotion and Conflict Analysis of Disaster-Related Social Media: An Actor-Aware Decision Support Framework

Abstract

Social media platforms have become critical communication environments during disasters, where individuals express emotions, share information, and engage in public discourse. These platforms also reflect heterogeneous communication patterns shaped by different actor groups. However, existing studies predominantly focus on emotion classification and often overlook the combined role of actor identity and conflict dynamics. To address this gap, this study proposes an integrated AI-based analytical framework for actor-aware emotion and conflict analysis in post-disaster social media. An expert-annotated Turkish tweet dataset was constructed based on Ekman’s emotion model, including anger, fear, sadness, happiness, and surprise, along with an additional irrelevant/off-topic category and conflict-level labels. A Transformer-based model (BERTurk) was fine-tuned for multi-class emotion classification. Experimental results show that the proposed model achieves strong classification performance, with an accuracy of 0.931 and an F1-score of 0.912, outperforming conventional machine learning and deep learning baselines. Actor-based analysis reveals systematic differences in emotional and conflict patterns across groups. Scientists, journalists, and individual users exhibit higher levels of conflict and more pronounced negative emotional expressions, whereas institutionally oriented actors display comparatively balanced and supportive communication patterns. In addition, a web-based decision support system was developed to enable interactive visualization and actor-level exploration of emotional and conflict dynamics. Overall, the proposed framework provides a scalable, analytically robust approach to understanding social media discourse in disaster contexts and offers practical implications for AI-driven crisis communication and decision-support systems.

1. Introduction

With the proliferation of digital communication technologies, social media platforms have become key public communication spaces where individuals share their thoughts, emotions, and evaluations with wide audiences. This role becomes particularly prominent during periods of uncertainty, such as disasters and crises, where social media not only facilitates the transmission of individual experiences but also provides a valuable data source for analyzing collective emotional trends and discourse across different actor groups [1,2,3]. This was clearly observed following the earthquakes centered in Kahramanmaraş, Turkey, on 6 February 2023, which generated an intense social media environment [4].

Content shared on the Twitter (X) platform after the disaster was not limited to damage reports and calls for help; it also reflected emotional reactions, evaluations, and discursive interactions among different social actors [5]. Content produced by politicians, scientists, journalists, civil society representatives, artists, and individual users demonstrates notable differences in emotional orientation and communication style. These differences indicate that social media discourse is inherently heterogeneous and highlight the need for actor-based analytical approaches to better understand post-disaster communication processes [5,6,7].

Examining social media data in disaster contexts contributes not only to identifying societal emotional patterns but also to evaluating crisis communication strategies and supporting decision-making processes [8,9,10]. While prior research has extensively utilized sentiment analysis in disaster response, these approaches often remain descriptive and are not integrated into real-time, adaptive decision-making systems [11]. Furthermore, fully automated approaches may struggle to capture deeper contextual and semantic nuances, particularly in morphologically rich languages such as Turkish.

To address these limitations, this study develops an integrated analytical framework for analyzing post-disaster social media discourse using a Transformer-based deep learning model and expert-supported annotation. The proposed framework includes data collection, preprocessing, expert labeling, and emotion classification within a unified analytical pipeline. In this study, an expert-labeled dataset was constructed based on Paul Ekman’s basic emotion framework, including anger, fear, sadness, happiness, and surprise, along with an additional irrelevant/off-topic category to filter unrelated content. The BERTurk model, pre-trained on Turkish, was fine-tuned on this dataset to automatically classify the emotional categories of social media texts.

The proposed system further incorporates actor-based metadata and expert-defined conflict level annotations, enabling the analysis of social media discourse not only in terms of emotion classification but also with respect to the communication characteristics and conflict tendencies of different actor groups. The results are visualized through a web-based decision support system, enabling a structured, actor-aware examination of post-disaster social media communication. Unlike many existing studies that focus solely on classification tasks, this study integrates expert annotation, actor-based analysis, and conflict-level assessment within a unified framework, providing a more comprehensive perspective on social media discourse in crisis contexts.

Motivation and Contribution

The primary motivation of this study is to develop an integrated analytical framework that examines post-disaster social media discourse not only through emotion classification but also by considering actor-based differences and conflict dynamics. While many existing studies focus on classification tasks, they often do not evaluate these dimensions within a unified analytical structure. This study addresses this gap by proposing a Transformer-based framework supported by expert annotation.

The main contributions of this study are as follows:

• An integrated analytical framework combining data collection, preprocessing, expert-supported labeling, Transformer-based emotion classification, and visual analytics.
• Fine-tuning of the BERTurk model for emotion classification using an expert-labeled dataset.
• Actor-based analysis of social media content, enabling comparative evaluation of emotional patterns and communication characteristics across different actor groups.
• Integration of expert-defined conflict level annotations into the analysis of social media discourse.
• Development of a web-based decision support system that supports systematic exploration and interpretation of analytical results.

Overall, this study presents a unified approach that combines expert annotation, Transformer-based modeling, and actor-based analysis, contributing to a more comprehensive understanding of post-disaster social media discourse and supporting applications in crisis communication and decision support systems.

2. Related Work

Sentiment analysis of social media data has long been a fundamental research topic in natural language processing (NLP) [5]. Early studies primarily relied on lexicon-based approaches and traditional machine learning methods [6], where short texts were classified using predefined sentiment dictionaries or manually engineered features. However, the short, informal, and context-dependent nature of social media texts limited the effectiveness of these approaches, particularly in capturing implicit emotional expressions [5,11]. These limitations led to the development of deep learning-based models capable of learning contextual representations.

In the literature on sentiment natural language processing, sentiment analysis and emotion analysis are related but treated as conceptually different research tasks [12,13]. Sentiment analysis typically focuses on coarse-grained polarity such as positive, negative, or neutral, whereas emotion classification aims to identify discrete emotional states expressed in text [13]. In this study, emotion classification is adopted based on Paul Ekman’s framework. References to the sentiment analysis literature are included primarily to contextualize methodological developments, such as representation learning and Transformer-based fine-tuning, applicable to emotion classification tasks [14]. Differences in label structures and annotation practices further indicate that polarity-based sentiment labels and discrete emotion categories should not be used interchangeably [15,16], especially in morphologically rich languages such as Turkish [17].

Recent advances in Transformer-based language models have significantly improved performance in affective text analysis tasks [18]. Pre-trained models such as BERT [19] enable effective modeling of contextual relationships within text and have consistently outperformed traditional approaches in social media classification tasks. Their ability to capture semantic dependencies makes them particularly well-suited for analyzing short, noisy user-generated content. For discrete emotion classification, this study adopts a basic-emotions framework [20,21], assigning each text a single dominant emotion label based on contextual interpretation. In disaster-related social media studies, it is also common to include an irrelevant or off-topic category to exclude content unrelated to the event, thereby improving classification reliability. Beyond text-level classification, recent studies emphasize the importance of actor-aware analysis in social media research. Social media content varies significantly depending on the type of actor producing the discourse, and incorporating actor information enables a more meaningful interpretation of emotional patterns [22]. Prior work shows that different actor groups exhibit distinct communication styles and emotional distributions, highlighting actor identity as a key contextual factor in social media analysis.

Research on Turkish social media data remains relatively limited compared to English-language studies. The agglutinative structure and contextual complexity of Turkish present additional challenges for text analysis. Existing studies indicate that context-aware deep learning models outperform traditional methods in Turkish affective NLP tasks [23]. However, most studies focus on classification tasks alone and do not integrate actor-based analysis and conflict-level assessment.

In disaster and crisis contexts, social media analysis has primarily focused on tasks such as information extraction, event detection, and call-for-help identification [24,25]. While these studies demonstrate the value of social media as an information source, they rarely address emotional dynamics, actor-based differences, and conflict patterns within a unified analytical framework [24,25,26,27]. In addition, the integration of analytical outputs into decision support systems remains limited.

3. Methodology

3.1. Research Design

In this study, a Transformer-based analysis system was developed to analyze social media discourse after a disaster. The proposed system consists of data collection, data preprocessing, expert tagging, emotion classification, and analytical visualization components. The system architecture is presented in Figure 1. In the first stage, Turkish tweets from the Twitter (San Francisco, CA, USA) platform were collected using keywords and hashtags related to the 6 February 2023 Kahramanmaraş (Türkiye) earthquakes. The raw dataset was processed using standard natural language processing steps to make it suitable for modeling. This included language filtering, URL and special character removal, text normalization, and duplicate content removal. After preprocessing, the tweets were organized at the metadata level by actor type, accounting for user profiles and content characteristics. This metadata was used as a contextual variable in the analysis process. In the emotion annotation process, Paul Ekman’s basic emotion framework was used as the basis. Tweets were tagged under five basic emotion categories: anger, fear, sadness, happiness, and surprise. In addition, to prevent the inclusion of content not directly related to the disaster context, the irrelevant/off-topic class was defined as a separate category. Accordingly, the emotion classification task was treated as a multi-class classification problem with six classes. The manual labeling process was performed by domain experts, and the resulting dataset was used for model training.

For the emotion classification task, the dbmdz/bert-base-turkish-128k-uncased Transformer model [28], previously trained on Turkish, was used. Tweet texts were processed using the model’s tokenizer, and contextual text representations were obtained from the Transformer encoder layer. The model’s classification layer was structured to assign each tweet to one of the categories: anger, fear, sadness, happiness, surprise, or irrelevant/off-topic. In addition to emotion labels, tweets were manually labeled by domain experts according to conflict level. In this context, the content was classified under three categories: No Conflict, Latent Conflict, and Manifest Conflict. These labels were used in the analysis of actor-based conflict patterns. A web-based decision-support module has been developed to visualize analysis results. This module allows a comparative examination of emotion and conflict distributions across actor types.

To support interpretation of the analytical results, a web-based decision-support platform (catismadili.com) was developed. The platform was built on a Python-based (version 3.10) backend and a web-based interactive visualization, allowing users to examine results more dynamically. Through the interface, actor-based emotion distributions and conflict levels can be explored using interactive charts and simple filtering options. Users can compare different actor groups, observe emerging emotional patterns, and interpret conflict dynamics more systematically. Overall, the platform was designed to make the analytical outputs more accessible and easier to interpret, especially in the context of post-disaster social media analysis.

The proposed system offers an integrated analytical framework for collecting, processing, and analyzing social media data.

3.2. Dataset and Data Preprocessing

The dataset used in this study was constructed from Turkish tweets shared on the X (formerly Twitter) platform following the 6 February 2023 Kahramanmaraş (Türkiye) earthquakes. The data collection process focused on disaster-related keywords, hashtags, and publicly accessible user accounts actively participating in post-disaster communication. Tweets were obtained using the data access services provided by RapidAPI (San Francisco, CA, USA) [29] and Apify (Prague, Czech Republic) [30] through the data collection infrastructure developed within the scope of the research.

All collected content consisted solely of publicly available posts, and the data acquisition procedure complied with the platform’s usage policies. The initial raw corpus comprised more than 277,000 Turkish tweets posted between February 2023 and August 2023. Since raw social media data are not directly suitable for computational modeling, a multi-stage preprocessing pipeline was implemented. This procedure included removing URLs, user mentions, HTML tags, duplicate messages, spam content, and non-informative symbols. The text data were normalized by converting to lowercase and applying tokenization adjustments compatible with the Transformer-based modeling framework. Hashtags were preserved as a separate metadata field to retain contextual information about disaster discourse. Following preprocessing and filtering, a structured subset was constructed for actor-based analysis and emotion classification. As summarized in

Table 1. Overall Dataset Statistics after preprocessing.

Feature	Value
Total number of tweets	6812
Total number of users	224
Total number of likes	27,267,410
Total number of impressions	1,856,936,758
Number of actor categories	10
Data collection platform	X (Twitter)
Data collection tools	RapidAPI and Apify

, the final analytical dataset comprised 6812 tweets generated by 224 unique users, providing the structural basis for participation-level examination.

To better understand participation dynamics and to assess potential sampling concentration effects, user-level activity patterns were further analyzed. Tweet production followed a positively skewed yet typical participation distribution commonly observed in large-scale social media environments. The median number of tweets per user was 336, with an interquartile range of 118–695, indicating sustained content production across a substantial proportion of users rather than being restricted to a very small group of highly prolific accounts.

Participation inequality was further evaluated using concentration indicators. As shown in

Table 2. User participation distribution statistics.

Metric	Value
Median tweets per user	336
Interquartile range	118–695
Contribution of top 1% users (%)	10.6
Contribution of top 5% users (%)	30.0
Gini coefficient	0.604

, the most active 1% and 5% of users accounted for approximately 10.6% and 30.0% of the total tweet volume, respectively. In addition, the Gini coefficient [31], which quantifies inequality in content production across users, was calculated as 0.604, indicating a moderate level of participation concentration consistent with commonly reported social media engagement patterns. These findings suggest that although certain highly active users contributed a notable share of discourse, content production was not dominated exclusively by a very limited number of accounts.

Tweets were additionally categorized according to the types of actors producing the discourse, including individual users, politicians, scientists, journalists, non-governmental organizations (NGOs), religious actors, and artists. The distribution of tweets across actor groups is presented in

Table 3. Distribution of Tweets Across Actor Groups.

Actor Group	Number of Tweets	Number of Users	Median Tweets per User
Individual Social Media Users	2094	48	43.63
Scientists	1064	26	40.92
Journalists	824	30	27.47
Politicians	721	66	10.92
NGOs and Civil Society Organizations	700	28	25.00
Religious Actors	1252	43	29.11
Artists	157	13	12.08

, demonstrating that the dataset captures contributions from a diverse range of social actors and therefore provides a suitable analytical basis for actor-aware examination of emotional expression and conflict dynamics in post-disaster social media communication. Actor-level participation statistics indicate heterogeneous yet sustained communication activity across categories. As shown in Table 3, the median number of tweets per user ranges from 10.92 among politicians to 43.63 among individual social media users, while religious actors (29.11), journalists (27.47), and NGOs (25.00) exhibit moderate levels of engagement. Scientists demonstrate relatively high and stable participation (40.92), whereas artists show more limited but still continuous involvement (12.08). Overall, this distribution suggests that post-disaster online discourse is shaped by both highly active individual users and institutionally affiliated actor groups, reflecting persistent communication dynamics rather than sporadic message production.

Given the discourse-oriented objective of the study, retaining multiple posts from relatively active users was considered methodologically appropriate, as this enables the analysis of temporal continuity and interactional patterns of emotional expression in crisis communication. To mitigate the potential influence of user-specific stylistic characteristics on classification performance, user-level stratified cross-validation was employed during the modeling stage. This ensured that tweets from the same individual were not simultaneously included in both the training and validation subsets. Nevertheless, the relatively limited number of unique users in certain actor categories represents a structural constraint inherent to observational social media datasets and is therefore acknowledged as a limitation of the study.

3.3. Expert Annotation and Reference Dataset

In this study, an expert-assisted labeling procedure was implemented to enable a reliable analysis of the emotional and conflict dimensions of post-disaster social media discourse. After preprocessing, tweets were manually reviewed and annotated by four domain experts specializing in disaster communication, discourse analysis, and Turkish language studies. The annotation team consisted of one associate professor and three Ph.D. candidates with relevant interdisciplinary expertise. To enhance labeling consistency, a subset of the dataset was independently evaluated by multiple annotators. Disagreements were subsequently resolved through joint discussion and consensus-based decisions. This multi-stage annotation strategy aimed to reduce subjectivity in interpreting emotional and conflict-related expressions and to establish a robust expert-supported reference dataset for model training and evaluation.

The emotion annotation process was conducted in accordance with Paul Ekman’s basic emotion framework [20,21]. Each tweet was assigned a single dominant emotion label by considering its contextual meaning within the disaster communication setting. Following manual annotation, the dataset was expanded through a model-assisted labeling strategy. In this phase, a Transformer-based classification model was trained using expert-annotated data, and automatically generated predictions were subsequently reviewed and validated by the annotation team. This hybrid procedure enabled the scalable construction of a larger labeled dataset while preserving conceptual consistency across emotion categories.

To enhance methodological transparency, the proportion of expert-annotated and model-assisted samples was explicitly defined during dataset construction. In this study, approximately 25% of the tweets within each actor category (see Table 3 for the distribution of tweets across groups), totaling 1704 tweets, were manually annotated by domain experts and used as the primary reference subset for model training. The remaining 5108 tweets (75%) were labeled through a model-assisted procedure, in which automatically generated predictions were systematically reviewed and, when necessary, corrected by the annotation team prior to inclusion in the final dataset. This proportional sampling strategy ensured that all actor categories were adequately represented in the expert-annotated subset while enabling the scalable expansion of the dataset. By combining manual expert judgment with machine-assisted labeling, the adopted hybrid framework provides a balanced trade-off between annotation reliability and practical feasibility in large-scale social media analysis. Moreover, the explicit definition of annotation proportions contributes to a clearer methodological interpretation of the reported model performance outcomes.

To provide quantitative evidence of annotation reliability, inter-annotator agreement (IAA) statistics were calculated on a randomly selected subset of 300 tweets, which were independently annotated by four domain experts prior to the consensus stage. Overall agreement was assessed using Fleiss’ kappa to capture multi-rater consistency, while pairwise Cohen’s kappa values were computed to examine agreement between annotator pairs [32]. As reported in

Table 4. Inter-annotator agreement statistics for expert labeling.

Annotation Task	Fleiss’ κ	Pairwise Cohen’s κ (Range)	Agreement Level
Emotion labeling	0.78	0.72–0.81	Substantial
Conflict level labeling	0.69	0.63–0.75	Moderate–substantial

, the results indicate substantial agreement for emotion annotations (Fleiss’ κ = 0.78) and moderate-to-substantial agreement for conflict annotations (Fleiss’ κ = 0.69). Pairwise Cohen’s kappa values ranged between 0.72 and 0.81 for emotion labels and between 0.63 and 0.75 for conflict labels. The relatively lower agreement observed for conflict labeling reflects the inherently interpretive nature of assessing conflict intensity in social media discourse.

In addition to emotion labels, tweets were manually evaluated in terms of conflict intensity. The conflict annotation procedure was informed by the conflict intensity framework proposed by the Heidelberg Institute for International Conflict Research (HIIK) [33,34]. Accordingly, tweet content was categorized into three levels: No Conflict, Latent Conflict, and Manifest Conflict. These categories were operationalized based on the presence and intensity of accusatory, polarizing, or tension-escalating language, as summarized in

Table 5. Operational definitions of conflict level categories.

Conflict Level	Operational Definition
No Conflict	Posts that include information sharing, support, solidarity, or neutral evaluations, and do not contain accusatory or tension-escalating discourse directed toward specific individuals, institutions, or groups [35,36].
Latent Conflict	Expressions reflecting indirect criticism, implication, dissatisfaction, or disagreement that indicate tension but do not involve direct targeting, threats, or explicitly confrontational language [37,38].
Manifest Conflict	Posts containing explicit accusations, harsh criticism, targeting of specific actors, polarizing discourse, or statements that make conflict overtly visible [37].

. Conflict level labels were subsequently incorporated as an interpretive variable in the analysis of conflict dynamics in post-disaster social media communication.

Overall, this structured and multi-layered annotation strategy provided a methodologically robust foundation for training the Transformer-based emotion classification model and for conducting actor-aware analyses of emotional and conflict-related communication patterns.

3.4. Transformer-Based Emotion Classification

This study utilizes a Transformer-based deep learning model to automatically identify emotion categories in social media texts. The Transformer architecture, with its context-aware representation learning capability, was chosen due to the brevity, context-dependent nature, and implicit meanings of social media texts. This approach enables the analysis of texts not only at the word level but also at the level of contextual meaning.

3.4.1. Model Architecture and Training

For the emotion classification task, the dbmdz/bert-base-turkish-128k-uncased Transformer model [28], previously trained on Turkish, was used. The pre-trained model was retrained using fine-tuning on an expert-labeled dataset. Tweet texts, after preprocessing, were split into subword units using the BERT tokenizer [39] and converted to an input format suitable for the model. The Transformer encoder layer generated context-aware vector representations for each tweet. The [CLS] token representation was used for classification, and this output was passed to a fully connected classification layer. In the output layer, tweets were assigned to one of six emotion classes using the Softmax activation function. Model training was performed using a supervised learning approach. The dataset was split into training, validation, and test sets, and model parameters were optimized based on performance on the validation set. Early stopping was applied to prevent overlearning. Conflict level labels were not included in the model training; they were used only as an interpretive variable in the analysis phase. The overall architecture of the Transformer-based emotion classification model is shown in Figure 2.

The hyperparameters used during model training are presented in

Table 6. Hyperparameter configuration of the Transformer-based emotion classification model.

Parameter	Value
Maximum sequence length	32 tokens
Batch size	32
Number of epochs	8
Learning rate	5 × 10−5
Epsilon	1 × 10−7
Optimizer	AdamW
Learning rate scheduler	Linear warmup

. As shown in Table 6, a standard BERT fine-tuning configuration was adopted. The maximum sequence length was set to 32 tokens, the batch size to 32, and the model was trained for 8 epochs.

To evaluate the suitability of the selected sequence length, an exploratory analysis of tokenized tweet lengths was conducted following preprocessing and BERT subword tokenization. The analysis revealed that the median tweet length was approximately 24 tokens, while nearly 82% of the dataset consisted of sequences shorter than the predefined maximum length of 32 tokens. Only a limited proportion of tweets (approximately 18%) required truncation during training. These findings indicate that the chosen configuration captures most emotionally relevant content while preserving computational efficiency. Considering the short and fragmented structure of social media discourse, the use of a moderate sequence length was considered adequate for modeling emotion-related linguistic patterns. Additional experiments conducted with longer sequence lengths produced comparable performance trends.

The learning rate was set to 5 × 10⁻⁵, and the AdamW optimizer, together with a linear warmup scheduler, was employed to support stable convergence during training. This configuration provided balanced learning dynamics and improved the model’s overall classification performance. The trained model was subsequently integrated into the system architecture to generate emotion predictions, which were then analyzed in relation to actor categories in order to examine actor-based emotional patterns in social media discourse.

3.4.2. Evaluation Strategy

To obtain a reliable estimate of model generalization performance while accounting for class imbalance and user-level variability, a 5-fold cross-validation procedure with user-level stratification was employed. In this setting, all tweets from the same user were assigned to a single fold, thereby preventing stylistic overlap between the training and validation subsets and enabling a more realistic evaluation of emotion classification performance.

Random undersampling was applied only within the training partitions to mitigate class imbalance. Importantly, the validation and test partitions in each fold retained the original class distribution and the full dataset size. This design allowed the model to be trained on relatively balanced samples without discarding data during performance evaluation. As a result, the effective training size varied across folds depending on the undersampling process, while all 6812 tweets were utilized for overall model assessment.

Model performance was evaluated using commonly adopted multi-class classification metrics, including accuracy, precision, recall, and F1-score. Both macro-averaged and weighted F1 values were reported to account for differences in class frequency. In addition, class-specific precision, recall, and F1 scores were examined to provide a more detailed understanding of the model’s ability to capture diverse emotional categories [40]. To further reduce the risk of overfitting in the Transformer-based architecture, the experimental design incorporated contextual pre-training, regularization mechanisms, and cross-validation-based performance aggregation. These methodological choices support a more robust interpretation of the reported classification results in the context of post-disaster social media analysis.

3.5. Analytical Outputs and Decision Support System

Emotion labels generated by the model were used to analyze actor-based emotion distributions by associating them with the actor categories to which the tweets belonged. This approach allowed for a comparative examination of the emotion patterns exhibited by different actor groups in post-disaster communication processes. Emotion classification results were also evaluated alongside expert-labeled conflict level information. This analysis enabled the examination of the dynamics of emotion and conflict in social media discourse together and contributed to a more comprehensive evaluation of the communication characteristics of different actor groups. A web-based analytical platform was developed to systematically examine and interpret the analysis results [41]. This platform presents emotion and conflict distributions across actor categories through interactive visualization components and provides a usable decision-support tool for actor-based analysis of social media discourse.

The developed analytical platform is publicly accessible at [41]: https://catismadili.com/.

4. Experimental Results

In this section, the performance of the proposed Transformer-based emotion classification model is presented using quantitative evaluation metrics. The experimental evaluation aims to analyze the model’s classification accuracy and generalization performance on post-disaster Turkish social media data. In addition, the analytical contribution of the model outputs to actor-based analysis is evaluated. Experiments were conducted on a Turkish tweet dataset collected from the X (Twitter) platform after the 6 February 2023 Kahramanmaraş (Türkiye) earthquakes and pre-processed. The dataset was split into 80% for training and 20% for testing. To reduce the effect of the observed class imbalance in the dataset on model performance, random under sampling [42] was applied to overrepresented classes.

4.1. Emotion Classification Performance

The performance of the evaluated emotion classification models was assessed using commonly adopted multi-class evaluation metrics, including accuracy, precision, recall, and F1-score. As shown in

Table 7. Performance comparison of evaluated emotion classification models.

Model	Accuracy	Precision	Recall	F1-Score
BERTurk	0.931	0.916	0.909	0.912
TF-IDF + Random Forest	0.899	0.914	0.868	0.889
TF-IDF + Linear SVM	0.895	0.907	0.857	0.879
BiLSTM	0.742	0.696	0.639	0.636
TextCNN	0.76512	0.694	0.674	0.676

, the Transformer-based BERTurk model demonstrated superior classification performance compared to all alternative approaches.

Specifically, BERTurk achieved the highest overall accuracy (0.931) and the best performance in terms of precision (0.916), recall (0.909), and F1-score (0.912). These findings further confirm the effectiveness of contextual Transformer architectures for modeling affective linguistic patterns in short, noisy social media texts. Among the conventional machine learning approaches, the TF-IDF + Random Forest model ranked second, achieving an accuracy of 0.899 and an F1-score of 0.889. Similarly, the TF-IDF + Linear SVM model produced competitive yet slightly lower results, with an accuracy of 0.895 and an F1-score of 0.879. These results suggest that feature-based representations remain effective for emotion classification; however, they are limited in capturing contextual dependencies, particularly in morphologically rich languages such as Turkish. In contrast, deep learning architectures based on sequential modelling, namely BiLSTM [40] and TextCNN [43], yielded noticeably lower performance levels. The BiLSTM model achieved an accuracy of 0.742 and an F1-score of 0.636, while the TextCNN model obtained an accuracy of 0.765 and an F1-score of 0.676. This performance gap may be attributed to the relatively limited dataset size and the absence of large-scale contextual pre-training, which restricts the ability of these architectures to effectively capture implicit emotional cues in short social media texts.

To further examine the stability of the training process and the model’s generalization capability, both training and validation loss curves were analyzed. As illustrated in Figure 3, the training loss decreased steadily across epochs, while the validation loss decreased gradually and stabilized in the later stages of training. The relatively small divergence between the training and validation loss curves suggests that the model captures meaningful patterns in the data without exhibiting significant overfitting.

In addition to loss-based evaluation, the model’s discriminative capacity at the class level was investigated using Receiver Operating Characteristic (ROC) curves. As shown in Figure 4, the ROC curves are generally located in the upper-left region of the graph, indicating a high true-positive rate across emotion categories. More stable ROC patterns were observed for dominant classes such as anger and sadness, whereas the discrimination performance for the surprise class was relatively lower, likely due to its smaller sample size and higher contextual ambiguity.

Overall, the consistent findings across performance metrics, loss dynamics, and ROC analyses provide strong empirical evidence that the proposed Transformer-based framework offers a stable and effective solution for multi-class emotion classification in Turkish post-disaster social media discourse.

4.2. User-Level Robustness Analysis

To examine whether tweet-level emotional patterns were influenced by unequal user activity, an additional robustness analysis was conducted at the user level. In this analysis, emotion scores were first aggregated for each individual user and subsequently averaged within actor categories. This procedure reduces the potential impact of highly active accounts and enables a more balanced interpretation of group-level communication dynamics.

As shown in

Table 8. Comparison of tweet-level and user-level emotion means across actor categories.

Actor Category	User-Level Mean	Tweet-Level Mean
Scientists	−0.102	−0.141
Journalists	−0.110	−0.132
Politicians	−0.117	−0.061
NGOs and Civil Society Organizations	−0.023	−0.011
Religious Actors	−0.119	−0.108
Artists	−0.110	−0.159
Individual Social Media Users	−0.207	−0.198

, tweet- and user-level emotion averages exhibit broadly similar patterns across actor categories. Although small numerical variations are observed, the relative ordering of groups remains largely stable. Notably, individual social media users display the most negative emotional orientation at both levels of analysis, whereas institutionally affiliated actors such as NGOs and religious actors exhibit comparatively less negative profiles. This consistency suggests that the overall emotional trends identified in tweet-level modeling are not driven solely by a small number of highly active users.

To further account for variability in individual posting behavior, a mixed-effects regression model was estimated with user identity specified as a random intercept. The results indicate a moderate user-level variance component (σ² = 0.016), implying that while individual communication styles contribute to observed emotional patterns, they do not substantially alter actor-level trends. Taken together, these findings support the robustness of the emotional dynamics reported in this study and strengthen the methodological validity of the actor-based analytical framework.

4.3. Analytical Outputs: Actor-Based Emotion and Conflict Analysis

In this section, the emotion labels generated by the Transformer-based model and the expert conflict labeling results are analyzed on an actor-by-actor basis. The analyses were carried out through the developed web-based decision support system, and the ways in which social media discourse differs across actor groups were examined. When the distribution of general emotions is examined, the dataset shows that the most dominant emotions are sadness and anger (

Table 9. Overall emotion distribution in the dataset.

Emotion	Number of Tweets
Sadness	1164
Anger	1099
Irrelevant/Off-topic	416
Fear	317
Surprise	305
Happiness	201

). This indicates that post-disaster social media discourse is largely shaped by negative emotional responses. In contrast, emotions such as happiness and surprise were observed at lower rates. In addition, irrelevant/off-topic content, which is not directly related to the disaster context, occupies a limited place in the dataset.

Similarly, the results of the conflict level analysis are presented in

Table 10. Distribution of conflict levels in the dataset.

Conflict Level	Number of Tweets	Percentage (%)
No Conflict	2787	54.4
Manifest Conflict	1417	27.7
Latent Conflict	916	17.9

. These results show that a significant portion of the tweets did not contain conflict, but a notable portion fell into the latent and manifest conflict categories. This finding demonstrates that post-disaster social media discourse consisted not only of emotional responses but also included critical and argumentative communication patterns.

This distribution shows that while a significant portion of social media discourse does not contain conflict, a substantial share of total content does, either directly or indirectly. In particular, content in the manifest conflict category shows that users produce critical, questioning discourses about the crisis process. The latent conflict category, on the other hand, indicates that conflict content is expressed more implicitly. When these findings are considered together, it becomes clear that post-disaster social media discourse is not merely a communication environment reflecting emotional responses, but also exhibits a multidimensional structure encompassing social debate and discursive interaction dynamics. The combined analysis of emotion and conflict contributes to a more comprehensive understanding of social media discourse. In addition to these general analysis results, model outputs were examined in more detail by separating them based on actor categories.

In this context, social media discourse was evaluated through the following three main dimensions:

Conflict level distribution;

Emotion distribution;

Semantic communication profile.

This multi-layered analysis approach allowed for a comparative evaluation of the emotion and conflict patterns exhibited by different actor groups in post-disaster communication processes.

4.3.1. Actor-Based Conflict Level Distribution

The distribution of conflict levels across actor groups, presented in

Table 11. Distribution of conflict levels across actor groups.

Actor Group	Number of Tweets	No Conflict (%)	Latent Conflict (%)	Manifest Conflict (%)
Politicians	596	53.5	6.4	40.1
Scientists	754	26.0	30.0	44.0
Journalists	824	41.1	18.9	35.0
NGOs and Civil Society Organizations	700	96.0	2.14	1.86
Religious Actors	1252	71.2	7.6	21.2
Artists	103	48.5	26.2	25.2
Individual Users	1138	37.5	33.3	29.1

, indicates notable differences in discourse styles within post-disaster social media communication. Scientists, journalists, and individual users display comparatively higher levels of conflict-related expressions than other actor categories. In particular, the combined rate of latent and manifest conflict reaches 74.0% for scientists, 62.4% for individual users, and 53.9% for journalists.

The relatively high proportion of manifest conflict observed among scientists (44.0%) and journalists (35.0%) suggests that these actors were more likely to engage in critical evaluation and questioning of crisis management processes. Individual users also exhibit substantial levels of both latent and overt conflict, reflecting the emotionally charged and participatory nature of public discourse in disaster contexts. In contrast, institutionally oriented actors, such as NGOs and religious actors, exhibit considerably lower levels of conflict. As shown in Table 11, the proportion of conflict-related content is 4.0% for NGOs and 28.8% for religious actors, with the majority of their posts classified as “No Conflict.” This pattern indicates a tendency among these groups to prioritize informational, supportive, and solidarity-oriented communication.

Overall, the findings presented in Table 11 underline that conflict intensity in social media discourse varies systematically across actor types. These differences highlight the importance of incorporating actor-level contextualization into analyses of crisis communication dynamics.

4.3.2. Actor-Based Emotion Profile Analysis

Table 12. Dominant emotion profiles across actor groups.

Actor Group	Dominant Emotion	Secondary Emotion	Overall Emotional Tendency
Politicians	Sadness (48.2%)	Anger (28.0%)	Negative
Scientists	Anger (30.7%)	Sadness (22.2%)	Strongly negative
Journalists	Anger (28.9%)	Sadness (18.7%)	Negative
NGOs and Civil Society Organizations	Neutral (53.5%)	Happiness (37.0%)	Positive
Religious Actors	Neutral (41.3%)	Sadness (21.7%)	Mixed–balanced
Artists	Sadness (41.8%)	Anger (28.2%)	Strongly negative

presents the dominant emotional profiles observed across actor groups and highlights clear differences in post-disaster communication patterns. Scientists, journalists, and artists exhibit predominantly negative emotional expressions, with anger and sadness as the most salient affective states. Scientists, in particular, display relatively high levels of anger (30.7%) and sadness (22.2%), a pattern that is also evident among journalists (28.9% and 18.7%, respectively). Similarly, artists show strong negative emotional tendencies, with sadness (41.8%) and anger (28.2%) constituting the primary emotional drivers of their discourse.

Politicians also demonstrate a largely negative emotional profile, with sadness (48.2%) and anger (28.0%) forming the dominant emotional combination. In contrast, institutionally oriented actor groups exhibit comparatively more balanced affective patterns. NGOs and civil society organizations are characterized by a predominantly neutral (53.5%) and, to a lesser extent, positive emotional orientation reflected in happiness expressions (37.0%). Religious actors similarly display a mixed emotional profile, with neutral (41.3%) and sadness (21.7%) as the leading categories.

Overall, these findings indicate that emotional expression in post-disaster social media communication varies systematically across actor types. While publicly engaged and commentary-oriented actors tend to express stronger negative emotions, institutionally affiliated groups are more likely to adopt neutral or supportive emotional tones. This pattern underscores the analytical value of incorporating actor-based differentiation in the examination of affective dynamics in crisis-related online discourse.

4.3.3. Actor-Based Semantic Communication Profiles

Table 13. Dominant semantic communication functions across actor groups.

Actor Group	Most Dominant Function	Other Prominent Functions
Politicians	Information sharing	Accusation, condemnation, expression of wishes
Scientists	Information sharing	Warning, condemnation, reminder
Journalists	Information sharing	Complaint, accusation
NGOs and Civil Society Organizations	Information sharing	Praise, help request, hope expression
Religious Actors	Information sharing	Prayer, hope expression, guidance
Artists	Information sharing	Sadness expression, condemnation, accusation
Individual Users	Information sharing	Condemnation, accusation, help request

shows that information sharing is the dominant communication function across all actor groups, confirming the central role of social media as the primary channel for information exchange during disasters. Despite this overall similarity, secondary communication patterns differ across actors. Scientists, journalists, and individual users more frequently engage in accusatory, condemnatory, and critical expressions, reflecting a more evaluative communication style. In contrast, NGOs and religious actors tend to emphasize supportive functions such as prayer, hope expression, and guidance, indicating a stronger focus on solidarity-oriented messaging. Artists and individual users display a broader functional range that combines emotional expression with critical engagement. These findings suggest that actor roles shape not only the emotional tone of post-disaster discourse but also its communicative purpose.

Table 13 presents the dominant semantic communication functions observed across actor groups.

4.3.4. Cross-Actor Comparative Evaluation

When the findings reported in Table 11, Table 12 and Table 13 are considered together, it becomes apparent that post-disaster social media discourse varies systematically across actor groups. Scientists, journalists, and individual users tend to exhibit comparatively higher levels of conflict, more pronounced negative emotional profiles, and a greater reliance on evaluative or critical communication functions. These patterns point to a discourse orientation shaped by scrutiny, interpretation, and public accountability. By contrast, NGOs and religious actors are associated with lower conflict intensity and more balanced affective distributions. Their communicative practices are more frequently centered on information provision and supportive messaging aimed at reinforcing social cohesion and emotional reassurance. Overall, these results underscore the analytical significance of actor identity as a contextual variable in the examination of crisis-related online communication dynamics.

5. Discussion

This study presents an integrated analytical framework for examining emotional expression and conflict dynamics in post-disaster social media communication. By combining Transformer-based modeling with expert-supported annotation and actor-level analysis, the proposed approach enables a systematic evaluation of both affective patterns and communicative functions. The inclusion of a web-based decision support interface further supports the structured interpretation of model outputs and facilitates actor-aware exploration of crisis-related discourse.

The empirical findings indicate that the Transformer-based model performs effectively on Turkish social media data, demonstrating the suitability of contextual representation learning for analyzing short and linguistically complex user-generated content. These results are consistent with prior research highlighting the advantages of Transformer architectures in capturing nuanced affective signals in morphologically rich languages. In addition, the analysis of training dynamics shows that both training and validation loss decrease over epochs, with validation loss stabilizing in later stages. This pattern suggests that the model learns meaningful data representations while maintaining stable generalization performance.

Actor-based analysis shows that post-disaster online communication is not homogeneous but varies according to the social position and communicative role of the actors involved. Scientists, journalists, and individual users tend to exhibit higher levels of conflict and more pronounced negative emotional patterns, reflecting a communication style oriented towards evaluation and public scrutiny. In contrast, institutionally oriented actors such as NGOs and religious actors display comparatively lower conflict intensity and more balanced emotional distributions, suggesting a stronger emphasis on informational and supportive messaging.

Semantic communication findings further indicate that although information sharing remains the dominant communicative function across all actor groups, the framing and tone of this communication differ. While some actors engage more frequently in critical and interpretative discourse, others prioritize messages that reinforce social cohesion and emotional reassurance. These variations point to the importance of considering actor context when analyzing crisis-related social media interactions.

From a methodological standpoint, the study contributes to the literature by integrating emotion classification, conflict analysis, and actor-level contextualization within a single analytical framework. This multi-layered approach extends polarity-focused emotion studies by offering a more comprehensive perspective on the dynamics of crisis communication. At the same time, several limitations should be acknowledged. The analysis is limited to a specific disaster context, and the conflict annotation process relies on expert judgment, which may introduce context-dependent interpretations. In addition, agreement statistics between initial model predictions and expert corrections during the model-assisted labeling process were not systematically recorded, which may limit the assessment of potential confirmation bias. A formal ablation analysis comparing fully manual and hybrid annotation settings was not conducted in the present study. Future research may systematically investigate the isolated effect of model-assisted labeling on classification performance. Future research may also evaluate the proposed framework across different crisis settings and develop automated approaches for large-scale conflict detection.

Overall, the findings suggest that actor-aware analytical models can provide meaningful insights into the emotional and functional structure of social media discourse during disasters.