An Empirical Study of Federated BERT for Decentralized Twitter Sentiment Analysis

Abstract

Twitter/x has become a key platform for analyzing public opinion on a large scale; however, traditional centralized approaches raise significant concerns regarding privacy and data governance. To address these challenges, this paper presents an empirical study of a federated learning approach based on a BERT model for decentralized sentiment analysis at the tweet level. This study focuses on evaluating the effectiveness of transformer-based models under realistic non-independent and identically distributed (non-IID) data distributions across distributed clients. The proposed approach enables collaborative model training without sharing raw tweet data, thereby preserving user privacy while leveraging knowledge from multiple sources. The model is evaluated over 100 communication rounds using the Sentiment140 dataset, distributed among four clients with heterogeneous data distributions. Experimental results demonstrate stable convergence and robust performance, with an accuracy of 95.00%, an F1 score of 95.00%, and a PR-AUC of 96.76%. It should be noted that the federated model performs within 1.2% of a centralized baseline, indicating minimal performance degradation despite data sharing constraints.

1. Introduction

The rapid proliferation of social media platforms has fundamentally transformed the global information landscape, enabling unprecedented levels of real-time communication and content sharing. twitter/x, in particular, serves as a critical channel for public discourse, political communication, and news dissemination. However, this openness also facilitates the rapid spread of harmful content, including hate speech, misinformation, spam, bot-generated content, and illegal material. These phenomena pose significant threats to social cohesion, democratic processes, and public safety, necessitating robust and scalable automated detection systems [1].

Over the past decade, the research community has developed numerous tweet-level detection systems targeting hate speech, misinformation, spam, bot activity, and illegal content. Early approaches relied on traditional machine learning techniques, including logistic regression, support vector machines (SVMs), Naïve Bayes classifiers, and ensemble methods such as Random Forests. While these methods offered interpretability and computational efficiency, their dependence on manually engineered features limited their ability to capture the semantic complexity, informal language, and rapidly evolving vocabulary characteristic of twitter/x. The subsequent adoption of deep learning architectures, particularly convolutional neural networks (CNNs) and recurrent models such as LSTMs and GRUs, addressed some of these limitations by enabling automatic feature extraction and sequential context modeling. More recently, transformer-based models, including BERT and RoBERTa, have established new performance benchmarks across multiple detection tasks by leveraging self-attention mechanisms to capture rich contextual and semantic dependencies [2].

Despite these advances, several structural challenges persist. First, model generalization remains problematic: systems trained on historical or domain-specific datasets frequently exhibit significant performance degradation when applied to new events, emerging narratives, or evolving linguistic patterns—a phenomenon commonly attributed to concept drift and annotation bias. Second, the informal, multilingual, and slang-heavy nature of twitter/x discourse continues to challenge even state-of-the-art transformer models, particularly in handling implicit hate speech, sarcasm, and coded language. Third, and critically, the centralized data collection paradigm underlying most existing approaches raises serious privacy, legal, and ethical concerns.

Centralized aggregation of large-scale user data exposes sensitive information and conflicts with emerging data protection regulations. Prior work has shown that decentralized architectures leveraging technologies such as blockchain and IPFS can effectively enhance data confidentiality, integrity, and access control in distributed environments [3].

Regulatory frameworks governing data ownership and cross-platform sharing increasingly restrict the feasibility of aggregating large-scale user data for model training, a constraint that is frequently overlooked in experimental evaluations. Together, these limitations highlight that improving tweet-level detection requires not only more sophisticated models, but also more realistic, privacy-respecting, and generalizable training frameworks [4].

To the best of the knowledge, no existing work has simultaneously addressed the challenges of contextual semantic modeling, temporal generalization, and user privacy preservation within a unified framework for tweet-level sentiment analysis on twitter/x. While federated learning has been explored in general natural language processing contexts, its integration with domain-adapted transformer architectures for decentralized tweet classification remains underexplored. Furthermore, existing federated approaches rarely account for the non-IID (non-independent and identically distributed) nature of social media data across distributed clients, nor do they evaluate performance under realistic privacy constraints. This gap motivates the development of a framework that combines the representational power of transformers with the privacy-preserving properties of federated learning, specifically tailored to the dynamic and distributed characteristics of twitter/x data [5].

To address the identified gap, this paper proposes a decentralized federated learning framework for tweet-level sentiment analysis in which transformer-based models are trained collaboratively across distributed clients without requiring access to raw user data, thereby preserving privacy while still leveraging knowledge from multiple sources. The approach integrates advanced transformer architectures within a federated setting, enabling the capture of rich contextual and semantic information inherent in informal twitter/x language, including slang, implicit expressions, and multilingual content. To ensure effective model training, a federated averaging aggregation strategy is employed, allowing locally fine-tuned model updates to be iteratively combined into a robust global model through multiple communication rounds, leading to stable convergence without centralized data sharing. Empirical evaluation further demonstrates that the proposed method achieves competitive classification performance compared to centralized baselines, while maintaining strong privacy guarantees and improving generalization across diverse and evolving twitter/x data distributions [6].

To address the identified gap, this paper makes the following key contributions:

• A decentralized federated learning approach for tweet-level sentiment analysis on twitter/x that enables collaborative model training across distributed clients without requiring access to raw user data.
• Integration of transformer-based architectures within a federated setting to capture rich contextual and semantic representations of informal twitter/x language, including slang, implicit expressions, and multilingual content.
• A federated averaging aggregation strategy that iteratively consolidates locally fine-tuned model updates into a robust global model through multiple communication rounds, achieving stable convergence without centralized data sharing.
• Empirical evaluation demonstrating that the proposed approach achieves competitive classification performance compared to centralized baselines, while maintaining privacy guarantees and improving generalization across diverse and dynamic twitter/x data distributions.

The rest of the paper is organized as follows: Section 2 presents the deep review of related work, including traditional machine learning approaches, deep learning methodology and transformer models applied to tweet-level detection tasks. Section 3 describe the proposed approach, including architecture, algorithm, dataset, preprocessing pipeline and evaluation metrics. Section 4 presents experimental results, with comparative analysis against baseline methods. Section 5 discusses the results, implications, limitations, and directions for future research. Section 6 concludes the paper.

2. Related Work

The detection of harmful content on social media platforms has been extensively studied across several interconnected tasks, including hate speech identification, misinformation detection, spam and bot classification, and illegal content moderation. This section reviews the progression of methods applied to these challenges, from traditional machine learning to deep learning and transformer-based architectures, before examining privacy-preserving approaches and identifying the gap that motivates the present work.

Early detection systems relied on classical machine learning methods operating on manually engineered features derived from text, metadata, or statistical representations [7]. Logistic regression, combined with lexical features and n-grams, offered simplicity and interpretability for spam and hate speech detection on small or structured datasets [8], but its linear nature limited its ability to capture contextual nuances and informal language variations. Support vector machines (SVMs) handled high-dimensional sparse representations more effectively, frequently outperforming logistic regression on noisy or imbalanced datasets [9,10], though their reliance on fixed features restricted adaptability to evolving language [11]. Naïve Bayes classifiers provided a computationally efficient probabilistic alternative suitable for large-scale filtering tasks [12], but their independence assumptions undermined performance on complex tasks such as hate speech or disinformation detection [13]. Ensemble methods, particularly Random Forests, partially addressed these limitations by modeling non-linear feature interactions across heterogeneous textual, temporal, and user-level signals [13,14], yet their dependence on manual feature engineering ultimately constrained their semantic modeling capacity.

The limitations of feature-driven approaches motivated a progressive shift toward deep learning. Convolutional neural networks (CNNs) were among the first deep architectures applied to tweet classification, learning local patterns from word or character representations efficiently [15]. While CNNs offered a good balance between speed and performance for short texts [8], their fixed-size receptive fields prevented them from capturing long-range dependencies. Recurrent architectures, particularly Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) networks, addressed this by modeling sequential dependencies across the full input [9]. Mossie and Wang [12] demonstrated the effectiveness of GRU-based classifiers for hate speech detection in a low-resource Amharic language setting, confirming that recurrent models generalize beyond high-resource English contexts. Nevertheless, RNN-based models carry practical drawbacks, including high computational cost and sensitivity to noise, limiting their scalability on large social media streams.

Transformer-based models have since become the dominant paradigm for social media text analysis. Models such as BERT and RoBERTa employ self-attention mechanisms that simultaneously capture relationships across the entire input sequence, yielding richer contextual representations than CNN or RNN approaches [13]. These architectures have demonstrated consistent performance gains across hate speech, misinformation, and bot detection tasks, particularly for subtle linguistic phenomena and multilingual content. In the broader context of content analysis systems, Cani et al. [16] conducted a rigorous, systematic benchmarking study across multiple architectures and datasets, demonstrating that no single model universally dominates and that performance claims must be grounded in multi-condition evaluation. This methodological observation applies directly to transformer-based tweet classification, where reported gains frequently fail to generalize across domains or temporal conditions [17]. Indeed, transformer models remain sensitive to concept drift, domain shift, and dataset-specific biases, with performance degrading notably when applied to new events or communities not represented in pre-training data [17,18].

Across specific detection tasks, these architectural limitations are consistently observed. In fake news detection, Aïmeur et al. [19] demonstrated through a comprehensive review that misleading content increasingly resembles credible information in tone and structure, rendering content-only approaches insufficient. Multimodal models incorporating contextual and visual signals have shown improved robustness [20], yet generalization to unseen events remains a persistent challenge [8,18]. In spam and bot detection, Alhosseini et al. [21] showed that incorporating social graph structure alongside user features substantially improved detection performance over feature-only approaches. However, Ng et al. [22] highlighted through the FATe framework that bot detection systems must also address fairness, accountability, and transparency, as increasingly human-like bot behavior and the risk of misclassifying legitimate users raise significant ethical concerns [22,23]. In illegal and extremist content detection, advanced language models have shown promise in identifying coded references to illicit activities [24], but rapid evolution of slang and symbolic language undermines generalization, while false positives carry serious legal and social consequences [24,25].

A structural limitation cutting across all these research areas concerns data centralization and user privacy. Most existing frameworks assume that large volumes of user-generated data can be aggregated into a single training corpus, an assumption that is increasingly untenable given platform access restrictions, data ownership regulations, and ethical obligations toward users [8,9,22]. These constraints are frequently overlooked in experimental evaluations despite being critical for responsible deployment. Recent studies therefore recommend approaches that promote decentralization, domain generalization, and privacy protection as necessary conditions for real-world applicability [26].

Federated learning (FL) has emerged as a promising approach to address both privacy and decentralization challenges. In the FL paradigm, a global model is trained collaboratively across multiple clients, each holding local data, without aggregating raw data at a central location. The seminal work by Saranya et al. introduced federated averaging (FedAvg), a simple yet effective algorithm that averages locally updated model parameters across clients [27]. FedAvg has since become a standard baseline for federated learning research.

Recent research has begun exploring FL for NLP tasks. However, the majority of federated NLP work focuses on language modeling or machine translation rather than sentiment analysis, and few studies specifically address the intersection of federated learning, transformer models, and social media analytics. This gap represents an opportunity to validate if federated learning offers a principled solution to the privacy challenge and transformer architectures provide state-of-the-art representational capacity; their integration tailored to the distributed, dynamic, and linguistically diverse characteristics of twitter/x data remains substantially underexplored.

3. Materials and Methods

This section describes the tools and methods used to design and test a decentralized approach to sentiment analysis on twitter/x, based on a federated transformer model. We discussed the data used for testing, the collaboration for training the model, and the tools employed. The primary objective is to analyze sentiment on twitter/x while ensuring the confidentiality of information. The goal is to develop a sentiment analysis approach for twitter/x based on a federated transformer model.

3.1. Dataset Overview and Data Processing

The experiments are conducted using the sentiment140 dataset [28]. It contains 1.6 million tweets extracted using the twitter/x API. The tweets have been annotated (0 = negative, 4 = positive) and they can be used to detect sentiment. The Sentiment140 dataset was constructed using a distant supervision approach based on emoticon patterns, in which tweets containing positive emoticons were labeled as positive and those containing negative emoticons were labeled as negative. This annotation methodology inherently produces a binary sentiment distribution with no neutral category, making the exclusion of neutral sentiment a direct consequence of the dataset’s construction rather than an arbitrary modeling decision.

To reflect realistic federated scenarios, the Sentiment140 dataset is partitioned across 4 clients following a non-IID (non-independent and identically distributed) strategy. Specifically, we use Dirichlet sampling with α = 0.5 to distribute class labels unevenly across clients, ensuring that the sentiment distribution varies by client. Client 1 receives ~50% positive tweets, Client 2 ~30%, and Clients 3–4 ~60–70%, simulating realistic platform diversity. This configuration tests the model’s robustness under data heterogeneity, a fundamental challenge in federated learning.

The dataset comprises six fields: (1) target, indicating the polarity of the tweet (0 = negative, 4 = positive); (2) id, the unique identifier of the tweet; (3) date, the timestamp of publication; (4) flag, the query keyword used for retrieval (set to NO_QUERY when no query was specified); (5) user, the username of the author; and (6) text, the raw textual content of the tweet.

Tweets are annotated with {positive, negative} labels using public labels when available, and split by client to reflect realistic data partitions; within each client, we further de-duplicate, make stratified splits (train/Val/test), and treat imbalance with class-weighted sampling.

A common preprocessing configuration is mandated across clients for fair comparison: BERT WordPiece tokenizer, max_seq_len = 128, truncation for overflow, right-padding with attention masks, and insertion of special tokens [CLS]; out-of-vocabulary fragments are handled by subword decomposition.

Raw text and user information are not transmitted across clients: only transient token IDs and local batch statistics are employed for training, and we implement per-client pseudonymization and optional differential privacy gradient noise addition during federated learning (training).

3.2. Federated Learning Setup

While tweets are publicly available, the environment of tweet analysis is inherently decentralized and privacy-constrained, making federated learning highly desirable. In the real-world scenario, tweet datasets are not typically stored in one place; instead, the dataset is distributed across different entities, like research organizations, moderation teams, and analysis platforms, with each entity having access to locally curated tweet datasets and, more importantly, proprietary annotations and metadata. Although tweets are publicly available, the associated metadata, like labels, user interaction patterns, and other contextual information, are typically institution-specific, privacy-constrained, and subject to legal and ethical regulations, making it impractical to share the dataset. Social media data are highly heterogeneous across different clients, with variations in language, regional slang, and class imbalance, making it highly impractical for centralized models to generalize across different clients. Federated learning offers the solution by allowing the collaborative fine-tuning of transformer-based models like BERT across different clients, with the ability to maintain local data and share only the updated models, thereby addressing the issue of privacy and improving the robustness of the models. In addition, federated learning offers the advantage of real-world applicability of the proposed solution in the scenario of distributed moderation and analysis systems.

As shown in the Figure 1, the proposed approach follows the federated learning paradigm via the standard FedAvg algorithm, allowing multiple clients to collectively train a shared model without the need to directly access their raw data. In the experimental configuration, we use 4 clients, each of which locally trains the model on their private dataset for a predetermined number of epochs. The training procedure is divided into 100 communication rounds, during which the clients share their locally updated model parameters with the central server. The server then combines these updates to form the next version of the global model. This process continues until convergence, ensuring that knowledge is properly shared among the clients while strictly enforcing data locality. Crucially, no raw textual data is shared between the clients and the server, thus properly addressing privacy concerns related to data leakage. Moreover, using multiple clients over multiple rounds improves scalability and robustness, while also decreasing communication overhead compared to the centralized training approach. The incorporation of FedAvg into this approach thus provides a good trade-off between efficiency, privacy preservation, and adaptability, making it more appropriate for sensitive tasks.

The pipeline includes tweet ingestion, preprocessing, embedding, and feature extraction, federated learning (FL) training, and global model inference. In this configuration, the clients train models on their local private datasets and transmit only the model updates to the server. The server then aggregates the parameters using the BERT-based algorithm to build the global model, as shown in Figure 2.

As depicted in Figure 2, the proposed pipeline integrates a BERT-based representation model within a federated learning (FL) paradigm following a FedAvg-style aggregation scheme. The process begins with tweet ingestion and preprocessing at the client level, where raw textual data are normalized and tokenized using a pre-trained BERT base uncased tokenizer. The tokenized sequences are then processed by the BERT encoder, consisting of multiple transformer layers, to produce contextualized representations, with the [CLS] token used as a compact sentence-level embedding for classification. Each client fine-tunes the BERT model locally on its private dataset by optimizing a cross-entropy loss function, without sharing raw tweets. At the end of each training round, only the locally updated model parameters are transmitted to the central server. The server aggregates these parameters using the federated averaging (FedAvg) algorithm to obtain a global BERT model, which is redistributed to clients for subsequent rounds of training. The performance of the model is checked using tests, and the final aggregated model is deployed for global inference. This BERT-FL formulation enables effective sentiment analysis while ensuring data privacy and scalability across decentralized data sources.

3.3. Federated Averaging

FedAvg aims to bring together multiple devices, clients, or machines and guarantee the confidentiality of each entity’s data to improve the accuracy of the model. This model is distributed across several devices, each updating it with its own data. Only model updates are sent to a server, not the text data itself. This ensures data confidentiality. In media analytics, data is dispersed, and confidentiality is paramount. This approach is therefore ideally suited, and the FedAvg strategy helps coordinate this process. Each device works independently to update the shared model, and the model parameters are exchanged with the server. This approach avoids sending raw text data. It is particularly useful in media analytics, where data is naturally dispersed, and confidentiality is critical. The FedAvg approach addresses these challenges and refines the BERT-based sentiment model. The model is improved thanks to the distributed client base, while preserving data locality.

Formally, let K denote the number of clients and ${\mathit{D}}_{\mathit{k}}$ the local dataset of client K, with ${\mathit{n}}_{\mathit{k}}=\left|{\mathit{D}}_{\mathit{k}}\right|$. At communication round t, the server broadcasts the global BERT parameters θt to all clients’ Fake Account [29]. Each client fine-tunes both the BERT encoder and the classification head by minimizing a local cross-entropy loss:

$${\mathcal{L}}_k(\theta ) ={\displaystyle \frac{1}{n_k}} {\displaystyle {\sum }_{i=1}^{n_k}}\mathcal{l}\left(f_{\theta }\right(x_i), y_i)$$

(1)

where $f_{\mathit{\theta }}$ denotes the BERT-based classifier. After a fixed number of local epochs, clients transmit their updated parameters ${\mathit{\theta }}_k^{t+1}$ to the server.

$${\mathit{\theta }}^{t+1}={\displaystyle {\sum }_{k=1}^K}{\displaystyle \frac{n_k}{{\sum }_{j=1}^K{n}_j}} {\theta }_k^{t+1}$$

(2)

The aggregated parameters are redistributed to the clients for the next round of training. This iterative process enables collaborative learning of contextualized text representations while ensuring that all tweet data remains local to each client, thus achieving a balance between model performance, scalability, and privacy preservation.

3.4. Experimental Environment

The experimental step and evaluation measures used in the performance of the proposed federated learning approach are presented in this section, including accuracy, precision, recall, F1-score, and PR-AUC [30].

Accuracy measures the total accuracy of the model’s prediction, and the equation is calculated using Equation (3).

$$Acc={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$

(3)

Precision is a measure that evaluates the ratio of positive predictions and is given by Equation (4):

$$P={\displaystyle \frac{TP}{TP + FP}}$$

(4)

Recall is the metric that measures the ratio of positive instances correctly detected by the model and is calculated by Equation (5).

$$R={\displaystyle \frac{TP}{TP+FN}}$$

(5)

The F1-score combines precision and recall through their harmonic mean and is determined by Equation (6):

$$F1={\displaystyle \frac{2 \times P \times R}{P + R}}$$

(6)

The PR-AUC estimates the trade-off between accuracy and recall for different classification levels.

$$PR-AUC={\int }_0^{1}Precision\left(r\right), dr$$

(7)

The study parameters and configuration are presented in

Table 1. Federated Learning Experimental Setup and Training Hyperparameters.

Parameter	Value
Base Model	BERT
Round	100
Number of Clients	4
Communication Rounds	100
Local Batch Size	16
Learning Rate	1 × 10−4
Local Batch Size	16
Optimizer	ADAM
Max Sequence Length	128
Training Split	75%
Validation Split	5%
Test Split	20%
Aggregation Strategy	FedAvg
Hardware	CPU Intel Core i5
Framework	Pytorch v2.2.1
Local Epochs Training	5

. The model training data is divided into 4 partitions, and each client uses different portion of the data and uses a BERT-based model for sentiment analysis over 100 rounds. The 75% of data is used for the training set, 20% for the test set and 5% is used for validation. The proposed study is evaluated using the F1 score, accuracy, recall, precision and PR-AUC, and these metrics provide robust evaluation of classification. To further analyze the training dynamics, loss curves are examined throughout the federated optimization process. Monitoring both training and validation loss enables an assessment of convergence stability and generalization performance and helps identify potential overfitting or underfitting effects across communication rounds.

4. Results

This section presents and analyzes the experimental results obtained using the proposed federated transformer-based approach for decentralized twitter/x sentiment analysis. The objective is to evaluate both the convergence behavior of the federated training process and the resulting classification performance across communication rounds.

Figure 3a illustrates the evolution of learning accuracy over time. This accuracy improves with each communication iteration in federated learning. This means that the clients, collaborating effectively, learn from each other. At the beginning, the accuracy of learning is 0.7745, after the first round of federated learning. Then, the accuracy of learning keeps getting better. By the round of federated learning, the accuracy is 0.8094, and converges to 0.9500 by Round 100. The advantage is that the accuracy of the learning does not degrade subsequently. Clients collaborate and share information efficiently throughout the federated learning process.

With each iteration, accuracy improves, demonstrating the effectiveness of federated learning when data is distributed. The overall model, which leverages customer data, remains high-performing. It combines the knowledge of all customers and continues to improve.

Analysis of classification accuracy reveals a gradual improvement. Initially, accuracy is 0.4458, recall is 0.4253, and the F1 score is 0.4153. These relatively low values are normal at the beginning of training, as the model is still relatively untrained. Distinguishing between classes is difficult. At iteration 25, these results improve significantly. Accuracy reaches 0.5562 and recall 0.6005, corresponding to an F1 score of 0.5775. A clear improvement in the model is therefore observed in the training phase. At iteration 50, accuracy is 0.6887 and recall 0.7155, resulting in an F1 score of 0.7018. The fact that accuracy and recall increase simultaneously shows that the model is improving. This improvement is not solely due to the classification threshold. It means that the model is learning to make decisions. This process continues throughout training. By round 75, both accuracy and recall exceed 0.82. By round 100, they reach 0.9500. The final F1 score of 0.9500 indicates that the classification is highly effective and well-balanced. Accuracy, recall, and the classification’s F1 score are all very high.

In the first round, the AUC-PR score is quite low (0.4600). This indicates that it is difficult to distinguish classes during the early training phases. As federated training progresses, the AUC-PR score gradually improves. It reaches 0.6035 in round 25 and 0.7235 in round 50. Improvements are observed in subsequent rounds. At round 75, the AUC-PR score reaches 84.89% and reaches 96.76% at round 100, which means that the model excels at distinguishing between classes. These improvements show that the federated transformer model understands sentiments better, including those that are rare or difficult to classify. The high AUC-PR score at the end of training demonstrates the model’s effectiveness and its performance even when the classes are unbalanced. The federated transformer model excels in much more than just overall accuracy. Its AUC-PR score, nevertheless, remains high.

Training and validation loss curves help us understand how well the federated approach is optimized. Figure 3 shows that both training loss and validation loss go down as the number of iterations increases. The training loss starts at 0.8659 at iteration 1. It comes down to 0.6761 at iteration 25. Then it drops to 0.4619 at iteration 50. The training loss keeps decreasing until it reaches 0.0740 at iteration 100. The validation loss follows a pattern. It starts at 0.9077 and steadily decreases. After the number of iterations, the validation loss reaches 0.1057. The training loss and validation loss keep decreasing as the iterations increase. The results demonstrate that the proposed federated approach presents stable convergence, as evidenced by the constant reduction in training and validation losses over rounds. The close alignment between training and validation losses throughout training indicates stable optimization and limited overfitting. The data is scattered and differs from one computer to another. When we combine all the information from these computers, the results remain very good, even with new, previously unseen data. This demonstrates the effectiveness of the training method and the combination of information from all the computers. The federated aggregation process works perfectly thanks to the training configuration and aggregation strategy used. This process is very efficient.

Table 2. Performance metrics across selected federated learning rounds.

Rounds	Accuracy	Precision	Recall	F1	PR-AUC	Train Loss	Val Loss
1	0.7745	0.4458	0.4253	0.4153	0.4600	0.8659	0.9077
25	0.8094	0.5562	0.6005	0.5775	0.6035	0.6761	0.7158
50	0.8563	0.6887	0.7155	0.7018	0.7235	0.4619	0.5223
75	0.9031	0.8214	0.8380	0.8296	0.8489	0.2743	0.3204
100	0.95	0.9500	0.9500	0.9500	0.9676	0.0740	0.1057

presents the performance indicators of federated towers, such as towers 1, 25, 50, 75 and 100. It can be seen in this table that the performance indicators of the towers improve progressively, and that this improvement occurs in a coordinated manner, for all the elements evaluated for the federated towers. Accuracy increases smoothly with no late-stage degradation. The system’s accuracy and recall improve gradually. This means the improvements are balanced and not simply due to threshold overshot. Training and validation losses decrease significantly. This indicates that the optimization is working correctly and that the system is not becoming overperforming on the training data, a phenomenon called overfitting, which negatively impacts the system’s accuracy and recall. Macro PR-AUC gains in later rounds reflect improved class separation. The consistency between tabulated results and the trends observed in the figures reinforces the reliability of the reported findings.

Table 3. Performance comparison between centralized BERT and federated BERT.

Metric	Centralized BERT	Federated BERT	Difference
Accuracy	0.9614	0.9500	−1.14%
Precision	0.9628	0.9500	−1.28%
Recall	0.9610	0.9500	−1.10%
F1-Score	0.9619	0.9500	−1.19%
$\mathrm{P}\mathrm{R}-\mathrm{A}\mathrm{U}\mathrm{C}$	0.9801	0.9676	−1.25%

compares the proposed federated BERT approach to a fully centralized BERT reference method. Both are trained on the entire Sentiment140 dataset with identical hyperparameters. The centralized reference method achieves 96.14% accuracy and an F1 score of 96.19%, representing the theoretical upper limit of performance achievable without considering data privacy constraints. The proposed federated approach achieves 95.00% accuracy and an F1 score of 95.00%, a performance difference of approximately 1.2% across all metrics. This minimal difference demonstrates that the federated approach proposed preserves the representational capacity of the BERT encoder while ensuring the privacy of tweets for large-scale sentiment analysis on twitter/x.

Overall, the experimental results show that the proposed federated transformer-based approach converges in a stable manner and achieves strong classification performance on decentralized twitter/x sentiment data. The consistent improvements across evaluation metrics, together with steady reductions in training and validation loss and enhanced class-level discrimination, confirm the effectiveness of the approach under the considered experimental setting.

5. Discussion

The outcome of the experiment shows that federated learning is a reliable method of conducting sentiment analysis on the tweet level, protecting both the privacy and the decentralization of the data. Consistent increases in the performance of the classifier on all measures prove that collaborative training of the model on different client devices allows it to learn robust sentiment representations.

The performance metrics, including precision, recall, and F1 score, increase consistently and in a well-coordinated manner across communication iterations. This trend confirms that the improvement in these metrics is linked to effective learning of representations and is not solely the result of threshold optimization. Furthermore, it should be noted that the balanced evolution of precision and recall provides further evidence of the effectiveness of the federated aggregation algorithm, which ensures equal participation and consideration of all contributions made by the various clients. Another interesting observation regarding the results can be made by analyzing the evolution of the PR-AUC metric. One would expect a sharp increase in the values of this indicator during the final communication cycles to be a sign of better class separation and better discrimination at different decision thresholds. In other words, a transformer-based model learns to effectively separate even the most difficult-to-recognize sentiment patterns, which is very important when working with real-world data (twitter/x).

From an optimization perspective, the simultaneous reduction of training and validation losses confirms the stability of the federated learning process. The close alignment between these losses suggests that the model generalizes effectively despite the non-IID distribution of data across clients. This result highlights the suitability of the federated averaging strategy for fine-tuning transformer models in decentralized environments, even when local datasets differ in size and distribution.

These findings underscore the practical relevance of federated learning for social media analytics under privacy constraints. By ensuring that raw tweet data remains localized and only model parameters are exchanged, the proposed approach aligns with data minimization principles and regulatory requirements. This characteristic is particularly important in real-world applications where legal and ethical considerations limit centralized data collection.

Limitations. Despite these promising results, several limitations should be acknowledged. The experimental setup involves a limited number of clients (four), which may not fully capture large-scale heterogeneity. Additionally, the study is conducted on a single dataset (Sentiment140) with binary sentiment labels, which may restrict generalizability to more complex or multilingual scenarios.

6. Conclusions

This paper presented an empirical study on the integration of federated learning with transformer-based architectures for decentralized sentiment analysis on twitter/x data. The primary objective was to evaluate whether high-performing language models such as BERT can be effectively deployed in a privacy-preserving setting without requiring centralized access to user-generated data.

The experimental results demonstrate that the proposed approach achieves stable convergence and strong classification performance across federated communication rounds. Performance metrics improved consistently throughout training, with the final model reaching high accuracy and F1-score values while maintaining robust generalization. The PR-AUC results further confirm the model’s ability to effectively distinguish between sentiment classes, including less frequent or ambiguous instances.

Notably, the federated setting resulted in only marginal performance degradation compared to the centralized baseline, with a difference of approximately 1.2% across key metrics. This finding indicates that federated learning can preserve the representational capacity of transformer models, even under non-IID data distributions. The effectiveness of self-attention mechanisms likely contributes to this resilience by enabling the model to capture contextual dependencies despite fragmented and distributed updates.

These results confirm that privacy-preserving sentiment analysis on social media platforms is both technically feasible and practically viable. The proposed approach achieves a strong balance between model performance and data privacy, making it suitable for real-world applications where data sharing is restricted by regulatory or ethical constraints.

Future work will focus on extending the approach to larger-scale federated environments, incorporating more diverse and multilingual datasets, and exploring advanced aggregation strategies to further improve robustness under heterogeneous conditions. Additionally, comparative evaluations with alternative privacy-preserving learning methods would provide a more comprehensive understanding of the trade-offs involved.