Systematic Fine-Tuning of Transformer Models for Domain-Specific Misinformation Detection in Spanish Social Media Text

Abstract

While social media platforms are primary vectors for misinformation, automated detection systems remain largely confined to English. This paper presents a transferable, three-stage framework for fine-tuning transformer models to detect domain-specific deceptive content in Spanish. The pipeline comprises: (1) corpus unification, merging fragmented datasets into a 61,674-article resource mapped into three classes (Real, Fake, Satire) to prevent stylistic confounding; (2) systematic model optimization, extensively benchmarking classical metaheuristics against eight transformer architectures (including mBERT, XLM-RoBERTa, and BETO) using strong regularization to mitigate overfitting; and (3) production deployment, encapsulating the optimized model as a containerized web application for real-time inference. Through rigorous experimentation, the Spanish-specific BETO encoder emerged as the strongest model for this task, achieving 89.18% overall accuracy. The model attains a near-perfect in-source F1-score on the satire class; however, a strict source-held-out test reveals that this performance is highly source-dependent—recall on satire from an unseen outlet drops to 0.08—indicating that single-source class construction leads the model to recognize the source rather than a generalizable category. We report this finding as a central methodological result: corpus design, and in particular the source diversity of each class, is the primary determinant of whether the framework generalizes. Adversarial robustness tests using named-entity masking and typo injection provide complementary evidence on the model’s reliance on semantic versus surface cues. The methodology is designed to be adaptable across domains: by substituting the training corpus, the same framework may in principle be retargeted to other digital threats, such as investment scams and phishing, provided that suitable labeled corpora are constructed and validated for each new domain. The complete framework, dataset, and application are released as open-source resources to support reproducible research and practical countermeasures against online misinformation.

1. Introduction

The proliferation of deceptive content on social media platforms represents one of the most pressing challenges at the intersection of machine learning and digital society [1]. The rapid expansion of social networks has transformed how information is created, disseminated, and consumed, creating unprecedented opportunities for the spread of misinformation, fraudulent schemes, and manipulative content. This phenomenon disproportionately affects vulnerable demographics with limited media literacy skills [2,3], undermining public trust in journalism, distorting democratic processes, and threatening public health outcomes [4,5]. Critically, the problem extends far beyond traditional fake news: social media platforms are increasingly exploited for investment scams—such as pages impersonating state-owned enterprises with fabricated profit schemes [6]—fraudulent e-commerce impersonations featuring fake testimonials and artificial urgency [7,8], and an alarming proliferation of fake storefronts impersonating major retail chains with fabricated liquidation sales [9,10,11,12]. Celebrity health misinformation—such as pages exploiting public figures’ names to promote fraudulent medical products [13]—further compounds the problem. These diverse forms of digital deception disproportionately target elderly and digitally illiterate populations, and their sheer volume on social media feeds underscores the urgent need for automated, scalable detection systems.

While transformer-based models have achieved state-of-the-art performance in English-language misinformation detection [14,15,16], a fundamental methodological gap persists: the absence of systematic, reproducible frameworks for adapting these general-purpose models to domain-specific deceptive content detection tasks. Most existing studies focus on reporting end-task accuracy without providing a transferable methodology that practitioners can apply to new domains, languages, or content types [17]. This gap is particularly acute for non-English languages and for emerging forms of social media fraud that lack established benchmark datasets.

The field of automated deceptive content detection has evolved through distinct methodological paradigms, reflecting advances in machine learning and natural language processing. Early research in the 2010s relied on classical machine learning approaches—Support Vector Machines (SVMs), Naïve Bayes, Random Forests—combined with hand-crafted linguistic features such as Term Frequency-Inverse Document Frequency (TF-IDF) and Bag-of-Words (BoW) representations [18,19,20]. While computationally efficient, these methods struggled to capture semantic nuances, sarcasm, and contextual meaning, limiting their effectiveness against sophisticated misinformation tactics.

The mid-2010s witnessed a paradigm shift toward deep learning architectures. Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) enabled automatic feature extraction from text, eliminating the need for manual feature engineering [21,22]. However, these models were constrained by their inability to process long-range dependencies and bidirectional context effectively.

The introduction of transformer architectures—particularly BERT [23] and its variants—revolutionized the field by capturing bidirectional contextual representations through self-attention mechanisms [24]. Recent surveys [15,17] have documented the superiority of transformer-based approaches across multiple languages, with models such as FakeBERT [14] leveraging datasets like LIAR [25] and FakeNewsNet alongside techniques including emotion-aware multitask learning [26] and hybrid CNN-RNN architectures [21].

However, despite these architectural advances, three interconnected challenges remain largely unaddressed in the literature. First, there is a lack of systematic fine-tuning methodologies: most studies report results from a narrow hyperparameter search without documenting the optimization trajectory or providing reproducible regularization strategies that can be transferred to new tasks [27,28]. Second, there is a scarcity of domain-specific resources for non-English languages, particularly for Spanish—a language with over 500 million native speakers—where datasets are fragmented and incompatible [15,29]. Third, a persistent deployment gap separates academic models from practical tools [19]: while papers report high accuracy, there is a striking absence of production-ready applications that communities can use for real-time verification.

This paper addresses these challenges by proposing a systematic and transferable framework for fine-tuning pretrained transformer models for domain-specific misinformation detection on social media. The framework is structured as a three-stage pipeline—corpus unification, systematic regularization, and production deployment—and is intended to be adaptable across domains: while validated here on Spanish-language fake news detection, the same methodology could in principle be applied to detect other forms of social media deception, such as investment scams (e.g., fraudulent cryptocurrency or petroleum investment pages), fake e-commerce sites impersonating legitimate retailers, and phishing campaigns, provided that appropriate labeled corpora are constructed for each target domain. We note that these additional domains are presented as motivating directions for future work rather than as empirically validated results in this study.

A progressive research design was adopted: a baseline is first established with classical NLP methods (TF-IDF representations optimized via five metaheuristic algorithms), followed by a fine-tuned transformer model whose systematic regularization methodology is documented in full.

The main contributions of this work are as follows:

• A Transferable Fine-Tuning Framework: A systematic three-stage pipeline (corpus unification, strong regularization, containerized deployment) is proposed for adapting pretrained transformer models to domain-specific deceptive content detection. The framework is explicitly designed to be transferable across languages, content domains, and social media platforms.
• A Systematic Regularization and Early Stopping Methodology: Through extensive experimentation across multiple architectures, we document a strong regularization strategy for fine-tuning the monolingual BETO encoder. The methodology achieved 89.18% overall accuracy and a 0.9095 Macro F1-Score on a challenging 3-class paradigm, with results reproduced across two independent deep learning frameworks. The training trajectories demonstrate how a dynamic early stopping mechanism, combined with this regularization regime, keeps the validation loss on a stable plateau and prevents the catastrophic overfitting commonly observed in high-capacity transformers, providing a reproducible recipe for practitioners facing similar optimization challenges.
• A Unified Spanish Corpus: A standardized corpus of 61,674 Spanish news articles was constructed from four academic datasets [27,29,30,31] enhanced with web-scraped satirical content, achieving a three-class distribution (35.3% fake, 50.2% real, 14.5% satire)—one of the largest resources for this task.
• A Containerized Prototype Web Application: The optimized model was deployed as a Docker-containerized web application capable of real-time URL analysis, demonstrating that the framework extends beyond academic benchmarks toward a practical tool for public misinformation verification [19]. We describe it as a prototype, since the source-held-out evaluation indicates limitations (notably for satire) that should be addressed before production use.

The complete framework is publicly available to encourage reproducibility. By substituting the training corpus, the same pipeline could in principle be adapted to detect investment scams, phishing pages, fraudulent e-commerce sites, and other forms of social media deception; demonstrating this adaptability empirically on additional domains is a primary direction for future work.

2. Related Work: Detection Paradigms and Methodological Gaps

The scholarly approach to deceptive content detection has evolved through several paradigms, shifting from socio-cognitive media literacy approaches to highly complex neural architectures. As summarized in

Table 1. Comparison of Methodological Paradigms in Deceptive Content Detection on Social Media.

Paradigm	Core Principle	Key Studies	Strengths	Limitations
Media Literacy & Societal Impact	Address root causes through critical thinking education	Higdon [2], Higdon [3], Tsfati et al. [4]	Addresses cultural and cognitive factors	Does not provide automated detection
Classical Machine Learning	Statistical features from text (TF-IDF, BoW)	Shu et al. [18], Ali et al. [19], Thota et al. [32]	Computationally efficient, interpretable	Limited semantic understanding
Deep Learning (CNN/RNN)	Automatic feature extraction via neural networks	Nasir et al. [21], Zhou & Zafarani [22]	Better feature learning, reduced manual engineering	Limited handling of long-range dependencies
Transformers (BERT-based)	Bidirectional context via self-attention mechanisms	Singh et al. [15], Alghamdi et al. [17], Kaliyar et al. [14], Rout et al. [16]	State-of-the-art performance, captures semantic nuances	Computationally expensive, requires large datasets
Multimodal & Emotion-Aware	Integrates text, images, and emotional features	Choudhry et al. [26], Zhen & Li [33], Shang et al. [34]	Holistic analysis, improved accuracy	Complex architectures, limited Spanish resources

, early computational efforts relied heavily on classical machine learning models utilizing static statistical features like TF-IDF. While these provided interpretable baselines, their inability to capture sequential semantic dependencies led to the adoption of deep learning architectures (CNNs and RNNs). Recently, transformer-based models have dominated the field by leveraging bidirectional self-attention to contextualize nuanced deceptive language.

2.1. Language Resources and the “Deployment Gap”

A significant bottleneck for advancing deceptive content detection—whether fake news, scam pages, or fraudulent content—is the availability of large-scale, comprehensive datasets, particularly for languages other than English. Furthermore, a persistent issue in the academic NLP community is the “deployment gap”: the divide between models achieving high accuracy in controlled research environments and the scarcity of practical, production-ready tools available to the public.

2.2. Systematic Fine-Tuning and State-of-the-Art Transformers

While models like BERT and its multilingual variants achieve strong results in NLP tasks, they require careful hyperparameter tuning—a process rarely documented systematically in the literature. Recent comprehensive reviews [15,17] highlight that while hybrid and multimodal architectures exist, properly configured transformers [16] remain the most robust approach for textual analysis. Specifically for the Spanish language, recent benchmarks emphasize the need to compare base multilingual models, such as mBERT [23] and XLM-RoBERTa [35], against language-specific encoders like BETO [36] and RoBERTa-bne [37] to reliably establish the strongest-performing approach [27]. This approach of systematic hyperparameter optimization for BERT-based models in Spanish NLP tasks has also been demonstrated in the biomedical domain [38], further motivating the reproducible fine-tuning methodology proposed in this work.

To synthesize the current landscape and highlight the specific gaps addressed by this research,

Table 2. Summary of key related works (Part 1): Corpus Creation & Transformer Application for Deceptive Content Detection.

Author(s) [Ref.]	Contribution	Key Finding/Performance	Limitation Addressed by This Work
Posadas-Durán et al. [29]	Created a pioneering Spanish corpus (971 articles) with a stylometric focus.	Established baseline stylometric markers for Spanish fake news.	Corpus is too small for modern transformer fine-tuning.
Kaliyar et al. [14]	Proposed FakeBERT, a deep learning model combining BERT with CNN layers.	Achieved 98.9% accuracy on English datasets.	Focused exclusively on English-language content.
Blanco-Fernández et al. [27]	Applied BERT/RoBERTa to a large, politically-focused dataset (57k articles).	High accuracy, but limited by domain specificity.	Lacked systematic regularization for out-of-domain transfer.
Martínez-Gallego et al. [28]	Explored different BERT variants (including BETO) for Spanish fake news detection.	BETO outperformed multilingual variants in preliminary tests.	No production-ready deployment or adversarial testing.
Rout et al. [16]	Proposed an enhanced attention-based transformer model for reliable fake news detection.	Improved reliability and attention focus.	Focused on English, without multi-class satire separation.

and

Table 3. Summary of key related works (Part 2): Metaheuristic, Classical, and Hybrid Approaches.

Author(s) [Ref.]	Contribution	Key Finding/Performance	Limitation Addressed by This Work
Yildirim [39]	Hybrid multi-thread metaheuristic approach for fake news detection.	Improved optimization speed and feature selection.	Classical models lack deep contextual semantic understanding.
Thota et al. [32]	Early deep learning approach using traditional neural networks for fake news detection.	Demonstrated NN superiority over classic ML classifiers.	Pre-transformer era; lacks bidirectional attention.
Gravanis et al. [40]	Systematic benchmarking of classical ML classifiers and feature representations.	Linguistic cues provide strong baseline signals.	Did not evaluate transferability to modern digital fraud domains.

provide a comprehensive breakdown of recent key studies. Table 2 details approaches centered on corpus creation and transformer models, while Table 3 categorizes works utilizing metaheuristic and classical hybrid architectures.

3. Materials and Methods

The methodology is structured as a transferable three-stage pipeline for adapting pretrained transformers to domain-specific deceptive content detection. While validated on Spanish fake news, each stage is designed to be reusable and adaptable to other detection domains. As illustrated in Figure 1, the pipeline comprises: (1) Corpus Unification—aggregating and balancing heterogeneous datasets; (2) Systematic Model Optimization—comparing classical and deep learning paradigms through a documented regularization trajectory; and (3) Production Deployment—containerizing the optimal model for real-time inference.

3.1. Stage 1: Corpus Unification and Processing

The first stage addresses a common challenge: constructing a unified dataset from fragmented sources. The unification methodology (schema standardization, content-based deduplication, targeted augmentation) is generalizable to any detection domain. A unified corpus was constructed by aggregating four publicly available datasets: the Spanish Fake News Corpus (572 articles) [29], the Acosta Dataset (598 articles) [30], the Tretiakov Dataset (2000 articles) [31], and the Spanish Political Dataset (57,231 articles) [27], yielding an initial pool of 60,401 articles.

Table 4. Exhaustive comparison of the characteristics of the corpora used in the construction of the unified dataset.

Comparative Aspect	Posadas-Durán	Acosta	Tretiakov	Blanco-Fernández	El Deforma
Initial Corpus Size	572 articles	598 articles	2000 articles	57,231 articles	8985 articles
Creation Year	2019–2021	2019	2022	2024	2025 (extraction)
Methodological Focus	Stylometric	NLP Baselines	Traditional ML	Transformers	Satirical Content
Thematic Domain	General	General	General	Political	Satirical/General
Class Distribution	Balanced	Balanced	Balanced	Balanced	Satire Only
Contribution (Pre-Dedup)	0.95%	0.99%	3.31%	94.75%	14.6% (added later)

details each source.

To ensure data quality and mitigate semantic overlaps that could cause data leakage, a robust deduplication protocol was implemented following the preprocessing guidelines established by Acosta [30]. Prior to deduplication, texts underwent normalization (lowercasing, removal of non-alphanumeric characters, and whitespace standardization). Content hashing on the normalized text identified and removed exactly 7712 near-duplicates from the initial 60,401 collected records. This precise filtering yielded exactly 52,689 unique articles (21,746 fake and 30,943 real).

Methodological Evolution: From Binary to Multiclass Classification. Initially, the research design conceptualized a binary classification task (Real vs. Fake), incorporating satirical articles as part of the “Fake” class to address class imbalance. However, extensive literature and preliminary evaluations revealed a critical construct-validity vulnerability: satire relies on irony, exaggeration, and parody markers, whereas malicious fake news relies on deceptive imitation of journalistic tone. Conflating the two forces the model to learn source-specific humor rather than robust misinformation cues, leading to miscalibration.

To address this, the methodology evolved into a three-class classification formulation: FAKE (0), REAL (1), and SATIRE (2). The final, rigorously structured corpus consists of exactly 61,674 records: the 52,689 deduplicated factual/deceptive articles, supplemented by an isolated class of 8985 satirical articles extracted via web scraping from regional sources. This structural evolution and the resulting class distribution, which transitioned from a highly imbalanced binary setup to a three-class architecture, is visually summarized in Figure 2.

This three-class architecture ensures the model learns to explicitly distinguish malicious misinformation from legitimate creative parody.

Regarding sequence length, although the BERT-based architectures used here support up to 512 subword tokens, exploratory analysis showed that the most discriminative deceptive markers (e.g., clickbait headlines and urgency triggers in the opening sentences) are heavily concentrated at the beginning of each article. We therefore tokenized inputs to a maximum length of 128 tokens for all fine-tuning experiments, which substantially reduced memory and compute requirements while preserving the leading content most relevant to classification. This choice avoids the overhead of long-sequence encoders (such as Longformer or BigBird) for this specific domain. At inference time, the deployed service uses the architecture’s native 512-token limit (see Section 4.8); the implications of this difference between the training and inference windows are discussed below.

In practical terms, this configuration reflects a deliberate efficiency choice rather than an empirically optimized one. Fine-tuning was performed with a 128-token window to reduce memory and compute cost, exploiting the observation that the leading content of an article carries most of the discriminative signal in our corpus. The deployed inference service, by contrast, tokenizes inputs with a 512-token limit, so that longer articles are not truncated as aggressively at prediction time. We did not run a controlled experiment to quantify how this train/inference mismatch affects predictions—for example, by comparing accuracy under 128- versus 512-token inference—so we refrain from claiming a performance benefit (or cost) for either setting. We flag this as a limitation: a systematic study of the effect of input length, and alignment of the training and inference windows, is left for future work. Practitioners adapting the system to domains where the salient signal is not concentrated at the start of the document should consider re-tuning with a longer training window.

3.2. Stage 2: Systematic Model Optimization

The second stage focuses on systematic fine-tuning documented as a reproducible optimization trajectory. Rather than reporting a single configuration, the full evolution of hyperparameter decisions is recorded, providing a transferable recipe for practitioners. The optimization begins by establishing a classical baseline.

Data Partitioning and Generalization Strategy

The unified corpus ($N=61,674$) was partitioned into three disjoint subsets (70% train, 10% validation, 20% test) using stratified random sampling to preserve the three-class distribution across all partitions. Concretely, partition was produced by two successive calls to scikit-learn’s train_test_split with a fixed seed: a first split separating 70% for training from a 30% temporary set and a second split dividing that temporary set into validation (one third, i.e., 10% of the total) and test (two thirds, i.e., 20% of the total), with stratification on the class label at both steps. The resulting per-class sizes are reported in

Table 5. Per-class partition sizes of the unified three-class corpus under the stratified 70/10/20 split. The same fixed partition (global seed 42) was used for every transformer architecture to ensure comparability.

Class	Train (70%)	Validation (10%)	Test (20%)	Total
FAKE	15,221	2175	4350	21,746
REAL	21,660	3094	6189	30,943
SATIRE	6290	898	1797	8985
Total	43,171	6167	12,336	61,674

. The test-set class supports (4350/6189/1797) are exact, as reported by the evaluation pipeline; the train and validation figures are derived from the stratified 70/10/20 ratio.

This hold-out strategy, combined with the regularization regime described above, is intended to ensure that the reported metrics reflect generalization to unseen examples rather than memorized patterns. We note, however, that this is a stratified random split over the merged corpus and therefore does not by itself control for source- or style-level leakage; a complementary source-held-out evaluation is discussed as a limitation and direction for future work in Section 5.5.

Deduplication and split ordering. Deduplication was performed before partitioning and over the merged factual/deceptive pool, so that near-duplicate articles cannot be split across the train, validation, and test sets. After normalization (lowercasing, removal of non-alphanumeric characters, whitespace standardization), content hashing removed 7712 near-duplicates from the 60,401 initial factual/deceptive records, yielding 52,689 unique articles, to which the 8985 satirical articles were then added to form the final three-class corpus of 61,674 records.

Seeds and repetition. To support reproducibility, a single global random seed (42) was fixed across Python, NumPy, and the deep learning backends prior to partitioning, weight initialization, and shuffling. We report results from this single seed; multi-seed runs with confidence intervals were not performed in this study due to the substantial cost of fine-tuning BETO, and are identified in Section 5.5 as a direction for future work.

A “hold-out” validation strategy combined with the regularization regime (the Final BETO configuration described below) is used so that reported metrics reflect generalization rather than memorization.

Classical Machine Learning Baseline. A baseline was established using TF-IDF feature representation [18], with dimensionality reduced to 800 features via Chi-squared test. Five metaheuristic algorithms—Multi-Start Simulated Annealing (MSA), Scatter Search (SS), Variable Neighborhood Search (VNS), Genetic Algorithm (GA), and Particle Swarm Optimization (PSO)—optimized the hyperparameters of a logistic regression model, maximizing F1-Score.

Deep Learning with Transformers. To overcome classical methods’ semantic limitations, a comprehensive benchmarking of eight transformer architectures was formulated. This battery included multilingual encoders (mBERT, XLM-RoBERTa, DistilBERT [41]) and Spanish-specific base and distilled models (

Table 6. Comparison of the 8 Transformer models systematically evaluated for the classification task.

Model	Parameters	Layers	Hidden Dim.	Language Scope	Reference
mBERT (base-multilingual)	110M	12	768	Multilingual (104)	Devlin et al. [23]
XLM-RoBERTa-Base	270M	12	768	Multilingual (100)	Conneau et al. [35]
XLM-RoBERTa-Large	550M	24	1024	Multilingual (100)	Conneau et al. [35]
RoBERTa-base-bne	125M	12	768	Spanish (BNE)	Gutiérrez-Fandiño et al. [37]
RoBERTa-large-bne	355M	24	1024	Spanish (BNE)	Gutiérrez-Fandiño et al. [37]
BETO (Spanish BERT)	110M	12	768	Spanish	Cañete et al. [36]
DistilBETO	67M	6	768	Spanish	Cañete et al. [36]
DistilBERT-multilingual	66M	6	768	Multilingual (104)	Sanh et al. [41]

). This extensive evaluation identified the Spanish-specific BETO encoder as the best-performing model for this domain.

For statistical reliability of the results and to guarantee model generalization to unseen data, a rigorous data splitting protocol was implemented, as illustrated in Figure 3.

Hardware Constraints and Framework Compatibility. Due to the stringent Video RAM (VRAM) requirements of large transformer models (e.g., XLM-RoBERTa-Large with 550M parameters), training the full battery of eight models was computationally unfeasible on local hardware equipped with 8GB of VRAM (NVIDIA RTX 2060 SUPER and RTX 4060 Laptop GPUs). Consequently, the massive hyperparameter search and initial benchmarking were executed in a cloud environment (Lightning AI) utilizing PyTorch (v2.8.0), PyTorch Lightning (v2.6.0), Optuna (v4.8.0), and HuggingFace Transformers (v5.6.2).

To ensure strict reproducibility and enable local deployment, the optimal configurations were subsequently reconstructed and validated on the local RTX 4060 Laptop GPU. This local environment was strictly pinned to TensorFlow-GPU (v2.10.0), Keras Tuner (v1.4.7), scikit-learn (v1.3.2), and Transformers (v4.24.0). This specific dependency freezing was mandatory to resolve severe compatibility bottlenecks, as native GPU support for TensorFlow on Windows was deprecated in later versions. The ability to reproduce the architecture across both PyTorch/Optuna (cloud) and TensorFlow/Keras (local) further supports the framework’s reproducibility.

Hyperparameter Fine-tuning and Reproducibility. Over 30 experiments and 500 GPU hours were conducted to identify configurations that mitigate overfitting—where the model memorizes training data but fails to generalize—as illustrated in Figure 4.

It is important to note that this hyperparameter optimization was carried out on the corpus in its original binary formulation (Real vs. Fake, prior to the introduction of the Satire class), using DistilBERT as the reference architecture. The resulting V11 regularization strategy was subsequently validated and confirmed on the final three-class corpus across all eight transformer architectures. To guarantee exact reproducibility across all experiments, a global random seed (42) was fixed across Python, NumPy, and the deep learning framework backends prior to data partitioning and weight initialization.

The hyperparameter search for the final three-class BETO model was conducted independently in each framework, both minimizing the validation loss as the primary objective. In the cloud environment, Optuna explored the space via categorical sampling; in the local environment, Keras Tuner (RandomSearch) explored an equivalent space. Owing to the substantial cost of fine-tuning BETO (each trial requiring several GPU-hours), the search was deliberately focused on a compact, high-impact grid of three trials per backend. The search space comprised learning rates ($1\times {10}^{-6}$, $2\times {10}^{-6}$, $5\times {10}^{-6}$), dropout rates (0.3, 0.4, 0.5), L2 penalties (0.01, 0.02, 0.05), and a Gaussian weight-noise factor (0.005, 0.01), with a batch size of 8 in the local run and 8–16 in the cloud run. Architecturally, the dropout probability was applied to both the hidden and attention layers via the model configuration, while the L2 penalty and the manual weight-decay callback acted on the encoder kernels to constrain capacity without distorting the pretrained attention weights. The complete search configuration for both backends is released in the public repository. The same fixed stratified partition (Table 5, seed 42) and the same regularization configuration were applied to all eight transformer architectures in the three-class benchmark, so that the comparison is controlled. For completeness, the configurations and results of the omitted intermediate exploratory versions (V7–V10), as well as the per-class precision, recall, and F1 of every model in the three-class benchmark, are provided in the public repository accompanying this paper.

The regularization strategy evolved through multiple configurations during an initial exploratory phase conducted on the binary corpus (Real vs. Fake) using DistilBERT as a fast, low-cost reference architecture.

Table 7. Evolution of hyperparameter configurations during the preliminary exploratory phase (binary Real vs. Fake task, DistilBERT reference architecture). These values document the search trajectory that established the qualitative anti-overfitting recipe; they are not the configuration of the final three-class BETO model (reported in the Final BETO configuration paragraph and the main text). Accuracy values correspond to the binary task.

Version	Learning Rate	Dropout	L2 Reg.	Batch Size	Val Loss Gap	Accuracy (%)
V1 (Baseline)	$3\times {10}^{-5}$	0.4	0.001	8	N/A	94.7
V2	$2\times {10}^{-6}$	0.4	0.01	4	0.018	94.3
V3	$2\times {10}^{-6}$	0.4	0.01	4	0.051	94.8
V4	$1\times {10}^{-5}$	0.3	0.01	8	0.037	95.8
V5	$1\times {10}^{-5}$	0.4	0.1	8	0.037	95.8
V6	$1\times {10}^{-5}$	0.5	0.5	8	0.051	94.8
V11 (binary milestone)	$\mathbf{5}\times {\mathbf{10}}^{-\mathbf{6}}$	0.7	0.5	4	0.058	95.36

documents the milestone versions of this exploratory trajectory; intermediate versions (e.g., V7 through V10) are omitted for brevity but followed the same path toward progressively stronger regularization. This exploratory phase established the qualitative “Anti-Overfitting” recipe later transferred to the final model: a low learning rate for stable convergence, elevated dropout, an explicit L2 penalty, a small batch size to inject stochastic gradient noise, and dynamic early stopping. We emphasize that the numerical values in Table 7 correspond to this preliminary binary exploration and not to the final three-class BETO model, whose validated configuration is reported separately below.

Final BETO configuration (three-class task). Applying this recipe to BETO on the final three-class corpus, the validated configuration was a learning rate of $1\times {10}^{-6}$, a dropout rate of 0.4 (applied to both hidden and attention layers), an L2 penalty of 0.02, a Gaussian weight-noise factor of 0.01, and a batch size of 8, optimized with AdamW (local TensorFlow run; the cloud PyTorch run used an equivalent configuration with a linear warmup schedule over 10% of training steps and a batch size of 16). Inputs were tokenized to a maximum sequence length of 128 subword tokens. This configuration is the one used for all reported BETO results and for the deployed model.

Justification for the anti-overfitting recipe. As observed in Table 7, intermediate configurations such as V4 and V5 initially reported higher raw accuracy (95.8%) on the binary task. However, an ablation analysis of the regularizers revealed that these configurations exhibited a growing validation-loss gap (validation loss diverging from training loss), an early indicator of overfitting.

To prioritize real-world generalization over artificial benchmark inflation, the strongly regularized recipe was retained and carried over to the final three-class BETO model. As detailed above, the validated BETO configuration used a learning rate of $1\times {10}^{-6}$, a dropout rate of 0.4, an L2 penalty of 0.02, and a Gaussian weight-noise factor of 0.01. This combination prioritizes stable convergence and constrains model capacity so that the network learns underlying linguistic patterns rather than dataset-specific noise, at the cost of a modest reduction in raw training accuracy.

3.3. Stage 3: Production Deployment

The final stage of the framework addresses the deployment gap by implementing the optimized model as a containerized web-application prototype. This stage is designed to be model-agnostic: the same containerized architecture can host any transformer model produced by Stage 2, regardless of the specific detection domain. A critical design principle was to minimize the technical barrier for end users: the application requires no machine learning expertise, no local model installation, and no command-line interaction—users simply submit a URL through a web browser. This accessibility is essential for the framework’s stated goal of empowering communities with limited technical resources to combat deceptive content on social media.

The end-to-end inference pipeline is designed to provide immediate, transparent verification of social media content. Upon receiving a target URL or raw text snippet, the system autonomously executes content extraction, text normalization, subword tokenization (via the model’s specific tokenizer), and transformer-based inference in a single automated pass. Notably, the system outputs not only the final multiclass prediction (REAL, FAKE, or SATIRE) but also the calibrated confidence scores derived from the final softmax layer (Figure 5). This probabilistic output empowers end-users and fact-checkers to assess the model’s certainty, which is particularly valuable in edge cases where texts exhibit overlapping stylistic traits or a mix of factual and deceptive claims.

The system consists of four integrated components, each designed for modularity and replaceability:

• User Interface: A responsive static web frontend built with HTML, CSS, and JavaScript where users submit URLs for analysis. The interface provides real-time visual feedback during the analysis process—including loading indicators and color-coded results (green for real, red for fake)—to ensure an intuitive user experience for non- technical users.
• API Backend: A Flask-based RESTful service exposing an /analyze endpoint that receives POST requests containing the target URL. The backend implements error handling for common failure scenarios such as unreachable URLs, non-textual content, and pages with insufficient text for meaningful classification.
• Inference Engine: The core analytical component loads the trained BETO model and tokenizer at startup to minimize per-request latency. Upon receiving a URL, it scrapes the page content by extracting <h1> and <p> tags via BeautifulSoup, applies text cleaning (removing HTML artifacts, normalizing whitespace), tokenizes the combined title and body with a [SEP] separator token, and computes probability scores through a softmax function. The engine truncates inputs exceeding 512 tokens—BETO’s maximum sequence length—ensuring graceful handling of long-form articles.
• Docker Deployment: The entire runtime environment—including Python 3.10 dependencies, PyTorch, the Hugging Face Transformers library, and the serialized model artifacts—is encapsulated in a Docker image built from a minimal base to ensure reproducibility, portability, and ease of deployment across platforms.

To support scalability, reproducibility, and adaptability across domains, the detection pipeline was encapsulated within a containerized microservice architecture. This modular approach logically separates the transformer inference engine from the client interface. This design is intended to facilitate adaptation workflows: when the framework is retargeted to a new domain (e.g., retraining the model on phishing data, investment scams, or emerging misinformation patterns), the model weights and tokenizer artifacts within the Docker image are updated, while the API endpoints and serving infrastructure remain unchanged. This decoupling between the machine learning model and the deployment environment is designed to allow the framework to be replicated across diverse hardware setups—from local research workstations to cloud-based institutional servers—with minimal environment-specific configuration. We emphasize that this architectural adaptability has been validated only on the Spanish fake news task; its effectiveness on other domains remains to be demonstrated.

3.4. Evaluation Metrics

The final task is a three-class classification problem (FAKE, REAL, SATIRE); we therefore evaluate the models using a multiclass formulation rather than the binary positive/negative convention. Per-class metrics are computed under a one-vs-rest scheme: for a given class c, a prediction is a true positive (TP_c) when both the predicted and true labels are c; a false positive (FP_c) when the model predicts c but the true label is another class; a false negative (FN_c) when the true label is c but the model predicts another class; and a true negative (TN_c) when neither the prediction label nor the true label is c. Aggregate scores are reported as macro-averages, i.e., the unweighted mean of the per-class values, so that the minority SATIRE class contributes equally to majority classes. We additionally report overall accuracy. Letting $K=3$ be the number of classes, the metrics are defined as follows.

• Accuracy: The overall proportion of correctly classified instances across all three classes. While a useful global indicator, it can be misleading under class imbalance, which is why macro-averaged metrics are emphasized.

  $$\mathrm{Accuracy}={\displaystyle \frac{{\sum }_{c=1}^K{\mathrm{TP}}_c}{N}}$$

  (1)

  where N is the total number of test instances.
• Macro Precision: The unweighted mean, across classes, of the per-class precision (the proportion of instances predicted as class c that truly belong to c).

  $${\mathrm{Precision}}_c={\displaystyle \frac{{\mathrm{TP}}_c}{{\mathrm{TP}}_c+{\mathrm{FP}}_c}},\mathrm{Macro}\text{-}\mathrm{P}={\displaystyle \frac{1}{K}}{\displaystyle \sum _{c=1}^K}{\mathrm{Precision}}_c$$

  (2)
• Macro Recall: The unweighted mean of the per-class recall (the proportion of instances truly belonging to class c that are correctly identified).

  $${\mathrm{Recall}}_c={\displaystyle \frac{{\mathrm{TP}}_c}{{\mathrm{TP}}_c+{\mathrm{FN}}_c}},\mathrm{Macro}\text{-}\mathrm{R}={\displaystyle \frac{1}{K}}{\displaystyle \sum _{c=1}^K}{\mathrm{Recall}}_c$$

  (3)
• Macro F1-Score: The unweighted mean of the per-class F1-scores, where each per-class F1 is the harmonic mean of that class’s precision and recall. This is the primary metric for model selection, as it is robust to the class imbalance of the corpus.

  $${\mathrm{F}1}_c=2·{\displaystyle \frac{{\mathrm{Precision}}_c·{\mathrm{Recall}}_c}{{\mathrm{Precision}}_c+{\mathrm{Recall}}_c}},\mathrm{Macro}\text{-}\mathrm{F}1={\displaystyle \frac{1}{K}}{\displaystyle \sum _{c=1}^K}{\mathrm{F}1}_c$$

  (4)
• Macro Specificity: The unweighted mean of the per-class specificity, defined in the one-vs-rest setting as the proportion of instances not belonging to class c that the model correctly excludes from c. It quantifies the model’s ability to avoid spuriously assigning instances to a given class.

  $${\mathrm{Specificity}}_c={\displaystyle \frac{{\mathrm{TN}}_c}{{\mathrm{TN}}_c+{\mathrm{FP}}_c}},\mathrm{Macro}\text{-}\mathrm{Spec}={\displaystyle \frac{1}{K}}{\displaystyle \sum _{c=1}^K}{\mathrm{Specificity}}_c$$

  (5)

Macro-averaging calculates each metric for every class independently and then averages without weighting by class frequency, ensuring that performance on minority classes (notably SATIRE) is evaluated equitably against majority classes. All transformer and classical baseline results are reported under this scheme; for completeness, per-class precision, recall, and F1 for the final model are also reported in Section 4.3.

4. Results: Framework Validation on Spanish Fake News Detection

4.1. Performance of the Metaheuristic Baseline (Stage 2a)

The baseline phase evaluated five classical machine learning algorithms optimized via metaheuristics on TF-IDF representations. The Genetic Algorithm (GA) performed best, achieving 72.03% accuracy and a 0.714 macro F1-score (Table 8). This suggests the GA’s evolutionary mechanism efficiently explores the high-dimensional TF-IDF space. Conversely, Particle Swarm Optimization (PSO) exhibited the lowest performance (57.67% accuracy). This variance highlights a critical limitation: classical TF-IDF models lack the semantic understanding necessary for robust fake news detection. The confusion matrices (Figure 6) visually confirm this instability through significant misclassification rates across all metaheuristic models.

Table 8. Preliminary performance comparison on the binary task (Real vs. Fake). These results correspond to the initial phase of the study and are not directly comparable to the three-class benchmark in Table 9.

Algorithm	Accuracy (%)	F1-Score (Macro)	Precision (Macro)	Recall (Macro)	Specificity (%)	Ranking
Transformer Reference (binary task)
DistilBERT (binary reference)	95.36	0.954	0.954	0.954	94.5	1st
Metaheuristic-Optimized Classical Approaches
Genetic Algorithm (GA)	72.03	0.714	0.740	0.720	57.6	2nd
Scatter Search (SS)	67.64	0.669	0.693	0.676	52.7	3rd
VNS	66.78	0.659	0.686	0.667	51.0	4th
Simulated Annealing (MSA)	60.86	0.586	0.638	0.608	37.4	5th
Particle Swarm Opt. (PSO)	57.67	0.489	0.736	0.575	16.3	6th

Table 8. Preliminary performance comparison on the binary task (Real vs. Fake). These results correspond to the initial phase of the study and are not directly comparable to the three-class benchmark in Table 9.

Algorithm	Accuracy (%)	F1-Score (Macro)	Precision (Macro)	Recall (Macro)	Specificity (%)	Ranking
Transformer Reference (binary task)
DistilBERT (binary reference)	95.36	0.954	0.954	0.954	94.5	1st
Metaheuristic-Optimized Classical Approaches
Genetic Algorithm (GA)	72.03	0.714	0.740	0.720	57.6	2nd
Scatter Search (SS)	67.64	0.669	0.693	0.676	52.7	3rd
VNS	66.78	0.659	0.686	0.667	51.0	4th
Simulated Annealing (MSA)	60.86	0.586	0.638	0.608	37.4	5th
Particle Swarm Opt. (PSO)	57.67	0.489	0.736	0.575	16.3	6th

Table 9. Benchmarking of State-of-the-Art Transformer Models on the 3-Class Test Set.

Model Architecture	Accuracy	Macro F1	Macro Precision	Macro Recall	Macro Specificity
BETO (V11 Final)	0.8898	0.9095	0.9129	0.9070	0.9331
XLM-RoBERTa-Large	0.8843	0.9061	0.9069	0.9053	0.9311
RoBERTa-Large-BNE	0.8837	0.9038	0.9081	0.9006	0.9293
XLM-RoBERTa-Base	0.8825	0.9025	0.9095	0.8984	0.9274
DistilBERT Multilingual	0.8795	0.9012	0.9027	0.9000	0.9280
mBERT	0.8730	0.8962	0.8973	0.8953	0.9242
DistilBETO	0.8586	0.8819	0.8889	0.8763	0.9141
RoBERTa-Base-BNE	0.6402	0.5791	0.6581	0.6651	0.7667

Table 9. Benchmarking of State-of-the-Art Transformer Models on the 3-Class Test Set.

Model Architecture	Accuracy	Macro F1	Macro Precision	Macro Recall	Macro Specificity
BETO (V11 Final)	0.8898	0.9095	0.9129	0.9070	0.9331
XLM-RoBERTa-Large	0.8843	0.9061	0.9069	0.9053	0.9311
RoBERTa-Large-BNE	0.8837	0.9038	0.9081	0.9006	0.9293
XLM-RoBERTa-Base	0.8825	0.9025	0.9095	0.8984	0.9274
DistilBERT Multilingual	0.8795	0.9012	0.9027	0.9000	0.9280
mBERT	0.8730	0.8962	0.8973	0.8953	0.9242
DistilBETO	0.8586	0.8819	0.8889	0.8763	0.9141
RoBERTa-Base-BNE	0.6402	0.5791	0.6581	0.6651	0.7667

4.2. Preliminary Analysis: Classical Baselines vs. a Transformer Reference (Binary Task)

Before conducting the final three-class benchmark, a preliminary analysis was performed on the initial binary formulation (Real vs. Fake) to motivate the shift from classical machine learning to transformer-based models. This analysis is not the main comparison of the paper and is reported here only as a sanity check that justified investing in transformer fine-tuning; the main results are those obtained under the three-class formulation in Section 4.3.

Important methodological note. All models in Table 8 (the five metaheuristic-optimized classical models and the DistilBERT reference) were trained and evaluated on the binary corpus (Real vs. Fake), which corresponds to the initial phase of the research design. They therefore use a different task formulation from the final three-class benchmark and must not be read as competing entries against the three-class results. Because the task, label space, and class priors differ, the binary and three-class numbers are not directly comparable, and we deliberately keep them in separate tables.

Within this preliminary binary setting, the DistilBERT reference outperformed the best metaheuristic algorithm (GA) by 23.33 percentage points in accuracy (95.36% vs. 72.03%). This gap illustrates the “semantic gap” of bag-of-words representations: while metaheuristics can optimize a decision boundary over keyword frequencies, they cannot compensate for the lack of contextual understanding inherent in TF-IDF features. This preliminary observation motivated the decision to adopt transformer fine-tuning, after which the task was reformulated into the methodologically sound three-class setting and a full eight-model benchmark was conducted (Section 4.3).

Given that the transformer reference clearly outperformed the classical metaheuristic approaches in this preliminary binary analysis, the study proceeded to the methodologically sound three-class formulation and evaluated an extended battery of eight transformer architectures to identify the optimal encoder for Spanish-language deceptive content.

4.3. Performance of the Systematically Fine-Tuned Transformer (Stage 2b)

To rigorously validate the transformer architecture and address the limitations of classical baselines, a comprehensive benchmarking of eight transformer models was conducted under the exact same three-class formulation (Real, Fake, Satire) and using the same regularization strategy and stratified split. The evaluated models included multilingual encoders (mBERT, XLM-RoBERTa-base, XLM-RoBERTa-large), general Spanish encoders (BETO, RoBERTa-base-bne, RoBERTa-large-bne), and distilled versions (DistilBERT, DistilBETO).

As detailed in Table 9, the Spanish-specific encoder BETO achieved the highest overall performance, indicating that monolingual pretraining captures the morphological and syntactic nuances of Spanish deception more effectively than the multilingual models evaluated here.

The optimized BETO model exhibited very high in-source discriminative power for the SATIRE class, which on the held-out El Deforma partition is recognized almost perfectly. We caution, however, that this in-source performance must be read alongside the source-held-out analysis in Section 4.5, which shows that it does not transfer to satire from a different outlet. The high in-source score indicates that, given satirical data from a single source, the three-class formulation cleanly separates that source’s satire from fake news; it does not establish that the model has learned a source-independent satire category.

Table 10. Per-class performance of the final deployed BETO model on the three-class held-out test set (local TensorFlow run; overall accuracy 0.8918, Macro F1 0.9095).

Class	Precision	Recall	F1-Score	Support
FAKE	0.91	0.77	0.83	4350
REAL	0.85	0.95	0.90	6189
SATIRE	0.99	1.00	1.00	1797
Macro avg	0.92	0.90	0.91	12,336
Weighted avg	0.89	0.89	0.89	12,336

reports the per-class precision, recall, and F1-score of the final deployed BETO model on the held-out test set ($N=12,336$), providing the class-level detail requested for full transparency.

The per-class breakdown reveals that the residual error of the model is concentrated in the FAKE/REAL boundary—FAKE recall of 0.77 indicates that a portion of fake articles is classified as real—whereas the SATIRE class is recognized almost perfectly. This pattern is consistent with the confusion matrix in Figure 7 and supports the construct-validity rationale for isolating satire as a separate class.

4.4. Adversarial Robustness and Source Leakage Mitigation

To directly address concerns regarding potential source leakage—where a model might trivially memorize specific publisher names, recurring political figures, or stylistic formatting rather than learning the underlying deceptive semantics—an adversarial robustness test under strong text perturbation was conducted on the completely held-out test set (12,336 records).

Using the optimal BETO (V11) model, a dual-perturbation pipeline was applied to the text prior to inference:

1.

Named Entity Recognition (NER) Masking: Utilizing the es_core_news_sm SpaCy pipeline, all identified Persons (PER), Organizations (ORG), and Locations (LOC) were dynamically replaced with generic masking tags (e.g., [ORG]).

2.

Typographical Noise Injection: To simulate real-world social media degradation, random character swaps and deletions were injected with a 5% probability per word.

Under this severe compound degradation, the model’s overall accuracy declined substantially, from 89.18% to 63.40%, yielding a Macro F1-score of 0.71. This drop is itself informative: it confirms that a meaningful portion of the model’s discriminative signal for the REAL/FAKE distinction depends on named entities, which is expected given that factual verification is inherently entity-dependent. A class-wise analysis, however, reveals a more nuanced picture. The SATIRE class remained markedly robust (F1-score: 0.96; Recall: 0.92) even when proper nouns were masked and typographical noise was injected. This result suggests that the model recognizes satirical content primarily through syntactic structures, irony markers, and stylistic exaggeration rather than through the specific entities or publishers mentioned. We are careful not to overstate this finding: the test perturbs entities and surface spelling but does not hold out the satire source itself, so it provides evidence against trivial entity memorization but cannot, on its own, rule out that the model has learned source- or style-specific cues correlated with a particular satirical outlet. A dedicated source-held-out evaluation, discussed in Section 5.5, would be required to establish source independence conclusively.

Conversely, the boundary between REAL and FAKE content experienced expected degradation (FAKE F1: 0.60; REAL F1: 0.56; REAL Recall: 0.44). This specific drop validates a core NLP principle: factual verification inherently relies on the entities involved. Masking “who” did “what” and “where” removes the critical factual anchors necessary for distinguishing truth from deceptive fabrication. Maintaining 63.40% accuracy and a 0.71 Macro F1-score when entirely blinded to proper nouns and subjected to spelling noise indicates that the model captures baseline semantic deception cues beyond proper-noun memorization. We emphasize, however, that robustness to entity masking is a weaker condition than robustness to a change in source: the adversarial test keeps the underlying corpus fixed and only perturbs it. The stronger source-held-out test reported in Section 4.5 shows that, for the SATIRE class in particular, the apparent robustness does not extend to an unseen satirical outlet, which qualifies the interpretation of this adversarial result substantially.

4.5. Cross-Source Generalization of the Satire Class

The adversarial test above perturbs the held-out test set but keeps its underlying sources fixed. To directly test whether the SATIRE class generalizes beyond the single outlet from which it was constructed, we conducted a strict source-held-out experiment. The satirical training data originate entirely from a single Mexican outlet (El Deforma). We therefore assembled an external evaluation set of 1000 satirical articles scraped from a different Spanish-language satirical outlet, El Mundo Today (Spain), which the model never saw during training and which differs in dialect (peninsular vs. Mexican Spanish), authorship, and editorial style. Crucially, this set is used for inference only and is not added to the training corpus. We applied the deployed BETO model to these articles, all of which carry the gold label SATIRE.

The result is stark. While the in-source SATIRE recall is 1.00 (El Deforma test partition), the recall on the out-of-source set collapses to 0.081: only 81 of 1000 genuinely satirical articles are recognized as SATIRE. Of the 919 misclassifications, 796 are assigned to FAKE and 123 to REAL, and the model does so with a mean confidence of 0.76—that is, it is confidently wrong.

Table 11. In-source vs. out-of-source recall for the SATIRE class. The near-perfect in-source recall does not transfer to an unseen satirical outlet, revealing strong source dependence.

Evaluation Set	Source	N	SATIRE Recall
In-source test partition	El Deforma (MX)	1797	1.00
Out-of-source (held-out)	El Mundo Today (ES)	1000	0.08

summarizes the contrast.

This finding is, in our view, one of the most important empirical contributions of the present study, and we report it without mitigation. It demonstrates that the near-perfect in-source satire performance does not reflect a generalizable, source-independent notion of “satire” that the model has learned; rather, it largely reflects recognition of the specific stylistic and formatting regularities of a single outlet. When confronted with satire from a different outlet and dialect, the model defaults overwhelmingly to FAKE—a behavior that is intuitive (both satire and fake news are “non-factual” text) but that confirms the construct-validity concern raised in review.

The result also reframes the central methodological message of this work. The framework is presented as transferable by substituting the training corpus; this experiment shows the precise sense in which that claim must be understood. The corpus is not an interchangeable input but the determinant of what the model actually learns. A class assembled from a single source teaches the model to recognize that source, not the underlying category. This is consistent with the rest of our results: the REAL and FAKE classes, which were assembled from four heterogeneous corpora by different authors, do not exhibit this collapse, whereas the SATIRE class—the only single-source class—does. Robust, generalizable detection therefore depends critically on constructing a diverse, multi-source corpus for every target class. We accordingly temper all claims about satire detection throughout the manuscript and identify multi-source satire corpus construction as essential future work (Section 5.5).

4.6. Overfitting Control Analysis

The implementation of a rigorous regularization strategy successfully mitigated overfitting across all transformer architectures. Rather than relying on a fixed number of epochs—which inherently varies depending on the data subset and initial weight instantiation—training was dynamically governed by an early stopping criterion. Training concluded autonomously when the validation loss plateaued and ceased to improve for eight consecutive passes. The optimal model weights were then automatically restored from the checkpoint that minimized the generalization gap (validation vs. training loss), ensuring peak performance before the onset of memorization.

4.7. Cross-Framework Reproducibility and Optimal Model (BETO) Analysis

To validate the structural integrity of the winning model, the BETO configuration was trained in two independent environments: a cloud setup using PyTorch and Optuna, and a local setup using TensorFlow and Keras. Both runs converged to the same Macro F1-Score of 0.9095, with overall accuracies of 88.98% (PyTorch) and 89.18% (TensorFlow). This close agreement across backends, optimizers, and tuning libraries indicates that the reported performance is a property of the regularization recipe and the data, rather than an artifact of a specific framework. The minor differences in per-epoch trajectories and in the selected best epoch (6 in PyTorch, 27 in TensorFlow) are attributable to backend-specific stochasticity and to the different optimizer schedules; both ultimately reached an equivalent generalization level.

The stability of this convergence is visually confirmed in Figure 8, where the local TensorFlow training session demonstrates a tightly coupled training and validation loss, decisively avoiding the overfitting plateau.

The training dynamics captured in Figure 8 illustrate the regularized convergence behavior of the model. During the first epoch, both training and validation losses decreased rapidly as BETO—pretrained natively on Spanish—quickly adapted to the task. Thereafter, the validation loss flattened into a stable plateau while the training loss continued to decrease only marginally, keeping the generalization gap bounded throughout the full 30-epoch run. The early stopping criterion monitored the validation loss and selected the weights from the epoch of minimum validation loss (epoch 27 in the local TensorFlow run; epoch 6 in the cloud PyTorch run) as the final model, rather than the weights from an arbitrary final epoch. The absence of a sustained upward divergence in the validation loss across the plateau indicates that the regularization strategy successfully suppressed the catastrophic overfitting commonly observed when fine-tuning high-capacity transformers without such constraints.

The generalization gap (difference between validation and training loss) at the selected checkpoint was 0.066, remaining below the 0.10 threshold commonly associated with the onset of overfitting, though not negligible. We report this value transparently: rather than indicating perfect convergence, it reflects a deliberate trade-off in which strong regularization keeps the gap bounded and stable across the training trajectory (Figure 8). Furthermore, addressing the claim of transferability, it is essential to clarify that this framework offers more than the standard machine learning practice of “retraining on new data.” The true innovation lies in the architectural decoupling and the empirical optimization template. By standardizing the regularization recipe, researchers targeting new domains—such as phishing or fraudulent e-commerce scams—may substantially reduce the hyperparameter-search effort. Additionally, the containerized Docker infrastructure handles the web-scraping and preprocessing pipelines, so that adapting the tool to a new domain primarily requires swapping the serialized model weights and tokenizer, with limited additional engineering effort.

4.8. Deployment Validation (Stage 3)

The deployed containerized application was empirically evaluated for end-to-end efficiency under a documented protocol. We measured the latency of the complete pipeline as experienced by a user—HTTP request, live web scraping, HTML parsing, tokenization, and forward-pass classification—by issuing POST requests to the running container and timing each response. The benchmark used 87 distinct real-world URLs spanning three content types (established news outlets, satirical articles, and fact-checking pages), each queried twice (174 requests total), after two discarded warm-up requests to exclude the one-time TensorFlow graph-tracing cost. The container performed CPU-only inference; the host was a 6-core/12-thread AMD CPU with 32 GB of RAM running the Docker image.

Across the 169 requests that produced a classification, the mean end-to-end latency was 1161 ms per URL (standard deviation 295 ms; median 1159 ms; p90 1418 ms; p95 1686 ms; p99 2099 ms; minimum 432 ms, maximum 2748 ms). Of the 174 requests, 92.5% returned a confident classification, 4.6% were flagged as low-confidence (maximum softmax probability below 0.50), and 2.9% failed at the scraping stage—including an intentionally unreachable URL and pages from which insufficient textual content could be extracted—confirming that the backend degrades gracefully on non-textual or unreachable pages rather than failing silently.

These figures show that the dominant cost is network-bound web scraping rather than model inference: the forward pass of the 110M-parameter BETO encoder is comparatively inexpensive, and the sub-second-to-low-second total latency is driven primarily by fetching and parsing live external pages. The latency is therefore a property of the deployment scenario (real-time scraping of arbitrary URLs) rather than of the classifier itself, and could be reduced substantially by caching, pre-fetching, or supplying text directly (the application also exposes a manual text-input mode that bypasses scraping). The architectural decision to decouple the frontend from the inference engine keeps the interface responsive while these operations complete. Overall, the deployment demonstrates that the framework operates at interactive, near-real-time latencies on commodity CPU hardware, without requiring specialized accelerators.

5. Discussion

This study proposed a systematic and transferable framework for fine-tuning pretrained transformer models for domain-specific deceptive content detection on social media, validated through a comprehensive case study on Spanish-language fake news. On the final three-class task, the regularized BETO model reached 89.18% accuracy and a 0.9095 Macro F1-Score. The classical metaheuristic baselines were evaluated only in the earlier, preliminary binary setting and are therefore not directly comparable to this three-class result; they serve to motivate the transition to transformer fine-tuning rather than as a final performance comparison. Beyond the raw metrics, it is crucial to analyze the framework’s strengths, the model’s boundaries, and the potential for domain transfer.

5.1. Strengths of the Proposed Framework

The primary technical contribution lies not in the final accuracy figure, but in the systematic and reproducible nature of the optimization process. By documenting the full hyperparameter trajectory (Table 7)—including failed configurations from the exploratory binary phase—this work provides a practical blueprint that practitioners can adapt for their own detection tasks. A regularization regime combining elevated dropout, an explicit L2 penalty, weight-noise injection, and a low learning rate was identified as critical for preventing overfitting when fine-tuning on domain-specific corpora of moderate size, a challenge that is universal across detection domains, not specific to fake news.

The resulting BETO model demonstrates strong semantic robustness. Unlike the TF-IDF baseline, which relies on keyword frequency and struggles with sarcasm or subtle linguistic cues, the transformer model effectively captures contextual dependencies. This is evidenced by its high macro specificity (93.19%), indicating that the model is effective at distinguishing deceptive content without over-flagging legitimate content—a common pitfall in automated moderation systems. The bounded generalization gap of 0.066 confirms that the regularization strategy kept overfitting under control.

5.2. Validation Against the State-of-the-Art

The empirical results generated by this framework validate and extend recent findings in the deceptive content detection literature. In the context of Spanish NLP, Blanco-Fernández et al. [27] demonstrated the high efficacy of BERT-based models on specific political datasets. Our findings align with theirs but expand the scope by demonstrating that a Spanish-specific encoder (BETO), coupled with our strong regularization strategy, maintains competitive performance (Macro F1 = 0.910) across a highly heterogeneous, multi-domain corpus.

Furthermore, our results corroborate the observations by Martínez-Gallego et al. [28] regarding the superiority of monolingual models over massive multilingual encoders. By directly benchmarking BETO against mBERT and XLM-RoBERTa (Table 9), we show that monolingual pretraining captures the morphological and syntactic nuances of Spanish deception more effectively. Finally, while English-centric models like FakeBERT [14] report extremely high binary accuracies (up to 98.9%), our multi-class formulation separates satire from fake news at the level of the in-source distribution; however, our source-held-out analysis (Section 4.5) cautions that such separation does not by itself guarantee generalization to unseen satirical sources, underscoring that high benchmark accuracy and real-world reliability remain distinct goals.

5.3. Comparative Analysis: Specialized Encoders vs. Generative LLMs

With the rapid emergence of Generative Large Language Models (LLMs) such as GPT-4 or LLaMA-3, a critical architectural question arises: why utilize specialized encoder-only models (like BETO) for deceptive content detection? While generative LLMs excel at zero-shot reasoning and text generation, they present severe limitations for real-time, production-scale classification.

First, generative models are computationally prohibitive; an 8-billion parameter LLM requires massive GPU VRAM for inference, making widespread public deployment economically unviable. In contrast, the optimized BETO encoder (110M parameters) runs efficiently on standard CPUs, with the model forward pass contributing only a small fraction of the end-to-end latency reported in Section 4.8 (which is dominated by network-bound web scraping). Second, LLMs are inherently probabilistic text generators, requiring complex prompt engineering and output parsing to extract structured classification logits, making them prone to hallucinations or format deviations. Finally, our framework shows that fine-tuned bidirectional attention (understanding the full context of a sentence simultaneously) yields deterministic accuracy (89.18%) that is fully reproducible and strictly constrained to the classification taxonomy. Thus, for scalable and widely deployable misinformation-detection tools, specialized encoders represent a more practical, reliable, and energy-efficient solution than generative LLMs.

5.4. Transferability to Other Social Media Deception Domains

A central design goal of this work is that the proposed three-stage framework should be adaptable beyond fake news detection. The architecture of the pipeline—corpus unification, systematic regularization, containerized deployment—was designed to minimize domain-specific coupling. To illustrate this intended adaptability, we outline below several scenarios in which the same framework could be applied by substituting the training corpus. We stress that these scenarios are illustrative and have not been empirically evaluated in the present study; each would require a dedicated labeled corpus and its own validation:

• Investment Scam Detection: Social media platforms, particularly Facebook and Instagram, are inundated with fraudulent pages promoting fake investment opportunities (e.g., petroleum investments, cryptocurrency schemes). These pages use persuasive language patterns that share rhetorical features with fake news—urgency, emotional manipulation, and fabricated testimonials. A concrete example is a page impersonating PEMEX (Mexico’s state oil company) that claims citizens can “earn 100,000 pesos” by joining a fictitious “social project” [6]—a classic social engineering tactic combining authority impersonation with unrealistic financial promises. A corpus of labeled scam pages could be constructed using the Stage 1 unification methodology, and the Stage 2 regularization recipe applied directly to fine-tune a detection model.
• Fraudulent E-Commerce Detection: Fake retail pages impersonating legitimate brands or promoting non-existent products represent a growing threat that disproportionately affects elderly and digitally illiterate populations. Real-world examples include pages selling fraudulent “anti-aging cosmetics” using fabricated before-and-after photos, fake WhatsApp testimonials, and artificial urgency tactics (e.g., countdown timers and “order now” buttons) [7], as well as pages advertising implausible products such as “smart radiation-proof reading glasses” claiming to sell over 100,000 units per day [8]. The textual patterns of these pages—exaggerated claims, pressure tactics, and suspicious product descriptions—are amenable to the same transformer-based classification approach.
• Phishing Content Detection: Phishing campaigns disseminated through social media messaging share linguistic features with misinformation: impersonation of authority, urgency, and deceptive intent. The framework’s containerized deployment stage (Stage 3) is particularly valuable here, as it enables real-time URL analysis that could be extended to phishing detection with minimal architectural changes.

For cross-lingual extension specifically, a multilingual base encoder (e.g., distilbert-base-multilingual-cased or mBERT) could replace BETO at Stage 2, since such models support over 100 languages. Extending the framework to a new language would then primarily require the curation of a language-specific corpus (Stage 1) rather than a redesign of the regularization strategy (Stage 2) or the deployment architecture (Stage 3). We note that the final model in this study is the Spanish-specific BETO encoder; a multilingual backbone would be a substitution made only when targeting languages beyond Spanish.

5.5. Limitations

The most important limitation concerns the source dependence of the SATIRE class. As shown in Section 4.5, the near-perfect in-source satire recall collapses to 0.08 on satire from an unseen outlet. Because the satirical training data derive from a single source (El Deforma), the model learned to recognize that source’s style rather than a generalizable notion of satire. This is the clearest illustration of a broader principle that frames our contribution: the framework is only as generalizable as the corpus is diverse, and a class built from a single source will not transfer. We did not retrain to “patch” this result, as doing so by simply adding the second source would risk merely teaching the model two sources rather than the category; the principled solution is the construction of a genuinely multi-source satire corpus, which we identify as essential future work.

Beyond this, the model has further limitations. It is static; it was trained on a closed corpus ending in early 2025 and is therefore susceptible to concept drift. It is strictly unimodal (text-only), failing when deception is embedded in multimedia or when analyzing extremely short texts. The reported results also rely on a single training seed; multi-seed runs with confidence intervals were not performed. Finally, while the framework is designed to be transferable across domains, the empirical validation here is limited to Spanish fake news; the additional domains (scams, phishing, e-commerce fraud) are motivating directions, not demonstrated results.

5.6. Future Work

Future work will focus on several critical areas. First and foremost, motivated by the source-dependence finding of Section 4.5, we will construct a genuinely multi-source satire corpus, drawing from several satirical outlets across different Spanish-speaking regions, and re-evaluate cross-source generalization; this is the direct path to a satire detector that recognizes the category rather than a single outlet. Second, the implementation of Continuous Learning (CL) pipelines is envisioned to update the model with fresh data periodically without catastrophic forgetting, mitigating concept drift. Third, the framework will be empirically validated on additional social media deception domains—such as investment scams and fraudulent e-commerce pages—each requiring its own multi-source labeled corpus. Fourth, multi-seed training with confidence intervals and calibration metrics will be added for production-grade reliability. Finally, the expansion of the web application to support browser extensions is planned, bringing the detection capability directly to the user’s browsing experience.

Finally, addressing the “black-box” nature of deep learning models remains a priority. Future iterations of the deployed web application will integrate explainability frameworks, such as SHAP (SHapley Additive exPlanations) or LIME, to provide users with calibrated probabilities and visual rationales (e.g., highlighting specific deceptive sentences), thereby enhancing user trust and media literacy. Further research will also incorporate multi-seed training runs and strict calibration metrics (Expected Calibration Error) to provide robust confidence intervals for production environments.

6. Conclusions

This study presents a systematic and transferable framework for fine-tuning pretrained transformer models for domain-specific deceptive content detection on social media. The framework is structured as a three-stage pipeline—corpus unification, systematic regularization, and containerized deployment—designed to be reusable across languages, content domains, and social media platforms.

The framework was validated through a comprehensive case study on Spanish-language fake news detection. A unified corpus of 61,674 articles was constructed from four heterogeneous sources, demonstrating the corpus unification methodology. Through a rigorous experimental process, the proposed regularization strategy for BETO achieved 89.18% accuracy on the three-class task with a bounded generalization gap of 0.066. As a preliminary reference point, the best metaheuristic-optimized classical baseline reached 72.03% on the simpler binary task; because the task formulations differ, these figures are not directly comparable, but the contrast motivated the adoption of transformer fine-tuning. The full optimization trajectory—including failed configurations from the exploratory phase—is documented to provide a reproducible recipe for practitioners.

Beyond the case study, a key design goal of the framework is its potential adaptability. The same three-stage pipeline could, in principle, be applied to detect other forms of social media deception—including investment scams, fraudulent e-commerce pages, and phishing content—by substituting the training corpus, although this remains to be validated empirically and constitutes a primary direction for future work. The deployment of the optimized model into a Dockerized web application demonstrates that the framework extends beyond academic benchmarks to deliver a practical verification tool. The open-source availability of the entire framework—corpus, model, and application—facilitates further adaptation, paving the way for more resilient detection systems that can help protect vulnerable populations against the evolving landscape of social media deception.