Comparative Performance of AI-Generated Fake News Detection Pipelines on Romanian News Content

Abstract

Fake news detection has become a major research topic at the intersection of artificial intelligence, data mining, and information security. In this paper, we evaluate the performance of English-trained algorithms on English translations of Romanian-sourced news articles, using a translation-mediated cross-domain evaluation design. The study is based on source code generated with the assistance of artificial intelligence systems for a set of machine learning and transformer-based models. The code was subsequently implemented in Google Colab. 2026, trained on international benchmark datasets, and tested on Romanian news content. This design allowed the rapid prototyping of multiple detection pipelines and the systematic observation of their behavior in a media environment different from that represented in the training corpora. The models were evaluated comparatively using standard classification metrics, including accuracy, precision, recall, and F1-score, complemented by additional indicators relevant to model robustness and practical usability. The experimental results revealed significant differences in performance across algorithms when applied to English translations of Romanian-language news content after training on international datasets. However, this study does not provide a direct comparison between model performance on the international benchmark datasets and the Romanian test corpus; therefore, the gap between the international training corpus and the Romanian-sourced test corpus is interpreted as an exploratory limitation and as a direction for future research. Based on these findings, we propose an empirical classification of the tested models according to their predictive effectiveness, their contextual robustness across linguistic environments, and their operational relevance as filtering tools for institutional monitoring. The results show that AI-assisted coding workflows can provide a viable starting point for reproducible misinformation research, but they also underline the limitations of directly transferring models trained on non-Romanian data to local media ecosystems. The study offers both a replicable evaluation framework and practical insights for institutions involved in strategic communication, public security, and the monitoring of information threats.

1. Introduction

The post-truth era has recently redefined the global information landscape, continuously transforming the public digital space from a place of democratic deliberation into a modern battlefield for operations whose ultimate goal is to exert influence. As information consumption slowly shifts toward social media platforms and feeds based on different algorithms, the scale, swiftness, and especially the diversity and sophistication of misinformation have outpaced traditional human-controlled verification methods. Fake news, defined as deliberate false information disguised and promoted as traditional journalism, is no longer merely a nuisance; it is a systemic threat to security, especially to the cognitive security of modern societies. The fundamental goal of this research is to bridge the gap between advanced computational linguistics and the specific needs of non-English media ecosystems. By proposing an AI-assisted framework, this study seeks to evaluate how detection models perform when directly applied to Romanian media and cultural contexts. This specific research is driven by the overall necessity to move beyond existing barriers and models towards more practical and refined solutions that can protect local information environments.

The accelerated digitalization of public communication is changing the way information is created, disseminated, and consumed by society. Social networks and online platforms have come to replace traditional mass media as the main sources of information, thereby creating an environment that enables the rapid spread of different types of content. This has transformed the online information environment into one that allows the proliferation of false and misleading content, a phenomenon whose consequences extend beyond individual-level effects, threatening public institutions or even social cohesion. Although, on average, people consider fake news less plausible than true news [1], misinformation proliferates where truth loses its value and where public opinion is shaped by political beliefs and emotions rather than facts.

1.1. The Global Scale of Digital Misinformation

Globally, the World Economic Forum’s Global Risks Report ranked misinformation and disinformation as the most severe near-term risk facing humanity for two consecutive years in 2024 and 2025 [2,3]. The scale of this problem was also examined in research by scholars [4], who found that social media users who frequently share content become less sensitive to the truth value of what they share, while a 2025 national survey of social media users in Australia found that 75% of respondents had encountered some form of misinformation. These findings suggest that the spread of misinformation is a feature of life online, driven by routine and the reward architecture of platforms.

What has changed in recent years is not just the growing presence of misinformation online but the methods by which it has come to be produced, disseminated, and incorporated into articles that appear trustworthy at first glance—a practice increasingly adopted by online publications whenever possible. According to NewsGuard [5], by March 2026, more than 3000 news sites were identified that dealt solely with the dissemination of erroneous information generated with the help of artificial intelligence. These platforms are not isolated cases, as they appear in search engine results alongside credible sources; yet they plagiarize articles from top publications and modify them, distorting the factual accuracy of the events they report.

1.2. Vulnerabilities in the Romanian Media Ecosystem

At the national level, Romania also faces such problems in terms of the rapid spread of disinformation, both among the population and through fake news channels. According to Eurobarometer data from 2025, 55% of Romanians report being exposed “often” or “very often” to disinformation and fake news within a given week, well above the EU average of 36%. Another Eurobarometer survey from 2025 on the Digital Decade found that 78% of Romanian citizens consider it important for public authorities to become involved in combating the spread of fake news. These figures highlight Romania’s vulnerability to disinformation [6].

This unstable environment is very well-expressed by the case of the 2024 Romanian elections. According to the Reuters Institute’s Digital News Report from 2025 [7], an independent candidate managed to win through a campaign based on disinformation propaganda during the first round of the elections, relying solely on his online campaign. This candidate gained strategic traction across social platforms such as Facebook, YouTube, and TikTok with anti-NATO, anti-European Union, and pro-Russia messages. However, the elections were annulled after documents were made public by the intelligence services revealing over 25,000 fake TikTok accounts, more than 5000 Telegram channels, and documented links to state-affiliated networks [8]. This event highlighted the need for automatic detection tools capable of operating in the Romanian media environment. Therefore, this context underscores the purpose of the present research: the comparative evaluation of a set of classifier models based on machine learning and transformer architectures, selected to represent the main stages of algorithmic evolution in natural language processing (NLP), on Romanian news content, as well as to assess the transferability of systems trained on international reference datasets in a non-English linguistic environment.

2. Literature Review

2.1. Definition of Fake News

The definition of the term fake news cannot be reduced to a single sentence due to its complex nature. Conceptualizations of the term are generally organized around two criteria: factuality, the degree to which content corresponds to verifiable reality; and intentionality, the deliberate intention to deceive an audience [9]—these two dimensions generating the greatest disagreements [10]. A strict definition treats fake news exclusively as articles that are both intentionally and verifiably false [11], while a broader formulation describes them as fabricated information that mimics the form of legitimate journalism [12]. Both formulations, however, have been criticized as insufficient, as a definition focused exclusively on falsehood omits manipulation of content; therefore, the definition may be complemented with the idea of information that is not entirely false in reality, but only manipulated or taken out of context [13]. This imprecision in defining the concept has led to the proposal of the term “information disorder” as a much more precise analytical framework [14].

In order to correctly understand and classify a piece of information, it is necessary to analyze three fundamental elements: the degree of truth of the content, the intention behind its production, and the actor who produces it [15]. On the first axis, the American Psychological Association [16] distinguishes misinformation, i.e., false content spread without the intent to deceive, from disinformation, which is intentionally fabricated, while scholars [17] have expanded the taxonomy with malicious information (malinformation): authentic information used with malicious intentions. On the axis of intent, satirical content presents a low degree of literal truth but no intent to deceive [18], while the essential identifying characteristic of fake news is the deliberate presentation of false content as if it were true [19]. The actor dimension has become more complex with the advent of large language models (LLMs), which can generate high-quality deceptive content on a large scale with a more formal and syntactically coherent linguistic profile, complicating automatic detection [20]. Content designed to “closely resemble the truth” [21] may circumvent both human skepticism and algorithmic classification, while the structural diversity of actors, state-sponsored networks, clickbait operations, and malicious actors require differentiated detection strategies, which generalist classifiers rarely achieve in practice [21,22].

For the purposes of this study, fake news refers to textual content, including journalistic articles, digital publications, and news-platform posts, that is intentionally fabricated or manipulated, produced, and then disseminated online with the deliberate purpose of deceiving the public. This definition aligns with formulations emphasizing the high degree of intentionality of fake news as “incorrect information intentionally disseminated to serve certain interests, political or economic in nature” [23], while also acknowledging the broader framework of information disorder as its epistemic context.

2.2. Detecting Fake News and Switching to Automated Systems

Fake news is deliberately designed to mimic legitimate journalism, leading even educated readers and AI-based systems without human oversight to fail regularly in distinguishing fabricated content from authentic content [24]. The automation of detection is therefore not merely a matter of efficiency but an operational necessity.

Natural language processing (NLP), a subfield of artificial intelligence that translates plain text into structured data that machines can analyze and classify [25], provides the technical basis for this automation. Thus, it allows for the automatic classification of items into predefined categories strictly based on their content [26]. Numerous processing tasks are required for machines to decode and understand human language [27], as it is extremely complex and varied given that people express themselves in multiple ways [28]. In such situations, NLP tools can not only identify the contexts of discussions and emotions but also extract manipulative linguistic patterns and detect irony or alarmist tones, essential elements to differentiate an authentic news story from disinformation. Three paradigms organize NLP-based detection: content-based methods, social-context-based methods, and evidence-based methods [29]. The present study operates entirely within the content-based paradigm, the most manageable and reproducible approach as it relies solely on characteristics that can be extracted from the article itself [30].

2.2.1. Traditional Models: Naive Bayes and KNN

Early research has been based on identifying characteristics such as clickbait patterns, punctuation density, and polarity of feelings. Naive Bayes is commonly cited as a classic example in the literature [31], and “assumes that the presence (or absence) of a certain characteristic is not related to the presence (or absence) of any other characteristic” [32]. Empirically, this limitation is well-documented, as in a comparative study that evaluated five algorithms on balanced fake news datasets, Naive Bayes scored an F1-score of 0.88, the lowest of all the models tested, while KNN achieved 92% accuracy but remained sensitive to sparse, high-dimensional representations generated by large corpora of text, limiting its scalability [33].

2.2.2. Ensemble Models: Random Forest and XGBoost

To overcome the limitations of individual classifiers, ensemble methods combine multiple decision trees to improve robustness. Random Forest aggregates predictions from random subsets of data by majority vote [34], while XGBoost constructs trees sequentially, each correcting for residual errors of its predecessor [20]. To address this limitation, the present research design directly analyzes the structural stability of these classifiers when applied to a different linguistic environment. The performance gap compared to classical models is substantial and reproduced in a direct comparison on the ISOT dataset, where both XGBoost and Random Forest achieved 100% accuracy, surpassing even deep learning methods [35]. As demonstrated by Essa, Omar, and Alqahtani [36], who tested traditional machine learning models, including XGBoost, on multiple real-world fake news datasets with various embeddings, performance degrades substantially when the linguistic environment changes—a pattern directly anticipated in the interlingual design of the present study.

2.2.3. Advanced Models: BERT and GCN Text

Deep learning has introduced artificial neural network architectures capable of capturing the meaning of text rather than merely its statistical properties; these architectures “have gained significant traction due to their superior performance in capturing complex patterns” [37]. Within this generation, transformer-based architectures demonstrate high effectiveness in fake news detection tasks [38] due to their architecture that allows attention to be paid to the relationships between words. The most prominent representative, BERT [39], reads text in both directions simultaneously to understand contextual relationships: “it is distinguished by its ability to consider the context of a word, both to the left and right of it within the sentence, unlike previous models that analyzed text only in one direction” [32]. The performance advantage over ensemble-type methods is empirically established in cross-lingual settings; in a direct comparison of the models, XGBoost achieved an accuracy of 81.0%, while BERT reached 98.0% on the same dataset, indicating that BERT-type architectures consistently outperform classical and ensemble models in classification tasks [40]. TextGCN models the relationships between words and documents these in the form of graph structures, allowing evidence to be aggregated and structural patterns of disinformation to be detected from multiple sources [41].

2.3. Performance Evaluation: Confusion Matrix and Classification Metrics

The confusion matrix breaks down predictions into four categories: true positives, true negatives, false positives, and false negatives [42]. From this, four metrics are derived. Accuracy, which is “the percentage of instances that are correctly categorized” [43], is the most intuitive but also the most misleading indicator under conditions of class imbalance: a classifier that always predicts the majority class achieves high accuracy without detecting a single instance of fake news. Precision “measures how well the model can identify positive instances, reduce false positive outcomes” [43] and quantifies the cost of false alarms, which is critical in implementation contexts where incorrectly labeling legitimate journalism as fabricated entails legal and reputational consequences. Recall, which measures “how well a pattern can identify each positive occurrence, thereby minimizing false negative outcomes” [43], captures the opposite risk of failing to detect real misinformation, which is the error with the greatest public consequences.

2.4. Trends in Research

Given the multidimensional nature of fake news and the architectural evolution of NLP detection flows, it is essential to recognize that the cross-lingual transferability of these systems to a morphologically complex and resource-constrained environment such as Romania’s requires a rigorous approach.

Benchmarking of fake news detection algorithms plays a key role in understanding their detection mechanisms and performance. Consequently, studies to this end by Buzea, Trausan-Matu, and Rebedea [44] established comparative benchmarks for the automatic detection of fake news in the Romanian online press. Their work demonstrated that transformer architectures generally outperform traditional models but face difficulties with subtle misinformation and decontextualized content, findings that directly support the H1 and H4 hypotheses of the present study. Furthermore, Bucos and Drăgulescu [45] demonstrated that machine translation augmentation substantially improves detection performance in Romanian content, while Moisi et al. [18] compared classical machine learning algorithms with transformer models on Romanian news, confirming the superiority of advanced architectures and emphasizing the critical importance of preserving diacritics in preprocessing to prevent cross-lingual transfer from degrading model performance.

In the present paper, by training the models on established international datasets and testing them on a localized corpus of 100 Romanian news sources (50 authentic and 50 fabricated), the study does not measure performance under ideal conditions but the practical applicability of global detection tools within the Romanian media ecosystem. Evaluation by standard classification metrics, such as accuracy, precision, recall and F1-score, complemented by a qualitative analysis of confusion matrices, will allow the study to test the premise that advanced models capture contextual nuances superior to classical algorithms (H1), while also verifying whether traditional statistical classifiers can maintain competitive results when the boundary between authentic journalism and disinformation is based on distinct lexical frequencies rather than complex semantic structures (H3).

3. Methodology

The purpose of this research is to comparatively evaluate the performance of several automated fake news detection pipelines, developed with the support of artificial intelligence systems and applied to Romanian journalistic content. The study aims to analyze the extent to which algorithms trained on international datasets can be transferred and used effectively to classify news articles in the Romanian media space. Therefore, the research is not just limited to evaluating the performance of individual algorithms but also investigates a broader process: generating code with the help of artificial intelligence, implementing it in Google Colab, training models on international databases, and then testing them on a Romanian news corpus.

Specific objectives

The research specifically pursues the following objectives:

• Development of an AI-assisted experimental framework for generating, deploying, and testing multiple fake news classification pipelines.
• Implementation in Google Colab of classic machine learning models and advanced natural language processing models.
• Training algorithms on international datasets used in disinformation detection research.
• Testing models on a corpus of Romanian articles that is culturally and contextually different from the data used in the training stage.
• Comparative evaluation of model performance through standard classification indicators: accuracy, precision, recall, and F1-score.
• Analysis of the robustness of models in conditions where training and test data differ in geographic origin, journalistic convention, and political context, specifically by evaluating models trained on internationally sourced English datasets and on English translations of Romanian journalistic content.
• Identification of the types of errors produced by each model by analyzing the real items classified as fake and fake items classified as real.

Research hypotheses

The research starts from the following hypotheses:

H1. 

Advanced natural language processing models, such as BERT and TextGCN, will outperform classical algorithms due to their ability to analyze contextual relationships between words.

H2. 

The performance of all models will be affected by the gap between the internationally sourced training data and the Romanian-sourced test content, as the training corpus does not fully reflect the thematic, political, and journalistic particularities of the Romanian media space, even when that content is evaluated in English translations.

H3. 

Classic machine learning models can achieve competitive results in situations where the difference between fake and real news is reflected mainly at the lexical level through the frequency and distribution of words.

H4. 

Most classification errors will occur in the case of articles in which legitimate journalistic language overlaps with the language of disinformation, especially in political, institutional, satirical, or alarmist texts.

Scope and methodological limitations

Although the study was designed to meet methodological and academic standards, the present study has several limitations that should be acknowledged. The first limitation concerns the gap between the training and test corpora. Although the Romanian test articles were translated into English prior to evaluation—meaning the models operated within the same language throughout—the internationally sourced training data do not reflect the journalistic style, political context, or thematic specificity of the Romanian media space. Additionally, the translation step itself may introduce semantic shifts that affect classification outcomes. The second limitation concerns the size of the Romanian test corpus. Although the set of 100 articles is balanced, it cannot cover the entire diversity of the Romanian media ecosystem. The third limitation concerns the dependence of the pipelines on AI-generated or AI-assisted code. Even though this code allows for rapid prototyping, it requires validation, adjustment, and methodological verification to avoid implementation or interpretation errors. Nevertheless, human validation remained essential throughout the use of AI-assisted forms. The fourth limitation is that the models evaluated do not verify the factual accuracy of the statements in the articles and only classify texts based on the patterns learned from the data. Therefore, an article can be classified as real, not because the information is verified, but because its language resembles that of authentic articles in the training data.

Research design and experimental framework

The study has an experimental, comparative, and mixed-methods design oriented toward evaluating the performance of several automated fake news detection pipelines. The models were developed with the support of the Google Gemini artificial intelligence systems, Gemini version 3.1 Pro, which assisted with the generation and adaptation of the source code. The resulting code was then implemented and run in Google Colab. 2026. This environment allowed rapid prototyping, standardization of experimental conditions, and comparison of algorithms under similar conditions. The comparative component consisted of testing several models, from classic machine learning algorithms to advanced transformer- and graph-based architectures, on the same corpus of Romanian articles. The mixed component combined quantitative evaluation, performed through metrics such as accuracy, precision, recall, and F1-score, with qualitative analysis of classification errors.

Six models were evaluated in the study that were selected to cover several algorithmic families:

• Naive Bayes—A probabilistic model frequently used in the classification of texts, based on the distribution and frequency of terms.
• K-Nearest Neighbors—KNN, a model based on the similarity between texts, which classifies documents according to their proximity to other labeled documents.
• Random Forest—An assembly-type model, consisting of several decision trees, capable of combining multiple classification rules.
• XGBoost—A boosting model based on successively built decision trees, with each stage seeking to correct the errors of the previous one.
• BERT—A transformer model, capable of analyzing words according to the context in which they appear.
• TextGCN—A model based on graphical networks, which analyzes the relationships between words and documents by means of a structure of nodes and connections.

Data used

The research used two levels of data: training data and test data. The training data consisted of international datasets used in research on the detection of fake news. Thus, three international databases were selected, totaling 195,701 news items, both real and fake. The sources used were as follows:

(a) WELFake Dataset—An aggregated dataset, constructed by combining four distinct news sources, containing articles labeled as either real or fake, and intended for textual classification tasks [46]. Available at: https://doi.org/10.5281/zenodo.4561253.

(b) ISOT Fake News Dataset—Developed by the University of Victoria, Canada, the dataset contains real articles taken from Reuters and fake articles identified by fact-checking organizations, and is frequently used as a reference in comparative auto-detection studies [47]. Available at: https://onlineacademiccommunity.uvic.ca/isot/2022/11/27/fake-news-detection-datasets/ (Accessed on 5 March 2026).

(c) Misinformation and Fake News Text Dataset 79K—A publicly available dataset on the Kaggle platform, comprising approximately 79,000 labeled text articles, which is used to train and test disinformation classification models [48]. Available at: https://www.kaggle.com/datasets/stevenpeutz/misinformation-fake-news-text-dataset-79k (Accessed on 6 March 2026).

To ensure optimal performance of the detection models, the three databases were subjected to a systematic preprocessing pipeline prior to training. The datasets were merged into a single centralized corpus, with article titles and full-body texts. A standardization and cleaning procedure was then applied, in which special characters, web links, and redundant whitespaces were removed. In the next step, deduplication and class balancing were performed; duplicate entries were identified and removed, and the corpus was balanced to an equal number of real and fake news articles, resulting in 124,580 unique articles. All preprocessing steps were implemented in Python and are provided in Appendix A.1—“Database processing code”. All three datasets are exclusively in English, and no translation was applied to the training corpus. The models were therefore trained entirely in English. Training and testing were performed on entirely separate datasets: the models were trained on the international corpus and tested on the independent Romanian corpus.

The test data were represented by a corpus of Romanian articles. The test set was balanced and included 100 items, of which 50 were labeled as false and 50 as true. The labels assigned to the fake articles in the corpus were drawn from fact-checking websites. Articles classified as authentic were selected from news sources with editorial credibility and were individually reviewed by the authors to verify their consistency with the factual context. Since the models were trained in English, it was necessary to translate the Romanian test texts into English prior to evaluation. The test articles were preprocessed using the same cleaning pipeline applied to the training corpus. The translation was performed using Google Translate, followed by a manual review by the authors to correct errors. The detailed description of the articles included in the test corpus, accompanied by the titles, sources and tags related to each article, is presented in Appendix A.2—“Training test database”.

The articles were selected from content published or circulated during the last three months prior to data collection and were included only if they were related to Romania or to topics directly relevant to the Romanian information space. Real articles were selected from Romanian mainstream online media sources, whereas fake articles were selected sequentially from fact-checking websites. Items that did not match the article-based design of the study, such as social media posts, video-only materials, or deepfake video cases, were excluded. The corpus was intended not to be statistically representative of the Romanian media ecosystem but to provide a pilot external test set for observing how models trained on international datasets behave when applied to Romanian-sourced news content.

Experimental procedure

The present study reports an external evaluation of the trained models on the Romanian test corpus. Although the models were trained using international benchmark datasets, the manuscript does not include a separate evaluation of model performance on those international datasets. Consequently, the study does not quantify a direct transfer-performance gap between international and Romanian data. Instead, it provides an exploratory assessment of how the tested pipelines behaved when applied to English translations of Romanian news content.

In the first stage, the models to be compared were selected. In the second stage, classification pipelines were generated or adapted with the support of artificial intelligence systems. The resulting code was organized and run in Google Colab, which provided approximately 12 GB of RAM and access to a shared T4 GPU. Standard CPU resources were used for classical and ensemble models, while the T4 GPU accelerator was used for BERT and TextGCN. The implementation was developed in Python 3.10, using the libraries: pandas, scikit-learn, XGBoost, transformers (Hugging Face), and torch (PyTorch). For BERT, the BERT-base-uncased variant was used, which is a model for English text. The complete hyperparameter configuration for all six models, including all relevant parameter values, is provided in

Table A4. Experimental procedures—hyperparameter configuration.

Algorithm	Parameter	Value Used
Naive Bayes	Model variant	MultinomialNB()
	TF-IDF max_features	50,000
	Random_state	42
KNN	N_neighbors	5
	TF-IDF max_features	10,000
Random Forest	N_estimators	100
	N_jobs	−1
	TF-IDF max_features	20,000
	Random_state	42
XGBoost	N_estimators	100
	Learning_rate	0.1
	Max_depth	6
	Tree_method	hist
	TF-IDF max_features	20,000
	Random_state	42
BERT	Model variant	Bert-base-uncased
	Tokenizer	BertTokenizerFast
	Max_length	256 tokens
	Padding	max_length
	Per_device_train_batch_size	8
	Num_train_epochs	1
TextGCN	TF-IDF max_features/input nodes	20,000
	Hidden layer	128 units + ReLU
	Dropout	0.5
	Output layer	2 units
	Optimizer	Adam (lr = 0.001)
	Loss function	CrossEntropyLoss
	Epochs	5
	Batch_size	256

.

Given the resource constraints of the cloud environment, the source code initially generated by the AI was reviewed and adapted before execution. The primary category of adaptations concerned memory management: for all classical and ensemble models (Naive Bayes, KNN, Random Forest, XGBoost), the maximum vocabulary size used by the TF-IDF vectorizer was explicitly limited, ranging from 10,000 to 50,000 of the most frequent terms, depending on the requirements of each algorithm. For transformer-based and neural models (BERT, TextGCN), additional constraints were applied to the tokenizer’s maximum sequence length and to the training batch size, with the number of training epochs reduced accordingly. For all models involving random processes, the random seed was set to 42. These adjustments were necessary to prevent session crashes due to RAM exhaustion and are provided in

Table A3. Experimental procedures—code adjustments.

Algorithm/Parameter	AI-Generated Value	Value Used
Naive Bayes: TF-IDF max_features	Unlimited	50,000
KNN: TF-IDF max_features	Unlimited	10,000
Random Forest: TF-IDF max_features	Unlimited	20,000
XGBoost: TF-IDF max_features	Unlimited	20,000
BERT: max_length (tokenizer truncation)	512 tokens (BERT’s architectural maximum)	256 tokens
BERT: per_device_train_batch_size	16	8
BERT: num_train_epochs	3	1
TextGCN: TF-IDF max_features	Unlimited	20,000
TextGCN: batch_size	512	256
TextGCN: epochs	10	5

.

The code generated by the AI system served as a functional starting point for each pipeline. Where the generated code executed without errors on the target dataset and environment, it was accepted with only minor adaptations, specifically with file path references. When runtime errors occurred, most frequently due to memory exhaustion, the error output was diagnosed and the parameter was adjusted manually until successful execution was achieved. The prompt submitted to the AI for this stage was:

“Write complete Python code for Google Colab that: (1) mounts Google Drive; (2) loads a CSV training file with columns text and label; (3) cleans nulls from those columns; (4) vectorizes the text using TF-IDF; (5) creates and trains the classifier; (6) saves the trained model and vectorizer as .pkl files to Google Drive. Print a progress message before each step.”

In the second stage, the trained models were applied to the Romanian test corpus of 100 news items. For each item, the algorithm generated a binary prediction: “fake” or “real”. The prompt submitted to the AI system for this stage was:

“Write complete Python code for Google Colab that: (1) mounts Google Drive; (2) loads the saved model and vectorizer; (3) loads the CSV test file; (4) preprocesses the text using the same cleaning procedure applied during training; (5) generates a prediction for each article; (6) saves the results to a CSV file. Print a progress message before each step.”

In the third stage, the predictions were compared with the ground-truth labels. Based on this comparison, performance metrics were calculated and confusion matrices were constructed. Subsequently, the classification errors were qualitatively analyzed to identify the linguistic and thematic patterns that influenced the performance of the models. The prompt submitted to the AI system for this stage was:

“Write Python code that: (1) loads the ground truth file and the predictions file; (2) aligns the two columns for comparison; (3) computes a full classification report including accuracy, precision, recall, F1-score, macro-F1 and weighted-F1; (4) plots a confusion matrix heatmap with axis labels and a title specifying the model and the test set”.

The quantitative assessment was based on standard indicators used in the automatic classification:

• Accuracy measures the total proportion of correct classifications.
• Precision indicates the proportion of correct predictions out of the total predictions made for a given class.
• Recall measures the model’s ability to identify all examples belonging to a class.
• F1-score represents the harmonic mean between precision and recall, and is useful for assessing the balance between the two.
• Macro-F1 computes the unweighted mean of per-class F1-scores, treating each class with equal importance.
• Weighted-F1 computes the same mean weighted by the number of instances per class. Since the test set is perfectly balanced (50 fake and 50 real articles), both metrics produce identical values across all models.

In addition to these metrics, the analysis included the values in the confusion matrix: true positives, true negatives, false positives, and false negatives.

In this study, the term “AI-generated” refers strictly to the use of artificial intelligence tools as implementation support for drafting, adapting, and debugging the source code of experimental pipelines. It does not imply that the AI-assisted coding process itself is evaluated as an independent research variable, nor that the study compares AI-generated code with human-written code; rather, the scientific focus remains on the comparative performance of the resulting fake news detection models under the same experimental conditions.

4. Results

This section presents the results obtained by testing the six pipelines for the automatic detection of fake news on a balanced Romanian corpus, consisting of 100 articles: 50 articles labeled as fake, and 50 articles labeled as real. Results are reported using standard classifications of accuracy, precision, recall, and F1-score and by the distribution of errors in the confusion matrix. In order to avoid terminological ambiguities generated by the different use of TP, TN, FP, and FN indicators depending on which class is treated as positive, the confusion matrices are presented below in descriptive format and directly report the combinations of actual labels and model prediction.

The analysis pursues two complementary levels. The first level is comparative and highlights the relative performance of the six models. The second level is interpretative and follows the types of articles that have generated classification errors, especially situations in which the vocabulary of legitimate journalism overlaps with the vocabulary of disinformation, satire, political discourse, or institutional communication.

4.1. Overall Comparative Performance of Models

Table 1. Comparative performance of the models on the Romanian test corpus.

Model	Accuracy	95% CI for Accuracy	Fake Precision	Recall Fake	F1 Fake	Real Precision	Recall Real	F1 Real	Macro-F1/Weighted-F1	Total Errors
Naive Bayes	0.71	0.61–0.79	0.80	0.56	0.66	0.66	0.86	0.75	0.70	29
KNN	0.49	0.39–0.59	0.49	0.94	0.65	0.40	0.04	0.07	0.36	51
Random Forest	0.77	0.68–0.84	0.74	0.84	0.79	0.81	0.70	0.75	0.77	23
XGBoost	0.57	0.47–0.66	0.54	0.88	0.67	0.68	0.26	0.38	0.52	43
BERT	0.73	0.64–0.81	0.69	0.82	0.75	0.78	0.64	0.70	0.73	27
TextGCN	0.72	0.63–0.80	0.68	0.84	0.75	0.79	0.60	0.68	0.72	28

summarizes the results obtained by the six models for the two classes analyzed, fake and real. Because the test set is balanced, accuracy provides a useful insight into overall performance; however, the interpretation of the results should be supplemented by an analysis of the recall and the F1-score for each class, as some models showed strong tendencies to favor one of the categories.

The results reported in this section refer exclusively to the Romanian test corpus. Therefore, the analysis should not be interpreted as a complete measurement of cross-lingual transfer performance but as an exploratory external test of models trained on international data and applied to English translations of Romanian-language news content.

Because the Romanian pilot test corpus contains only 100 articles, 95% confidence intervals were calculated for model accuracy using the Wilson score interval. Accuracy was treated as a binomial proportion, corresponding to the number of correct predictions out of the total number of tested articles. These intervals provide a simple estimate of uncertainty around the reported accuracy values and support a more cautious interpretation of the differences observed between models.

The results indicate important differences between the models. Random Forest achieved the highest accuracy at 77%, and the lowest total number of errors of 23. BERT and TextGCN performed similarly, with accuracies of 73% and 72%, respectively, while Naive Bayes achieved 71% accuracy. XGBoost and KNN performed significantly worse, with accuracies of 57% and 49%, respectively.

These results show that (Figure 1), under the conditions of the present experiment, advanced natural language processing models did not automatically surpass classical machine learning models. Although BERT and TextGCN have superior contextual or relational modeling capabilities, Random Forest produced the most balanced overall performance. This observation is important for the interpretation of cross-lingual transfer, because models trained on international datasets may lose their architectural advantage when applied to a Romanian corpus with different linguistic and thematic particularities.

The distribution of errors complements the picture provided by precision. Naive Bayes favored the real class, generating 65 real predictions and only 35 fake predictions, which explains the high number of fake articles classified as real. KNN had the opposite behavior, classifying 95 of the 100 articles as fake, which led to the almost complete non-recognition of authentic journalism. XGBoost also showed a sharp trend towards the fake class, with 81 fake predictions. Random Forest, BERT, and TextGCN had more balanced distributions, although all three retained a slight orientation toward identifying fake content.

4.2. Analysis of Results by Model

4.2.1. Naive Bayes

Naive Bayes achieved an accuracy of 71% (

Table 2. Real articles classified as fake by Naive Bayes.

Item No.	Title of the Article	Real Label	AI Prediction
6	INSCOP Survey: 46% of Romanians want to give up changing the time	Real	Fake
7	The strategy that empties our pockets. How the tiny difference of one ban manipulates our perception	Real	Fake
8	The end of a historic robbery. Why the theft in the Netherlands was felt as a personal loss by Romanians	Real	Fake
9	A 2-m long snake was found in the house of a woman from Galati county.	Real	Fake
11	The places in Romania gripped by the gold rush	Real	Fake
24	A man won 44 million euros at the 6/49 Lottery and told his wife nothing for several days	Real	Fake
34	How Romania’s meat production evolved in January 2026. Differences between pork, poultry and beef sectors	Real	Fake

), misclassifying 29 of the 100 articles. Of the 50 fake items, the model correctly identified 28 and misclassified 22 as real. Of the 50 real items, the model correctly identified 43 and erroneously classified seven as fake.

The rating ratio indicates a high precision for the fake class of 0.80 but a lower recall of 0.56. This means that the model is relatively reliable when labeling an item as fake but fails to detect a considerable portion of the fake items in the test set. For the real class, the recall of 0.86 indicates a good ability to recognize authentic items, but the precision of 0.66 shows that a significant proportion of predictions labeled real include a substantial number of fake items.

The analysis of real articles classified as fake shows that Naive Bayes is sensitive to potentially sensationalist or controversial vocabulary. Articles about surveys, important financial gains, unusual events, or cultural and historical themes have been interpreted as having features associated with disinformation. This limitation can be explained by the way the model works: the text is treated as a collection of words, and the classification is based on lexical frequencies and probabilities, without analyzing context, journalistic intent, or source credibility.

In the case of fake articles classified as real, the main problem was the use of common or seemingly legitimate vocabulary. Satirical texts, articles built around distorted real events, and materials using the names of institutions, such as the Romanian Government or NATO, were harder to distinguish from authentic journalism. Technical and legislative vocabulary, such as ‘modular reactors’, ‘engineering and design studies’, ‘draft law’, or ‘decisive vote of MPs’, contributed to this confusion, as the model cannot verify whether the terms are used in a factual or manipulative context.

Therefore, Naive Bayes performs relatively well in situations in which the lexical differences between fake and real articles are clear, but it becomes vulnerable when the two categories use similar vocabulary. The model is too permissive with some of the fake content, as 44% of the fake items in the test set were classified as real.

4.2.2. K-Nearest Neighbors (KNN)

KNN achieved the lowest performance among the models analyzed, with an accuracy of 49% and 51 classification errors. The model correctly identified 47 of the 50 fake items, but it misclassified 48 of the 50 real items. Thus, although the recall for the fake class is very high, 0.94, the recall for the real class is only 0.04, which indicates an almost complete inability to recognize authentic journalism.

The model generated 95 fake predictions and only five real predictions. This distribution indicates an extreme tendency to associate almost any item with the fake class. Methodologically, the result suggests an incompatibility between the structure of the KNN algorithm and the classification task applied in this cross-lingual context. The model does not build its own interpretative rules but classifies the texts according to their proximity to the previously observed examples in a multidimensional space of lexical characteristics.

The error of classifying real articles as fake (

Table 3. Fake articles classified as real by KNN.

Crt. No.	Title of the Article	Real Label	AI Prediction
60	Donald Trump confirms almost identically what Călin Georgescu was saying (Davos speech, green energy)	Fake	Real
89	IT WAS NOT ROMANIA THAT CANCELED THE ELECTIONS IN ROMANIA—The shocking hypothesis of a Donald Trump influencer and Mike Benz	Fake	Real
92	Năsui’s bomb. Why wasn’t Iohannis at the reception from the NATO Summit	Fake	Real

) can be explained by the overlap in high-frequency vocabulary between the two classes. Terms such as “Romania”, “energy”, “government”, “crisis”, “prices”, “elections”, and “people” appear in both real and fake news. In such a lexical space, KNN fails to distinguish between the classes, especially if one of the groups is denser or better represented in the training data. Articles that were atypical, short, or focused on unusual events amplified this problem, as their specific terms could not be associated clearly enough with relevant neighbors.

The three fake articles classified as real share vocabulary that resembles legitimate political analysis. The presence of proper names, quotes, geopolitical references, and terms such as “hypothesis”, “economy”, “press”, or “geopolitics” have positioned these texts closer to the cluster of real news than that of conspiracy content. The result confirms that KNN did not learn an operational definition of disinformation but instead reacted to lexical proximities.

Overall, KNN has little relevance to the task under review. Its performance reflects not only low precision but also a severe imbalance of predictions, which makes it unsuitable for operational uses in detecting fake news.

4.2.3. Random Forest

Random Forest achieved the best overall performance in the experiment, with an accuracy of 77% and 23 errors. The model correctly identified 42 of the 50 fake items and 35 of the 50 real items. It produced eight fake → real errors and 15 real → fake errors, indicating a relatively balanced distribution compared to KNN and XGBoost.

The ranking report shows an F1-score of 0.79 for the fake class and 0.75 for the real class. The precision for the real class is the highest of all models with 0.81, which indicates that when Random Forest assigns the real label, the prediction is generally reliable. At the same time, the 0.84 recall for the fake class shows a good ability to detect fake content.

Some of the real articles misclassified by Random Forest coincide with those misclassified by Naive Bayes, especially articles about surveys, unusual events, situations with sensationalist features, or cultural–historical themes. This overlap suggests the existence of difficult lexical areas for several types of models and not just for a single algorithm. In the case of Random Forest, the difficulty arises when the decision trees encounter combinations of characteristics associated in the training data with disinformation, even though the analyzed text is authentic journalism.

Three confounding factors were particularly noticeable (

Table 4. Fake articles classified as real by Random Forest.

Crt. No.	Title of the Article	Real Label	AI Prediction
55	Romanians and Europe will freeze this winter	Fake	Real
73	The socialist government of Spain goes even further with censorship.	Fake	Real
74	Right-hand drive cars will no longer be able to circulate on public roads, the senators voted	Fake	Real
77	Bolojan sold the Tarnița hydroelectric plant to France. It is good for the French and not good for Romanians	Fake	Real
80	Romania, forced to organize the second round of the presidential election, according to the 2024 election calendar	Fake	Real
88	Alex Jones’s bomb: In Romania a war related to Ukraine can explode to block Trump. NATO staged a coup d’état against Romanians	Fake	Real
91	Will NATO deploy NUCLEAR WEAPONS in Romania?	Fake	Real
99	Bolojan went personally to the hunger strikers to savor a hamburger.	Fake	Real

). The first is the length of the article. Very short texts offer few features for stable splits, whereas very long texts increase the likelihood of signal words associated with fake news. The second factor is represented by cultural or historical topics, where the vocabulary about legends, curses, spirits, deposits, or narrative stories can resemble the rhetoric of conspiracy texts. The third factor is the articles on regulations and legal provisions, where the institutional and procedural vocabulary can be confused with the language of theories about abuse of power or social control.

In the case of fake articles classified as real, Random Forest was especially vulnerable to misinformation built on seemingly plausible legislative, political, or institutional events. The names of public figures, references to institutions, geopolitical mentions, and narrative structures close to political journalism created a lexical profile that the model associated with real articles. The manipulation takes place here at the semantic and pragmatic level, not just at the lexical level, and the model cannot factually verify the statements.

Despite these limitations, Random Forest offered the best balance between detecting fake content and recognizing authentic journalism. Compared to the other models, its result suggests that a classical assembly-type architecture can remain competitive under cross-lingual conditions, especially when the classification task depends on lexical distributions and recurrent combinations of features.

4.2.4. XGBoost

XGBoost achieved 57% accuracy, with 43 articles misclassified. The model correctly identified 44 of the 50 fake items, but it correctly classified only 13 of the 50 real items. It generated 81 fake predictions and only 19 real predictions, which indicates a strong trend of overlabeling articles as fake.

This distribution is reflected in the high recall for the fake class with 0.88 and the very low recall for the real class with 0.26. The precision for the fake class is 0.54, which means that almost half of the fake predictions are actually real articles mislabeled. For the real class, the precision of 0.68 suggests that the model may be relatively correct when assigning this label, but it does so too rarely to be considered balanced.

The 37 real items classified as fake indicate a major transfer and calibration problem. Among these errors are articles that have caused difficulties for the other models as well: surveys, articles with analytical language, materials with cultural or historical topics, texts with large volumes, or formulations with sensationalist potential. This suggests that XGBoost has internalized certain patterns associated with disinformation too strongly and applied them excessively to the Romanian corpus.

Other problematic cases involved articles (

Table 5. Fake articles classified as real by XGBoost.

Crt. No.	Title of the Article	Real Label	AI Prediction
55	Romanians and Europe will freeze this winter	Fake	Real
69	The satans from the CNA banned Realitatea Plus. Dastardly attack by the system on the only TV where Georgescu appears	Fake	Real
74	The socialist government of Spain goes even further with censorship.	Fake	Real
74	Right-hand drive cars will no longer be able to circulate on public roads, the senators voted	Fake	Real
80	Romania, forced to organize the second round of the presidential election, according to the 2024 election calendar	Fake	Real
99	Bolojan went personally to the hunger strikers to savor a hamburger.	Fake	Real

) with metaphors, legislative language, or alarmist tones, including expressions such as “radical law”, “phasing out”, “ban”, “crisis”, or “chaos”. In these situations, the model interpreted the political, economic, and institutional vocabulary as a signal of disinformation, even though some texts were real articles.

The fake articles classified as real used either names, institutions, too apparently precise procedures, or technical and journalistic language that masked the fabricated nature of the information. In addition, the model had difficulty identifying subtle satire, as phrases such as ‘prime minister’ or ‘hunger strike’ can function as legitimate signals if analyzed without access to the pragmatic level of the text.

XGBoost’s performance is mostly relevant as a negative result. Although the model generally performs well in many classification tasks, it did not bring an advantage over Random Forest in this experiment. The result suggests either insufficient parameter calibration or high sensitivity to the distribution of the drive data, which led to an excessive orientation towards the fake class. In the form tested, XGBoost is less useful in an operational context in which it is also necessary to protect authentic journalism from misclassification.

4.2.5. BERT

BERT achieved 73% accuracy with 27 classification errors. The model correctly identified 41 of the 50 fake items and 32 of the 50 real items. It generated 59 fake predictions and 41 real predictions, having a relatively balanced distribution compared to KNN and XGBoost.

The classification report indicates an F1-score of 0.75 for the fake class and 0.70 for the real class. The 0.82 recall for the fake class shows a good ability to identify fake items, while the 0.78 precision for the real class indicates high reliability when the model classifies an item as genuine. However, the model misclassified 18 real items as fake and nine fake items as real. In the case of real articles classified as fake, two types of limitations were apparent. The first concerns articles built around polls, statistics, or public opinion topics. For example, the article about the INSCOP survey on giving up the time change contains percentages, divided opinions, and a topic with political potential, elements that can appear both in authentic journalism and in manipulative content. The model cannot verify the legitimacy of the source or the validity of the cited data.

The second limitation concerns the length of the texts (

Table 6. Fake articles classified as real by BERT.

Crt. No.	Title of the Article	Real Label	AI Prediction
71	The spring planting campaign begins. The SRI will plant 200,000 microphones in Romanians’ homes	Fake	Real
76	The tax on the sun, a new villainy. And not the last one	Fake	Real
95	The desperation of a dinosaur, PSD with AUR teeth	Fake	Real
96	The Government will grant vouchers for the purchase of cherries	Fake	Real
97	The corn quotation grows after Simion and AUR started ordering popcorn	Fake	Real
73	The socialist government of Spain goes even further with censorship.	Fake	Real
74	Right-hand drive cars will no longer be able to circulate on public roads, the senators voted	Fake	Real
80	Romania, forced to organize the second round of the presidential election, according to the 2024 election calendar	Fake	Real
99	Bolojan went personally to the hunger strikers to savor a hamburger.	Fake	Real

). BERT typically processes a maximum of 512 tokens, which means that long items can be classified based on the first part of the text. In the case of the article about “gold fever” in Romania, the initial part is more narrative and closer to the folkloric register, while the analytical elements that support the journalistic character appear later. This structure can lead to misclassification, as the model does not have effective access to the entire discursive architecture of the article.

Fake articles classified as real highlight a major difficulty of the model in relation to satire, irony, and absurdity. Five of these articles are satirical or strongly political in nature. BERT can correctly interpret the local grammatical and semantic coherence of sentences, but it does not have a sufficient mechanism to assess the plausibility of the situations described. Examples such as biodegradable microphones planted in homes, vouchers worth thousands of lei for cherries, or influencing corn prices through the consumption of popcorn by politicians require real-world knowledge and a pragmatic interpretation of ironic intention.

Compared to models based strictly on frequencies or decision trees, BERT benefits from more advanced contextual analysis. However, its performance remains dependent on the diversity of training data and the adequate representation of local forms of satire, political opinion, and disinformation. The results indicate a relatively stable model, but not one robust enough to outperform Random Forest under the conditions of this experiment.

4.2.6. TextGCN

TextGCN achieved 72% accuracy, with 28 articles misclassified. The model correctly identified 42 of the 50 fake items and 30 of the 50 real items. It generated 62 fake predictions and 38 real predictions, which indicates a moderate trend of favoring the fake class.

The performance indicators show an F1-score of 0.75 for the fake class and 0.68 for the real class. The 0.84 recall for the fake class indicates good misinformation detection capability, but the 0.60 recall for the real class shows that 40% of authentic items were misclassified as fake. The precision for the real class of 0.79 is still high, which means that real predictions are relatively reliable, but they occur less often than would be necessary in a balanced system.

The main explanation for the errors comes from the architecture of the model. TextGCN constructs a graph in which documents and words are represented as nodes, and the links reflect co-occurrence or association relationships. The classification decision depends not only on the internal vocabulary of an article but also on how its terms are connected to other nodes in the network. Therefore, a real article that uses terms commonly found in disinformation may receive negative signals through the propagation of the information within the graph.

This logic explains the relatively high number of real articles classified as fake. Terms such as “crisis”, “European”, “Romania”, “war”, “government”, or “energy” can appear in both categories. If these words are strongly connected to fake articles, they can transfer a negative lexical signal to the real articles. Thus, the advantage of the model, its ability to capitalize on the relationships between documents and words also becomes, in this context, its main vulnerability.

In the case of fake articles classified as real, TextGCN struggled with satire and disinformation built on real political events. Absurd combinations, such as “biodegradable microphones”, very large vouchers for cherries, or global corn prices allegedly being influenced by popcorn orders, cannot be evaluated by simply analyzing the relationships between the nodes. Each word may have a coherent position in the graph, but the pragmatic combination of terms remains absurd. Detecting these nuances requires a real-world plausibility assessment that the TextGCN architecture does not provide.

The result of TextGCN is close to that of BERT, but the causes of errors are different. If BERT is limited by incomplete contextual processing of long texts and the difficulty of interpreting satire, TextGCN is mostly affected by lexical contamination by propagation of signals in the graph. This is relevant for the use of the model in media ecosystems where the same political or institutional terms circulate in both legitimate journalism and disinformation.

4.3. Comparative Synthesis and Typology of Errors

The comparative analysis shows that the results do not fully confirm the expectation that advanced deep learning models would automatically outperform classical models. Random Forest achieved the best overall performance, followed by BERT, TextGCN, and Naive Bayes. XGBoost and KNN produced unbalanced results, especially by over labeling articles as fake.

The results allow the identification of five recurrent types of errors. The first type consists of real articles with sensationalist or alarmist vocabulary. These include news about unusual events, very large values, controversial surveys, economic crises, or local situations with an emotional impact. Models tend to associate these elements with disinformation even though they can also appear in authentic journalism.

The second type also involves authentic articles misclassified as fake, but the triggering mechanism is different. These articles cover political events, electoral dynamics, party disputes, coalitions, or geopolitical affairs using the vocabulary of political journalism: party names, names of public figures, references to institutional crises, elections, or governmental decisions. This register overlaps substantially with the vocabulary of political disinformation, making the two categories difficult to distinguish.

The third type of error is represented by fake articles that use institutional, political, or technical vocabulary. References to governments, NATO, legislative procedures, energy infrastructure, elections, or public policies increased the likelihood that some false texts will be interpreted as real. This category is particularly important, because contemporary disinformation does not always appear in the form of obviously absurd texts and can mimic a legitimate journalistic style.

The fourth type consists of fake articles. To the human reader, the satirical character may be obvious, but models have difficulty interpreting non-literal intent. BERT and TextGCN, although more architecturally advanced, classified several satirical articles as real because they were grammatically or lexically coherent.

The fifth type of error is related to the effects of cross-lingual transfer. The models were trained on international datasets and tested on Romanian articles, which implies differences in language, journalistic style, political themes, and cultural registers. The results suggest that these differences affect both classical and advanced models but through different mechanisms: some models overreact to lexical frequencies, whereas others to graph structures or limited contextual fragments.

Table 7. Error categories.

Model	Total Errors	Sensationalist Vocabulary in Authentic Journalism	Political Register Overlap in Authentic Journalism	Institutional Vocabulary in Fabricated Content	Failure to Identify Satirical Content
Naive Bayes	29	7	0	18	4
KNN	51	27	21	3	0
Random Forest	23	11	4	7	1
XGBoost	43	24	13	5	1
BERT	27	14	4	4	5
TextGCN	28	13	7	3	5

reports, for each of the six models, the total number of misclassifications and the number of errors falling into each of the four categories, which are strictly related to the nature of the articles. Categories “1. Sensationalist vocabulary in authentic journalism” and “2. Political register overlap in authentic journalism” represents real news predicted as fake. Categories “3. Institutional vocabulary in fabricated content” and “4. Failure to identify satirical content” represent fake news predicted as real. The fifth type of error, related to cross-cultural transfer, is not represented as a separate column in the table because it constitutes a background condition rather than a classifiable feature of the articles themselves.

The four categories presented reflect the editorial classification that already existed in the source publications and fact-checking platforms from which the corpus was assembled.

For authentic news articles that were misclassified as fake, the categorization follows the topic of the article, as can be inferred directly from the title and source. Articles covering political events, electoral dynamics, party disputes, or institutional decisions were distinguished from articles covering non-political topics with sensationalist vocabulary. This distinction reflects the observable thematic content of each article.

The qualitative explanations of the misclassifications are interpretative and based on the observed linguistic and thematic characteristics of the misclassified articles. They should not be understood as definitive causal proof but as plausible error patterns that require further validation on a larger corpus and through additional statistical testing.

4.4. Practical Implications and Cautious Interpretation

In the context of this study, operational relevance is evaluated by analyzing how each model’s distribution of prediction errors—including false positives and false negatives—affects its practical utility for institutional deployment. From the perspective of the present exploratory experiment, Random Forest obtained the highest observed performance and the most balanced distribution of errors among the tested models. However, this result should not be interpreted as a recommendation for operational deployment given the limited size and non-representative nature of the Romanian test corpus.

BERT and TextGCN are promising in their ability to capture contextual or relational information, but the results show that these advantages are not sufficient in the absence of a better adaptation to the Romanian journalistic conventions, contextual political language, and local specificity of disinformation.

Naive Bayes may function as a baseline or reference model, but its major risk lies in classifying a large number of fake articles as real. KNN and XGBoost, in the tested form, are less useful for practical applications because they exhibit severe prediction imbalances. KNN classifies almost all articles as fake, while XGBoost excessively labels many real articles as fake.

For institutions involved in information threat monitoring, strategic communication, or public security, these findings should be interpreted only as preliminary evidence of the behavior of the tested models. The results suggest that automatic classifiers may be useful as experimental screening tools, but they cannot be considered reliable operational systems without validation on larger, more diverse, and representative Romanian news datasets.

Furthermore, when these operational insights are contextualized alongside existing Romanian fake news detection benchmarks, the implications of the empirical results become clearer. While Buzea et al. [44] demonstrated high performance using deep learning architectures, and Bucos and Drăgulescu [45] highlighted the necessity of data augmentation for transformer models handling smaller Romanian datasets, our findings introduce an important operational counterpoint. The findings confirm their baseline observations that transformer architectures (such as BERT) encounter significant bottlenecks when confronted with specific localized Romanian disinformation without extensive local fine-tuning. However, by demonstrating that traditional ensemble methods like Random Forest can achieve the highest observed predictive stability on a localized corpus without requiring complex data-augmentation loops, this study complements prior benchmarks, shifting the discussion from raw architectural optimization toward the practical feasibility of efficient institutional deployment.

5. Conclusions

Section 4 shows that the performance of fake news detection models depends not only on algorithmic complexity but also on the compatibility between the training data, the test corpus, and the type of content analyzed. Random Forest obtained the highest observed performance within the limited Romanian test corpus used in this study, but this result should be interpreted descriptively and cautiously and not as evidence of operational robustness.

Naive Bayes demonstrated moderate performance but with significant vulnerabilities in detecting sophisticated fake items. KNN and XGBoost highlighted important limits of calibration and transfer.

A comparison of the results with the research hypotheses shows that the hypotheses are confirmed to different degrees. H1 is only partially confirmed, as the advanced BERT and TextGCN models achieved competitive results but did not outperform Random Forest, which had the best overall performance. H2 is confirmed, as all models were affected by the gap between the internationally sourced training data and the Romanian-sourced test corpus, with performance limited by differences in journalistic convention, political context, and topic distribution rather than language itself. H3 is confirmed because classical models, especially Random Forest and Naive Bayes, achieved competitive results, demonstrating that lexical distributions can remain relevant in the classification of fake news. However, this confirmation needs to be qualified, given the poor performance of KNN and XGBoost. H4 is confirmed because the analysis of errors has shown that the main difficulties arise when legitimate journalistic language overlaps with the language of disinformation, especially in political, institutional, satirical, or alarmist articles.

Overall, the results support the idea that automatic detection of disinformation in the Romanian media space requires locally adapted models, representative training sets, and assessments that combine quantitative indicators with qualitative error analysis. Without these elements, models risk confusing authentic journalism with disinformation or allowing false articles to pass that effectively mimic the style of legitimate media.

In this context, a relevant direction for future research aims to develop and validate a “gold standard” data universe on a larger scale, designed exclusively for the Romanian media ecosystem. The expansion of the volume of data would also allow for the training of LLMs (large language models) that are optimized for the needs and specificity of local language and culture. Also, the proper integration of new automated verification mechanisms is an essential step in improving the accuracy of detecting hard-to-detect disinformation, which is increasingly taking ultra-sophisticated forms. Through the integration of real-time fact-checking features, future iterations could serve as foundational components for assisting fact-checkers in analyzing text structures rather than just automated tools for blocking content, and could assess not only how a message is written but also whether or not the central claims in that message are supported by verified, truthful, and factual data. The continued development of abusive disinformation systems presents a real danger, which requires increased attention and adaptability to keep up and prevent new models used for misinformation, contributing to the broader academic development of automated media literacy frameworks.

A further limitation concerns the absence of a direct comparison between the models’ performance on the international benchmark datasets and their performance on the English translation of the Romanian test corpus. As a result, the study cannot precisely quantify the transfer-performance gap between international and Romanian data. Future research should address this limitation by reporting model performance on both international and Romanian datasets, comparing the results statistically, and evaluating whether performance decreases when models are transferred to the Romanian media context.

The Romanian test corpus includes only 100 articles and cannot be considered representative of the full Romanian media ecosystem. Therefore, the observed performance of Random Forest, although the highest among the tested models, does not justify recommending the model for real-world operational deployment. Any operational use would require validation on substantially larger, more diverse, and continuously updated Romanian news datasets, including different media sources, topics, genres, and forms of misinformation.

Practical Recommendations

Based on the exploratory results, automatic fake news detection models should be used only as preliminary screening tools and not as autonomous decision-making systems. Even the best-performing model in this study, Random Forest, produced errors, while BERT and TextGCN also showed limitations in cases involving satire, political language, institutional discourse, and alarmist vocabulary.

For public security, strategic communication, and institutional monitoring, these models may help prioritize content for human review, especially during elections, crises, or periods of intensified disinformation. However, any practical use should involve expert verification, contextual analysis, and fact-checking. Before institutional deployment, the models must be validated on larger, more diverse, and representative Romanian datasets.