1. Introduction
In recent years, advancements in artificial intelligence (AI) have brought significant attention to detecting fake content, a critical area addressing the challenges posed by AI-generated texts. The increasing prevalence of Generative Pre-trained Transformer (GPT) models exemplifies the potential of AI to produce human-like textual content, blurring the lines between authentic and machine-generated texts. GPT models are a revolution in the field of Natural Language Processing (NLP) with their ability to produce human-like texts [
1,
2]. Training the GPT models requires huge data sets and high processing power. During the pre-training stage, the language model is run through a large data set and learns the language’s general rules, patterns, and relationships. This gives the model a huge knowledge base and, therefore, its success in carrying out the assigned tasks [
3]. In contrast to merely using word frequency, GPT models rely on contextual meanings in the language. These uses encompass understanding the different meanings of particular words and the grammatical structure of the language [
4].
Transformer-based language models can analyze syntactic and semantic features powerfully [
5]. This cognitive capacity helps in acquiring the natural fluency of language to produce texts closer to the natural flair of writing. The recent versions of GPT are even better able to offer texts that appear to be more naturally flowing, making it harder to distinguish between those written by human authors. Such situations may lead to new challenges regarding the reliability and accuracy of information [
6]. Circulation of fake texts or information pollution may further mislead decisions taken on incorrect information, unfavorably affecting individuals and organizations [
7,
8]. However, verifying all such information based on human assessment and expert judgment can be both time-consuming and costly. However, textual authenticity and expert confirmation of authenticity by way of manual inspection bear the high risk of an error rate ranging from 0.7 to 0.9, and they call for an average of 20 h of evaluation. Therefore, there is an urgent need for the development of effective methods for automatic detection of artificial intelligence-generated texts in both industry and academia [
9]. This creates a reliability problem, especially in information-based platforms such as news, academic studies, and social media, and facilitates the spreading of fake or manipulative content [
10,
11]. Reliable methods need to be developed that can successfully detect artificially written texts through the use of language and style analysis [
12,
13]. In the development of such models, studies are being conducted that combine natural language processing with machine learning algorithms and show highly effective results in fake text detection, particularly in feature sets based on different languages and styles. The models developed utilize a wide range of metrics, including style, language structure, grammatical complexity, word frequency, sentence length, and semantic connections within texts, resulting in improved performance in identifying both human and AI texts. Fake content detection having high accuracy could be achieved by algorithms through the separation of stylistics and semantic features in texting, the most prominent when combined with language processing techniques like Latent Semantic Analysis (LSA), Term Frequency-Inverse Document Frequency (TF-IDF), and word embedding [
14].
Problem Statement
A significant risk to information security arises from the similarities between AI-generated texts and real human texts, which makes it very difficult to detect AI-written text. With the rapid proliferation of false or misleading information on the internet, especially with the emergence of advanced language models such as GPT, rapid and accurate manual checking of such content becomes an enormous challenge [
15]. Detection of these texts with traditional methods can be time-consuming and prone to errors. Therefore, more advanced, reliable, and efficient methods are needed to automatically detect fake texts. While the high similarity of texts generated by artificial intelligence to real human texts makes this detection difficult [
12], the rapid spread of content containing false information with the development of models such as GPT threatens information security. It is known that manual fact-checking incurs problems in speed and difficulty because of the amount of online content, while detection processes using conventional means bring forth limitations and are definitely not reliable in terms of accuracy. Further, some widely used online detectors such as GPTZero and ZeroGPT have reported limited multilingual consistency [
16].
The present study addresses critical limitations and challenges in the detection of AI-generated fake texts by introducing an innovative hybrid model that combines a Transformer-based architecture with a graph-based model. Existing methods struggle with the high degree of similarity between AI-generated texts and human-written texts, making it difficult to reliably distinguish between the two. This issue is compounded by the lack of multilingual support and inconsistencies in existing tools, which limit their applicability across diverse linguistic contexts. Language modeling, stylometry, syntactic analysis, semantic parsing, and deep learning techniques are utilized to assess the structural differences between fake and real texts [
17]. The model thus developed is fitted together with Transformer-based structures, and the semantic depth of the language is analyzed through attention mechanisms and vector representations. In addition, advanced methodologies such as hyperparameter optimization, model adjustment methods, and k-fold cross-validation are applied to boost classification accuracy [
18]. The generalization ability of the model and its performance in fake text detection are statistically tested with datasets from different languages. The main contributions of this paper are summarized as follows:
The proposed model leverages advanced Natural Language Processing (NLP) techniques such as language modeling, stylometry, syntactic analysis, and semantic parsing to address the structural and semantic intricacies of fake and real texts. By incorporating these techniques into a unified framework, the model effectively evaluates subtle textual features that are often overlooked by traditional methods. The hybrid model of transformer models and Graph Convolutional Networks enables the analysis of semantic depth through attention mechanisms and vector representations, offering a more nuanced understanding of text authenticity.
The proposed model presents high accuracy in Turkish, English, and Arabic languages, and can successfully detect fake texts of different languages.
One of the significant innovations of this study is its focus on multilingual applicability. The model has been tested on datasets in Turkish, English, and Arabic, which are strategically selected to represent languages with varying levels of resources in NLP. This direction is motivated in part by the limited multilingual coverage reported for some existing online detectors, underscoring the relevance of the proposed framework in linguistically diverse contexts.
Three new datasets have been introduced, specifically for Turkish, English, and Arabic, including data with limited availability such as Turkish and Arabic.
As shown in
Figure 1, this study is composed of four main sections, including the introduction. Following the introduction, the second section presents related studies, the third section focuses on the materials and methods comprising the proposed model, and the final section concludes the study.
2. Literature Review
The literature studies primarily deal with the use and effectiveness of various techniques in AI and NLP for detecting fake texts. Deep learning models and algorithms generally go a long way toward reliable automatic detection of fake texts. These techniques analyze language structure, content, and images found in texts for signs of fakeness. A brief summary of the literature related to this study is presented in
Table 1. The scientific value of the research proposal is being justified by superior AI-based systems that can comprehend and discover, indeed, true fakes in a fraction of the time and without human assistance, compared to traditional methods. Subsequently, studies have tackled how these fake texts propagate through many platforms (e.g., the internet, academic articles, product reviews) and how they may be countered effectively. Such studies will further provide critical information geared toward developing strategies for achieving social good through fake texts, information security, and reliability.
Celikten et al. proposed a new model that improves the text classification architecture [
27]. The model was tested on a custom dataset using LSTM, Bi-LSTM, Attention mechanism, Bi-GRU, and fully connected networks. The results obtained indicate that the model achieves 90% accuracy on the test set. Further studies are needed to know how the performance of the model will change as the input data increases in size. Mitchell et al. propose a new method for detecting texts generated by large language models [
19]. In the present study, a curvature-based criterion termed as DetectGPT is defined, followed by text detection using random perturbations. The experiments show that DetectGPT is more discriminative than existing zero-shot methods, allowing an increase in AUROC from 0.81 to 0.95 for the detection of fake news articles. However, the approach warrants testing on a larger and more diverse dataset, and the generality across different text types needs to be explored. Verma et al. developed the “Ghostbuster” model to detect AI-generated texts [
28]. The model has achieved high accuracies on datasets such as creative writing, news, and student essays. However, its effectiveness on a wide variety of text types has not yet been fully tested. Cingillioglu et al. developed a successful model for detecting AI-generated essays using SVM and the n-gram Discrepancy Bag-of-Words model on 230 trials of a student essay dataset [
29]. The 92.7% accuracy rate obtained suggests that the model is effective, but it is not yet clear how it will perform on larger datasets. Gaggar et al. developed models that use RoBERTa and SVM to detect machine-generated texts in various domains [
30]. High accuracies were achieved in tests on multiple datasets. However, it was noted that further optimization is needed to correctly classify texts from very different domains. Wang et al. They developed the SeqXGPT model using convolutional and self-attention networks to detect AI-generated text at the sentence level [
31]. The SeqXGPT-Bench dataset appeared to have achieved very high accuracy rates. However, the work should also consider the generalization ability of the model on texts in languages other than English. Jawahar et al. addressed the problem of the distinction between texts generated by text generative models and those written by humans [
20]. In this paper, TGMs were shown to be misused for propagating fake news and fake product reviews on various datasets like WebText, Amazon Product Reviews, RealNews, and Common Crawl. The paper has, however, suggested the development of a detector which may discriminate between a Text Generator Model (TGM)-generated text and one authored by a human. However, it is emphasised that the performance of these detectors needs to be evaluated on texts written in different languages, and future studies have to turn their attention towards more general usage scenarios. Desaire et al. developed a classifier that classifies ChatGPT- and other major language model-generated texts against the texts of 10 chemistry journals for discrimination [
21]. The investigation developed a method for the detection of AI-generated texts based on the XGBoost classifier using texts generated by both GPT-3.5 and GPT-4. The results obtained demonstrate that the proposed method can distinguish between human and AI-written texts with an accuracy of 99%, therefore allowing for tracking of AI usage in the scientific literature. However, according to this investigation, this method is restricted to chemistry texts, and it is recommended to develop broader applicability by validating it on texts of many areas. A recent systematic review on machine learning for detecting deceptive activities on social media highlighted common methodological challenges, including class imbalance, evaluation practices, and multilingual validation, which were considered when designing and evaluating the proposed framework [
32].
Related work also includes LLM watermarking and watermark-robustness methods, which provide a complementary, generator-side approach to AI-text identification rather than post hoc supervised authorship classification; such methods were not benchmarked in our experiments but are noted as an important adjacent research direction.
3. Materials and Methods
This study aims to develop an innovative system that combines language modeling and stylometric analysis techniques for the detection of fake texts generated by artificial intelligence. Categorical distribution of human and AI-generated text data sources in
Figure 2. Initially, three large-scale datasets with different languages were constructed using a diverse range of sources, including social media platforms, academic journals, news websites, and AI-generated texts. The dataset underwent preprocessing to preserve the natural structural and linguistic features inherent to each language. This step ensured that the data retained its authenticity for effective analysis. The proposed system incorporates a Transformer-based language model, which was integrated with a graph-based model to analyze both the contextual meaning and stylistic nuances of texts. This hybrid architecture leverages the strengths of both models to enhance the accuracy and reliability of text classification. To evaluate the model’s multilingual capability, extensive testing was conducted using datasets in Turkish, English, and Arabic. The system’s performance in distinguishing AI-generated texts from human-authored texts was statistically analyzed to validate its effectiveness in a multilingual context.
3.1. Data Collection
The dataset used in this study was collected in three different languages, Turkish, English, and Arabic, to provide multilingual support. During the data collection process, certain categories were created to distinguish between human and AI-generated texts [
33]. Human data were collected from forum sites, academic papers, theses, and news websites. While AI data were compiled from AI chatbots, namely ChatGPT (OpenAI, San Francisco, CA, USA), Gemini (Google LLC, Mountain View, CA, USA), and Copilot (Microsoft Corporation, Redmond, WA, USA) [
34]. The human data were labeled as “0.0” in the dataset, while all AI-generated texts were labeled as “1.0.”
The dataset, created and written in Microsoft Excel (Microsoft Corporation, Redmond, WA, USA) format, enabled a Python (Python Software Foundation, Wilmington, DE, USA) script to automatically partition the data, with the aim of enhancing the model’s performance using specified sentence lengths. The script further divides the text into sentences of certain lengths before bringing each piece into the Excel file.
As seen in the Algorithm 1, the Python script (Python 3.14) allows the texts generated by artificial intelligence to be divided into pieces of various lengths. The “split_into_chunks” function used here groups the sentences by separating the text with dots and spaces, and each group creates pieces of a certain length. The pieces can be 5, 6, 7, or 8 sentences long, and each piece is written in the lines of the Excel file. All chunks derived from the same source document were subsequently assigned to the same train, validation, or test partition to prevent cross-split leakage. This process allows the data to be divided into smaller and more manageable pieces, thus optimizing the learning process of the model [
35]. This script plays an important role in training the model by automating the data collection and processing process. The datasets generated in this study were organized to include text and generated columns by labeling human data as “0.0” and AI-generated text as “1.0”, and examples from datasets in different languages are shown in
Figure 3.
| Algorithm 1 AI text splitting and Excel writing algorithm |
- 1:
Input: text: Original text; chunk_length: Sentences per chunk (default 6, allowed 5–8); label: Class label (0.0 human, 1.0 AI) - 2:
Output: Excel file with text chunks and class labels - 3:
Begin - 4:
Split text by ". " into sentences - 5:
Group sentences into chunks of chunk_length, ending with a period - 6:
Try loading the language-specific workbook (e.g., data-arabic.xlsx) - 7:
if not found then - 8:
Create new workbook and add headers ["text", "generated"] - 9:
end if - 10:
for each chunk in chunks do - 11:
Write chunk to column 1 and label to column 2 - 12:
end for - 13:
During subsequent train–test splitting, assign all chunks from the same source document to the same partition - 14:
Save workbook - 15:
End
|
To ensure topic consistency while preserving linguistic diversity, human-written texts were collected from multiple domains, and the prompts used to generate AI-written texts were derived from the titles and subject matter of the corresponding human-written documents. Consequently, both classes covered comparable topics across diverse domains, reducing the likelihood that the model relied solely on domain- or topic-specific characteristics rather than authorship-related patterns.
To ensure the reproducibility and broader accessibility of our study, the complete dataset used for the GCN-DistilBERT hybrid model has been shared on Hugging Face (Hugging Face, Inc., New York, NY, USA). The dataset files in Arabic, English, and Turkish are provided in Excel format for ease of use. The dataset can be accessed at
https://huggingface.co/datasets/aycabostancioglu/Arabic-English-Turkish-AI-Human-Text (accessed on 15 July 2026). The repository includes detailed explanations and examples in the README.md file to guide researchers and practitioners in exploring and utilizing the data effectively.
3.2. Data Preprocessing
In this section, the cleaning and augmentation techniques applied for the preparation of the dataset are discussed in detail. Preprocessing steps, lemmatization, character cleaning, data augmentation strategies, and graph feature extraction are explained in order to make the texts suitable for the language model and graph-based structure [
36].
Approximately 3000 text samples were collected for each language (English, Turkish, and Arabic). An 80/20 train–test split was applied before augmentation. Then, 10% of the training partition was held out as a validation set for early stopping; data augmentation was applied only to the remaining training samples, while the validation and test sets were held out unchanged. Final metrics were computed on held-out test sets of 1765 (English), 1995 (Turkish), and 1691 (Arabic) samples. The slight variation in test set sizes across languages reflects differences in the number of samples collected and chunked per language.
On the training set only, data augmentation techniques were applied to increase the generalization ability of the model and to prevent over-learning [
37]. In this context, various strategies were applied to diversify the training dataset. The “Word Replacement” method randomly swaps the places of the two chosen words in the text and diversifies the structure of the text without damaging it [
38]. The “Word Deletion” technique deletes random words in a way that maintains semantic integrity in the text, helping the model focus on important words. The “Synonym Replacement” technique created variants without damaging the semantic structure of the text by replacing certain words with their synonyms. These processes, in conjunction with the original dataset, helped in building a wider and balanced training set, thus augmenting the model’s learning capacity based on data. The data augmentation enabled the model to become flexible toward various types of data while also reducing bias and variance during the learning process.
In data cleaning, processes are carefully laid out on raw data to render them suitable for easy portability into the model to avoid wrong predictions or outcomes. This process forms a very important step towards enhancing the quality of data with which the model would be trained to derive valid information out of it [
39]. The basic operations applied in the data cleaning program are explained below:
- –
Removal of Non-Alphanumeric Characters: Unnecessary symbols are removed from the texts, retaining only letters, numbers, and spaces.
- –
Reducing Repeated Characters: Repeated character sequences are written down just once by reducing the number of characters in repetitive instances.
- –
Deletion of Single Characters: Words consisting of a single character that are void of meaning are removed from the texts.
- –
Convert to Lowercase: Everything has been converted to lowercase for some uniformity.
- –
Cleaning Unnecessary Spaces: Excessive spaces have been merged and the texts have been brought into a more compact form.
- –
Removal of HTML Tags and Punctuation: The special characters and HTML tags are cleaned.
- –
Lemmatization: Morphological variation is minimized by putting words to their roots.
The functions mentioned above (e.g., remove_non_alphanumeric, remove_single_characters, etc.) are used to clean unnecessary symbols and incorrect characters from the text. These functions contribute to the learning process of the model by purifying the data. In particular, the remove_non_alphanumeric function prevents the model from encountering meaningless symbols by keeping only alphanumeric characters in the text [
40]. The remove_single_characters function eliminates single-character words (e.g., “a”, “I”), allowing meaningful words to stand out. As a result, these preprocessing steps optimize the learning process of the model, reduce noise, and provide more accurate predictions. These steps increase the cleanliness level of the text, significantly improving the overall accuracy and efficiency of the model.
3.3. Feature Extraction
It offers a hybrid combination of Graph Convolutional Networks (GCNs) and transformer-based models, where feature extraction plays a critical role in capturing both the structural relationships found in graph data and the contextual information found in text sequences. DistilBERT is particularly useful for extracting meaningful features, enabling the model to effectively combine graph-based relational features with sequential contextual information.
Figure 4 shows how this approach leverages the strengths of both types of models, effectively combining graph-based relational features with sequential contextual information in text to capture complex dependencies in data and improve overall performance.
In our experiments, text is represented as a sequential chain graph: each preprocessed text chunk is one node, and undirected edges connect consecutive chunks in reading order within each train, validation, or test partition. In the proposed architecture, each text chunk constitutes one node , and undirected sequential edges connect consecutive chunks , yielding a sparse adjacency matrix with . Each node is associated with a feature vector derived from the corresponding chunk. That graph-based representation gives this model both local and global dependencies with operational ease.
For reproducibility, the experiments follow a fixed graph protocol. In each train, validation, or test partition, one node corresponds to one preprocessed text instance (chunk row). Nodes are linked only along the document order in that partition, forming an undirected chain graph with binary, unweighted edges (
), and no separate edge-weighting scheme is applied. Message passing uses symmetric degree normalization through the standard GCN formulation in Equation (
1), where
balances contributions from neighboring nodes. Each node is initialized with a 16-dimensional feature vector, passed through a two-layer GCN (
) trained on the training graph, and the resulting 8-dimensional node embedding
is concatenated with the DistilBERT
CLS representation before classification. For morphologically rich Turkish (agglutinative suffixation) and Arabic (root-and-pattern structure), DistilBERT encodes local contextual cues within each chunk, while the sequential GCN propagates document-level dependencies across consecutive chunks via message passing, complementing transformer representations without language-specific syntactic parsers.
Graph Convolutional Network (GCN) is the central element of the graph-related component [
41]. It intelligently aggregates information from neighboring nodes, ensuring an accurate representation of the graph while reducing the computational burden typically associated with classical GNN models. The GCN updates the node representations via a propagation process, in which the graph structure is preserved. This process is implemented through multiple layers, such as GCNCon1 and GCNCon2, each performing a step of graph convolution. At each layer
k, each node
aggregates information from its neighbors
, as shown in Equation (
1):
where
is the hidden state of node
i at layer
k,
is the adjacency matrix entry indicating the relationship between nodes
i and
j,
is the learnable weight matrix for layer
k, and
is a non-linear activation function, such as Rectified Linear Unit (ReLU). As seen in
Figure 4, for Feature Extraction, the GCNCon1 layer performs the initial graph convolution operation by transforming the raw node features into an intermediate representation. The GCNCon2 layer then further refines these embeddings, capturing higher-order relationships and improving the quality of graph-level representations. The normalization constant
is typically calculated to account for the degree of the nodes to ensure that each node aggregates information in a balanced manner. This iterative message-passing process allows each node to capture global structural information as the number of layers increases.
After K layers of graph convolutions, the node embeddings are aggregated to form a graph-level representation . Various aggregation methods can be applied, such as the following:
- –
Max Pooling: This selects the maximum value across nodes for all feature dimensions.
- –
Mean Pooling: This computes the average across all the node embeddings.
Graph-level embedding
is a rich approach where the structural features of the graph are combined with the sequential features of the transformer. The model leverages a hybrid architecture that combines DistilBERT, a compact and efficient version of Bidirectional Encoder Representation from Transformers (BERT) [
42], with a Graph Convolutional Network to efficiently handle sequential text data and graph structured data. In the proposed architecture, transformer-based DistilBERT takes sequential data as input,
, where
T denotes the length of the sequence. It also processes graph data with a GCN. This hybrid model offers great performance as it captures both contextual information from text and structural relationships from a graph.
3.4. Classification Process
DistilBERT works on the input sequence by first expressing each token
as a dense vector
, utilizing a pre-trained embedding layer. These embeddings are then enriched with positional encodings to preserve the token order of the sequence, as shown in Equation (
2):
where
is the positional encoding for the
i-th position, which is crucial for the model to determine and preserve the sequence order, which is a mandatory requirement for text classification operations. The transformer architecture uses self-attention to extract relationships between tokens of the sequence, regardless of their being in positions of either. The attention mechanism computes attention scores for words, as shown in Equation (
3):
where
Q,
K, and
V are query, key, and value matrices, respectively, and
is the dimension of the key vectors. This mechanism of attention enables the model to weigh the significance of one word in relation to others and, thus, permits long-distance dependencies that are a part of the sequence to be learned as well.
The self-attention mechanism is deployed in multiple layers of the transformer to produce contextual embeddings , where each contains information about the token as well as its contextual relationship to the other tokens in the sequence. The final sequence representation, , is usually found either through the embedding of the last token or by a mechanism of mean pooling or using the [CLS] token, which aggregates information from the whole sequence.
The hyperparameters are the most important factors influencing the performance of the hybrid model. Hyperparameters are determined and adjusted to optimize the training efficiency and accuracy of the model. This includes training speed, the number and size of sessions, and various optimizations to improve model performance.
Table 2 provides a detailed explanation of all the fundamental hyperparameters and their corresponding values. These parameters govern the various stages of the model’s training process. In fact, the learning cost (
) controls the size of each iteration when moving to the minimum performance. A small initial value
= 2 × 10
−5 was chosen to allow for tuning of the model parameters at the beginning of the training phase. The batch size of 64 indicates that this is the number of samples the classifier needs to calculate the gradients and update the model [
43]. Smaller batch sizes introduce more noise and variation in the gradient adjustment, which can improve accuracy, but larger batch sizes provide greater stability. The number of iterations indicates how often the model is fed the data. This was considered 25 times, which gives the model enough time to learn the correct patterns of the material.
Optimization algorithms are an important part of tuning model parameters to reduce the loss function [
44]. This study uses the Adam optimization algorithm that analyzes each parameter based on the gradient of the first and second terms. This approach successfully solves deep learning problems and simplifies learning rate adjustments. The numerical expression of Adam optimization is given in Equation (
4):
where
represents the model’s parameters at step
t,
is the learning rate,
is the first moment (the mean of the gradients),
is the second moment (the variance of the gradients), and
is a small constant (typically
) to prevent division by zero. At each stage, Adam continuously adjusts the moments for faster and efficient learning. Assorted techniques include learning rate schedulers and gradient clamps. This one here is a linear scheduler, decreasing the learning rate over time while allowing fine-tuning of model parameters as training progresses. Gradient clipping is adopted to avoid the explosion of gradients by limiting them to some predefined value [
45]. This technique provides more stable training while avoiding instability due to excessively large updates. So, all parameters of training, together with optimization procedures, are put in place for the model to learn while the risk of overfitting is minimized. Using gradient accumulation enhances the memory efficiency of the training because the gradients are accumulated over several smaller batches before updating the parameters. Mixed-precision FP16 training was put to good use to speed up the process and reduce memory requirements, thus making efficient use of resources.
Table 2 shows the technical configuration and training parameters. The table gives pertinent details that allow a person to comprehend what specific settings were used throughout training and reproducing the experiments more easily.
The hybrid GCN-Distilled Bidirectional Encoder Representations from Transformers (DistilBERT) model combining GCNs and transformer-based models for text classification tasks is shown in Algorithm 2. Simultaneously, a Graph Convolutional Network (GCN) processes graph-structured data to produce node-level embeddings
. DistilBERT encodes each text chunk independently and produces a sequence-level representation
from the
[CLS] token;
is not injected into the transformer encoder but is concatenated with
before the final classification layer, as specified in Equations (
5) and (
6). The final output of the GCN,
, captures structural relationships among sequentially connected chunks, while the DistilBERT component captures the contextual information of each chunk.
The hybrid model, as seen in Equation (
5), integrates both representations to establish a single feature vector:
The neural network concatenates the vector
as described in Equation (
6), it is then transferred into the single dense layer next which culminates with the application of a softmax activation function leading to the final predictions:
where
and
signify the parameters of the output layer, being the learnable weights and biases.
Apart from the cross-entropy loss, which one may see in Equation (
7), the model is conditioned on finding the true labels
and the predicted probabilities
:
where
N is the number of samples,
is the true label, and
is the predicted probability for the
i-th sample.
| Algorithm 2 Algorithm describing the hybrid GCN-DistilBERT model. |
Require: Train set , Validation set , Test set , GCN parameters , tokenizer, DistilBERT model, number of epochs E, batch size B, learning rate
- 1:
Step 0: Dataset Partition and Augmentation - 2:
Apply an 80/20 split to obtain and - 3:
Hold out 10% of as validation subset for early stopping - 4:
Apply data augmentation to the remaining training samples only - 5:
Step 1: Graph Construction - 6:
for each dataset do - 7:
Define each preprocessed text instance (chunk) as one node - 8:
Construct undirected chain edges with binary adjacency - 9:
Initialize a 16-dimensional feature vector for each node and store the graph - 10:
end for - 11:
Step 2: Feature Extraction - 12:
Initialize GCN model with parameters - 13:
Train GCN on for E epochs: - 14:
for epoch to E do - 15:
for each batch in train loader do - 16:
Forward pass: - 17:
Compute loss and update parameters - 18:
end for - 19:
end for - 20:
Extract GCN features for , , and - 21:
Step 3: DistilBERT Encoding and Fusion - 22:
for each dataset do - 23:
Tokenize texts and obtain [CLS] representation - 24:
Concatenate with before classification - 25:
end for - 26:
Step 4: Dataset Preparation - 27:
Define a custom dataset to handle tokenized data, GCN features, and labels - 28:
Create train, validation, and test datasets - 29:
Step 5: Model Training - 30:
Initialize DistilBERT model for sequence classification - 31:
Define training arguments and initialize Trainer - 32:
Train model on for E epochs - 33:
Use early stopping on and save the best model
|
To optimize model parameters, as seen in Equation (
8), an adaptive learning rate, namely, the Adam optimizer, is used that guides the learning rate to be based on each parameter:
where
represents the model parameters,
is the learning rate, and
is the gradient of the loss function with respect to the parameters.
This hybrid architecture combines the strengths of transformers for sequential data, such as DistilBERT, with the relational modeling capabilities of GCNs for graph-structured data. Applications such as sentiment analysis, question answering, and relational text classification can benefit from this approach because the model can account for both the text sequence and the structural relationships presented in the underlying graph. The model is flexible enough for a variety of domains and input data types. With simultaneous use of sequential and graph-based input, it is suitable for complex multimodal learning tasks such as social network analysis, recommender systems, and biological data processing, where text and graph-structured data are abundant.
4. Experimental Results and Discussion
In this section, the results of the experiments conducted to evaluate the performance of the model are presented and these results are analyzed. As shown in
Table 3, various models have been proposed in the literature for text classification and the detection of content generated by artificial intelligence. A hybrid model developed has achieved text classification success with 90% accuracy by using network structures such as Long Short-
Term Memory (LSTM), Bi-LSTM, Attention mechanisms and Bi-GRU [
27]. The model called Ghostbuster aimed to detect texts generated by artificial intelligence and this model achieved significant success with 97.5% accuracy [
28]. Similarly, 92.7% accuracy rate was achieved on student essays written with Support Vector Machine (SVM) and n-gram based models proposed by Cingillioglu et al. (2023) [
29]. With RoBERTa-based models, successful results such as 97.11% Area Under Curve (AUC)–Receiver Operating Characteristic (ROC) and 99% F1-score have been achieved on multiple datasets [
30]. On the other hand, with the model called SeqXGPT, it has achieved significant success with 97.6% Macro-F1 and 99.2% precision using convolutional and self-attention networks to detect AI-generated sentences [
31].
Our proposed model uses a hybrid DistilBERT and GCN (Graph Convolutional Network) model to detect fake content based on language in multilingual datasets. This model achieved 99% accuracy for English and 98% accuracy for Turkish and Arabic. These results are quite competitive when compared to existing models in the literature. Particularly, it combines linguistic and stylometric analyses and achieves strong and consistent performance across the three evaluated languages, yielding accuracy comparable to or better than most literature baselines tested under the same multilingual protocol.
To understand the results of the experiments conducted on English, Turkish, and Arabic datasets in more detail, loss graphs, accuracy curves, confusion matrices and ROC curves are presented in
Figure 5. Final evaluation was performed on held-out test sets comprising 1765 (English), 1995 (Turkish), and 1691 (Arabic) samples, as shown in the confusion matrices. The experiments were performed using a hybrid model, consisting of Graph Convolutional Networks and DistilBERT, which can efficiently process text data and learn contextual representations. The performance of this model on each of the languages is provided in
Table 4. To address uncertainty in the reported point estimates,
Table 5 reports Wilson 95% confidence intervals for test-set accuracy and two-proportion
z-tests comparing GCN+DistilBERT with the BERT baseline evaluated on the same held-out partitions.
As seen in
Table 4, high accuracy rates (98 percent and above) were achieved on all language datasets. In particular, the English dataset exhibited excellent performance with a sensitivity value of 1.0, meaning zero false negatives. Slightly lower precision values were observed on the Turkish and Arabic datasets, but these decreases were minimal and did not affect the overall performance of the model.
The loss and accuracy curves show how the model performed whilst being trained on the datasets, determining its generalization ability. This shows that the model converged properly during training, indicating that overfitting can be avoided. The analysis of the curves also proved that the model performed evenly on both training and validation data without creating problems of underfitting or overfitting. The confusion matrices of English, Turkish, and Arabic show the prediction abilities of the models in detail. These confusion matrices have a high number of true positives (TPs) and true negatives (TNs) and only a small quantity of misclassifications (i.e., contracting false positives and false negatives). The Receiver Operator Characteristic (ROC) curve was used to measure each model’s discriminative ability for each class. This determined the area under the ROC curve (AUC), which illustrates how much the model can correctly classify; the closer the AUC is to 1.0, the better would be the model. All models’ AUC values are high and close to 1.0, meaning they maintained good sensitivity and specificity.
The experimental results highlight the overall performance of the model and its effective learning ability. Data augmentation methods for growing datasets further strengthened learning abilities, with model accuracy still above 98 percent; the balance between sensitivity and precision was kept quite well. The GCN and DistilBERT combination provided a very powerful model able to sequence linguistic structures through graph-based and transformer-based embedding techniques. In summary, it was observed that the proposed hybrid model approach provides high accuracy rates by effectively generalizing on multilingual datasets and achieves successful results in modern natural language processing tasks.
The results in
Table 6 show how well the proposed model performs for the binary classification task of AI-generated texts compared to human-written texts across Turkish, English, and Arabic datasets. The GCN+DistilBERT model, where graph-based learning is used with transformative embeddings, is found to be the superior model in terms of optimal solution across all datasets. The results obtained for the Turkish dataset are a remarkable accuracy of 98.8%, while the precision, recall, and F1 score metrics for this model are 98.7%, 99%, and 98.9%, respectively. These scores prove that it can find a middle ground between classifying one text as AI-generated and the other as human-written. The time taken for this model to learn is only a few minutes, which shows its efficiency in computational power compared to many other models.
In the English dataset, since the structural behavior of the language and the quality of the datasets really render high performance to the model, GCN+DistilBERT stood its ground. While GCN+BERT achieved a slightly higher accuracy on the English (99.8% vs. 99.6%) and Arabic (98.1% vs. 98.0%) datasets, GCN+DistilBERT reduced total training time by approximately 45% and 40%, respectively (1.32 vs. 2.41 min for English; 2.58 vs. 4.29 min for Arabic), offering a favorable accuracy–efficiency trade-off. This efficiency is particularly crucial in large-scale or real-time text classification tasks.
To isolate the contribution of individual components, an ablation study was conducted on the English, Turkish, and Arabic datasets (
Table 7). The full configuration corresponds to the proposed GCN+DistilBERT model. Removing the GCN branch (DistilBERT only), disabling feature fusion, or omitting preprocessing and augmentation consistently reduced accuracy relative to the full model, with data augmentation showing the largest impact on the English dataset (accuracy dropping from 0.996 to 0.910). Varying GCN depth confirmed that the default two-layer configuration used in the full model achieved the best trade-off among the tested settings. These results indicate that graph-based features, preprocessing, and augmentation each contribute to the overall performance of the hybrid architecture.
To assess robustness beyond the in-house multilingual corpora, we conducted a cross-dataset evaluation on an external English AI/human text collection (Kaggle;
). A stratified sample of 2500 instances (30–250 words per document) was drawn; 500 were used for domain-adaptation fine-tuning and 2000 were held out for testing. The GCN+DistilBERT model was first trained on
data-eng.xlsx, then lightly fine-tuned on the Kaggle adaptation split (2 epochs, max sequence length 128). On the held-out cross-dataset test set, the model achieved 92.2% accuracy, 89.0% precision, 95.0% recall, and 92.4% F1 (
Figure 6). Although this is lower than the 99.6% in-distribution English accuracy, it demonstrates that the hybrid architecture retains strong discriminative ability on unseen writing styles and an external data source after modest domain adaptation.
Human-edited, heavily paraphrased, or complex-prompt AI outputs were not collected as a separate class; our labels compare unedited human texts with direct chatbot responses under topic-aligned prompts. Training-only augmentation (word replacement, deletion, synonym replacement) and cross-dataset evaluation on Kaggle (
Figure 6) provide partial robustness to surface perturbations and style shift, but dedicated benchmarks on adversarially edited or paraphrased AI text remain future work.
One of the biggest advantages of the GCN+DistilBERT model is its potential to process large datasets much more efficiently than other transformer models. Increasing dataset sizes greatly increases the computational costs of using models such as BERT, Robustly Optimized BERT Approach (RoBERTa), or Efficiently Learning an Encoder that Classifies Token Replacements Accurately (ELECTRA), making them doubly impractical for online or large-scale applications. In contrast, using a more lightweight yet efficient transformer model such as DistilBERT will provide significant reductions in processing times without sacrificing model accuracy. In big data analysis scenarios where time efficiency becomes as important as classification performance, the hybrid model may become more promising. Moreover, the addition of GCN provides this potential advantage in advancing the process and leads to structural insights that increase the efficiency of the semantics from DistilBERT.
The findings highlight the benefits of combining GCN with transformer models for text classification tasks. The optimal architecture combines the ability of GCN to represent syntactic relationships with the semantic depth of transformer embeddings, creating a model that effectively distinguishes AI-generated from human-written text. In the future, accessible combinations may evoke such approaches for other languages or domains, ranging from journalism to academic writing or creative literature, especially as the depiction of AI-generated materials becomes more urgent. Moreover, as dataset sizes continue to grow, the efficiency of GCN+DistilBERT compared to heavier transformer models will become clear. This will solidify GCN+DistilBERT as a practical and robust model for dealing with today’s challenges in NLP. This synergy of structural and semantic learning paves the way for more interpretable and scalable NLP solutions, ensuring adaptability in an ever-evolving AI landscape.
Despite strong in-distribution results, this study has several limitations: AI outputs were stored as a single mixed class without per-generator labels; heavily human-edited or paraphrased AI text was not benchmarked; cross-dataset validation was limited to an external English corpus; detector-specific and watermarking baselines were not evaluated; and the model addresses authorship rather than factual veracity.