Policy-Based Spam Detection of Tweets Dataset

Abstract

Spam communications from spam ads and social media platforms such as Facebook, Twitter, and Instagram are increasing, making spam detection more popular. Many languages are used for spam review identification, including Chinese, Urdu, Roman Urdu, English, Turkish, etc.; however, there are fewer high-quality datasets available for Urdu. This is mainly because Urdu is less extensively used on social media networks such as Twitter, making it harder to collect huge volumes of relevant data. This paper investigates policy-based Urdu tweet spam detection. This study aims to collect over 1,100,000 real-time tweets from multiple users. The dataset is carefully filtered to comply with Twitter’s 100-tweet-per-hour limit. For data collection, the snscrape library is utilized, which is equipped with an API for accessing various attributes such as username, URL, and tweet content. Then, a machine learning pipeline consisting of TF-IDF, Count Vectorizer, and the following machine learning classifiers: multinomial naïve Bayes, support vector classifier RBF, logical regression, and BERT, are developed. Based on Twitter policy standards, feature extraction is performed, and the dataset is separated into training and testing sets for spam analysis. Experimental results show that the logistic regression classifier has achieved the highest accuracy, with an F1-score of 0.70 and an accuracy of 99.55%. The findings of the study show the effectiveness of policy-based spam detection in Urdu tweets using machine learning and BERT layer models and contribute to the development of a robust Urdu language social media spam detection method.

1. Introduction

Online social media is an essential tool for communication in the high-tech society of today. Social media sites such as Facebook, Instagram, and Twitter have made it possible to communicate with people around the world. Twitter is a well-known social media platform that enables communication and global awareness. As an interactive platform, Twitter’s main objective is to continuously engage its users through tweets. Unfortunately, this objective has been hampered by spammers who use Twitter to disseminate harmful information [1] and unsolicited communications. This regularly detracts users from general subjects and lowers the quality of the Twitter experience for actual users. On Twitter, spam has unquestionably grown to be a major problem. Users can report accounts they believe to be spam on Twitter as well. Studies already published use machine learning (ML) and deep learning (DL) techniques to find and isolate suspected spam Twitter accounts. Twitter uses natural language processing (NLP) and tweet-ranking algorithms to analyze thousands of tweets per second. Over time, it becomes better at predicting what tweets the user will find most interesting. These algorithms gather as much information about us as possible. This information includes what you prefer to watch, how you react to status updates, and even the links you click. The platforms then utilize machine learning to make extremely accurate predictions of your future desires [2,3]. Deep learning, on the other hand, employs a technology that improves machines’ capacity to detect and magnify even the tiniest patterns [4]. Machine learning can be used for investigation so that attackers can look at their target’s traffic patterns, defenses, and potential vulnerabilities.

Spammers pose a significant risk to the internet [5] and spam tweets detection has become a serious issue. Moreover, with the rapid rise of internet users, the number of tweet account spammers has also increased. They are being used for unlawful and unethical behavior, phishing, and fraud [5]. Likewise, harmful links attached via spam tweets get access to our payment details or perform other harmful acts. This paper presents effective ways of the most common algorithms to develop an ML- and DL-based model that can determine if the tweet in Urdu is spam or non-spam. We have scraped Twitter to construct an Urdu dataset. This study applies ML and DL methods to the processed datasets to determine the best algorithm for spam tweet detection based on policy and timestamp features to achieve the highest precision and accuracy.

1.1. Motivation

There is a need for additional research on the Urdu language due to the rising number of Urdu speakers in different nations. In the realm of speech processing, there is a demand for transcribing audio files to text files, such as Urdu audio files to Urdu text files. Aside from this, we have numerous additional needs for Urdu language processing in NLP, speech processing, and possibly other domains. A substantial number of Urdu datasets are available; however, most of them are of lower quality and limited in size as compared to English datasets. It is crucial that we collect an Urdu dataset and make the tools and dataset open source in order to expand the research in this area.

The main motivation of this work is, thus, to compile a high-quality Urdu tweets dataset to distinguish between spam and non-spam tweets utilizing ML and DL algorithms. Compared to English, there are fewer high-quality datasets available for spam review detection in Urdu. Urdu is a less commonly used language on social media platforms such as Twitter, which makes it harder to collect large amounts of relevant data.

Table 1. Comparison of available English and Urdu datasets for spam review.

Dataset Name	Language	Number of Tweets/Messages	Size of Dataset	Quality
Twitter Sentiment Analysis Dataset [6]	English	1.6 million	Large	High
Twitter Spam Classification Dataset [7]	English	25,000	Medium	High
SemEval-2015 Task 10 [8]	English	N/A	N/A	High
CrowdFlower Twitter Spam Corpus [9]	English	10,000	Small	Medium
Stanford Large Movie Review Dataset [10]	English	50,000	Large	High
Urdu Sentiment Corpus [11]	Urdu	5000	Small	Medium
UrdConv Corpus [12]	Urdu	22,000	Medium	Medium
COMSATS Urdu SMS Spam Corpus [13]	Urdu	10,000	Small	Medium
Social Media Urdu Corpus [14]	Urdu	6500	Small	Low

compares the available English and Urdu datasets for spam review detection.

In Table 1, the dataset size corresponds to the number of tweets. Large quantity datasets represent a large number of tweets inside the dataset or corpus, whereas small quantity datasets represent a smaller number of tweets within the corpus. In this context, quality refers to the dependability of the annotations and the neatness of the dataset. High-quality datasets have been meticulously annotated and cleaned, whereas low-quality datasets may contain inaccurate or noisy annotations or be difficult to manipulate due to uneven formatting or missing data.

1.2. Challenges

A major challenge for our study was selecting the Urdu language for the research, given there has been little work done on the raw Urdu dataset, whether it be an Urdu collection of datasets or experimentation with the dataset. In addition, while collecting features from our dataset, we analyzed Twitter’s policies. One of the policies was to limit the number of tweets per day to 2400 [15], so we could have 100 tweets every hour. Any number exceeding the standard limit may be deemed spam. We were also tasked with extracting features from a limited number of features, such as a person’s name, user ID, and tweet. We meticulously evaluated the data and determined if the aggregation of user count was based on some check statement of policy. As such, the policy specifies a maximum of 100 tweets every hour.

1.3. Research Contribution

The research contributions provided by this paper are as follows:

• Included a total number of 1,100,000 plus (+) unique real-time Urdu tweets of multiple users. The dataset is publically available on Kaggle [16];
• Proposed a model to process real-time tweet couplets with feature extraction of Urdu couplets considering timestamp and policy features;
• Evaluated the model by applying machine learning and deep learning training and testing on the Urdu dataset to achieve improved accuracy.

The rest of the paper is organized as follows. The Introduction is followed by a Literature Review, which highlights the latest related work in the domain. It also highlights the research gap in existing work. Section 3 described machine learning and deep learning algorithms. Section 4 presents the proposed algorithm, preprocessing, feature extraction details, and architecture of the model. The results and discussion are presented in Section 5. Finally, Section 7 concludes the paper.

2. Literature Review

Today, news-related spam is pervasive on social media platforms, with Twitter being the most popular. It is now harder for users to spot fake tweets. In addition, only a few studies have analyzed spam tweets using machine learning and deep learning techniques, as mentioned in the following, with the majority of studies focusing on English, Chinese, Arabic, and Turkish. It has been highlighted that a comprehensive study examining spam review detection in Urdu tweets does not exist. Therefore, it is vital to develop reliable and precise ML and DL techniques employing Urdu tweets. This section examines the available research in order to investigate the Urdu language techniques using social website datasets.

Existing studies are analyzed according to the type of dataset utilized, such as online or offline, the type of algorithm, and the results achieved by each algorithm. In order to improve accuracy, researchers are performing experiments on a variety of languages, leveraging diverse data sets, and presenting results with differing degrees of complexity. Ge et al. [17] conducted forensic analysis on a vast distribution of Urdu corpus. It is tested with Latent Dirichlet Allocation (LDA) and cosine similarity to detect textual similarity. Anwar et al. classify text using an individual and profile-based LDA classification strategy, achieving an f1-measure of 92 percent [18]. Mashooq et al. [19] explored sentiment analysis in Urdu comprehensively, covering future potential, taxonomy development, and associated issues. Hussain et al. [20] make a significant addition to the field of spam identification in Roman Urdu scripts by providing insights and approaches for recognizing misleading product evaluations in the realm of spam review detection in Roman Urdu scripts. Similarly, Hussain et al. [21,22] present a complete analysis of linguistic and spammer behavioral strategies for detecting spam reviews individually and in groups, providing useful insights and methods for efficiently addressing the problem. Duma et al. [23] introduce a unique strategy that integrates review text, overall ratings, and aspect ratings to efficiently detect false reviews, highlighting the potential of a deep hybrid model in enhancing the accuracy of fake review detection.

Combining ML and NLP techniques, such as multinomial naïve Bayes (MNB), support vector machine (SVM), expectation–maximization algorithm (EM), and stop words, lemmatization, and stemming [24], are used to identify fake reviews. Mekala et al. [25] demonstrate high precision by utilizing the approach of stylistic characteristics and term weight measurement. Saha et al. [26] achieved 96 percent accuracy using MLP on a dataset of social text from social media platforms. Similarly, Benzebouchi et al. [27] apply word2vec and then multilayer perceptron (MLP) to the English corpus, which raises accuracy to 95.83 percent. It is simple and computationally inexpensive to extract user-profiles and message content-based information using the Twitter API. Due to the scale and complexity of the Twitter user graph, collecting this information takes time and money. However, they can be utilized to identify spam. Assuming that all spam tweets contain URLs, some studies utilize URLs embedded in tweets to identify spam on Twitter [28]. Similarly, Khanday et al. is conducting a study to identify propagandistic nodes using social network analysis [29].

Based on the reviewed literature, it has been observed that several researchers have considered classifying spam tweets as language-specific evaluations, such as Chinese, Roman Urdu, English, and Turkish, therefore, it is necessary to evaluate and identify spam tweets in an Urdu dataset.

Table 2. Research gap analysis in the existing literature.

Papers	Urdu Dataset	Techniques	Feature	Policy Feature	Timestamp Feature
[30]	×	LSTM	Features on frequent or important Corpus	×	×
[31]	×	Transfer Learning	Feature mapping using CNN	×	×
[32]	✓	Naïve Bayes, SVM	Content feature	×	×
[33]	×	Naïve Bayes	TFIDF feature extraction	×	×
[34]	×	Linguistics Approaches	Linguistic and behavioral features	×	×
[35]	×	Machine Learning	TF-IDF feature extraction	×	×
[36]	✓	Deep Learning	CNN-LSTM n-gram feature	×	×
[37]	✓	BERT	Linguistic feature	×	×
[38]	×	LSTM, BERT	Word n-gram features	×	×
[39]	×	CNN	Feature mapping using CNN	×	×
[40]	✓	Logistic Regression	TFIDF feature extraction	×	×
[41]	✓	Machine Learning	Latent semantic indexing (LSI) extracted features	×	×
Proposed Research	✓	SVM, Logistic Regression, Naïve Bayes, BERT	Policy and timestamp features	✓	✓

Key: ✓ represents that feasure is available; × represents that feasure is not available.

analyzed the research gap in the existing literature. Our proposed study, therefore, utilizes Urdu tweets of Twitter users and a proposed method utilizing policy and timestamp features. Moreover, we used different supervised classification model, which includes naïve Bayes, logistic regression, k-nearest neighbors, and long short-term memory, and finally compared their accuracy.

3. Machine Learning and Deep Learning Algorithms

Artificial intelligence (AI) includes both machine learning and deep learning. In brief, machine learning refers to AI that can autonomously adapt with minimal human intervention. Deep learning is a kind of machine learning that use artificial neural networks to simulate the human brain’s learning process. This section explains existing machine learning and deep learning algorithms, which are utilized in this study.

3.1. Multinomial Naïve Bayes

The foundation of naïve Bayes classifiers is Bayesian classification techniques. These depend on Bayes’ theorem, an equation that illustrates the relationship between the conditional probabilities of statistical values. In this scenario, we are attempting to determine the likelihood of a label based on a set of observed features. It is based on a simple (but naïve) application of the Bayesian formula for conditional probability after getting basic statistics from a specified training dataset. It is also popular for creating a baseline categorization performance, which other, more complex techniques must improve upon.

One most top rated algorithms for categorical classification is naïve Bayes. Moreover, our dataset is a binary classified algorithm that includes label 0 for spam and label 1 for non-spam. So, we can write the equation as follows:

$$\mathrm{P}(\mathrm{spam}|\mathrm{policy}\_\mathrm{feature})=\frac{\mathrm{P}\left({\mathrm{policy}}_{\mathrm{feature}}\right)\times \mathrm{P}\left(\mathrm{spam}\right)}{\mathrm{P}\left(\mathrm{policy}\_\mathrm{feature}\right)}$$

(1)

$$\mathrm{P}(\mathrm{ham}|\mathrm{policy}\_\mathrm{feature})=\frac{\mathrm{P}\left(\mathrm{policy}\_\mathrm{feature}\right)\times \mathrm{P}\left(\mathrm{ham}\right)}{\mathrm{P}\left(\mathrm{policy}\_\mathrm{feature}\right)}$$

(2)

3.2. Logistic Regression

In order to accomplish binary classification, logistic regression models a dependent variable (Y) in terms of one or more independent variables (X). In other words, it is a generalized model that predicts the chances of an event occurring. Specifically, logistic regression employs linear regression to represent the logit function. Equation (3) below represents the logistic regression cost function.

$$\mathrm{Cost}\left({\mathrm{h}}_{\mathsf{\theta }}\left(\mathrm{x}\right),\mathrm{y}\right)=\left\{\begin{array}{c}-\log ({\mathrm{h}}_{\mathsf{\theta }}\left(\mathrm{x}\right)) \mathrm{if} \mathrm{y}=1\\ -\log (1-{\mathrm{h}}_{\mathsf{\theta }}\left(\mathrm{x}\right)) \mathrm{if} \mathrm{y}=0\end{array}\right.$$

(3)

Logistic regression is another popular and highly effective algorithm for binary classification tasks. It is widely used in various domains, including spam detection and sentimental analysis.

3.3. Support Vector Machine

Support vector machine (SVM) is a supervised machine learning method applicable to classification and regression issues. However, its most frequent application is in categorization issues. Using the SVM technique, each data point is represented as a point in n-dimensional space (where n is the number of features), with the value of each feature corresponding to a certain coordinate. Consequently, we employ a support vector machine classifier in our situation. It consists of an RBF kernel with a count vectorizer, term frequency, and inverse document frequency classification process. Equation (4) represents the SVM cost function.

$$\underset{\theta }{\min } C{\displaystyle \sum }_{i=1}^m\left[y^{\left(i\right)}cos{t}_1\left({\theta }^T{x}^{\left(i\right)}\right)+\left(1-y^{\left(i\right)}\right)cos{t}_0\left({\theta }^T{x}^{\left(i\right)}\right)\right]+\frac{1}{2}{\displaystyle \sum }_{i=1}^n{\theta }_j^2$$

(4)

3.4. Long Short-Term Memory (LSTM) and Gated Recurrent Neural Network (RNN)

The incorporation of gating units permits the expansion of the family of gated recurrent neural networks (RNN) to include long short-term memory (LSTM). In particular, the system can be expanded to include three gate components. First, the forget gate can be utilized to control a direct copying or a complete state clearing. The input gate follows a similar technique to evaluate if the state should be updated in consideration of the current input signal. At each time step, the amount of data to keep from the perturbation input signal and the previous state signal is learned. In addition to learning long-term time requirements by retaining information, the system must occasionally learn to clear data from its current state.

A sequential model with layers of embedding, LSTM, dropout, and dense with ReLu and sigmoid activation functions for two successive levels of dense and dropout layers was utilized for LSTM. For LSTM Urdu sentences, we employed a tokenizer that was initialized, fitted to tweet input, and created a number sequence.

4. Methodology

This section presents the proposed methodology of the proposed approach. It describes data collection details, data pre-processing, feature extraction, architecture, and implementation details of the proposed model.

4.1. Data Collection

The dataset is extracted from Twitter using only tweets written in Urdu. The dataset is obtained via Twitter’s API. We utilized the retrieved dataset from snscrape, which provides an API for retrieving different attributes such as username, URL, tweet content, etc.

Table 3. Data statistics.

Name	Spam	Ham
Count	7319	1,213,068
Mean	0.6%	99.4%
Standard Deviation	7.72%	7.72%
Maximum	900	198
Minimum	208	1

provides the dataset statistics, e.g., collection of spam and ham tweets in terms of mean, standard deviation, and so on, to understand the distribution of the dataset.

Figure 1 represents the distributed total of tweets classified on usernames as number of tweets of spam and ham for a user. “Spam” and “ham” refer to the content of the tweets, with “spam” tweets being unwanted or unsolicited messages and “ham” tweets being legitimate or desired messages. The distribution of these tweets as data points shows how many of each type of tweet a user has sent over a certain period of time. This information can be used to help identify patterns in the user’s behavior or to classify their tweets as either spam or ham for filtering purposes.

Our proposed research has fetched data using snscraper, a Python library that provides services for fetching data of multiple features such as username, tweet content, and so on. Data using snscraper had been fetched for nearly 2 h continually. This dataset is preprocessed to remove English alphabets, characters such as punctuation, exclamation mark, and other symbols. Based on Twitter’s policy [42], it only permits 2400 tweets per day, which equates to 100 tweets each hour. As a result, we choose a threshold of 200 tweets, because we get tweets using snscraper continuously for two hours, which translates to 200 tweets per user. Using this timestamp and policy functionality, we can now determine whether or not a tweet is spam.

4.2. Preprocessing

Preprocessing is done both manually and programmatically. Numerous machine learning library functions, such as the strip function, the replace function, and tokenization for removing punctuation, numbers, and irrelevant phrases or words, were employed for text cleansing processing. During tokenization, punctuation is removed and the sentence is parsed into words, which are then reassembled into entire sentences using the join function and space, as well as stop words being deleted. Each sentence is saved after purifying or pre-processing the data and assigning labels to binary labels, such as spam label as ‘0’ or non-spam label as ‘1’. The saved dataset is then processed further during the following implementation session.

4.3. Features Extraction

This study uses the vector space model (VSM), a method for expressing words as vectors. It is a typical information retrieval technique that allows judgments to be made regarding which terms are comparable. A vocabulary is composed of every word in our corpus. Our suggested research utilizes a feature engineering of Twitter policy from which a check statement is generated to determine whether a tweet is spam or ham. Twitter now states that a user can send no more than 100 tweets per hour. Therefore, it offers us the ability to determine that a user who tweets more than 100 times is likely a spammer.

As mentioned previously, this research gathered tweets continuously for two hours using snscraper. Due to the fact that we had retrieved tweets for two hours, we made the assumption that a check statement was necessary for classifying a tweet as spam or ham. As demonstrated by the following Equation (5), where TR represents the tweet rate from a user:

$$User Tweet Rate \left(TR\right)=\frac{\#\mathrm{of}\mathrm{Tweetsx}\mathrm{Time}\mathrm{in}\mathrm{Hour}}{\#\mathrm{of}\mathrm{Users}}$$

(5)

In Equation (5), # of tweets represents the total number of tweets posted on Twitter during a specific time period, # of users represents the total number of users who posted tweets during the same time period, and time period represents the duration of the time period, usually measured in hours.

4.4. Implementation

Our research used the sklearn libraries. Support vector machine (SVM), multinomial naïve Bayes (MNB), and logistic regression (LR) algorithms are employed to increase accuracy. Algorithm 1 presents the steps to label tweets as ham/spam. As illustrated in Figure 2, we represent spam/ham tweets against the number of tweets. It displays spam/ham distinct username’s count against the number of tweets and Figure 3 spam/ham distinct label’s count against the number of tweets. Figure 4 represents the number of tweets count against distinct usernames count. Algorithm 2 represents the procedure of categorizing Urdu language spam tweets. Certainly, our input is a specific Urdu tweet without any label, and our output is either a spam or not-spam label. Now, for each tweet in our corpus of tweets obtained from Twitter using the snscraper Python module, we applied policy and timestamp features. The dataset is imbalance as shown in Figure 5. Policy and timestamp features examine whether a user has surpassed Twitter’s 200-tweet limit within two hours of the timestamp. So, based on the retrieved characteristic, a spam or non-spam label is assigned to the tweets, as illustrated in Figure 6. Following the application of feature engineering, we selected a method or estimator from naïve Bayes, logical regression, SVM, and BERT. We divided the dataset into training and testing using a method named train_test_split from sklearn with a test size of 30%. After dividing the dataset and fitting it to the algorithm or estimator for these algorithms, e.g., SVM, we used a pipeline method of sklearn library comprising TF-IDF, count vectorizer, and ML classifiers (multinomial naïve Bayes, support vector classifier, RBF, logistic regression, and BERT) after preprocessing. Now, after training each unique estimated trained model, e.g., SVM, we predict spam or non-spam classification labels on the remaining 30% testing dataset. Finally, calculated metrics accuracy, precision, recall, and f1-score using a confusion matrix by comparing actual truth and predicted values.

Algorithm 1: Labeling Ham/Spam Tweets in the Urdu Language

0 def labeler_function (row, username_count): 1 if row[username_count] > 200: 2 label = Spam 3 else: 4 label = Ham 5 return label 6  7 dataset_label = dataset.apply (lambda row: labeler_function (row, username_count))

Algorithm 2: Spam Tweets in Urdu Language

0 Input: Tweet $T_i$ 1 Output: Spam or Not-Spam 2 for each tweet $T_i$ in Tweets do 3 for Policy/Timestamp feature in Policy/Timestamp features 4 assign/label 0 to spam and 1 to non-spam 5 end for 6 end for 7 for each estimator in estimators (Naïve Bayes, Logistic Regression, SVM, BERT) 8 uniqueEstimtorArray = estimator.fit(labeled dataset using Policy/Timestamp features) 9 end for 10 if uniqueEstimtorArray.Predict ($T_i$ == 1) 11 label $T_i$ = Non-Spam 12 elseif uniqueEstimtorArray.Predict ($T_i$ == 0) 13 label $T_i$ = Spam 14 end if

4.5. Dataset Imbalance

As shown in Figure 5, our dataset contains more spam than genuine tweets sent by users. In cases where one class (such as spam) contains many more examples than the other (such as ham), a precision–recall (PR) curve might be a more useful evaluation of a classification model’s performance than metrics such as accuracy. As the threshold for determining whether a tweet is spam or ham is adjusted, the PR curve illustrates the trade-off between precision (the fraction of identified spam tweets that are genuinely spam) and recall (the fraction of actual spam tweets that are accurately classified as spam). A classifier with high accuracy for the majority class (say, ham) but low precision and recall for the minority class would be unsuitable for use on an uneven dataset (e.g., spam). We can better judge the classifier’s overall performance by looking at the PR curve, which shows how well it performs for both classes.

4.6. Architecture

Figure 6 depicts the architecture of the suggested technique. It demonstrates how the dataset is preprocessed and then machine learning is used on the data once it has been collected. During preprocessing, the dataset is cleaned of non-Urdu characters such as punctuation, numbers, and Unicode that are not part of the language. Replace to get rid of numbers, tokenize to get rid of punctuation marks, and use the strip function to get rid of extra lines and Unicode that are not part of the Urdu language. A full sentence is constructed from the available tokens after tokenization.

5. Experimental Results

This section presents the results and discussion of the implementation of the proposed methodology. We have collected 1,100,000 plus (+) unique real-time tweets from multiple users. While using multinomial naïve Bayes, we used a pipeline of TF-IDF and count vectorizer for the classification approach to overcome low accuracy; likewise, we used a pipeline of count vectorizer and TF-IDF for RBF kernel support vector machine.

Table 4. Confusion matrix for naïve Bayes.

Name	Spam	Ham
Spam	91	2114
Ham	36	363,876

,

Table 5. Confusion matrix for RBF kernel SVM.

Name	Spam	Ham
Spam	0	2205
Ham	0	363,912

,

Table 6. Confusion matrix for logistic regression.

Name	Spam	Ham
Spam	590	1615
Ham	83	363,829

and

Table 7. Confusion matrix for BERT.

Name	Spam	Ham
Spam	0	1098
Ham	0	85,883

represent the confusion matrix of naïve Bayes, SVM, logistic regression and BERT.

We employed a sequential model for BERT consisting of dropout and dense layers, using ReLu and softmax activation functions for the two layers that came after the dropout layer. We employed a BERT tokenizer that was on the Tweets data to generate a series of numbers for the Urdu sentences model. Logistic regression achieves both 99.55% accuracy and 94% precision, as seen in

Table 8. Accuracy results of the trained models.

Model/Ensemble	Library	Accuracy	F1-Measure	Recall	Precision
Multinomial Naïve Bayes, CV, TF-IDF	SK-Learn	99.40%	0.54	52.0%	86.0%
RBF Kernel SVM, CV, TF-IDF	SK-Learn	99.38%	0.5	50.0%	50.0%
Logistic Regression, CV, TF-IDF	SK-Learn	99.55%	0.7	63.0%	94.0%
BERT	Transformers	99.00%	0.5	50.0%	49.0%

.

Figure 7 shows the confusion matrix of (a) logistic regression, and (b) naïve Bayes and (c) SVM and (d) BERT with true–false and positive–negative values of spam with other ham tweets. Table 7 represents the accuracy results of training and testing of the models using existing ML and DL models. Results show that logistic regression performs best in achieving higher accuracy. In binary classification, logistic regression is superior to SVM, multinomial naïve Bayes, and BERT for a number of reasons. Several factors, including the amount and complexity of the dataset, the quality of the data, and the choice of hyperparameters, may influence the relative performance of these methods. Logistic regression is a reasonably simple algorithm, has a low variance, and is, therefore, less susceptible to overfitting, and is quick to train. This makes it suitable for scenarios in which fresh data are continuously generated and the model must be continually updated.

6. Future Research Directions

There are a number of important topics for future research in the domain of Twitter spam review identification. The following items indicate prospective research and development areas:

• Examination of online bullying and harassment: There is a need to examine the frequency, features, and impacts of online bullying and harassment on Twitter in greater detail. This entails studying patterns and trends, comprehending the motivations behind such activities, and investigating the impact on victims and the greater online community;
• Methods for detection and prevention: Methods for detecting and preventing cyberbullying and cyber-harassment must be developed. These strategies should take Twitter’s standards for identifying and responding to harassment and abusive behavior into account. It is essential to develop automated methods that can detect and indicate incidents of harassment in real-time, allowing for prompt interventions;
• Machine learning and natural language processing: To identify and classify instances of online harassment and cyberbullying on Twitter, it is necessary to construct sophisticated machine learning models and natural language processing techniques. These models should be consistent with Twitter’s guidelines regarding abusive behavior and harassment, allowing for the proper identification and classification of harmful content [43,44];
• Dataset expansion and feature extraction: To improve the robustness and generalizability of the models, it is essential to expand the existing dataset utilized for spam identification. The emphasis of research should be on gathering more diverse and representative samples of spam and non-spam tweets. Additionally, studying various feature extraction techniques can result in a more accurate representation of the distinctive qualities of spam content;
• Comparison with advanced baseline approaches: While the current study employed simple baseline approaches, future research could investigate more complicated and advanced strategies for spam identification. Comparing the performance of advanced classifiers, such as deep learning models or ensemble approaches, can shed light on their efficacy and potential for improvement over conventional classifiers;
• Efforts should be made consistently to optimize the performance of spam identification classifiers. This involves fine-tuning the models, refining the feature selection procedure, and researching creative techniques to improve spam detection’s accuracy, precision, and recall.

By pursuing this research direction, we can obtain a greater knowledge of online harassment, enhance the efficacy of detection and prevention approaches, and improve Twitter’s overall safety and user experience.

7. Conclusions

This study has compiled a large database with more than 1,100,000 different real-time tweets from different users. These tweets have been carefully collected and processed to make sure the data is clean and to extract the important features. According to Twitter’s rules, users cannot send more than 100 tweets per hour, so any tweets that go over this limit are considered spam. Predictions are made using testing data with machine learning and BERT layer models. We used the snscrape fetch dataset, which has an API for retrieving a wide variety of attributes (username, URL, tweet content, and so on). After extracting features based on Twitter policy, we train and test our dataset to conduct spam analysis. The best F1-score (0.7) and highest accuracy (99.55%) are achieved by the logistic regression. The results of this study highlight the effectiveness of policy-based spam detection in Urdu tweets using ML and BERT layer models. The findings contribute to the development of robust spam detection techniques specifically tailored for the Urdu language on social media platforms. Future studies will focus on Word2Vec word embedding, one Hot Embedding, and FastText to increase performance and reduce bias. Convolutional neural networks (CNN) and bilingual BERT can be utilized in future studies to improve accuracy.