1. Introduction
Social media has rapidly transformed communication patterns across the Arab world, reshaping how people express emotions, share opinions, and engage in events [
1]. Among these platforms, Twitter stands out, with over 27 million daily tweets, offering users a space for interaction but also exposing them to cyber violence, including cyberbullying and online hate speech [
2,
3]. Despite the existence of regional and international legislative frameworks, violent digital content continues to rise in Saudi Arabia, where social media is widely used for both positive and harmful exchanges [
4]. Prior research emphasizes the urgent need for effective online mechanisms to mitigate abusive behaviors and protect vulnerable users [
1,
3]. Studies confirm that toxic content has severe psychological and social effects and is closely linked to hate crimes and rights violations [
5]. Therefore, developing automated systems to detect and limit such behavior is critical to safeguarding society and supporting national digital safety initiatives [
6].
In this study, cyber violence refers to the intentional use of digital technologies to cause psychological, social, or financial harm to individuals, organizations, or communities. It encompasses two primary categories [
7]:
- •
Cyberbullying: Involves harassment, threats, defamation, or ridicule carried out through online or mobile communication channels. It particularly affects youth, leading to severe emotional and psychological distress [
8].
- •
Online Hate Speech: Refers to derogatory or discriminatory language directed at individuals or groups based on characteristics such as nationality, gender, religion, social class, or region [
9]. In the context of this study, regional hate speech specifically addresses intra-Saudi variations across regions such as Najdi, Hijazi, and Qassimi.
The growth of cyber violence emerges from multiple social elements that interact with technological systems. People can create harmful content through internet anonymity, as they face no accountability, and social media platforms create echo chambers that strengthen extremist beliefs, while their algorithms boost sensational content for increased views [
3]. The current digital environment of Arabic social media platforms experiences increased digital aggression because of these social and technological elements.
The Arabic language presents special linguistic challenges because it exists in three forms: Classical Arabic (CA), Modern Standard Arabic (MSA), and Arabic dialects (AD) [
1]. Dialects play a significant role in informal communication, particularly on social media platforms like Twitter. Saudi Arabia ranks tenth in the world in terms of Twitter interaction and leads Arabic-speaking countries with 15.67 million active users [
10]. This has been confirmed by a study conducted by Mubarak and Darwish, which shows that most dialect-classified Twitter content is in Saudi dialects, demonstrating the need for dialect-based resources [
11]. Saudi Vision 2030 [
12] emphasizes digital transformation and improving quality of life, which underscores the importance of maintaining a trusted and safe digital environment for communication [
13].
Although notable progress has been made in English cyber violence detection, Arabic language resources remain limited [
14]. Moreover, much of the existing work still frames the task as binary classification (e.g., hate vs. non-hate) [
15] and often focuses on narrowly defined hate domains, such as religion- or gender-based hate speech. In addition, many publicly available corpora suffer from severe class imbalance, where a few dominant categories overshadow rarer forms of cyber violence content; this can bias learning and limit generalizability across contexts [
15,
16]. The problem is further complicated by the frequent use of emojis, hashtags, and coded expressions that mask hostility or shift meaning [
3]. At the operational level, platforms that rely heavily on manual moderation face clear scalability constraints in high-volume environments [
14]. Collectively, these limitations underscore the need for a balanced, multi-dimensional Saudi dialect corpus that captures both cyberbullying and diverse hate-speech categories, enabling more robust, fair, and reliable model performance across real-world Saudi social media settings.
This study aims to address this challenge by developing a comprehensive Arabic corpus for cyber violence detection, with a particular emphasis on Saudi dialects. The corpus is designed to support the development of advanced AI models for digital safety, enabling proactive monitoring, early warning of cyberbullying and online hate speech trends, and more effective protection of individuals and communities in online spaces. Accordingly, the study demonstrates both technical feasibility and a scalable strategic foundation for national-level digital protection.
This study makes the following key contributions to Arabic cyber violence research:
- •
Digital Security Enhancement: We introduce the first large-scale Saudi-dialect corpus for cyber violence detection (88,687 tweets), enabling the development and benchmarking of effective models for identifying cyber violence in Saudi social media, and supporting decision-makers with evidence-driven insights to strengthen online safety and moderation.
- •
Hierarchical Annotation: Introduces a multi-level framework distinguishing between benign, cyberbullying, and hate speech tweets, with detailed hate categories.
- •
Balanced Dataset Design: Ensures category parity to mitigate imbalance found in prior corpora.
- •
Comprehensive Model Evaluation: We systematically evaluate three major NLP approaches (machine learning-, deep learning-, and transformer-based models) and demonstrate their comparative performance on Saudi dialectal content.
- •
Exploratory Data Analysis (EDA): We provide a comprehensive statistical characterization of the SD-CVD corpus, reporting label distributions, subtype frequencies, and word- and character-level length profiles. This analysis offers actionable insights that support corpus reuse, reproducibility, and future benchmarking.
- •
Complementary Features: Incorporates sentiment and emoji-derived features to improve model performance
The remainder of this paper is structured as follows.
Section 2 reviews related work on Arabic sentiment analysis and cyber violence detection.
Section 3 presents the adopted methodology, including data collection, preprocessing, annotation framework, exploratory data analysis, and corpus optimization.
Section 4 outlines the experimental setup and evaluation metrics.
Section 5 presents and discusses results, and
Section 6 concludes with recommendations for future work.
2. Related Work
Research on Arabic abusive and hateful content has progressed along two main lines: resource creation and model development, with an increasing focus on the Saudi dialect. Early sentiment corpora such as AraSenTi-Tweet (17,573 Saudi tweets) [
11] and AraCust (20,000 telecom tweets) [
17] demonstrated the importance of domain-specific corpora and highlighted the overlap between sentiment and offensive expressions.
A shift toward explicit offensiveness emerged with Alshalan and Al-Khalifa (2020), who introduced a Saudi hate speech corpus (9316 tweets), where CNN achieved F1 = 0.79 [
18]. Similarly, Mubarak et al. (2023) collected large-scale emoji-anchored data covering vulgarity, violence, and hate, demonstrating how transformer-based models capture cultural nuances [
3]. Multi-label approaches like Zaghouani et al. (2024) expanded scope through So Hateful!, a dataset annotated for hate, offensiveness, humor, and spam, with AraBERT outperforming other models [
19].
Subsequent work introduced broader Arabic corpora. Ahmad et al. (2024) compiled a 403,688-tweet Jordanian dataset and compared TF-IDF, Word2Vec, and AraBERT representations, evaluating their performance across seven classifiers [
5]. Charfi et al. (2024) advanced this work with the balanced, multi-dialect ADHAR corpus, addressing race, religion, and ethnicity [
15]. Within Saudi Arabia, Asiri and Saleh (2024) introduced SOD, the most comprehensive Saudi corpus (24,000 tweets), featuring hierarchical labels, with data augmentation performed best, reaching and achieving F1 ≈ 0.91 using AraBERTv0.2-Twitter [
1].
Alghamdi et al. (2021) proposed a two-level machine-learning framework for detecting violent content in Saudi Arabic tweets [
20]. The first stage classifies tweets as violent or non-violent, and the second stage further distinguishes violent tweets as either cyberbullying or threatening. Using SVM and Naive Bayes with different feature extraction and preprocessing settings, SVM achieved stronger results, reporting an F1-score of 76.06% for Level 1 and 89.18% for Level 2 [
20], while newer transformer models, such as Arabic BERT-Mini [
21] and SaudiBERT [
22], achieved high performance with reduced computational cost. Studies integrating sentiment and emoji features (e.g., Refs. [
23,
24]) or evolutionary optimization [
6] demonstrated performance improvements, with surveys confirming that data balance and label design remain critical [
25]. Complementary emotion-aware modeling that leverages affective cues has also improved hate-speech detection in multi-target settings [
26].
Fine-grained multi-label detection gained attention through works like Alghamdi et al. (2021) (two-level violent tweet classifier, F1 ≈ 0.89) [
20] and Al Anezi (2022), who introduced a seven-class DRNN model achieving 84% accuracy [
27]. Collectively, these studies confirm that dialect-specific, balanced corpora, and hybrid architectures are vital for improving Arabic cyber violence and hate speech detection.
Table 1 provides an overview of representative Arabic and Saudi-dialect corpora for sentiment and hate-speech detection. For each dataset, we report the main task, best-model performance, and documented limitations, enabling a concise comparison of coverage and research gaps.
The reviewed studies directly informed the design choices and research directions adopted in this work. While earlier efforts contributed valuable resources and modeling insights, a careful analysis of the related literature revealed several recurring limitations. Saudi dialect–specific resources remain comparatively limited [
1,
18], particularly for cyber violence and fine-grained harmful-content modeling. Moreover, many prior datasets and benchmarks continue to rely on binary classification, which can yield unreliable predictions for complex cyber violence phenomena [
26], and on highly imbalanced label distributions that bias learning toward majority classes and reduce generalizability [
15]. These gaps motivated the development of SD-CVD: a large-scale, balanced Saudi-dialect corpus with hierarchical annotations targeting cyberbullying and fine-grained hate speech subtypes across multiple Saudi dialects.
Prior research has also shown that linear models with explicit lexical features can remain highly competitive for text classification tasks [
17], which guided our inclusion of strong TF–IDF-based baselines. At the same time, recent advances in Arabic transformer models motivated the incorporation of BERT-based benchmarks to ensure alignment with contemporary NLP practices. Furthermore, our annotation guidelines, error analysis, and evaluation protocol were shaped by observations reported in earlier studies regarding annotation ambiguity, dialectal bias, and sentiment overlap. Overall, the proposed corpus and experimental framework build on existing empirical findings while explicitly addressing open challenges identified in the literature.
5. Discussion
Arabic’s rich morphology and wide dialect variation make cyber violence detection especially difficult in informal tweets. The SD-CVD corpus addresses key gaps in prior Arabic resources by focusing on multiple Saudi dialects, enabling more robust and culturally informed detection. Its balanced distribution across eleven classes supports fair training and reliable evaluation, reducing bias caused by class imbalance. In addition, the corpus follows a rigorous collection and annotation process with multiple native Arabic annotators and reported inter-annotator agreement, strengthening the reliability and overall quality of the dataset.
Extensive experiments across ML-, DL-, and transformer-based models consistently confirm that SD-CVD provides an effective benchmark for accurate cyber violence classification. The results confirm the advantage of BERT-based models for Saudi dialect cyber violence detection, where AraBERTv02-Twitter achieved the top weighted F1-score (0.882), followed by CAMeLBERT-MSA (0.869). The gap is expected given AraBERTv02-Twitter’s closer pretraining domain (Arabic Twitter), which better reflects informal spelling, dialect markers, and emoji-heavy discourse typical of SD-CVD. These findings reinforce prior evidence that domain- and dialect-matched pretraining is crucial for robust Arabic harmful-content detection [
1,
3].
These outcomes are consistent with the findings of Ahmad et al. [
5], who reported that SVM and LR achieved competitive results on a large-scale Jordanian dialect corpus, while fine-tuned transformer models provided superior contextual understanding when sufficient training data were available. Similarly to their study, our results confirm that carefully engineered textual features remain highly effective as baselines, but pretrained transformers provide additional gains when handling fine-grained and context-dependent cyber violence categories.
In contrast, Anezi [
27] proposed a deep recurrent neural network and reported high accuracy on smaller, balanced datasets. However, as observed in our experiments, deep learning architectures based on fixed sentence embeddings did not surpass either SVM or BERT-based models, particularly when applied to a large, multi-dialect corpus with subtle interclass boundaries. This suggests that deep architectures may require either end-to-end contextual learning or significantly richer supervision to outperform linear and transformer-based approaches.
Similarly, Mubarak et al. [
3] demonstrated that transformer-based models such as AraBERT outperform classical classifiers for Arabic offensive and hate speech detection, especially when emojis and dialectal cues are involved. Our findings reinforce this observation, as the best results were achieved by AraBERTv02-Twitter, which is explicitly pretrained on Arabic Twitter data and dialectal content.
Overall, the comparative analysis confirms that transformer-based models represent the state of the art for Saudi dialect cyber violence detection, while traditional ML models such as SVM remain strong, interpretable, and computationally efficient baselines. Deep learning models without contextual pretraining showed comparatively weaker performance, supporting previous conclusions that dataset scale, representation choice, and annotation granularity critically influence model effectiveness [
3,
27].
Beyond the supervised models evaluated in this study, recent work on Large Language Models (LLMs) has introduced alternative directions for social media moderation that go beyond traditional supervised classifiers. In contrast to task-specific fine-tuned models, LLM-based approaches often rely on instruction prompting, reasoning mechanisms, or logic-augmented inference to detect cyber violence content. One representative example is FOLAR and related reasoning aware co-design frameworks, in which LLM representations are combined with logical constraints to achieve more robust and consistent stance detection and harmful-content detection on social media platforms [
41]. Similarly, LogiMDF, proposed by [
42], integrates LLM predictions through logical multi-decision fusion, resulting in improved moderation reliability in complex online discussions and a reduction in spurious predictions.
Although recent studies have explored Large Language Models (LLMs) as an alternative direction for social media moderation, their direct application to Arabic dialectal cyber violence detection remains challenging and was therefore not evaluated in this study. Despite their strong generalization capabilities, LLMs often suffer from dialectal and cultural bias, as most are predominantly trained on English and Modern Standard Arabic, leading to reduced sensitivity to Saudi dialects, idiomatic expressions, and culturally grounded insults [
43]. In addition, hallucination and over-generalization can introduce false positives, which is particularly problematic for benign or sarcastic content and for fine-grained cyber-violence detection tasks [
44]. Furthermore, deploying LLMs for large-scale monitoring of user-generated content raises ethical and privacy concerns, including unintended disclosure of sensitive information and the reinforcement of social biases [
45].
Author Contributions
Conceptualization, S.E.; methodology, S.E. and S.B.; software, A.A.; validation, A.A.; formal analysis, A.A.; investigation, A.A.; resources, A.A.; data curation, A.A.; writing—original draft, A.A.; writing—review and editing, S.E. and S.B.; visualization, A.A.; supervision, S.E.; project administration, S.E. All authors have read and agreed to the published version of the manuscript.
Funding
This research was funded by the General Authority for Defense Development (GADD) under Project No. 262.
Institutional Review Board Statement
In this study, all collected and processed data were fully public and did not contain any personal identifiers that could reveal or trace the identity of any individual. Therefore, in accordance with the Saudi Personal Data Protection Law (PDPL), prior ethical approval or IRB clearance was not required.
Informed Consent Statement
In this study, all collected and processed data were fully public and did not contain any personal identifiers that could reveal or trace the identity of any individual. Therefore, in accordance with the Saudi Personal Data Protection Law (PDPL), informed consent was not required.
Data Availability Statement
The dataset used in this study consists of Arabic social media posts that may contain offensive or harmful language. Due to ethical restrictions, the dataset cannot be made publicly available. However, it can be obtained from the corresponding author upon reasonable request.
Acknowledgments
This research was supported by the General Authority for Defense Development (GADD), under Pro-ject No. 262. The authors gratefully acknowledges this financial support, which covered the full re-search and development costs of the SD-CVD project.
Conflicts of Interest
The authors declare no conflict of interest.
References
- Asiri, A.; Saleh, M. Sod: A corpus for saudi offensive language detection classification. Computers 2024, 13, 211. [Google Scholar] [CrossRef]
- Kaur, S.; Singh, S.; Kaushal, S. Deep learning-based approaches for abusive content detection and classification for multi-class online user-generated data. Int. J. Cogn. Comput. Eng. 2024, 5, 104–122. [Google Scholar] [CrossRef]
- Mubarak, H.; Hassan, S.; Chowdhury, S.A. Emojis as anchors to detect arabic offensive language and hate speech. Nat. Lang. Eng. 2023, 29, 1436–1457. [Google Scholar] [CrossRef]
- Mousa, A.; Shahin, I.; Nassif, A.B.; Elnagar, A. Detection of arabic offensive language in social media using machine learning models. Intell. Syst. Appl. 2024, 22, 200376. [Google Scholar] [CrossRef]
- Ahmad, A.; Azzeh, M.; Alnagi, E.; Al-Haija, Q.A.; Halabi, D.; Aref, A.; AbuHour, Y. Hate speech detection in the arabic language: Corpus design, construction, and evaluation. Front. Artif. Intell. 2024, 7, 1345445. [Google Scholar] [CrossRef] [PubMed]
- Shannaq, F.; Hammo, B.; Faris, H.; Castillo-Valdivieso, P.A. Offensive language detection in arabic social networks using evolutionary-based classifiers learned from fine-tuned embeddings. IEEE Access 2022, 10, 75018–75039. [Google Scholar] [CrossRef]
- Council of Europe. Cyberviolence. Council of Europe Website, 2025. Available online: https://www.coe.int/en/web/cyberviolence (accessed on 18 December 2025).
- Ministry of Health. Available online: https://www.moh.gov.sa/HealthAwareness/EducationalContent/BabyHealth/Pages/Bullying.aspx (accessed on 25 March 2025).
- KAICIID. Stop Hate Speech. 2019. Available online: https://www.kaiciid.org/resources/publications/quick-guide-hate-speech-prevention (accessed on 3 November 2025).
- Statista. Number of Active Twitter Users in Selected Countries. Available online: https://www.statista.com/statistics/242606/number-of-active-twitter-users-in-selected-countries/ (accessed on 17 October 2025).
- Al-Twairesh, N.; Al-Khalifa, H.; Al-Salman, A.; Al-Ohali, Y. Arasenti-tweet: A corpus for arabic sentiment analysis of saudi tweets. Procedia Comput. Sci. 2017, 117, 63–72. [Google Scholar] [CrossRef]
- Saudi Vision 2030. Available online: https://www.vision2030.gov.sa/en (accessed on 19 December 2025).
- Dhiaa, M.; Atta, R.; Mohammed Abbas, A.; Kamel, M.; Mohammed, M.; Ali, H.; Ali, J.; Fayez, G.; Issa, M.; Dalal, A.; et al. A machine learning approach to cyberbullying detection in arabic tweets. Comput. Mater. Contin. 2024, 80, 1033. [Google Scholar] [CrossRef]
- Duwairi, R.; Hayajneh, A.; Quwaider, M. A deep learning framework for automatic detection of hate speech embedded in arabic tweets. Arab. J. Sci. Eng. 2021, 46, 4001–4014. [Google Scholar] [CrossRef]
- Charfi, A.; Besghaier, M.; Akasheh, R.; Atalla, A.; Zaghouani, W. Hate speech detection with adhar: A multi-dialectal hate speech corpus in arabic. Front. Artif. Intell. 2024, 7, 1391472. [Google Scholar] [CrossRef] [PubMed]
- Alzaqebah, M.; Jaradat, G.M.; Nassan, D.; Alnasser, R.; Alsmadi, M.K.; Almarashdeh, I.; Jawarneh, S.; Alwohaibi, M.; Al-Mulla, N.A.; Alshehab, N.; et al. Cyberbullying detection framework for short and imbalanced arabic datasets. J. King Saud Univ.-Comput. Inf. Sci. 2023, 35, 101652. [Google Scholar] [CrossRef]
- Almuqren, L.; Cristea, A. Aracust: A saudi telecom tweets corpus for sentiment analysis. PeerJ Comput. Sci. 2021, 7, e510. [Google Scholar] [CrossRef]
- Alshaalan, R.; Al-Khalifa, H. Hate Speech Detection in Saudi Twittersphere: A Deep Learning Approach. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, Barcelona, Spain, 12 December 2020; pp. 12–23. [Google Scholar]
- Zaghouani, W.; Mubarak, H.; Biswas, M.R. So hateful! building a multi-label hate speech annotated arabic dataset. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), Torino, Italy, 20–25 May 2024; pp. 15044–15055. [Google Scholar]
- Alghamdi, D.; Al-Motery, R.; Alma’Abdi, R.; Alzamzami, O.; Babour, A. Automatic detection of cyberbullying and threatening in saudi tweets using machine learning. Int. J. Adv. Appl. Sci. 2021, 8, 17–25. [Google Scholar] [CrossRef]
- Almaliki, M.; Almars, A.M.; Gad, I.; Atlam, E.-S. Abmm: Arabic bert-mini model for hate-speech detection on social media. Electronics 2023, 12, 1048. [Google Scholar] [CrossRef]
- Qarah, F. Saudibert: A large language model pretrained on saudi dialect corpora. arXiv 2024, arXiv:2405.06239. [Google Scholar] [CrossRef]
- Alfreihat, M.; Almousa, O.S.; Tashtoush, Y.; Al-Sobeh, A.; Mansour, K.; Migdady, H. Emo-sl framework: Emoji sentiment lexicon using text-based features and machine learning for sentiment analysis. IEEE Access 2024, 12, 81793–81812. [Google Scholar] [CrossRef]
- Althobaiti, M.J. Bert-based approach to arabic hate speech and offensive language detection in twitter: Exploiting emojis and sentiment analysis. Int. J. Adv. Comput. Sci. Appl. 2022, 13, 972–980. [Google Scholar] [CrossRef]
- Subramanian, M.; Sathiskumar, V.E.; Deepalakshmi, G.; Cho, J.; Manikandan, G. A survey on hate speech detection and sentiment analysis using machine learning and deep learning models. Alex. Eng. J. 2023, 80, 110–121. [Google Scholar] [CrossRef]
- Chiril, P.; Pamungkas, E.W.; Benamara, F.; Moriceau, V.; Patti, V. Emotionally informed hate speech detection: A multi-target perspective. Cogn. Comput. 2022, 14, 322–352. [Google Scholar] [CrossRef]
- Al Anezi, F.Y. Arabic hate speech detection using deep recurrent neural networks. Appl. Sci. 2022, 12, 6010. [Google Scholar] [CrossRef]
- Aldjanabi, W.; Dahou, A.; Al-Qaness, M.A.A.; Elaziz, M.A.; Helmi, A.M.; Damaševičius, R. Arabic offensive and hate speech detection using a cross-corpora multi-task learning model. Informatics 2021, 8, 69. [Google Scholar] [CrossRef]
- Plaza-Del-Arco, F.M.; Molina-González, M.D.; Ureña-López, L.A.; Martín-Valdivia, M.T. A multi-task learning approach to hate speech detection leveraging sentiment analysis. IEEE Access 2021, 9, 112478–112489. [Google Scholar] [CrossRef]
- Hashmi, E.; Yayilgan, S.Y.; Hameed, I.A.; Yamin, M.M.; Ullah, M.; Abomhara, M. Enhancing multilingual hate speech detection: From language-specific insights to cross-linguistic integration. IEEE Access 2024, 12, 121507–121537. [Google Scholar] [CrossRef]
- Alhazmi, A.; Mahmud, R.; Idris, N.; Abo, M.E.M.; Eke, C. A systematic literature review of hate speech identification on Arabic Twitter data: Research challenges and future directions. PeerJ Comput. Sci. 2024, 10, e1966. [Google Scholar] [CrossRef]
- Moons, F.; Vandervieren, E. Measuring agreement among several raters classifying subjects into one or more (hierarchical) categories: A generalization of fleiss’ kappa. Behav. Res. Methods 2025, 57, 287. [Google Scholar] [CrossRef]
- Mubarak, H.; Rashed, A.; Darwish, K.; Samih, Y.; Abdelali, A. Arabic offensive language on twitter: Analysis and experiments. arXiv 2020, arXiv:2004.02192. [Google Scholar]
- Alrashidi, B.; Jamal, A.; Alkhathlan, A. Abusive content detection in arabic tweets using multi-task learning and transformer-based models. Appl. Sci. 2023, 13, 5825. [Google Scholar] [CrossRef]
- Cohen, S.; Presil, D.; Katz, O.; Arbili, O.; Messica, S.; Rokach, L. Enhancing social network hate detection using back translation and gpt-3 augmentations during training and test-time. Inf. Fusion 2023, 99, 101887. [Google Scholar] [CrossRef]
- Jovel, J.; Greiner, R. An introduction to machine learning approaches for biomedical research. Front. Med. 2021, 8, 771607. [Google Scholar] [CrossRef] [PubMed]
- Nahar, K.M.O.; Alauthman, M.; Yonbawi, S.; Almomani, A. Cyberbullying detection and recognition with type determination based on machine learning. Comput. Mater. Contin. 2023, 75, 5308–5319. [Google Scholar] [CrossRef]
- El Koshiry, A.M.; Eliwa, E.H.I.; El-Hafeez, T.A.; Omar, A. Arabic toxic tweet classification: Leveraging the AraBERT model. Big Data Cogn. Comput. 2023, 7, 170. [Google Scholar] [CrossRef]
- Qarah, F. EgyBERT: A large language model pretrained on Egyptian dialect corpora. arXiv 2024, arXiv:2408.03524. [Google Scholar] [CrossRef]
- Inoue, G.; Alhafni, B.; Baimukan, N.; Bouamor, H.; Habash, N. The interplay of variant, size, and task type in Arabic pre-trained language models. arXiv 2021, arXiv:2103.06678. [Google Scholar] [CrossRef]
- Dai, G.; Liao, J.; Zhao, S.; Fu, X.; Peng, X.; Huang, H.; Zhang, B. Large language model enhanced logic tensor network for stance detection. Neural Netw. 2024, 183, 106956. [Google Scholar] [CrossRef] [PubMed]
- Zhang, B.; Ma, J.; Fu, X.; Dai, G. Logic-augmented multi-decision fusion framework for stance detection on social media. Inf. Fusion 2025, 122, 103214. [Google Scholar] [CrossRef]
- Naous, T.; Ryan, M.; Xu, W.; Ritter, A.; Van Durme, B. Having Beer After Prayer? Measuring Cultural Bias in Large Language Models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), Bangkok, Thailand, 11–16 August 2024. [Google Scholar]
- Huang, T. Content moderation by large language models: From accuracy to legitimacy. Artif. Intell. Rev. 2025, 58, 320. [Google Scholar] [CrossRef]
- Gallegos, I.O.; Rossi, R.A.; Barrow, J.; Tanjim, M.M.; Kim, S.; Dernoncourt, F.; Yu, T.; Zhang, R.; Ahmed, N.K. Bias and fairness in large language models: A survey. Comput. Linguist. 2024, 50, 1098–1179. [Google Scholar] [CrossRef]
Figure 1.
Three-level hierarchical annotation framework: Level 1 distinguishes Cyber Violent from Benign tweets; Level 2 labels Cyber Violent tweets as Cyberbullying or Online Hate Speech and Benign tweets as Positive, Negative, or Neutral; and Level 3 further decomposes Online Hate Speech into seven fine-grained discrimination classes.
Figure 2.
Annotation team selection process.
Figure 3.
Screenshots of the tweet annotation interface used to label the SD-CVD corpus. The interface is presented in Arabic; the heading translates to “Tweet Classification System.” The screenshots illustrate the workflow: (right) annotator login, (left) displaying a tweet and indicating whether it contains cyber-violence, and (middle) assigning the primary and secondary class labels, with options to save the annotation (green), undetermined classification (orange), or delete the tweet (red).
Figure 4.
Class distribution in the SD-CVD corpus.
Figure 5.
Confusion matrix of the Support Vector Machine (SVM) model.
Figure 6.
Confusion matrix of the Multilayer Perceptron (MLP)model.
Figure 7.
Confusion matrix of the AraBERTv02-Twitter model.
Figure 8.
Comparative F1-score performance of machine learning (ML)-, deep learning (DL)-, and BERT-based models.
Table 1.
Summary of Saudi dialect dataset studies.
| Citation | Dataset Size | Main Task | Model Performance | Limitation |
|---|
| [1] Asiri and Saleh (2024) | 24,000 Saudi tweets | Offensive and hate speech detection | AraBERT with data augmentation achieved up to 0.91 F1-score (binary) and 0.83 (multiclass). | Only AraBERTv0.2-Twitter was used; no comparison with other transformer-based models. |
| [19] Zaghouani et al. (2024) | 15,965 tweets | Arabic text classification tasks (sentiment, emotion, offensive language, hate target/type identification) | AraBERT outperformed classical baselines with an F1-score of 0.66. | Annotator/regional bias, subjectivity in implicit hate, under-represented dialects, and class imbalance. |
| [5] Ahmad et al. (2024) | 403,688 tweets | Multiclass hate detection | Fine-tuned AraBERT-based models reached an aggregate F1-score of approximately 0.60. | Focused on Jordanian Arabic; labels are relatively coarse-grained and sentiment based. |
| [18] Alshalan and Al-Khalifa (2020) | 9316 Saudi tweets | Hate speech detection | CNN achieved the best performance with an F1-score of 0.79. | Binary classification limits fine-grained hate categorization. |
| [3] Mubarak et al. (2023) | 12,698 annotated Arabic tweets | Offensive language and hate speech detection | QARiB achieved F1 ≈ 82.3 for offensive language detection; AraBERT achieved F1 ≈ 80.1 for hate speech detection. | Emoji-based sampling may over-represent certain offensive styles while under-representing others, reducing dataset representativeness. |
| [20] Alghamdi et al. (2021) | 2000 Saudi tweets | Cyberbullying and threat detection | SVM achieved 76.06% F1 at Level 1 (violent vs. non-violent) and 89.18% F1 at Level 2 (cyberbullying vs. threats). | Small dataset; keyword-based collection may bias toward explicit violence; no comparison with transformer-based architectures. |
| [11] Al-Twairesh et al. (2017) | 17,573 Saudi tweets | Arabic sentiment analysis | SVM with term-based features achieved F1: 62.27% (two-way), 58.17% (three-way), and 54.69% (four-way). | Moderate inter-annotator agreement (κ = 0.60); performance drops in multiclass settings; limited robustness for implicit/ambiguous sentiment. |
| [17] Almuqren and Cristea (2021) | 20,000 telecom tweets | Arabic sentiment analysis (telecom customer satisfaction) | SVM achieved 91% accuracy. | Binary sentiment (positive/negative) limits granularity, moderate agreement (κ ≈ 0.60), and domain-specific (telecom) may reduce generalizability. |
| [27] Al Anezi (2022) | 4203 comments | Arabic hate speech detection (multiclass) | Deep RNN achieved 99.73% accuracy (binary), 95.38% (three-class), and 84.14% (seven-class). | Small dataset for deep learning; potential overfitting; and lacks comparison with transformer-based models. |
| [15] Charfi et al. (2024) | 4240 tweets | Arabic hate speech detection | AraBERT reached 94% accuracy/F1 for hate vs. non-hate and 95% accuracy/F1 for hate category classification. | Relatively small/static dataset; limited cross-dataset generalization; evolving hate expressions. |
Table 2.
Examples of lexicon components for data collection.
| Lexicons | List | Translation |
|---|
| Keywords | غبي، فاشل، جاهل، تفو عليك، حمير، همجي بقر، عالة على المجتمع، وجهك يسد النفس، خنزير، مغفل، سافل، مجانين، حقودين، ثور، نجس، عيال الشوارع، خنازير، كلب، يلعن، بلا كرامة | Stupid, loser, ignorant, shame on you, donkeys, savage, cows, burden on society, your face is repulsive, pig, fool, despicable, lunatics, spiteful, bull, filthy, street kids, pigs, dog, curse, dishonorable. |
| Emojis | 🦍 👊 💩 🔪 💣 🤮 👟 💀 🐸 🐶 🐻 ❄️ 🐷 🙈 🦉 🐖 🐭 | Gorilla, fist/punch, pile of poo, kitchen knife, bomb, face vomiting, sneaker, skull, frog, dog face, polar bear, pig face, see-no-evil monkey, owl, pig, mouse. |
| Hashtags | # النساء_ناقصة_عقل ، #فاترحل_بنغلادش ، #الموت_ لإسرائيل ، # بنت_شوارع ، #مشاهير_الفلس ، #احرقوهم ، #سجن_صيدنايا | #Women_Are_Lacking_Mind, #Bangladesh_Go_Away, #Death_To_Israel, #Street_Girl, #Bankrupt_Celebrities, #Burn_Them, #Saydnaya_Prison |
Table 3.
Definition of cyber violence categories and subcategories.
| Categories | Subcategories | Definitions |
|---|
| Cyber Violence | Cyberbullying | Harassment, threats, or defamation through online or mobile platforms, exposing individuals—particularly youth—to digital violence [8]. |
| Cyber Violence | Online Hate Speech | Discriminatory or derogatory language directed at individuals or groups based on characteristics such as nationality, gender, religion, or region [9]. |
| Online Hate Speech | Incitement to Violence | Posts legitimizing or calling for violence or threats [21,25,27]. |
| Online Hate Speech | Gender Discrimination | Derogatory remarks reinforcing gender stereotypes or bias [31]. |
| Online Hate Speech | National Discrimination | Ridiculing or targeting others based on nationality or cultural prejudice [2,24]. |
| Online Hate Speech | Social Class Discrimination | Verbal attacks or mockery of individuals based on economic status [4]. |
| Online Hate Speech | Tribal Discrimination | Offensive remarks toward individuals or groups from specific tribes. |
| Online Hate Speech | Religious Discrimination | Hostile comments elevating or demeaning individuals based on religion [6,19,30]. |
| Online Hate Speech | Regional Discrimination | Discriminatory language directed toward regional identities in Saudi Arabia such as Najdi, Hijazi, Eastern, or Qassimi. |
Table 4.
Exploratory data analysis for the SD-CVD Corpus.
| Statistic | Benign | Cyber Violence |
|---|
| Negative | Neutral | Positive | Cyber Bullying | Gender | Incitement to Violence | National | Regional | Religious | Social Class | Tribal |
|---|
| Count | 5903 | 7061 | 8045 | 7042 | 7219 | 7255 | 5585 | 4255 | 5559 | 4415 | 4095 |
| Word length |
| Mean | 20.55 | 22.46 | 16.06 | 18.28 | 23.63 | 13.36 | 20.45 | 17.28 | 20.41 | 20.50 | 15.02 |
| Median | 16 | 18 | 12 | 14 | 19 | 10 | 17 | 17 | 17 | 20 | 15 |
| Min | 3 | 3 | 2 | 3 | 3 | 2 | 3 | 3 | 3 | 3 | 3 |
| Max | 91 | 322 | 299 | 128 | 184 | 116 | 128 | 64 | 151 | 87 | 76 |
| Character length |
| Mean | 112.55 | 129.24 | 98.24 | 102.76 | 134.19 | 80.02 | 119.98 | 93.58 | 115.20 | 113.14 | 80.11 |
| Median | 89 | 106 | 71 | 77 | 109 | 59 | 95 | 92 | 94 | 107 | 77 |
| Min | 12 | 9 | 9 | 7 | 11 | 8 | 11 | 14 | 11 | 10 | 13 |
| Max | 475 | 1964 | 1604 | 856 | 1022 | 1338 | 810 | 376 | 919 | 505 | 705 |
Table 6.
Weighted performance metrics for SVM.
| Algorithm | Accuracy | Precision | Recall | F1-Score |
|---|
| SVM | 0.854 | 0.852 | 0.854 | 0.853 |
Table 7.
Weighted performance metrics for LR.
| Algorithm | Accuracy | Precision | Recall | F1-Score |
|---|
| LR | 0.850 | 0.848 | 0.850 | 0.849 |
Table 8.
Weighted performance metrics for RF.
| Algorithm | Accuracy | Precision | Recall | F1-Score |
|---|
| RF | 0.769 | 0.768 | 0.769 | 0.767 |
Table 9.
Weighted performance metrics for LSTM.
| Algorithm | Accuracy | Precision | Recall | F1-Score |
|---|
| LSTM | 0.813 | 0.813 | LSTM | 0.813 |
Table 10.
Weighted performance metrics for RNN.
| Algorithm | Accuracy | Precision | Recall | F1-Score |
|---|
| RNN | 0.812 | 0.814 | 0.812 | 0.812 |
Table 11.
Weighted performance metrics for MLP.
| Algorithm | Accuracy | Precision | Recall | F1-Score |
|---|
| MLP | 0.831 | 0.830 | 0.831 | 0.830 |
Table 12.
Weighted performance metrics for CNN.
| Algorithm | Accuracy | Precision | Recall | F1-Score |
|---|
| CNN | 0.571 | 0.560 | 0.571 | 0.561 |
Table 13.
Weighted performance metrics for AraBERTv02-Twitter.
| Algorithm | Accuracy | Precision | Recall | F1-Score |
|---|
| AraBERTv02-Twitter | 0.883 | 0.883 | 0.883 | 0.882 |
Table 14.
Weighted performance metrics for CAMeLBERT-MSA.
| Algorithm | Accuracy | Precision | Recall | F1-Score |
|---|
| CAMeLBERT-MSA | 0.871 | 0.869 | 0.871 | 0.869 |
| Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |