Combining the Strengths of LLMs and Persuasive Technology to Combat Cyberhate

Abstract

Cyberhate presents a multifaceted, context-sensitive challenge that existing detection methods often struggle to tackle effectively. Large language models (LLMs) exhibit considerable potential for improving cyberhate detection due to their advanced contextual understanding. However, detection alone is insufficient; it is crucial for software to also promote healthier user behaviors and empower individuals to actively confront the spread of cyberhate. This study investigates whether integrating large language models (LLMs) with persuasive technology (PT) can effectively detect cyberhate and encourage prosocial user behavior in digital spaces. Through an empirical study, we examine users’ perceptions of a self-monitoring persuasive strategy designed to reduce cyberhate. Specifically, the study introduces the Comment Analysis Feature to limit cyberhate spread, utilizing a prompt-based fine-tuning approach combined with LLMs. By framing users’ comments within the relevant context of cyberhate, the feature classifies input as either cyberhate or non-cyberhate and generates context-aware alternative statements when necessary to encourage more positive communication. A case study evaluated its real-world performance, examining user comments, detection accuracy, and the impact of alternative statements on user engagement and perception. The findings indicate that while most of the users (83%) found the suggestions clear and helpful, some resisted them, either because they felt the changes were irrelevant or misaligned with their intended expression (15%) or because they perceived them as a form of censorship (36%). However, a substantial number of users (40%) believed the interventions enhanced their language and overall commenting tone, with 68% suggesting they could have a positive long-term impact on reducing cyberhate. These insights highlight the potential of combining LLMs and PT to promote healthier online discourse while underscoring the need to address user concerns regarding relevance, intent, and freedom of expression.

1. Introduction

Social media platforms like Facebook, X (formerly Twitter), and WhatsApp are among the most popular venues for content creation and sharing. They provide users with a fast and efficient means to disseminate information, making them valuable sources of information [1,2]. However, these same platforms can also be exploited to spread harmful content, including cyberhate, misinformation, and cyberbullying. Such negative content not only degrades the quality of interactions but can also lead to adverse experiences for users and, in some cases, even spark criminal activities or violence [3,4].

Cyberhate—often referred to as online hate speech—encompasses expressions of hostility or incitements to hatred directed toward individuals or groups based on characteristics like ethnicity, religion, nationality, or sexual orientation [5,6,7]. Although there isn’t a universally accepted definition or legal framework for cyberhate, its rapid proliferation across social media has attracted significant attention from researchers, technology companies, and programmers over the last decade as they seek solutions.

A recent Anti-Defamation League (ADL) report noted that 41% of Americans have encountered cyberhate or harassment [8,9]. Figure 1 displays the distribution of reported incidents based on individuals’ self-identified characteristics. Similarly, the European Union Kids Online Survey (2017–2019) documented an increase in cyberbullying and cyberhate [10,11]. In response, initiatives like the EU’s 2016 Code of Conduct mobilized companies such as Facebook, YouTube, and Microsoft to curb illegal cyberhate [12]. National laws with criminal penalties have also emerged to address this issue, yet cyberhate persists [13], often masked by coded language or seemingly innocuous phrases that evade traditional detection methods.

Recent advances in machine learning have produced models to identify cyberhate [14,15,16,17], but their real-world utility remains limited. Many perform well on specific datasets but falter when applied to new contexts [18,19,20], highlighting a critical need for generalizable solutions. Additional research has explored the dynamics and diffusion of cyberhate across social media [21], though relatively few studies have proposed methods to prevent its spread [12]. Other investigations have examined how users perceive and propagate cyberhate, along with their attitudes and behaviors related to the issue [22,23]. Notably, there is a gap in studies that aim to promote healthier online behavior as a means to counter the spread of cyberhate [24].

Fortunately, software can be a powerful tool for encouraging behavioral change among users, effectively motivating bystanders to actively counter cyberhate. Research indicates that bystanders play a critical role, as their inaction when witnessing cyberhate incidents can be perceived normalizing hostility [25,26]. Thus, fostering bystander intervention is vital for mitigating the negative impacts on targeted groups [27], particularly through visible counter-arguments in public comments [28,29,30].

To promote bystander action against cyberhate, it is essential to design social media platforms that effectively encourage their interventions. Information systems can be purposefully crafted to drive behavioral change and are commonly known as persuasive systems or persuasive technology (PT) [31,32,33]. PT refers to digital systems intentionally designed to influence user attitudes or behaviors in a non-coercive way. The success of PT in fields such as health, education, and wellness [34,35,36,37,38,39]—along with evidence that incorporating persuasive strategies increases engagement and behavior change [40,41]—suggests that similar techniques can be applied to motivate users to speak out against cyberhate, thereby lessening its prevalence and negative impact.

The recent emergence of large language models (LLMs), defined as AI systems trained on extensive text data to perform tasks like text classification and content moderation such as GPT-3 and Gemini, has also marked a significant advancement in Natural Language Processing (NLP) [42]. These models have proven highly effective in a range of NLP tasks, from text classification to sentiment analysis, due in part to their training on vast amounts of data, which enables them to acquire an extensive understanding of human language [43,44,45]. Furthermore, interest in using LLMs for cyberhate detection is growing. For example, one study found that GPT-3 could identify sexist and racist language with an accuracy of around 85% using few-shot learning [46]. Although that study was limited by its narrow focus and small dataset, further investigations, such as Li et al.’s evaluation of ChatGPT’s ability to classify harmful content, have shown comparable results to expert human annotations [47]. Another study by He et al. demonstrated that LLMs and prompt-based learning could slightly outperform existing baselines in detecting toxic content [48].

To extend these efforts, the authors investigate the feasibility of combining the strengths of LLMs and Persuasive Technology to motivate users to take a prosocial role in addressing and limiting the spread of cyberhate. The study introduces the Comment Analysis Feature, which uses a prompt-based fine-tuning approach to classify user comments as cyberhate or non-cyberhate and generates context-aware alternative statements to encourage positive communication. A case study evaluated the feature’s real-world performance, exploring users’ perceptions of this self-monitoring strategy, assessing detection accuracy, and measuring the impact of alternative statements on user engagement and perceptions. The findings of this study are meant to guide software engineers in designing persuasive social media platforms that are cyberhate aware. The study’s primary contribution lies in its dual focus: leveraging LLMs’ contextual precision for cyberhate detection while employing a persuasive design to nudge behavioral change. This approach addresses a critical gap in existing research, which often prioritizes detection over prevention.

2. Related Work

Toxic content (e.g., cyberhate) is rapidly spreading on the internet in various forms such as racism on social media [49,50], bullying in online gaming, replies to posts on social media platforms [51,52], trolling [53], threats of violence, sexual harassment, and more [54]. These attacks constitute a subset of abusive behavior rooted in hate and harassment—part of a broader category of threats involving attempts to cause emotional harm to a target, such as stalking, doxxing, sextortion, and intimate partner violence [55].

The issue of combating cyberhate has garnered significant attention from researchers across various fields, including natural language processing (NLP), machine learning [56], psychology, and human-computer interaction. Traditional approaches to managing cyberhate primarily focused on rule-based systems and keyword filters, which are often rigid and prone to high false positive and negative rates due to the complexity and evolving nature of cyberhate. Additionally, recent advancements in NLP, particularly the development of large language models (LLMs) like GPT-3 and BERT, have significantly improved the ability to detect cyberhate with higher accuracy. LLMs are capable of understanding context, sarcasm, and nuanced language, which are critical in identifying subtle forms of cyberhate [57].

In addition, Zhou et al. [18] proposed a deep learning fusion approach combining CNNs and LSTMs to enhance context understanding for cyberhate detection. Furthermore, Swamy et al. [19] examined the challenge of generalizability across datasets, highlighting the issue of domain adaptation in cyberhate detection models. Caselli et al. [58] emphasized the mitigation of bias in training data and its impact on model fairness. Moreover, multilingual and cross-platform detection has gained traction, with Liu et al. [59] demonstrating how transformer-based models can adapt to varying linguistic contexts. These contributions collectively underscore the importance of building flexible, context-aware cyberhate detection models.

Almaliki et al. [9] introduce the Arabic BERT-Mini Model (ABMM) for the detection of cyberhate on social media. This study employs the Bidirectional Encoder Representations from Transformers (BERT) model to analyze Twitter data, classifying the content into three categories: normal, abuse, and cyberhate.To evaluate the effectiveness of the model, a series of experiments were conducted.

Liu et al. [59] demonstrated the effectiveness of LLMs in identifying cyberhate across various social media platforms, highlighting their ability to generalize across different contexts and languages. However, despite these advances, LLMs can still struggle with ambiguous or context-dependent content, where human interpretation plays a crucial role.

Elmezain et al. [11] propose a hybrid model that combines transformer architectures with SVM to classify images in a cyberbullying dataset. Initially, seven different CNN architectures are evaluated to identify the most effective ones. Feature extraction is then performed using the top four models—ResNet50, EfficientNetB0, MobileNet, and Xception. Then, the concatenated features are optimized and fed into the SVM classifier for the final classification.

Almars et al. [1] introduce a Hybrid Attention Neural Network (HANN) for rumor detection on social media. HANN is designed to capture the most relevant and distinguishing features across different classes while also offering interpretability of the model’s decisions. The proposed framework integrates two deep learning components: Convolutional Neural Networks (CNNs) and Bidirectional Long Short-Term Memory (Bi-LSTM) networks, both enhanced with attention mechanisms.

Studies like those conducted by Caselli et al. [58] have pointed out the biases inherent in training data, which can lead to biased outcomes in cyberhate detection. These models can sometimes reinforce stereotypes or overlook hate speech directed at minority groups. Furthermore, LLMs may be manipulated through adversarial attacks, where subtle changes in input can lead to incorrect classifications. These limitations indicate the need for supplementary approaches to enhance the robustness and fairness of cyberhate detection systems.

Persuasive technology, which involves the design of digital tools and interventions aimed at changing attitudes or behaviors, has been explored as a means to promote positive online interactions [60]. Oinas et al. [61] foundational work on the principles of persuasive technology has been widely applied in health, education, and environmental domains, demonstrating its potential to influence user behavior effectively. In the context of cyberhate, persuasive technology has been utilized to encourage empathy, reduce aggressive behavior, and promote respectful communication. For example, interventions like the “ReThink” software developed by Jain (2021) have shown success in reducing cyberbullying by prompting users to reconsider their messages before posting them [62,63,64].

Despite these advancements, several gaps remain in the current body of research. Most existing studies have focused on either detection (LLMs) or prevention (persuasive technology) of cyberhate [65], with few exploring the integration of both approaches. Additionally, Many models perform well on benchmark datasets but struggle to generalize across contexts, languages, or platforms, and often lack transparency in how decisions are made. In addition, these systems typically operate in isolation from the end user, limiting their potential to promote awareness or behavior change. Furthermore, there is a need for more research on the ethical implications of using AI and persuasive technology in this domain, particularly concerning user privacy, consent, and the potential for manipulation. Future work should also explore the development of standardized frameworks and metrics for evaluating the effectiveness of combined LLM and persuasive technology interventions, ensuring they are both scalable and adaptable to various online environments.

In contrast, our approach integrates real-time, context-aware suggestions with a self-monitoring strategy, combining detection and intervention within a single interface. This dual focus—classification and persuasion—aims to not only flag cyberhate content but also support users in adopting more prosocial communication practices, addressing a critical gap in the current literature.

3. Methodology

This paper presents an empirical investigation into the potential integration of persuasive technology and LLMs capabilities within the design process of social media platforms, with the goal of motivating users to actively combat the propagation of cyberhate. Particularly, the integration of a self-monitoring strategy in the form of a Comment analysis feature that employs LLMs to analyze and visually represent to users how their generated textual content (such as posts, replies, etc.) might be interpreted by others. As users draft their comments and right before they post them, LLM-based comment analyzer will analyze their input based on their choice of words, writing style, and punctuation, differentiating between tones such as hateful and natural, and subsequently generating appropriate recommendations. Users will then have the option to accept or reject these suggestions.

Self-monitoring is an effective persuasive technique [66], as it enables individuals to develop a clearer insight into their behavioral tendencies [67] and supports self-regulation [68]. In this context, the implementation of a comment analysis feature can play a valuable role by helping bystanders to improve their self-awareness regarding the tone and impression of their online comments to challenge cyberhate. To evaluate the effectiveness of this integration, a case study methodology was selected [69]. Case studies are often more appropriate than controlled experiments when assessing the broader implications of a tool or method. While they may not match the scientific rigor of formal experiments, case studies offer practical insights to determine whether a particular approach could be advantageous for an organization or project. Consequently, they are widely regarded as a fitting research strategy in software engineering contexts [70].

In the following sections, we will provide an overview of how the Comment Analysis Feature processes user comments, identifies instances of cyberhate, and generates appropriate responses. After that, we will discuss the case study design, development, and evaluation.

3.1. Comment Analysis Feature

This study introduces a feature capable of analyzing user input to detect instances of cyberhate, with the goal of motivating users to combat cyberhate by enhancing their self-awareness regarding the tone and impression of their online comments. Prompt engineering is consider essential factor for effectively guiding LLMs to deliver relevant, coherent, and high-quality responses [71]. In this context, a prompt refers to a natural language text that is provided to a language model (LLM) to generate specific output [72,73]. In related work, prompt-based techniques have been categorized into four main types: [71,74,75,76]: (1) Zero-shot prompts, where the LLM generates a response without being given any additional examples or information provided; (2) One-shot prompts, which provide the model with a single example to guide its response; (3) Few-shot prompts, where several examples are provided to help the model understand the expected pattern and generate similar types of responses; and (4) Template-based prompts, which use structured and consistent formats to deliver clear instructions and relevant context to the model.

In this study, our proposed method utilizes a template-based prompts approach combined with large language model techniques to tailor the model for the specific task of cyberhate detection and recommendation. The prompt utilized in our feature was carefully crafted to frame the query within the relevant context of cyberhate, ensuring that the LLM could accurately process and classify the input based on this specific template and suggest several alternatives for hate speech. Additionally, prompts were iteratively tested and evaluated based on the quality of the model’s responses. Feedback from two experts was used to refine the prompt by adjusting its wording, structure, and level of detail. An overview of our proposed feature is illustrated in Figure 2. The gemini-1.5-pro-latest version is employed to generate responses using structured prompt templates (https://gemini.google.com/ (accessed on 20 November 2024)). Each prompt was sent to the model with the following parameter settings:

• Max_output_tokens: This parameter controls the number of tokens (i.e., the length) generated by the model. We set it to maximum 100 tokens (words) to ensure concise responses, making it ideal for recommendations while avoiding long or irrelevant content.
• Temperature: This parameter defines the level of creativity and randomness in token selection. In this study, the temperature is set to 0.7, providing a balanced trade-off between logical coherence and creativity.
• Top_P and Top_K: These parameters determine how the model selects tokens during response generation. The Top_P parameter is set to 0.95, which means the model samples from the top 95% of the probability distribution. While, the Top_K parameter is set to 40, restricting selection to the top 40 most probable tokens within the Top_P range.

To effectively combat hate speech online, two prompt templates were implemented to identify hate speech behavior. The first prompt is designed to classify user comments as either hate speech or non-hate speech. Additionally, a second prompt template was integrated to assist the LLM suggest alternative statements for cyberhate, tailored to the context of the user’s original input. The process begins by analyzing user comments to determine whether they contain hateful language. Given a user input (comment) i, a predefined prompt (template) $P_1$: “Does this sentence contain inappropriate language or hate speech?”, and another predefined prompt (template) $P_2\left(n\right)$: “Based on the context, suggest n polite and modified versions of the statement, focusing on making the language more friendly”. The objective is to classify a user input (comment) i into one of two predefined classes: $Y=\{\mathrm{hateful}, \mathrm{not\; hateful}\}$, and subsequently generate a list of n recommendations tailored to i if the comment contains aggressive or hate speech.

To achieve this, the feature follows these steps to identify cyberhate:

1.

Input Analysis: The feature first receives user input i

2.

Hate Speech Classification: The feature then evaluates user input i by concatenating it with the prompt $P_1$:

$$R_1=\mathrm{LLM}\left(i,P_1, \mathrm{parameter\_config}=\left\{\begin{array}{cc}\mathrm{temperature}\hfill & =0.7\hfill \\ \text{top\_p}\hfill & =0.95\hfill \\ \text{top\_k}\hfill & =40\hfill \\ \text{max\_output\_tokens}\hfill & =100\hfill \end{array}\right\}\right)$$

where LLM is a function that processes prompt template $P_1$ and user input i, returning a response $R_1\in \{\mathrm{Yes},\mathrm{No}\}$, which indicates whether i contains inappropriate language or cyberhate (hateful) or normal (not hateful). The configuration parameters here govern the model’s response generation behavior.

3.

Contextual Recommendation: According to the output of $R_1$:

• If $R_1=\mathrm{Yes}\mathrm{or}\mathrm{True}$: The feature generates n alternative sentences (suggestions) using $P_2\left(n\right)$ template for user offensive input i:

  $$R_2=\mathrm{LLM}\left(i,P_2\left(n\right),\text{parameter\_config}=\left\{\begin{array}{cc}\mathrm{temperature}\hfill & =0.7\hfill \\ \text{top\_p}\hfill & =0.95\hfill \\ \text{top\_k}\hfill & =40\hfill \\ \text{max\_output\_tokens}\hfill & =100\hfill \end{array}\right\}\right)$$

  where $R_2=\{a_1,a_2,\dots ,a_n\}$, and each $a_j$ is an alternative suggestions for i. These suggestions aim to preserve the meaning while removing offensive or harmful words from user input.
• If $R_1=\mathrm{No}\mathrm{or}\mathrm{False}$: The feature acknowledges and posts the conversation:

  $$Ack=“\mathrm{Acknowledgment}:\mathrm{Post}\mathrm{conversation}.”$$

Output Generation: Finally, the feature identifies the classification output and generates a list of suggested alternatives, if applicable.

$$\mathrm{Output}=\left\{\begin{array}{cc}R2\hfill & \mathrm{if}{R}_1=\mathrm{Yes}\mathrm{or}\mathrm{True}\hfill \\ Ack\hfill & \mathrm{if}{R}_1=\mathrm{No}\mathrm{or}\mathrm{False}\hfill \end{array}\right.$$

3.2. Study Design

The aim of conducting this case study is to assess the impact of integrating a self-monitoring persuasive strategy and LLMs capability within the design process of social media platforms, with the goal of enhancing users’ behavior to limit the spread of cyberhate. To reach this aim, the authors developed an interactive news website that employs a self-monitoring strategy in the form of a real-time textual analysis and feedback using a comment analysis feature. The comment analysis feature is powered by Google Gemini AI model that analyzes user-generated textual comments and replies and then provides real-time feedback and suggestions to the users on the tone and potential impact of their comments to motivate them to reconsider their language choices, adopt more acceptable language to limit cyberhate spread. Users will then have the option to accept or reject these suggestions. An interactive news website was adopted to avoid platform-specific variables such as algorithms or user networks that are present on social media platforms (e.g., Facebook, X, etc.). This ensured ethical data collection but limits real-world replication (see Section 6). The design, development, and evaluation of the interactive news website are detailed in the following subsection:

3.2.1. News Website Development

The core elements of the website are as follows:

• Website Content:
  The website included news articles collected from trusted news sources, covering a wide range of tones and topics, from neutral subjects to more controversial and emotionally charged topics. The participants can post comments, reply to others, and engage in discussions related to news articles. Figure 3 shows the home page of the news website.
• LLM-Based Comment Analysis Feature:
  The website employs a comment analysis feature powered by Google Gemini AI model to analyze the participants’ comments before they post them to detect any cyberhate instances. Figure 4 presents examples of how LLM-Based Comment Analyzer detects and categorizes different types of hateful content in user comments.
• LLM-Based Feedback Feature:
  As shown in Figure 4, the comment analyzer not only identifies hateful instances in user comments but also suggests alternative, non-hateful revisions. If a comment includes hateful or aggressive input, the comment analyzer will provide the participants with suggestions of a none-hateful and more friendly form of their comments. It will offer textual recommendations to help the participants modify their input to sound more positive. For example, if a comment is detected as potentially harmful, the comment analyzer can suggest a rephrasing to make it less hateful and less aggressive. The participants will then have the option to accept or reject these suggestions. Figure 5 shows examples of some LLM-based suggestions to help the participants modify their comments.
• Flagging on Comments: Three different flag indicators were assigned to the participants’ comments to measure their interactions with the proposed self-monitoring strategy (comment analysis feature) and the LLM-based suggestions. These flags were as follow: The Blue Flag was assigned to standard comments, where the participants posted their comment without requiring intervention from LLM. The Green Flag was used for comments that were detected by the comment analyzer as cyberhate and the participants accepted the suggested revisions by the LLM. Lastly, the Red Flag was assigned to hateful comments where the participants declined to adopt the LLM-based suggestions for modification. This classification enabled a structured analysis of user engagement with the proposed features and their willingness to adjust their language accordingly. Figure 6 presents examples of flagged comments.

3.2.2. Study Procedure

The participants were asked to interact with the news website to read articles and participate in comment sections. As participants post comments, they interact with the LLM-Based comment analysis feature integrated into the website. When the LLM analysis identified potentially offensive or harmful language in a comment, it suggested alternative, more neutral phrasing. The participants could choose to accept or reject these suggestions. They were asked to comment on news articles that interest them as they normally do to let them express their opinions honestly and naturally. A post-study survey was then employed to gather their feedback, shedding light on the perceived effectiveness, user experience, and potential for broader application of such interventions in online environments. The participants were recruited to participate in the study through ClickWorkers (https://www.clickworker.com/ (accessed on 27 January 2025)), an online platform specializing in the recruitment of research participants. As an incentive, a payment of €1 was offered to each participant upon completing the study.

3.2.3. Study Evaluation

The evaluation of this study focuses on understanding the effectiveness of integrating Large Language Models (LLMs) and persuasive technology into a news website to reduce cyberhate. By tracking and analyzing the participants’ comments made before and after they receive the recommendations, the study measures whether they modify their comments to adopt less hateful or aggressive tones in their comments. The post-study survey gathers insights into how the participants perceive the LLM-based intervention and whether they find it useful for revising their comments and adopting more constructive language. Additionally, their feedback is gathered on the quality of the detection capability and the alternative suggestions received by the LLM and whether it provided appropriate and effective rephrasing for potentially harmful comments.

3.2.4. Survey Design

The survey designed for this study consisted of 20 diverse types. Initially, it was tested on a small group of 10 respondents who met the study’s sampling criteria. Feedback and responses from this initial group were used to improve and refine the questionnaire before distributing it to a larger audience. To ensure participants were familiar with the topic, a summary of the study was included at the beginning of the questionnaire. Additionally, respondents were briefed on the purpose of their participation and how their input would be used. The complete survey questionnaire used in this study is provided as Supplementary File S1 to enhance transparency and facilitate replication. A copy is also available at: https://shorturl.at/LUlcz (last accessed on 5 March 2025).

3.2.5. Sampling

A convenience sampling approach was employed to recruit 122 participants aged 18 or older, representing diverse cultural backgrounds. The characteristics of the participants were analyzed using cross-tabulation and frequency counts. Comparisons were made across variables such as age, gender, nationality, and educational level. The analysis revealed a high degree of diversity among the respondents, which strengthens the potential generalizability of the study’s findings. An overview of the participants’ characteristics is provided in

Table 1. An overview of participant demographics and characteristics.

		Age Groups					Gender			Cultural Background
		18–25	26–34	35–54	55 or Above	Total	Male	Female	Total	European	Middle Eastern	Total
level of education	No Schooling	0	0	1	0	1	1	0	1	1	0	1
	High school	12	0	1	3	16	13	3	16	9	7	16
	Associate degree	4	10	5	0	19	13	6	19	13	6	19
	Bachelor’s degree	24	20	13	8	65	35	30	65	24	41	65
	Master’s degree	0	3	8	3	14	5	9	14	11	3	14
	Doctorate degree	0	0	2	0	2	0	2	2	2	0	2
	Others	2	1	0	2	5	3	2	5	3	2	5
	Total	42	34	30	16	122	70	52	122	63	59	122

.

3.2.6. Analysis

First, the frequencies of detected cyberhate comments and the accepted or rejected suggestions were quantified. The questionnaire responses were prepared and refined, removing 29 incomplete or inconsistent forms. To ensure a robust analysis, we calculated acceptance ratios (accepted suggestions/suggestions received) for participants and conducted correlation analyses using Spearman’s rank correlation coefficient to explore relationships between variables such as willingness to express opinions and helpfulness of suggestions. An LLM classification performance and a post-hoc analysis of a random sample of flagged comments were performed to estimate the true-positive and false-positive rates of LLM detections, with results integrated into the findings. Descriptive analysis was also conducted to represent the collected responses [77].

4. Findings

4.1. LLM Intervention Clarity and Acceptance

The study’s findings suggest that the LLM-based suggestions to replace cyberhate comments were generally easy for people to understand. A total of 81% of participants indicated that the suggestions were easy to comprehend whereas only a very small proportion reported some difficulty (0.82%) in understanding those suggestions. This overall ease of understanding is encouraging, suggesting that the LLM communicates in a manner that is accessible to most users. Nevertheless, the minority who struggled with clarity signal an opportunity to further refine the simplicity and presentation of these suggestions, particularly for users with varying language proficiencies.

This finding is consistent with the work of Thomas Davidson et al. [78], who emphasize that clarity and interpretability are crucial in systems designed for cyberhate detection. In addition, ref. [79] highlights the importance of designing user-friendly interventions that can be seamlessly integrated into online interactions. However, the results show a moderate rate of acceptance for these LLM-based suggestions. To assess acceptance, we calculated individual acceptance ratios, revealing an average ratio of 0.35 across participants, with significant variation (SD = 0.21). Of the 80.3% participants who received at least one suggestion, 36% of participants accepted only one suggestion during the study and 17% accepted three suggestions, while 15% did not accept any suggestions. These findings suggest that although many users are open to modifying their comments based on the LLM’s feedback, a significant portion remain resistant. When participants chose to reject the LLM’s alternative phrasing, they cited several key reasons:

• The suggestion was not relevant to their opinions (51%)
• The suggestion was too different from what they wanted to express (38%)
• They did not think their comments were hateful (23%)
• They did not agree with the tone of the LLM suggestion (20%)

These observations align with insights from Gillespie [80], who examines how users often react to content moderation depending on their perceptions of fairness, transparency, and alignment with their own values. He emphasizes that effective moderation systems should be designed in a way that fosters user trust and engagement rather than creating a sense of imposed control.

These responses indicate that the LLM is often successful at flagging potentially harmful language but struggles to capture the nuanced intent behind a user’s original comment. Nearly a quarter of participants did not consider their comments hateful, suggesting that the system may misinterpret context and tone. This indeed aligns with [81] where it highlighted the issue of users rejecting interventions that they feel misrepresent their real intentions. Furthermore, Chandrasekharan et al. [82] underscore the difficulty of designing moderation systems that accurately interpret nuanced user intent. Future iterations of the LLM interventions should focus on improving the alignment between the LLM’s suggestions and the user’s original intent, as well as providing clearer explanations for why a particular suggestion is being offered.

4.2. LLM Frequency of Intervention

The frequency with which participants received LLM-based suggestions varied considerably. Among the 80.3% participants receiving suggestions, about 33.61% received suggestions once during the study, 20.49% received suggestions twice, and 15.57% received suggestions three times. Notably, 10.66% of the participants were presented with five or more suggestions, indicating that LLM was actively intervening when it detects language that may potentially include cyberhate instances. The minimum comment requirement was five per participant, but exact counts were not recorded, a limitation noted in Section 6.

This variability likely reflects differences in individual commenting behavior—some participants could be more prone to using language that triggered the LLM’s detection system. The fact that 10.66% of participants received five or more suggestions suggests that a subset of users may require more frequent interventions, which could be an area for further investigation taking into account that the effectiveness of moderation tools is often a function of their frequency of use; too many interventions can lead to fatigue, while too few may fail to curb harmful behavior.

In addition, to assess participant engagement patterns, we classified respondents into two groups based on their survey responses, those demonstrating constructive engagement (defined by high ratings for suggestion helpfulness and low rejection frequency) and those demonstrating resistant engagement (characterized by low suggestion helpfulness and high rejection frequency). The analysis revealed a statistically significant difference in suggestion acceptance ratios between the two groups (M = 0.45, SD = 0.18 for constructive engagers vs. M = 0.22, SD = 0.15 for resistant engagers), t(96) = 6.87, p < 0.001, indicating that constructive engagers were more receptive to the provided suggestions.

4.3. LLM Impact on Awareness and Behavior

The LLM’s-based suggestions appeared to have a varied impact on participants’ willingness to express their opinions. Focusing on the 80.3% participants who received suggestions, about 44% of them reported that the suggestions increased their willingness to express their opinions, while a similar percentage (45%) saw no change. However, 11% felt that the interventions decreased their willingness to speak freely. Spearman’s rank correlation coefficient was also used to assess the relationship between participants’ perceived helpfulness of LLM-generated suggestions and their willingness to express their opinions. The results revealed a positive correlation ($\rho =0.52$, $p<0.001$), indicating that participants who found the suggestions more helpful were also more likely to report increased willingness to express their opinions.

This suggests that while the LLM’s interventions encouraged some participants to engage more openly, others felt constrained by the provided suggestions. The fact that a significant portion of participants felt that the suggestions decreased their willingness to express their opinions is a critical finding. It suggests that the LLM’s interventions may have inadvertently stifled some users’ freedom of expression, particularly if they felt that their comments were being unfairly flagged or altered.

This mixed outcome highlights the delicate balance between mitigating cyberhate and preserving free expression. Gillespie [80] discusses how moderation tools can sometimes have a chilling effect on free expression if users feel that their contributions are being unfairly moderated. Thus, ensuring transparency and fairness in automated interventions is critical.

In addition, a significant portion of participants 40.98% believed that the LLM’s interventions improved their language and commenting tone to some extent, and 23.77% reported significant improvements. However, 26.23% saw no effect, and a small percentage (2.46%) felt that their commenting tone worsened as a result. This suggests that the LLM’s suggestions actively contributed to a behavioral shift in how participants expressed themselves. This improvement is likely a result from the intervention prompting participants to rethink and adjust their language. By providing immediate feedback and constructive alternatives, the LLM served as a real-time nudge for self-regulation, aligning with the goal of adopting self-monitoring technique as a persuasive strategy. Additionally, this finding emphasizes the system’s role as a behavioral intervention tool, not merely a content moderation mechanism.

However, the fact that over a quarter of participants felt that the intervention had no effect indicates that the LLM’s impact may be limited in certain contexts or for certain types of comments. Additionally, the small percentage of participants who felt that the intervention worsened their tone suggests that there may be cases where the LLM’s suggestions inadvertently led to less effective or less authentic communication. Suler’s [83] discussion of the “online disinhibition effect” provides context here, as it explains how online interactions can become either more positive or negative, depending on various factors. In this case, the LLM’s interventions sometimes succeed in moderating the participants commenting tone, but there is still a room for improvement.

4.4. Long-Term Impact on Cyberhate and Freedom of Speech

A substantial majority of the participants (68.03%) believed that the LLM-based interventions (using a self-monitoring strategy) could have a positive long-term impact on reducing cyberhate, while 13.11% were skeptical and 18.85% were unsure. This overall optimism suggests a willingness among users to embrace technological solutions for improving online discourse, despite some uncertainties about their implementation and potential overreach. Schmidt and Wiegand [84] provide further support for this optimism, arguing that automated moderation tools have the potential to reduce cyberhate—provided they are designed with both effectiveness and ethical considerations in mind. However, the skepticism and uncertainty expressed by a significant minority highlight the need for further research and development to address concerns about the effectiveness and ethical implications of LLM-based interventions to combat cyberhate.

Additionally, when the participants were asked about the impact of the LLM’s suggestions on their freedom of expression, 47.54% of participants felt that the interventions did not infringe on their ability to speak freely, describing them as “a helpful nudge to ensure respectful discourse”. However, 36.07% reported that the suggestions felt like censorship, and 16.39% were unsure. These responses underscore the tension between promoting respectful communication and protecting free speech.

The perception of the LLM’s interventions as censorship is a critical concern, as it suggests that some users may feel that their ability to express themselves freely is being curtailed. This underscores the importance of designing LLM-based intervention systems in a way that is transparent, user-controlled, and respectful of individual autonomy. Future research should explore ways to mitigate these concerns, such as by providing users with more control over the intervention process or by offering clearer explanations for why a particular suggestion is being made. As Crawford and Paglen [85] discuss, designing algorithmic systems involves complex ethical trade-offs, and ensuring that such systems respect users’ rights is essential.

4.5. LLM Classification Performance

To evaluate the classification performance of the comment analysis feature, we used the dataset from our previous work [9], which consists of approximately 9000 tweets categorized into three classes: normal, abusive, and hate speech. Due to the rate limit restrictions of the Gemini model, approximately 600 tweets were used in the evaluation. Furthermore, since our feature analysis task is a binary classification problem, we annotated normal tweets as 0, and both hate and abusive tweets as 1.

Table 2. Tweet Count by Class.

Class	Tweet Count
Non-Hate (0)	338
Hate (1)	260

shows the number of tweets in each class.

Each tweet was sent to the Gemini model without revealing its actual label, allowing us to assess the model’s performance based on its predictions. Figure 7 displays the confusion matrix for the LLM model, indicating its classification test performance. The matrix provides a detailed breakdown of the instances of each class that the model identified as hate and non-hate. According to the results, the model performs well across all classes. Moreover, the results demonstrates superior performance achieving a precision of 0.93, recall of 0.93, F-score of 0.93 and accuracy of 0.94. Furthermore, A manual expert post-hoc analysis of 50 flagged comments estimated a 92% true-positive rate and 8% false-positive rate for LLM detections, suggesting reasonable accuracy but room for improvement. In summary, these results indicate the potential of integrating a LLM into our feature analysis method to effectively identify and address instances of cyberhate. This integration improves the ability to analyze user behavior, classify harmful language, and provide constructive alternatives (suggestions).

5. Discussion

The discussed findings in Section 4 highlight several important insights into the effectiveness and challenges of using LLM-based interventions as a self-monitoring persuasive strategy to limit cyberhate spread. On the one hand, the overall ease of understanding and moderate acceptance rates suggest that many users can and do engage positively with LLM-based interventions. This supports the notion that well-designed interventions—those that prioritize clarity and relevance—can contribute to a more respectful online environment. However, the data also reveal persistent challenges.

A significant portion of users rejected the LLM-based suggestions due to perceived irrelevance or a mismatch with their intended tone. These findings underscore the importance of context in using LLM-based interventions to be finely tuned to distinguish between genuinely harmful language and benign expressions that might be misinterpreted as cyberhate. This nuance is critical, as misclassification can lead not only to user dissatisfaction but also to a perceived infringement on their freedom of expression.

The mixed impact on users’ willingness to express their opinions further complicates the picture. While some users appreciated the corrective LLM-based suggestions and felt encouraged to communicate more responsibly, others experienced a chilling effect, feeling that their autonomy was undermined. This duality reflects broader debates in the literature about the balance between moderation and free speech, with scholars such as Gillespie [80] and Roberts [86] emphasizing the need for moderation systems that are both effective and respectful of individual rights. In this study, the intervention was designed to be assistive rather than prescriptive, providing optional suggestions without enforcing censorship. Participants retained full control over whether to accept or reject language revisions. To address this, interventions should prioritize transparency, offer opt-out features, and clearly communicate why alternative suggestions are made. These steps can help build trust and mitigate perceptions of overreach while encouraging more respectful online discourse.

Additionally, the variability in the frequency of LLM-based suggestions points to the need for adaptive systems that can calibrate the level of intervention based on individual user behavior. Over-intervention can lead to fatigue and disengagement, whereas under- intervention might fail to curb cyberhate effectively. Future research should explore adaptive models that optimize intervention frequency based on real-time feedback and user context. Finally, the overall optimism regarding the long-term impact of LLM-based interventions as a self-monitoring persuasive strategy to limit cyberhate spread is tempered by significant concerns about potential overreach and censorship. This tension suggests that ongoing refinement, transparency, and user control will be critical components of any successful LLM-based intervention strategy moving forward.

6. Validity Threats

Despite the authors’ careful adherence to the principles of conducting empirical studies, there are still several potential threats to the validity of this study as follows:

• The persuasive suggestions generated by the LLM may implicitly assume a universal standard of politeness, potentially overlooking cultural, contextual, and rhetorical variations in communication. As a result, individual participants may interpret these suggestions differently based on their personal beliefs, language proficiency, or communication norms. Future work should prioritize fairness by incorporating diverse datasets, multi-perspective evaluations, and explainability features to ensure that interventions are culturally sensitive and do not disproportionately moderate marginalized voices. To improve interpretability and reduce unintended variation in user responses, future studies will also include standardized participant training and calibration sessions prior to interaction, along with behavioral analytics (e.g., tracking changes in commenting patterns) to supplement self-reported perceptions with objective measures of intervention impact.
• Participants may have provided socially desirable responses rather than their true opinions, particularly regarding sensitive topics like cyberhate. To minimize this threat, the author ensured anonymity in responses and emphasized that there were no right or wrong answers, encouraging honesty.
• While the news website allowed for testing of the LLM-based intervention, we acknowledge that it does not fully replicate real-world social media dynamics. Key features such as user networks, content algorithms, and evolving community standards were beyond the scope of this study. However, this study environment enabled a focused evaluation of user responses to self-monitoring strategies. Future research will explore integration with real-world social platforms to validate and extend these findings in more dynamic and socially nuanced contexts.
• Conducting the study on a custom-built news website may limit the generalizability of findings to other social media platforms with different dynamics and policies. To mitigate this, future experiments will replicate the study across multiple platforms with varying user interfaces and community guidelines. Cross-platform comparisons will help determine the consistency of LLM-based intervention effects.
• While the adopted methodology successfully identified and described users’ perceptions regarding our proposed approach, it is possible that certain significant aspects that could impact their behaviors in this context were not fully captured.
• A common concern when using questionnaires is whether respondents interpreted and understood the questions as intended. To address this, a pilot study was conducted with 10 participants who met the study’s inclusion criteria. Based on their feedback, some questions were reviewed and improved to ensure a shared understanding among all respondents.
• The sample size of the study, consisting of 122 respondents, can be considered medium-sized. A larger sample would enable the findings to be more generalizable to larger population groups. Further investigation of the study’s results on a larger population will be conducted in future research.
• The study utilized a convenience sampling method, which may not fully represent the broader population. To mitigate this issue, the author recruited participants from diverse cultural backgrounds and demographics to enhance the generalizability of the findings.
• The exact number of comments posted per participant was not recorded, limiting our ability to fully decouple the effects of the comments volume. In future work will track this precisely.
• Resource constraints prevented human labeling of all suggestions, though our sample analysis (92% true-positive) provides an initial estimate.

7. Conclusions

This study presented a novel approach that integrates the advanced contextual capabilities of LLMs with PT to address the persistent challenge of cyberhate. By employing a self-monitoring strategy (i.e., comment analysis feature) that is designed to enhance user self-awareness and encourage prosocial language choices. The empirical evaluation of users perceptions regarding our proposed approach revealed that a significant majority of participants found the LLM-based suggestions clear and beneficial, and many adjusted their language accordingly, demonstrating the potential effectiveness of this intervention. However, our findings also highlight important challenges, particularly in striking a balance between effective moderation and preserving freedom of expression, as well as ensuring high accuracy in the detection of cyberhate. These challenges underscore the need for further refinement of both the underlying models and the intervention strategies. The study’s primary contribution lies in its dual focus: leveraging LLMs’ contextual precision for cyberhate detection while employing persuasive design to nudge behavioral change. This approach addresses a critical gap in existing research, which often prioritizes detection over prevention. However, the mixed outcomes underscore the complexity of designing interventions that respect user autonomy while mitigating harm.

Moving forward, this work advocates for three key directions: (1) longitudinal studies to assess sustained behavioral impact and user fatigue, (2) refinement of LLM accuracy and contextual sensitivity to reduce mismatches between user intent and algorithmic interpretation, and (3) ethical frameworks to guide the deployment of persuasive technologies without compromising free speech. By centering user agency and fostering collaborative design, such tools could empower communities to self-regulate harmful content while preserving the open discourse vital to digital platforms. Ultimately, this research underscores the transformative potential of combining AI-driven analysis with human-centered persuasive strategies—a synergy that could redefine how social media platforms address cyberhate, not as mere moderators but as facilitators of healthier digital ecosystems.