Leveraging LSTM Neural Networks for Advanced Harassment Detection: Insights into Insults and Defamation in the U-Tapis Module ^†

Abstract

The prevalence of online harassment necessitates sophisticated automated systems that can accurately classify offensive content. In this work, we present a text classification system based on Long Short-Term Memory (LSTM) networks to categorize text into Neutral, Insult, and Defamation classes, thereby providing a more granular understanding of abusive behavior in digital environments. The system was evaluated using two labeled datasets—150 samples generated by ChatGPT and 1000 samples from internet sources—achieving an accuracy of 85% on both. Notably, the model demonstrated strong performance in identifying Defamation, exhibiting high precision and recall. These findings underscore the effectiveness of LSTM networks in capturing complex linguistic features, highlighting their potential for improving content moderation tools and curbing online harassment.

1. Introduction

In the rapidly evolving digital era, cases of cyberspace harassment are becoming increasingly complex, which requires swift and accurate handling [1]. Traditionally, the analysis of such cases has involved linguistic experts tasked with evaluating language use, including slander and defamation, under the Undang-Undang Informasi dan Transaksi Elektronik (UU ITE). Badan Reserse Kriminal (Bareskrim) and the police frequently engage with linguistic experts to analyze examination reports (BAP) to determine whether a case falls under the categories of slander, defamation, or other violations.

In a private interview with M. Niknik on 18 September 2024, it was stated that, in response to technological advancements, the police have developed a website allowing the public to file complaints, including harassment cases. While this platform has facilitated the reporting process, the current system still faces limitations in terms of speed and efficiency. A primary challenge lies in the three-hour time frame set to determine whether a report can proceed. This constraint often requires linguistic experts to be available at any time, even during unconventional hours, potentially affecting the accuracy of analysis due to fatigue or time limitations [2].

Recognizing the critical role of linguistic accuracy and efficient text analysis in addressing harassment cases, efforts have been made to enhance automated systems for language processing. As cases involving language use continue to rise and technology advances, Universitas Multimedia Nusantara has strived to remain responsive by developing a system for detecting Indonesian language errors. This system aims to support automation in journalism while offering potential applications in contexts such as harassment detection [3,4,5]. Existing language-screening systems include modules for detecting errors in word usage, compound words, and spelling [6,7,8].

In forensic linguistics, texts or messages can serve as concrete evidence [9,10,11]. However, the exploration and development of the U-Tapis system have yet to include a module capable of detecting harassment elements within a text. Developing such a module would not only strengthen U-Tapis as a pioneer of automation in journalism, but would also contribute significantly to the prevention and resolution of harassment cases in Indonesia.

Harassment can be broadly classified into categories such as slander, abuse, insults, and defamation. According to M. Niknik in 23 September 2024, in a private interview, among these categories, insults and defamation are among the most frequently encountered in Indonesia. This phenomenon underscores the importance of developing applications capable of detecting insults and defamation within the text [2].

Detecting a text containing harassment, particularly insults and defamation, requires a comprehensive approach. A significant challenge is the accurate classification of the text into harassment subcategories, given the diversity of language used, including both formal and informal styles. Insults and defamation often overlap in linguistic characteristics, necessitating models that can identify specific patterns within textual data [12,13].

In the realm of computer science, this research is highly significant as it integrates forensic linguistics with technology-based automation. Prior studies, including those by Pawit Widiyantoro (2025) [14], Nadaa (2024) [15], Tiara (2022) [16], and Alifqi (2023) [17], have made notable progress in detecting various forms of online harassment. However, these studies did not achieve perfect accuracy, and even those with decent results typically perform only a binary classification—identifying whether a text constitutes harassment (or related categories such as cyberbullying or hate speech) without addressing the nuanced subcategories of harassment. In contrast, our research specifically targets the differentiation between insults and defamation, which are among the most frequently encountered harassment subcategories in Indonesia [18,19].

This research employs Long Short-Term Memory (LSTM), a deep learning architecture renowned for its ability to process sequential data. LSTM has demonstrated exceptional performance in identifying patterns within unstructured text, such as hate speech and negative sentiments, making it highly suitable for this research problem [20,21]. Its strength lies in its capacity to model linguistic complexities, including informal language nuances and overlapping harassment subcategories, while maintaining high accuracy [22]. Unlike previous approaches that offer only a general harassment detection, the proposed system aims to precisely distinguish between insults and defamation, thereby addressing the research gap in detailed subcategory classification.

The objective of this research is to evaluate the accuracy of the LSTM method integrated into the U-Tapis project module for detecting insults and defamation in harassment cases, with a focus on automating text classification to improve the efficiency of online reporting systems. This approach addresses the computational demands of processing unstructured text while ensuring robustness in identifying linguistic complexities. By integrating advanced natural language processing (NLP) techniques to identify specific language patterns, this research not only meets practical needs—such as accelerating online reporting systems—but also contributes to the development of more adaptive, large-scale text analysis technologies, essential in an era of exponential digital data growth [23,24].

2. Materials and Methods

Figure 1 provides a detailed overview of the methodology applied in this research, from the initial steps to the final processes. The diagram aims to offer a clear understanding of the procedures followed in this research and how each stage is interconnected to achieve the research’s objectives.

2.1. Problem Identification

In this stage, an interview was conducted with M. Niknik, a lecturer and research coordinator for the U-Tapis project. In addition to her academic role, M. Niknik is a linguistics expert frequently involved in forensic linguistic cases. During the interview, M. Niknik explained that the harassment reporting system in Indonesia has advanced to the point where individuals can report cases through a dedicated website. The system then decides whether or not the case will proceed for legal action. However, while this system was designed to streamline the process, it has inadvertently placed a burden on the professionalism of linguists [2].

One major challenge is the three-hour time limit for determining whether a case can move forward legally. Within this time frame, the police consult linguists for an opinion on whether a given statement qualifies as harassment. Another issue is that linguists are often contacted outside of standard working hours. This situation highlights the need for a system that can automatically determine whether a sentence falls into the category of harassment, thus supporting the legal process and reducing the workload on linguistic professionals.

2.2. Literature Review

The literature review stage involves collecting both theoretical and practical knowledge to support the development of an NLP model for detecting harassment insults. The literature includes concepts such as the definition of harassment, insults, defamtaions, NLP, supervised learning, text preprocessing, and the LSTM algorithm.

2.3. Data Collection and Preprocessing

2.3.1. Data Collection

The data collection in this study involved 18,000 sentences generated by the ChatGPT language model using specific query methods. These sentences were then categorized into three groups: those containing insults, those containing defamation, and those that were neutral (i.e., not containing harassment). The labeling of the data was performed by ChatGPT based on predetermined rules, regulations, and definitions. These definitions were derived from the relevant literature, expert-validated case examples, and the Indonesian Electronic Information and Transactions Law (UU ITE). This approach ensures that the generated data aligns with the context and boundaries of harassment, particularly in the categories of insults and defamation.

The use of synthetic data generated by ChatGPT offers several advantages that have been recognized in the literature. First, it helps address the scarcity of authentic data, which is often difficult to obtain for legal or privacy reasons. Second, the flexibility in designing query parameters enables researchers to maintain the relevance and purpose of the generated data. Studies such as those by Xu Guo and Yiqiang Chen (2024) have demonstrated that synthetic data from generative AI can effectively replace authentic data for specific tasks [25]. In the context of NLP, Ghanadian et al. (2024) utilized synthetic data generated by large models such as ChatGPT to detect suicidal ideation, thereby significantly improving model performance [26].

2.3.2. Text Preprocessing

Preprocessing textual data is a crucial step in preparing raw input for machine learning models. For this research on insult and defamation detection using Long Short-Term Memory (LSTM) networks, a systematic text preprocessing pipeline was implemented to ensure data consistency and facilitate effective model training. The key stages in this pipeline included text cleaning, data splitting, tokenization, conversion of text to sequences, and sequence padding.

As shown in Figure 2, the preprocessing began with the normalization of slang words. Informal or abbreviated terms commonly found in Indonesian text, such as “gak” (informal for “tidak”) and “dgn” (abbreviation for “dengan”), were replaced with their standard forms to ensure linguistic consistency.

The next step was text cleaning, which involved a series of actions to eliminate noise and irrelevant elements from the data. All text was converted to lowercase to ensure uniformity and prevent unnecessary distinctions due to capitalization. URLs, email addresses, and other unnecessary patterns were removed, followed by the elimination of special characters, numbers, and punctuation marks. Extra whitespace, including leading, trailing, and consecutive spaces, was stripped to produce a cleaner, standardized input.

After cleaning, the text was processed further without stemming or stopword removal. Unlike traditional preprocessing approaches, this research retained the original word forms and common words such as “dan,” “atau,” and “tetapi” to preserve contextual and linguistic patterns critical for classification. This decision was based on observations that stemming or removing stopwords could hinder the model’s ability to capture important semantic and syntactic relationships. Tokenization was then performed, converting the processed text into tokens—individual words or phrases that form the building blocks of natural language processing models. To reduce computational complexity and improve model efficiency, the vocabulary was limited to the 10,000 most frequent words in the dataset, with out-of-vocabulary words replaced by a special token.

The tokenized text was subsequently transformed into sequences of integers, where each word was mapped to a unique index in the vocabulary. To address the varying lengths of text samples, all sequences were pre-padded or pre-truncated to a fixed length of 100 words. Padding augmented shorter sequences with zeros, while truncation shortened longer sequences, ensuring a uniform input shape compatible with the LSTM model.

Finally, the preprocessed data was split into two subsets: 80% for training and 20% for testing. This stratified division preserved the representativeness of the dataset across all subsets, enabling the model to learn effectively from the training data while maintaining a separate set for validation and final performance assessment.

2.4. Model Construction

The model construction and hyperparameter tuning process in this research consisted of several systematic steps to ensure robust performance in classifying insult, defamation, and neutral texts.

2.4.1. Model Construction

As shown in Figure 3, the process began with the initialization of a sequential neural network model, enabling the addition of layers in a linear stack. The first layer was an embedding layer, configured to map the input words to dense vector representations in a 128-dimensional space. This embedding layer served to capture semantic relationships between words, ensuring that the model could learn meaningful patterns from the input text.

Following the embedding layer, the first and second LSTM (Long Short-Term Memory) layers were added to extract temporal and contextual features from the input sequences. The number of hidden units in the LSTM layer, dropout rate, and recurrent dropout rate were set as hyperparameters to be optimized during tuning. Dropout was applied to prevent overfitting by randomly deactivating a fraction of neurons during training.

The output layer was a dense layer with three units, corresponding to the three target classes: insult, defamation, and neutral. A softmax activation function was applied at this layer to normalize the outputs into probabilities, ensuring that the sum of the probabilities across the three classes equaled one. This configuration allowed the model to output class probabilities for each input text sample.

The model was then compiled with categorical crossentropy as the loss function, which is appropriate for multi-class classification tasks. The Adam optimizer was used to adjust the model’s weights iteratively during training. Accuracy was chosen as the primary evaluation metric to assess the model’s performance.

2.4.2. Hyperparameter Tuning

To optimize the model’s configuration, hyperparameter tuning was conducted using a grid search approach with the keras_tuner library. The search focused on key hyperparameters that significantly influence the model’s performance. The parameters explored during the tuning process included the following:

• The dimensionality of the embedding layer (embedding_dim), tested with values of 50, 100, and 200.
• The number of units in the first and second Bidirectional LSTM layers (lstm_units_1 and lstm_units_2), tested with values of 128 and 256 for each layer.
• Dropout rates for the first and second LSTM layers (dropout_1 and dropout_2), as well as the dense layer (dropout_dense), tested with values ranging from 0.2 to 0.5 in increments of 0.1.

The grid search explored a total of 20 configurations, evaluating each based on validation accuracy. Each trial involved training a model composed of the following layers:

• An embedding layer with a vocabulary size of 10,000 and sequence length of 100.
• Two Bidirectional LSTM layers, each followed by dropout layers to prevent overfitting.
• A dense layer with 64 units and ReLU activation, followed by a dropout layer.
• A softmax output layer for classifying sentences into neutral, insult, and defamation categories.

To ensure the model converged effectively and avoided overfitting, additional callbacks were incorporated during training:

• EarlyStopping: Monitored the validation loss and stopped training after three consecutive epochs without improvement, restoring the best weights to prevent overfitting.
• ReduceLROnPlateau: Reduced the learning rate by a factor of 0.5 if the validation loss did not improve for two consecutive epochs, facilitating smoother convergence.

After evaluating all configurations, the optimal hyperparameters were selected based on validation accuracy. The final model was trained using these hyperparameters for up to 20 epochs with a batch size of 32. This tuned model demonstrated improved performance and robustness during the evaluation phase, effectively balancing complexity and generalization.

2.4.3. Test and Evaluate Model

The final model was trained using the optimal hyperparameter configuration on the training dataset. The performance of the final model was evaluated using several key metrics, namely the confusion matrix, accuracy score, precision, recall, and F1-score. These metrics were chosen to provide a thorough evaluation of the model’s performance, particularly in ensuring that all classes were treated equitably. By leveraging these metrics, the research ensures that the final model is both well-tuned and generalizable to unseen data, making it effective in classifying texts into the categories of insult, defamation, and neutral.

2.5. Testing and Evaluation

The evaluation of the experimental results was conducted manually by calculating and comparing the statistical metrics of the trained model. Additionally, the model was tested using a separate dataset distinct from the training data. The evaluation dataset consisted of 1150 sentences, with 150 sentences sourced from GPT 4o LLM and 1000 sentences sourced from the internet. Each dataset contained diverse examples representing the three categories: neutral, insult, and defamation.

2.6. Website and Interface Development

After completing the testing and evaluation phase, the research proceeds to the creation of a web-based interface. The workflow for the web development process is illustrated in Figure 4. Once the interface design or reference is finalized, the API is developed using Python 3.10.13. The Flask framework is employed in the web application to handle input data and generate output in JSON format. On the frontend, Vue.js is used for data processing, enabling the visualization of words and sentences. The interface displays paragraphs with identified errors alongside their corrected forms, providing an intuitive and user-friendly representation of the results.

2.7. Report and Documentation Making

At the final stage, a comprehensive report and documentation will be prepared, detailing the results of the nd the implementation of the developed model. This documentation will include information on the research findings, model implementation, and the evaluation of the model’s accuracy.

3. Results

This section is divided into four subsections: dataset collection, preprocessing, the design and interface of the web-based application, and the testing and evaluation of the LSTM model. The web-based interface was developed to assist law enforcement in conducting preliminary detection of linguistically relevant cases. Following this, an in-depth analysis of the model’s testing and evaluation is presented, detailing its performance and accuracy in detecting harassment cases, including insults and defamation.

3.1. Dataset

The dataset used in this research consists of 18,000 labeled text samples, evenly distributed across three categories neutral, insult, and defamation with 6000 samples in each category. The dataset was generated using GPT, leveraging its advanced language modeling capabilities to produce linguistically diverse and contextually appropriate examples for each classification category. This approach facilitated the creation of a well-balanced dataset, enabling the LSTM model to learn nuanced differences between categories effectively. The use of GPT-generated data also allowed for the inclusion of controlled and structured samples, which were instrumental in testing and evaluating the system’s performance under various linguistic scenarios.

3.2. Preprocessing

The preprocessing stage significantly enhanced the dataset’s quality, ensuring optimal performance of the LSTM model. Slang normalization was a key component, where a slang-to-formal mapping was applied using a colloquial Indonesian lexicon. This step allowed the conversion of informal expressions into their standard forms, enabling the model to better capture the intended meaning of the text.

Text cleaning involved converting all characters to lowercase and systematically removing unnecessary elements such as URLs, special characters, numbers, and punctuation. By focusing on linguistically relevant content, this process reduced noise and improved data clarity.

Unlike traditional preprocessing techniques, stemming and stopword removal were deliberately omitted in this research. This decision was based on the observation that stemming and removing stopwords often eliminate critical linguistic patterns and nuances. These patterns, such as specific word choices or syntactical structures, are particularly important for distinguishing between categories like insult and defamation, where subtle variations in language play a significant role.

Tokenization transformed the cleaned text into sequences of integer tokens, limiting the vocabulary to the 10,000 most frequent words for computational efficiency. Padding and truncation standardized the length of input sequences to 100 tokens, ensuring uniformity across the dataset. Additionally, class labels were encoded numerically, facilitating seamless integration with the LSTM model.

3.3. Website Implementation

The developed web-based interface provides an intuitive and user-friendly platform for text classification, specifically designed to assist in the detection of linguistic harassment. Figure 5 illustrates the initial interface displayed when users access the website. To begin, users input a sentence in the designated input field on the home page. Once the input is complete, they click the “Mulai Deteksi” button, which triggers the system to process the text and classify it accordingly.

In addition to the main hero section, the system provides an about us section, as shown in Figure 6 and Figure 7. This section introduces the purpose and background of the platform, explaining the motivation behind the creation of U-Tapis. The name U-Tapis reflects the system’s core function—acting as a filter to screen out harmful or harassing language from online interactions. This section not only enhances user trust but also communicates the broader social and ethical objectives of the project.

Finally, after users submit their input, the classification results are displayed as shown in Figure 8. This section clearly presents whether the input sentence contains elements of linguistic harassment or not. The result page is designed to be straightforward and easy to interpret, ensuring that users can immediately understand the outcome of the detection process.

3.4. Testing and Evaluation

This subsection assesses the performance and effectiveness of the developed system in accurately classifying text into neutral, insult, and defamation categories. To achieve this, the system was evaluated using a total of 1150 data samples: 1000 labeled data samples sourced from the internet and 150 labeled data generated by ChatGPT.

3.4.1. Testing with 150 GPT Datasets

The evaluation of the LSTM model with 150 labeled data samples generated by ChatGPT assesses its ability to accurately classify text into the categories of neutral, insult, and defamation. The evaluation results are presented in

Table 1. Confusion matrix for the LSTM algorithm (ChatGPT data).

True Label	Neutral (0)	Insult (1)	Defamation (2)
Neutral (0)	54	0	0
Insult (1)	15	32	2
Defamation (2)	5	0	42

.

The confusion matrix demonstrates that the LSTM model classified most samples accurately, achieving an overall accuracy of 85%. The detailed metrics of precision, recall, and F1-score for each category are provided in

Table 2. Classification report for the LSTM algorithm (ChatGPT data).

Label	Precision	Recall	F1-Score	Support
Neutral (0)	0.73	1.00	0.84	54
Insult (1)	1.00	0.65	0.79	49
Defamation (2)	0.95	0.89	0.92	47
Accuracy: 0.85 (85%)

.

3.4.2. Testing with 1000 Internet-Sourced Datasets

The evaluation of the LSTM model with 1000 labeled data samples from the internet also assesses its classification accuracy across the three categories. The evaluation results are shown in

Table 3. Confusion Matrix for the LSTM algorithm (Internet-sourced data).

True Label	Neutral (0)	Insult (1)	Defamation (2)
Neutral (0)	356	24	32
Insult (1)	15	274	31
Defamation (2)	19	27	222

.

The confusion matrix demonstrates that the LSTM model correctly classified most samples, achieving an overall accuracy of 85%. The detailed metrics of precision, recall, and F1-score for each category are provided in

Table 4. Classification report for the LSTM algorithm (Internet-sourced data).

Label	Precision	Recall	F1-Score	Support
Neutral (0)	0.91	0.86	0.89	412
Insult (1)	0.84	0.86	0.85	320
Defamation (2)	0.78	0.83	0.80	268
Accuracy: 0.85 (85%)

.

4. Discussion

The above results highlight the robustness of the LSTM-based approach for text classification in the Indonesian language domain. Both datasets—one with 150 ChatGPT-generated samples and one with 1000 real-world internet samples—yielded an accuracy of 85%. The model performed consistently across categories, although slight confusion was observed between neutral and insult instances, as well as in the classification of borderline defamation texts.

Several factors may contribute to this performance level. First, the deliberate omission of stemming and stopword removal appears to help preserve nuanced linguistic patterns critical for distinguishing offensive and defamatory statements. Additionally, the balanced distribution of training data (6000 samples per category) and the inclusion of GPT-generated samples likely enriched the model’s understanding of less frequent or nuanced language constructs.

However, some misclassifications underline the importance of deeper context. For instance, subtle or indirect insults and defamation may not be captured by direct lexical cues alone. Future research might incorporate advanced language models (e.g., Indonesia-specific large language models) or multimodal data to capture contextual information more effectively. Expanding the training samples with more real-world data—particularly from social media contexts—may also refine the model’s ability to handle informal registers and code-switching patterns.

5. Conclusions

This research evaluated a text classification system based on Long Short-Term Memory (LSTM) networks for categorizing text into neutral, insult, and defamation categories. The system achieved an accuracy of 85% on two distinct datasets: 150 ChatGPT-generated samples and 1000 internet-sourced samples. Results indicate that the model is effective at handling diverse linguistic expressions and exhibits particularly strong performance in identifying defamation, though challenges persist in accurately distinguishing subtle insults from neutral statements.

Notable future directions include the following:

• Exploring methods to enhance context understanding for borderline cases of insult or defamation.
• Integrating large language models specific to the Indonesian language to improve semantic representation and classification performance.
• Incorporating more real-world data from social media and other informal domains to address a broader range of linguistic styles.

Such enhancements will help the system adapt to evolving language use and better serve real-world law enforcement or content moderation applications.