A Comparative Evaluation of Transformers and Deep Learning Models for Arabic Meter Classification

Abstract

Arabic poetry follows intricate rhythmic patterns known as ‘arūḍ’ (prosody), which makes its automated categorization particularly challenging. While earlier studies primarily relied on conventional machine learning and recurrent neural networks, this work evaluates the effectiveness of transformer-based models—an area not extensively explored for this task. We investigate several pretrained transformer models, including Arabic Bidirectional Encoder Representations from Transformers (Arabic-BERT), BERT base Arabic (AraBERT), Arabic Efficiently Learning an Encoder that Classifies Token Replacements Accurately (AraELECTRA), Computational Approaches to Modeling Arabic BERT (CAMeLBERT), Multi-dialect Arabic BERT (MARBERT), and Modern Arabic BERT (ARBERT), alongside deep learning models such as Bidirectional Long Short-Term Memory (BiLSTM) and Bidirectional Gated Recurrent Units (BiGRU). This study uses half-verse data across 14 m. The CAMeLBERT model achieved the highest performance, with an accuracy of 90.62% and an F1-score of 0.91, outperforming other models. We further analyze feature significance and model behavior using the Local Interpretable Model-Agnostic Explanations (LIME) interpretability technique. The LIME-based analysis highlights key linguistic features that most influence model predictions. These findings demonstrate the strengths and limitations of each method and pave the way for further advancements in Arabic poetry analysis using deep learning.

1. Introduction

Language is a fundamental component of human intelligence and plays a crucial role in cognitive processes. Among it various forms, poetry stands out as a refined and sophisticated artistic expression. It transcends national boundaries and cultural variations, maintaining its popularity across generations and significantly influencing the development of human society [1].

Poetry represents the earliest form of literary expression in the Arabic language, serving as a powerful medium for articulating Arab self-identity, collective memory, and aspirations. Arabic poetry is typically classified into classical and contemporary forms [2]. Much of the research on Arabic prosody—also known as ‘arud’—has historically focused on morphology and phonetics, a trend that has continued for several years. Analyzing poetic meters is essential to determining whether a piece follows a consistent rhythmic pattern or contains metrical deviations [3].

Modern and traditional Arabic differ in their usage of vowels: short vowels are indicated through diacritic in writing, while long vowels are explicitly represented [4]. In the eighth century, the renowned philologist, Al-Farahidi, pioneered the study of ancient Arabic poetry by poetry by developing a system of poetic meters [5]. The metric structure in Arabic poetry encompasses sixteen distinct meters. Several key terms are used in Arabic prosody, including:

• Tafilah: A distinct metrical foot
• Bayt: A poetic line composed of two half-verses
• Sadr: The initial segment of a half-verse
• Ajuz: The latter segment of a half-verse
• Arud: The final part of the Sadr
• Darb: The final part of the ‘Ajuz’

Due to the diversity and complexity of the Arabic language, categorizing Arabic poetry requires advanced natural language processing (NLP) techniques [6]. The morphological, syntactic, and semantic intricacies of Arabic often exceed the capabilities of traditional rule-based and statistical approaches.

The recent advancements in artificial intelligence (AI)—particularly in deep learning (DL) and transformer-based models—have significantly improved outcomes in tasks such as text classification and sentiment analysis. Applying these cutting-edge methods to Arabic poetry classification offers a promising pathway toward improved understanding and model accuracy [7,8,9].

However, the Arabic NLP landscape remains challenging due to the language’s structural complexity and regional variations. Nonetheless, ongoing research and the development of specialized tools and datasets have facilitated more robust and efficient NLP solutions [10]. As the field continues to grow, the application of Arabic NLP is becoming more widespread across domains, such as healthcare, social media analysis, and even poetry generation [11,12].

Recent studies have increasingly adopted machine learning (ML) and deep learning (DL) techniques to tackle the complex task of poetic meter classification. Recurrent neural networks (RNNs) and various ML and DL models have been employed to classify poems based on their metrical patterns [13,14,15].

For meter categorization, handcrafted features such as word length, phonological patterns, and n-grams have traditionally been used with classic ML techniques, including Support Vector Machines (SVMs), Random Forests, and Decision Trees [16,17,18]. While these approaches are efficient, their adaptability across different languages and poetic forms is limited, as they depend heavily on manual feature engineering. In contrast, DL models like BiGRU and BiLSTM can automatically learn auditory and rhythmic patterns directly from raw text. These models have demonstrated strong performance across diverse literary traditions, effectively capturing both sequential and semantic patterns without requiring extensive preprocessing [19,20,21].

Transformer models have revolutionized NLP since the introduction of the ’Attention is All You Need‘ architecture in 2017 [22]. These models process input data in parallel using self-attention mechanisms, enabling them to capture long-term dependencies and contextual relationships within text. As a result, transformers have become foundational in many advanced NLP applications, including text classification.

Recent advancements in NLP—particularly the emergence of transformer-based models such as Bidirectional Encoder Representations from Transformers (BERT) and its Arabic counterparts—have significantly enhanced Arabic text processing. Comparative analyses have showed improved accuracy in text classification across various domains with the adoption of updated BERT models [23,24].

In this study, we used a public dataset for Arabic meter classification, MetRec [25], and evaluated it using several transformers and DL models. We employed the BERT base Arabic model (AraBERT), Arabic Efficiently Learning an Encoder that Classifies Token Replacements Accurately (AraELECTRA), Multi-dialect Arabic BERT (MARBERT), Modern Arabic BERT (ARBERT), Computational Approaches to Modeling Arabic BERT (CAMeLBERT), and Arabic-BERT for meter classification. In addition to transformer-based models, we used DL architectures such as Bidirectional Long Short-Term Memory (BiLSTM) and Bidirectional Gated Recurrent Units (BiGRU). These networks are capable of capturing long-range dependencies in sequential data, allowing the classification model to learn both spatial and contextual features—an essential requirement for handling the intricacies of Arabic poetry.

We also assessed feature importance and model behavior using the Local Interpretable Model-agnostic Explanations (LIME) interpretability technique. The findings provide meaningful insights into the effectiveness of different architectures for Arabic poetry classification, supporting the development of more advanced NLP solutions for the Arabic language.

The major contributions of this study are as follows:

• We compare and evaluate the performance of various pretrained transformer models—including Arabic-BERT, AraBERT, MARBERT, AraELECTRA, CAMeLBERT, and ARBERT—alongside BiLSTM and BiGRU DL models.
• We evaluate half-verse poems and tune the DL models using different hidden layer configurations, and adjust batch sizes for the transformer models.
• We apply multiple encoding strategies, including pretrained tokenizers (WordPiece and SentencePiece) for transformer models and character-level encoding for DL models.
• We investigate model behavior and feature significance using LIME to gain insights into the model’s decision-making processes.
• We contribute to the expanding of Arabic NLP by assessing interpretability and applicability of different modeling strategies for poetry classification.

The remainder of this paper is structured as follows: Section 2 presents a comprehensive literature review of transformer models and DL methodologies. Section 3 outlines the proposed techniques and model architecture. Section 4 describes the experimental results, followed by an in-depth discussion in Section 5, which also highlights potential avenues for further research. Finally, Section 6 concludes this study.

2. Literature Review

Deep learning methods have been extensively utilized in numerous studies to address Arabic meter classification. Notably, the work by Al-shaibani et al. [26] demonstrated significant progress by modeling the sequential nature of poetic lines using bidirectional recurrent neural networks (RNNs). Their approach effectively captured rhythmic patterns in Arabic poetry, providing a strong foundation for further exploration using advanced models such as Transformers.

Several efforts have aimed to develop algorithms for identifying meters in contemporary Arabic poetry [4]. Some approaches focus on detecting features of classical poetry, including rhythm, punctuation, and alignment, although however, they merely differentiate poetic from non-poetic texts [27]. Other techniques attempt to identify specific meters by converting verses into ‘Arud’ script using Mel Frequency and Linear Predictive Cepstral Coefficients characteristics [28].

Achieving performance levels of up to 75% accuracy, Berkani et al. [29] emphasized that automated meter recognition requires intricate data processing. Their method involves syllables segmentation, phoneme conversion, and the alignment of resulting patterns with predefined meter templates. Building on El-Katib’s approach, Khalaf et al. [30] were noteworthy in automating Arabic meter classification. Their method supports educators and students in understanding prosody and accurately identifying poetic meters, even in complex scenarios.

Albaddawi et al. [31] achieved high accuracy in meter detection by employing a model architecture comprising an embedding layer, five Bi-LSTM layers, a softmax activation function, and an output layer. Similarly, Abandah et al. [32] developed a neural network focused on enhancing diacritical understanding, which is crucial for improving both understanding and meter recognition accuracy.

Talghalit et al. [33] introduced a robust method based on leveraging contextual semantic embeddings and transformer models for Arabic biomedical search application. Their methodology involved preprocessing biomedical text, generating vector representations, and fine-tuning several transformer variants—including BERT, AraBERT, Biomedical BERT, RoBERTa, and Distilled—using a specialized Arabic biomedical dataset. Their model achieved an F1-score of 93.35% and an accuracy of 93.31%. Despite this strong performance, the authors identified key limitations, including Arabic’s complex morphology and the scarcity of domain-specific datasets. They emphasized the necessity of enriched contextual embeddings and expanded datasets to further improve transformer model performance in Arabic biomedical NLP applications.

Combining AraBERT’s contextual comprehension with Long Short-Term Memory (LSTM) for sequence modeling, Alosaimi et al. [34] proposed a sentiment analysis model. Its performance was benchmarked against traditional ML and DL models using various vectorization strategies across four Arabic benchmark datasets. Al-Onazi et al. [13] applied LSTM and convolutional neural network models, potimized using the Hawks optimization method, on the MetRec dataset for meter classification, achieving an accuracy of 98% with precision and recall scores of 86.5%.

In sentiment analysis, ensemble learning—where multiple models are combined to enhance classification accuracy—has proven highly effective. Studies show that integrating multiple deep learning models can outperform individual architectures in Arabic sentiment analysis tasks [35,36]. For instance, Zhou et al. [37] employed a hybrid model combining convolutional and LSTM networks, where the convolutional layers captured local features and LSTM layers captured global context.

AraGPT-2, the Arabic adaptation of Generative Pre-trained Transformer-2, aims to generate synthetic text to improve Arabic classification tasks. Refai’s et al. [38] outlined a three-part methodology: applying similarity metrics to evaluate sentence quality, generating refined Arabic samples using AraGPT-2, and assessing sentiment classification using AraBERT. Their research also addressed class imbalance issues prevalent in Arabic sentiment datasets.

Al Deen et al. [10] explored Arabic Natural Language Inference (NJI) using transformer models such as AraBERT and RoBERTa. The study introduced a novel pretraining strategy combining Named Entity Recognition, allowing for better identification of conflicts and entailments. Using a custom dataset derived from publicly available resources and linguistically enriched pretraining, AraBERT achieved an accuracy of 88.1%, outperforming RoBERTa enhanced with language-specific features.

In another study, Qarah et al. [39] developed a linguistic model for Arabic poetry analysis based on a BERT architecture pretrained from scratch using an extensive Arabic poetry corpus. This model incorporated 768 hidden units, 10 encoder layers, and 12 attention heads per layer, along with a vocabulary of 50,000 words, enabling it to capture the full a breadth of poetic forms and idiomatic expressions in the Arabic literature.

A hybrid methodology integrating deep learning with text representation techniques has been proposed to enhance the classification of Arabic news articles [9]. After comprehensive preprocessing steps including text cleaning, tokenization, lemmatization, and data augmentation, a proprietary attention embedding layer was employed to capture contextual relationships in the text. With data augmentation, the model surpassed the state-of-the-art Arabic language model AraBERTv2, achieving a classification accuracy of 97.69% [40].

In a study by Alshammari et al. [41], the authors introduced a novel AI text classifier designed to address the specific challenges in detecting AI-generated Arabic texts. This approach optimized two transformer models: the Cross-lingual Language Model and AraELECTRA [42]. Achieving an accuracy of 81%, the proposed models outperformed existing AI detectors. CAMeLBERT, when trained on an Arabic poetry dataset, achieved a performance accuracy of 80.9% [43].

Alzaidi et al. [44] enhanced Arabic text representation by utilizing FastText word embeddings, followed by an attention-based BiGRU classification model. This approach included a complete preprocessing pipeline and incorporated hyperparameter tuning to efficiently explore the search space and improve model performance. Similarly, Badri et al. [45] applied NLP techniques to detect hate speech in the Arabic literature across various dialects, leveraging BERT-based pretrained models to capture the linguistic complexity of the Arabic language.

Al-Shaibani and Ahmad [46] proposed a dotless Arabic text representation to reduce vocabulary size while maintaining comparable NLP performance. Their approach demonstrated up to a 50% reduction in vocabulary, with successful results across tasks such as language modeling and translation. Recent efforts in Arabic text-to-speech synthesis have also focused on enhancing emotional expressiveness, particularly in applications designed to assist visually impaired people [47].

Transformer-based architectures have also been adapted for domain-specific applications. For instance, Wang et al. [48] developed a graph transformer model for detecting fraudulent financial transactions. While their work targets the financial sector, it highlights the increasing trend of tailoring transformer models for intricate and domain-specific frameworks. Mustafa et al. [49] explored the use of explainable AI methods—such as LIME and SHapley Additive exPlanations (SHAP)—to clarify model decision-making in Arabic linguistic applications. The integration of explainability tools is considered essential for broader adoption and trust in transformer-based models within domain-specific contexts.

CAMeLBERT, designed specifically for Arabic NLP, is capable of handling linguistic variations, including Classical Arabic (CA), Dialectal Arabic (DA), and Modern Standard Arabic (MSA). It has demonstrated great performance in MSA tasks, particularly in emotion analysis and named entity recognition. Studies indicate that when paired with DL techniques, fine-tuned CAMeLBERT models outperform other models in sentiment analysis [50], achieving high accuracy, recall, and F1-scores in named entity recognition [51,52]. Its capacity to manage morphological and linguistic diversity also makes it suitable for dialect detection, translation, and the processing of the CA literature—an essential step in applying NLP techniques to historical texts [53]. Although BERT has gained prominence in NLP, studies [54,55,56] suggest that its application to Arabic text classification remains limited. However, transformer models specifically trained on extensive Arabic corpora—such as CAMeLBERT, AraBERT, and MARBERT—have shown superior performance. This growing body of research not only reveals promising avenues for addressing current challenges but also highlights the potential for expanding the efficacy of Arabic language applications through the use of specialized transformer architectures.

3. Materials and Methods

The proposed study comprises several phases: data preprocessing, data splitting into training and testing sets, model implementation, evaluation using various metrics, model testing, and interpretability analysis using the LIME technique. The workflow is illustrated in Figure 1.

3.1. Dataset and Train-Test Split

This study utilizes the MetRec Arabic poetry dataset, which comprises 55,440 verses classified across 14 m [26]. The symbol ‘#’ denotes the separation between the left and right parts of each verse. These halves are vertically concatenated to generate half-verse data, resulting in a total of 110,880 half-verses, as depicted in Figure 2.

The dataset exhibits moderate class imbalance: 10,000 verses are associated with the saree, kamel, mutakareb, ramal, baseet, taweel, wafer, and rajaz meters; 2524 with mutadarak; 8896 with munsareh; 2812 with madeed; 4244 with mujtath; 10,002 with khafeef; and 2402 with the hazaj meter. Figure 3 presents a word cloud generated from the dataset, while Figure 4 illustrates sample verses along with their associated meter labels.

After removing unwanted characters, alphabets, and punctuation, the cleaned data are split into training, validation, and testing sets using a 60:20:20 ratio. The training set contains 66,528 half-verses, while both the validation and test sets each contain 22,176. The meter-wise distribution across these splits is shown in Figure 5.

3.2. Data Preprocessing

The performance of the proposed models depends significantly on effective data preprocessing. Given the linguistic complexity of Arabic and differing model architectures, two preprocessing strategies were employed: character-level encoding for deep learning (DL) models and tokenization for transformer-based models. In both approaches, label encoding was applied to the target categories.

3.2.1. Tokenization

In natural language processing (NLP), tokenization—the process of breaking text into smaller components—is a crucial first step. Depending on the method, tokens may represent characters, subwords, or words. Among the various tokenization techniques, WordPiece and SentencePiece have become particularly popular for their adaptability to multiple languages and applications [57].

WordPiece, developed for the BERT architecture, is a subword tokenization method [58,59]. It constructs a vocabulary from the characters in the text and identifies frequently co-occurring token pairs to merge into new tokens. This iterative process continues until either no additional pairs can be merged or a pre-defined vocabulary size is achieved, maximizing the likelihood of representing the training data efficiently.

SentencePiece, developed by Google, is another subword tokenization tool. Unlike WordPiece, it applies a data-driven, language-independent strategy by treating the input as a series of characters. SentencePiece supports Byte Pair Encoding (BPE) and Unigram Language Models to produce subword tokens suitable for various NLP applications [57].

Table 1. Tokenization methods for various pretrained transformer models.

Model	Tokenizer
AraBERT	SentencePiece
AraELECTRA	WordPiece
ARBERT	WordPiece
MARBERT	WordPiece
Arabic-BERT	SentencePiece
CAMeLBERT	WordPiece

summarizes the tokenization methods employed by the pretrained transformer models used in this study.

For the proposed study, we employed pretrained tokenizers alongside the corresponding models. The code snippet for invoking the tokenizer is shown in Figure 6. The maximum sequence length was set to 60 and padding was enabled.

3.2.2. Character Encoding

Character encoding is the process of converting characters into a format that computers can easily access and manipulate. In this study, we addressed the complexity of the Arabic language—rich in morphological structures—by implementing character-level encoding. After removing special characters and spaces from the dataset, a vocabulary was created based on unique characters [26]. Each character was then assigned an index to form a mapping dictionary. This mapping translates every text input into a sequence of numerical indices, thus preserving distinct characters. The maximum sequence length was derived from the longest text in the dataset to ensure consistency in input size through sequence padding. This encoding method is particularly effective for managing spelling variations and noisy text, as it retains character-level patterns. The code snippet for character encoding is depicted in Figure 7.

3.2.3. Label Encoding

Label encoding is especially efficient when the target variable comprises discrete classes as it assigns a unique integer to each category. In this study, we used label encoding to map each of the 14 m classes to an integer ranging from 0 to 13, enabling faster processing during model training.

3.3. Model Architectures

Several models were used for classifying Arabic meters, including ARBERT, MARBERT, CAMeLBERT, Arabic-BERT, AraELECTRA, BiGRU, and BiLSTM.

3.3.1. Transformer Models

Transformers are suitable for high-performance tasks as they can handle large volumes of data and complex structures efficiently. The workflow of the transformer model used in this study is shown in Figure 8.

AraBERT is a transformer model pretrained specifically for Arabic [60]. It was developed to address the challenges of Arabic NLP, particularly the language’s intricate morphology and dialectal variations. AraBERT has demonstrated outstanding performance in various tasks, including sentiment analysis [61]. Built on the original BERT architecture, AraBERT significantly enhances the precision and efficacy of Arabic NLP applications.

AraELECTRA is another transformer model fine-tuned for Arabic language [42], extending the ELECTRA architecture. Unlike conventional masked language models, ELECTRA employs a discriminator network that distinguishes between real and replaced tokens generated by a generator. This approach improves training efficiency while maintaining performance.

CAMeLBERT is a robust Arabic transformer model designed to process different Arabic language forms, including dialectal Arabic, Classical Arabic, and Modern Standard Arabic (MSA) [43]. Unlike AraBERT, CameLBERT is trained on a more diverse corpus, enabling it to capture stylistic nuances in poetry. Its BERT-based architecture is enhanced through extensive fine-tuning for a better comprehension of poetry forms and morphological patterns.

MARBERT and ARBERT are advanced transformer models tailored for Arabic NLP tasks [62]. These models extend the BERT architecture with a bidirectional attention mechanism, allowing them to evaluate context by examining both left and right surroundings. MARBERT was trained on a large corpus comprising various Arabic dialects and MSA, while ARBERT focused primarily on MSA.

Arabic-BERT further expands the BERT architecture by focusing on Arabic-specific linguistic features, particularly those of MSA [63]. All the pretrained transformer models used in this study consist of 12 hidden layers and 12 attention heads in their architecture.

3.3.2. Deep Learning Models

Deep learning architectures such as BiLSTM and BiGRU are commonly used for handling sequential data, including natural language processing, time-series forecasting, and speech recognition. Both methods effectively capture long-terms dependencies and mitigate the vanishing gradient issue inherent in conventional RNNs.

BiLSTM (Bidirectional Long Short-Term Memory) enhances contextual understanding by processing input sequences in both forward and reverse directions [14,64]. The LSTM model includes three main gates: input, output, and forget gates.

$$f_t=\sigma (v_f{I}_t+w_f{L}_{t-1}+a_f)$$

(1)

where f_t is the forget gate at time stamp t, σ is the sigmoid activation function, v_f and w_f denote the weight matrices for f_t, I_t signifies the data input, a_f is the error vector, and L_t−1 denotes the hidden state result from the previous time step.

$$i_t=\sigma (v_i{I}_t+w_i{L}_{t-1}+a_i)$$

(2)

where i_t is the input gate, v_i and w_i denote the weight matrices for i_t, and a_i is the error vector.

$$\widetilde{{CM}_t}=tanh(v{I}_t+w_c{L}_{t-1}+a_c)$$

(3)

where $\widetilde{{CM}_t}$ is the candidate memory cell, a_c is the error vector, and v_c and w_c are the associated weight matrices.

$${CS}_t=f_t·{CS}_{t-1}+i_t·\widetilde{{CM}_t}$$

(4)

where CS_t is the current cell state, CS_t−1 represents the memory state from the previous timestamp, f_t, i_t, $\widetilde{{ CM}_t}$ is calculated from Equations (1), (2), (3), respectively.

$$O_t=\sigma (v_o{I}_t+w_o{L}_{t-1}+a_o)$$

(5)

where O_t is the output gate, v_o and w_o denote the weight matrices for O_t, and a_o is the error vector.

$$L_t=O_t·\tanh \left({CS}_t\right)$$

(6)

where L_t is the hidden state at time O_t, calculated from Equation (5), and CS_t from Equation (4). The BiLSTM analyzes the input sequence in both directions.

$$L_t=\overrightarrow{L_t} ☉ \overleftarrow{L_t}$$

(7)

where the $\overrightarrow{L_t}$ is the forward output and $\overleftarrow{L_t}$ is the backward output of the LSTM. The symbol ‘ʘ’ denotes on operation such as multiplication, averaging, summation, or concatenation.

In contrast, the BiGRU is a simplified variant of LSTM, with fewer parameters and enhanced computational efficiency. Like the BiLSTM, it processes sequences bidirectionally [26]. It includes two gates: the reset gate and the update gate.

$${RS}_t=\sigma (v_r{I}_t+w_r{G}_{t-1}+a_r)$$

(8)

where RS_t is the reset gate, v_r and w_r are the weight matrices for RS_t, G_t−1 is the hidden state from the previous time step, and a_r is the bias vector.

$$P_t=\sigma (v_p{I}_t+w_p{G}_{t-1}+a_p)$$

(9)

where P_t is the update gate, v_p and w_p denote the weight matrices, and a_p is the error vector.

$$\widetilde{G_t}=tanh(v_h{I}_t+w_h({RS}_t·G_{t-1})+a_h)$$

(10)

where $\widetilde{G_t}$ is the new hidden state, a_h is the error vector, and v_h and w_h are the weight matrices. RS_t is obtained from Equation (8).

$$G_t=\left(1-P_t\right)·G_{t-1}+\widetilde{G_t}·P_t$$

(11)

where G_t is the final updated hidden state, P_t is calculated from Equation (9), and $\widetilde{G_t}$ is from Equation (10).

$$G_t=\overrightarrow{G_t} ☉ \overleftarrow{G_t}$$

(12)

where the $\overrightarrow{G_t}$ is the forward output and $\overleftarrow{G_t}$ is the backward output of the GRU. The symbol ‘☉’ denotes operations such as multiplication, averaging, summation, or concatenation.

The BiLSTM and BiGRU model architecture is illustrated in Figure 9. The approach begins by converting a character sequence into an integer index using a character encoder, where each index corresponds to a specific vocabulary character. The input layer is shaped according to the training dataset. The embedding layer transforms input tokens into dense vectors of a predetermined size, set to 128 in this study. The BiGRU or BiLSTM layer then processes the sequence bidirectionally, capturing dependencies from past and future tokens. If the number of layers exceeds one, the return-sequence parameter is set to True. The dimensionality of the bidirectional layer is fixed at 256. To mitigate overfitting, a dropout layer with a rate of 0.2 is employed. A fully connected (dense) layer follows, mapping to the number of output classes—14 in this case. The Softmax function is used as the activation function, and the final prediction corresponds to the category with the highest probability in the Softmax output.

3.4. Model Evaluation and Parameter Tuning

The system environment used in this study is as follows:

• Windows 10 operating system
• 16 GB RAM
• Intel Core i7 processor
• GPUs—Nvidia GeForce GTX 1080 Ti
• Deep learning libraries: TensorFlow 2.7, Transformers 4.48.1
• Additional evaluation libraries: Scikit-learn 1.0, PyArabic 0.6.14, lime 0.2.0.1

The models were compiled prior to training. The Adaptive Moment Estimation (Adam) optimizer was employed due to its memory efficiency and notable effectiveness [65]. The loss function used was sparse categorical cross-entropy, appropriate for labels formatted as a sparse matrix.

The tuning parameters were model-specific. Deep learning (DL) models were optimized by varying the number of hidden layers, whereas the transformer model was tuned based on batch size. This study employed EarlyStopping and ReduceLRonPlateau as regularization strategies to mitigate overfitting. Training ceased when the model achieved optimal performance based on the monitored parameters, before overfitting could occur.

The EarlyStopping parameter monitored validation loss and stopped training if the loss remained constant or increased over eight consecutive epochs. The ReduceLRonPlateau technique reduced the learning rate by a factor of 0.1 if the loss value remained unchanged over two epochs.

Table 2. Parameters used in the proposed study for each model.

Model	Parameter	Values	Tuning
Transformer model	Batch size	[32, 64, 128]	Yes
	Epoch	50	No
	Learning rate	5 × 10−5	No
Bidirectional model	Batch size	128	No
	Layers	[1, 2, 3, 4, 5, 6, 7]	Yes
	Epoch	50	No
	Learning rate	1 × 10−3	No

presents the parameters used in this study.

3.5. Performance Evaluation

The model were evaluated using the test dataset. Performance assessment included analyzing the confusion matrix and classification report for each model. For multi-class classification, this study evaluated standard metrics such as accuracy, F1-score, recall, and precision. The formulas for each metric are provided below.

$$Precision={\displaystyle \raisebox{1ex}{$T_{pos}$}\!\left/ \!\raisebox{-1ex}{$T_{pos}+F_{pos}$}\right.}$$

(13)

$$Recall/Sensitivity={\displaystyle \raisebox{1ex}{$T_{pos}$}\!\left/ \!\raisebox{-1ex}{$T_{pos}+F_{neg}$}\right.}$$

(14)

$$Accuracy={\displaystyle \raisebox{1ex}{$T_{pos}+T_{neg}$}\!\left/ \!\raisebox{-1ex}{$T_{pos}+T_{neg}+F_{pos}+F_{neg}$}\right.}$$

(15)

$$F1-score={\displaystyle \raisebox{1ex}{$2 \ast T_{pos}$}\!\left/ \!\raisebox{-1ex}{$\left(2\ast T_{pos}\right)+F_{pos}+F_{neg}$}\right.}$$

(16)

A classification is considered accurate when a verse is correctly categorized into its respective meter (true positive, Tpos). For instance, if a verse written in the saree meter is correctly predicted as saree, it constitutes a true positive. A verse deviating from a specified meter but is accurately recognized as not conforming to that meter is a true negative (Tneg). Conversely, if a verse from one meter is misclassified as belonging to another, this results in a false positive (Fpos) for the predicted meter and a false negative (Fneg) for the actual meter. For instance, if a kamel meter verse is classified as madeed, it counts as an Fpos for madeed and an Fneg for kamel.

3.6. Explainability with LIME

LIME (Local Interpretable Model-Agnostic Explanations) is an interpretability technique that uses a locally approximated interpretable model to explain the predictions of complex models [66]. It can highlight the words or characters that significantly influence the categorization decision in the classification model. LIME approximates the model’s decision in a relatively small area surrounding a specific instance by perturbing the input data and tracking how the model’s prediction changes.

In the context of Arabic meter classification, LIME facilitates the identification of specific words or characters that strongly influence the model’s prediction. This helps to interpret and validate the patterns recognized by the transformer models. The final training parameters for the model are described in

Table 3. Training parameters used in the proposed study for each model.

Parameter	Values
Optimizer	Adam
Loss function	Sparse categorical cross entropy
Metrics	Accuracy, F1-score, Recall, Precision
EarlyStopping	Patience: 8 epocs, Monitor: Validation loss
ReduceLRonPlateau	Patience: 2 epocs, Monitor: Validation loss
Model explainability	LIME

. The tuning parameters are mentioned in Table 2.

4. Results

The dataset used in this study is the MetRec data, comprising 55,440 verses and a total of 110,880 half-verses. There are no studies based on half-verses and transformer models. The data are split into training, validation, and testing sets in a 60:20:20 ratio. The class labels are encoded numerically from 0 to 13.

4.1. Deep Learning Models Results Comparison

In the deep learning (DL) models, the text data are encoded using a character-encoding approach. Both BiLSTM and BiGRU models were fine-tuned with varying numbers of hidden layers. The complete performance results are presented in

Table 4. The performance of the DL models in the proposed study.

Model	Hidden Layers	Training Time (in Seconds)	Validation Accuracy (%)	Testing Time (in Seconds)	EarlyStopping Epochs	Testing Accuracy (%)	Recall	Precision	F1-Score
BiGRU	1	184.51	86.34	10.09	14	86.42	0.86	0.86	0.86
	2	439.77	89.16	16.57	18	88.88	0.89	0.89	0.89
	3	636.27	90.07	26.29	18	90.06	0.90	0.90	0.90
	4	1236.31	90.21	31.20	27	90.34	0.90	0.90	0.90
	5	1090.65	90.44	36.79	18	90.39	0.90	0.90	0.90
	6	2295.92	90.25	47.03	34	90.04	0.90	0.90	0.90
	7	3135.84	90.53	56.34	39	90.09	0.90	0.90	0.90
BiLSTM	1	200.68	86.09	8.59	14	85.83	0.86	0.86	0.86
	2	390.16	88.88	15.88	14	88.68	0.89	0.89	0.89
	3	712.73	88.79	23.63	17	89.13	0.89	0.89	0.89
	4	907.06	89.84	32.77	16	89.66	0.90	0.90	0.90
	5	1144.01	90.20	38.75	16	90.05	0.90	0.90	0.90
	6	1632.94	90.37	47.11	18	90.53	0.90	0.91	0.91
	7	2022.12	90.56	54.70	20	90.33	0.90	0.90	0.90

.

The BiGRU model achieved its best performance at 5 layers, reaching a testing accuracy of 90.39%, with F1-score, recall, and precision scores all equal to 0.90. The BiLSTM model performed best at 6 layers, attaining a testing accuracy of 90.53%, with recall of 0.90, and both precision and F1-score reaching 0.91.

The EarlyStopping parameter terminated the training process if the validation loss failed to increase or remained stable for eight epochs. Therefore, training stopped at the epochs indicated in the table for each layer.

The performance per class for the BiLSTM model with 6 layers is shown in

Table 5. Class-wise results for the BiLSTM model with 6 hidden layers.

Meters	Encoded Meters	Precision	Recall	F1-Score
saree	0	0.82	0.82	0.82
kamel	1	0.88	0.88	0.88
mutakareb	2	0.92	0.93	0.93
mutadarak	3	0.90	0.88	0.89
munsareh	4	0.92	0.93	0.93
madeed	5	0.80	0.85	0.82
mujtath	6	0.94	0.96	0.95
ramal	7	0.93	0.93	0.93
baseet	8	0.96	0.96	0.96
khafeef	9	0.93	0.93	0.93
taweel	10	0.95	0.96	0.96
wafer	11	0.94	0.94	0.94
hazaj	12	0.91	0.90	0.91
rajaz	13	0.80	0.77	0.79

. The baseet and taweel meters achieved the highest F1-scores (0.96), while the rajaz meter recorded the lowest (0.79). The corresponding confusion matrix is illustrated in Figure 10.

4.2. Transformer Models Results Comparison

In the transformer-based model, input text was encoded using tokenizers. The models were fine-tuned by varying the batch size. Full performance results are summarized in

Table 6. Performance of the transformer models in the proposed study.

Model	Batch Size	Training Time (in Seconds)	Validation Accuracy (%)	Testing Time (in Seconds)	EarlyStopping Epochs	Testing Accuracy (%)	Recall	Precision	F1-Score
AraBERT	32	2561.87	84.36	66.83	10	85.24	0.85	0.85	0.85
	64	2017.41	79.91	60.01	10	79.95	0.80	0.79	0.79
	128	1505.03	78.76	61.73	10	78.85	0.78	0.78	0.78
Arabic-BERT	32	2167.33	81.44	81.31	9	62.71	0.81	0.81	0.81
	64	1566.64	78.93	62.03	8	79.04	0.79	0.79	0.79
	128	1148.03	76.08	57.26	8	75.92	0.76	0.76	0.76
AraELECTRA	32	2204.53	87.59	60.23	14	87.14	0.87	0.87	0.87
	64	1901.09	83.46	56.83	10	83.72	0.83	0.83	0.83
	128	1623.86	79.63	55.83	12	79.60	0.79	0.79	0.79
ARBERT	32	2299.88	72.92	58.11	9	73.21	0.73	0.74	0.73
	64	1546.04	78.01	57.02	8	78.37	0.78	0.79	0.78
	128	1094.12	77.57	53.76	8	77.23	0.77	0.77	0.77
CAMeLBERT	32	2431.07	90.59	35.18	12	90.26	0.90	0.90	0.90
	64	1769.59	90.49	54.08	12	90.62	0.91	0.91	0.91
	128	1232.03	90.57	52.28	12	90.21	0.90	0.90	0.90
MARBERT	32	1875.58	83.95	59.64	7	83.43	0.83	0.83	0.83
	64	1348.81	82.79	57.97	7	82.69	0.82	0.83	0.82
	128	929.73	83.07	62.54	7	83.35	0.83	0.83	0.83

.

The CAMeLBERT model, with a batch size of 64, delivered the best performance among the transformer models, achieving 90.62% testing accuracy along with F1-score, recall, and precision scores of 0.91.

Class-wise results for CAMeLBERT with batch size 64 are provided in

Table 7. Class-wise results for CAMeLBERT with batch size 64.

Meters	Encoded Meters	Precision	Recall	F1-Score
saree	0	0.84	0.84	0.84
kamel	1	0.89	0.88	0.89
mutakareb	2	0.93	0.93	0.93
mutadarak	3	0.90	0.86	0.88
munsareh	4	0.91	0.92	0.91
madeed	5	0.83	0.83	0.83
mujtath	6	0.93	0.96	0.94
ramal	7	0.91	0.92	0.92
baseet	8	0.97	0.96	0.96
khafeef	9	0.93	0.92	0.93
taweel	10	0.96	0.97	0.96
wafer	11	0.93	0.95	0.94
hazaj	12	0.91	0.92	0.91
rajaz	13	0.80	0.79	0.80

. Similar to the BiLSTM model, baseet and taweel attained the highest F1-score of 0.96, while rajaz had the lowest at 0.80. The confusion matrix is shown in Figure 11.

5. Discussion

This study analyzed various models, including pretrained transformer models and bidirectional deep learning (DL) models, using half-verse data comprising 14 m. Among the tested models, CAMeLBERT achieved the highest accuracy of 90.62%, marginally outperforming the BiLSTM model, which reached 90.53%.

Tokenizers were also evaluated using the BiLSTM model. Specifically, SentencePiece (AraBERT) and WordPiece (CAMeLBERT) tokenizers were assessed. The model was tested with two layers, as depicted in

Table 8. Performance of the BiLSTM model using transformer tokenizers.

Tokenizer	Layers	F1-Score	Accuracy (%)	Recall	Precision
AraBERT	1	0.45	45.66	0.45	0.45
	5	0.52	52.28	0.52	0.53
CAMeLBERT	1	0.40	40.98	0.40	0.41
	5	0.40	40.24	0.39	0.41

. However, transformer tokenizers did not yield significant performance improvements with BiLSTM. Consequently, character encoding was preferred for the DL models.

BiLSTM leverages the benefits of long-term memory and enhances traditional LSTM by processing sequences bidirectionally [67]. Compared to GRUs, LSTMs with memory cells are more effective at capturing long-range dependencies. Al-Shathry [68] conducted further research using a balanced dataset of full-verse poems. By randomly selecting 1000 verses for each of the 14 m, their BiGRU model achieved 98.6% accuracy and scores of 0.90 for recall, precision, and F1-score.

Transformer models have notable advanced NLP challenges, including Arabic meter categorization. These models use deep contextual embeddings to identify intricate linguistic patterns specific to Arabic [34,69]. Although CAMeLBERT was trained on diverse datasets—including one related to poetry—it achieved only 80.9% accuracy on that set [43]. The exceptional performance of transformer models lies in their self-attention mechanism, which allows for more efficient modeling of long-range dependencies, outperforming recurrent networks such as BiLSTM and BiGRU. They are particularly adept at capturing complex linguistic characteristics [55].

Arabic poetry is often governed by systematic metrical structures, which can be effectively represented using half-verses. Analyzing half-verses preserves essential rhythmic and structural features.

Table 9. Comparison with previous studies using the same half-verse dataset.

Reference, Year	Model	Verse Length	Accuracy (%)	F1-Score
[26], 2020	BiGRU 5 Layers	110,880	88.80	-
The proposed study	BiLSTM 6 Layers	110,880	90.53	0.91
	CAMeLBERT		90.62	0.91

presents a comparative analysis of this study with previous research using the same half-verse dataset.

Model explainability using LIME is visualized in Figure 12, Figure 13 and Figure 14 for three representative sample texts. LIME highlights the words that most influence the model’s prediction by underlining them in different colors. These visualizations demonstrate how particular syllables or phrases contribute to the identification of a specific meter, along with the associated probabilities for the top predicted meters.

5.1. Practical Implications

The findings of this study offer valuable practical applications for Arabic poetry analysis and AI-driven literary tools. The high accuracy achieved using half-verse data shows that meter classification can be effectively automated without requiring full verses, enhancing its real-world applicability. Moreover, the ability of DL and transformer models to learn metrical patterns supports the development of educational technologies that allow researchers and students to rapidly analyze and classify Arabic poetic meters.

This research contributes to Arabic NLP, particularly in tasks that demand rhythmic-sensitive analysis. Demonstrating that half-verses suffice for precise classification reduces processing demands and broadens access to AI-driven solutions in Arabic literary and language technologies.

5.2. Limitations and Future Work

While the proposed study employs half-verse data to achieve a higher level of accuracy in Arabic meter classification, it has certain limitations. One of the restrictions is its reliance on a specific dataset which may not adequately represent the diversity of Arabic poetry. Future research should explore the potential for expanding the dataset to encompass a larger diversity of poetic styles and emotions to assess the generalizability of the models.

Additionally, exploring multimodal approaches that combine textual and audio features could provide deeper insights into Arabic poetry. Such interdisciplinary research would bridge computational linguistics, phonetics, and literary analysis, potentially leading to more nuanced and human-like poetic classification systems.

6. Conclusions

This study explored Arabic meter categorization using various deep learning and transformer models, including AraBERT, AraELECTRA, Arabic-BERT, ARBERT, MARBERT, CAMeLBERT, BiGRU, and BiLSTM models. To the best of our knowledge, no prior research has evaluated this combination of methodologies, making our contribution novel in the field.

By applying these methodologies, we introduced new perspectives on processing and categorizing Arabic poetic meters. CAMeLBERT achieved the highest accuracy (90.62%), closely followed by BiLSTM (90.53%). These results underscore the effectiveness of transformer models in capturing the complex metrical structures in Arabic poetry, marking a significant advancement and opening new avenues for Arabic NLP research.