Enhancing Arabic Speech Therapy with AI: Binary Fluent/Disfluent Classification of Arabic Stuttered Speech

Abstract

Stuttering and related speech disorders can interrupt the natural flow of speech through repetitions, prolonged sounds, and pauses or delays, affecting millions of people worldwide. Although considerable progress has been made in artificial intelligence-based Automatic Speech Recognition (ASR) technology, most of the current models remain mainly designed for high resource and dominant lingua franca languages, e.g., English, and underperform for Arabic. This paper presents insights into stuttering classification in Arabic using Whisper ASR from OpenAI, trained to classify speech segments as fluent or disfluent. As a necessary first step, we frame the task as a binary (fluent vs. disfluent) classification. Fine-grained recognition of specific disfluency types (such as repetitions, prolongations, and blocks) is left to future work. The most salient aspect of this study is the construction of a marked stuttering speech corpus in which speech segments of fluent and disfluent speech were collected from real clinical cases. A systematic comparative framework is built between the full Whisper family (Tiny–Large) and the Wav2Vec2.0 family (Base–XLarge) under identical conditions, providing the first benchmark for Arabic stuttering classification. We find that Whisper outperforms Wav2Vec2.0 at every scale, including the smallest variants, and remains reliable even for low-resource deployment. This confirms that the Whisper encoder is suitable for clinical Arabic speech-disorder workflows.

1. Introduction

Millions of individuals around the world of various ages and different languages suffer from some of the worst communication disorders, one of them being the most severe of them all: stuttering. Fluency is disrupted, and speech becomes convoluted. Stuttering is the inability to ward off misguided involuntary repetitions, long stretches of speech, and speech blocks. Current studies suggest that 5% of the world’s population stutters, and of this population, up to 2.5% are children under 5 years of age [1]. Stuttering is complex and must be accurately diagnosed where the speech therapist must analyze and interpret the detailed speech samples provided. In doing this, stuttering ranges and fluctuations are common, and constellations are much more abundant. Therefore, clinicians often find themselves in a state of conflict. Artificial Intelligence (AI) is being developed to assist in automating stuttered speech to classify speech as fluent or disfluent [1].

Although AI-based speech recognition systems have greatly progressed, most systems remain focused on English speakers. Although there are multilingual systems, low-resourced languages, such as Arabic, still struggle with insufficient training data and poor adaptation [2]. Numerous approaches to stuttering detection operate under the assumption of merely complex transcription models, and so, fall short. This gap is created by the absence of Arabic speech datasets, the scarce deployment of AI systems in the domain of speech therapy [1], and the issue of classifying speech disfluencies [3]. Given the scarcity of major high-quality Arabic speech datasets, especially those about stuttering, the present study makes a significant contribution to the field by compiling one of the largest and most clinically relevant Arabic stuttering corpora to date. This resource meets an essential need in speech-language research, providing an excellent starting point for future work. Clinical, linguistic, and computational studies are all to be developed based on this resource.

The primary objective of this paper is to propose a specialized AI-based system for Arabic stuttering classification, capable of detecting and classifying stuttering in Arabic speech. The purpose of this system is to analyze speech inputs, transcribe them, and classify each segment as fluent or disfluent, thereby improving automated assessment and diagnosis of stuttered speech and therapy. As a first step, the present work addresses the binary fluent/disfluent task, leaving fine-grained classification of specific disfluency types to future work. This research will contribute to the field in the following ways:

• Establish the first systematic, controlled comparison of the full Whisper family (Tiny–Large) against the full Wav2Vec2.0 family (Base–XLarge) for binary fluent/disfluent classification of Arabic stuttered speech, under identical preprocessing, speaker-independent splits, and evaluation metrics.
• Expand and refine the dataset by incorporating 10 h of annotated Arabic stuttering speech and an additional 10 h of fluent speech. This resource serves as a foundation for future AI research and applications in speech therapy and disorder detection.
• Employ the Whisper model by adapting the encoder-freezing plus lightweight classification-head paradigm specifically to Arabic, with Arabic-tuned preprocessing (single-speaker filtering, 10-s overlapping windows, Wiener-filtered denoising) and a Whisper-assisted, human- and clinician-verified annotation pipeline to address low-resource label scarcity.

The rest of the paper is organized as follows. In Section 2, the foundational concepts and reviewed works are discussed. Section 3 introduces the proposed method. In Section 4, we conduct a detailed experimental analysis to evaluate the performance of the proposed method. Section 5 discusses the findings, their clinical implications, limitations, and future work. Finally, Section 6 concludes the paper.

2. Literature Review

Stuttering is a speech disorder characterized by interruptions in fluency such as repetitions, prolongations, and silent pauses, which can hinder communication and reduce self-confidence. Traditional assessment methods by speech-language pathologists rely heavily on manual annotation of speech samples and behavioral evaluations, which are accurate but time-intensive and prone to subjectivity. Stuttering assessment is not limited to measuring repetitions, prolongations, or pauses. A clearer understanding also depends on the speaker’s background, the context in which disfluency appears, and the possible factors behind the disorder. This broader perspective can guide researchers and developers in building more useful tools for people who stutter and in exploring areas of stuttering research that remain underdeveloped.

Modern stuttering-detection systems build on pre-trained Automatic Speech Recognition (ASR) models. Whisper is a transformer-based encoder–decoder ASR system trained on 680,000 h of weakly supervised multilingual data [2], and it is considered as a family of models (Tiny to Large). Wav2Vec2.0, by contrast, is a self-supervised model that learns contextualized representations directly from raw audio waveforms [4]. Both architectures have become common backbones for downstream speech tasks, including disfluency and stuttering detection, and they form the two model families compared in this study.

These backbones can be specialized to new languages and tasks through fine-tuning. Open toolkits such as the Whisper fine-tuning framework [5] streamline adapting the pre-trained Whisper models to specific languages and datasets, supporting scalable deployment. Work targeting Arabic speech disorders, however, remains scarce. Boughariou et al. [6] propose a machine-learning-guided framework for detecting speech disorders in spontaneous Tunisian-dialect speech, combining rule-based preprocessing with classifiers that handle code-switching between Modern Standard Arabic and French. Their study is one of the few addressing an Arabic dialect, but it does not employ ASR-based representations and is restricted to Tunisian speech.

Both backbones have been adapted for stuttering and related tasks, though with different trade-offs. Wav2Vec2.0 yields powerful self-supervised features but is comparatively demanding to fine-tune. Vaessen and van Leeuwen, for instance, adapt it to speaker recognition [7]. Whisper is cheaper to specialize, because much of its pre-trained encoder can be kept frozen while only the upper layers and a small head are trained. This selective layer-freezing strategy preserves accuracy while cutting the number of trainable parameters [8], and layer-wise analyses of the Whisper encoder show that its deeper layers carry the disfluency-related information that such adaptation should target [9]. Beyond the choice of backbone, performance also depends on careful audio preprocessing, analog-to-digital conversion, fast Fourier transform and Short-Time Fourier Transform (STFT) analysis, Mel-Frequency Cepstral Coefficient (MFCC) extraction, and denoising [10], where Wiener filtering followed by MFCC extraction is a common front-end for noisy speech [11].

Not all stuttering-detection systems rely on ASR backbones. A parallel line of work uses dedicated acoustic architectures. FluentNet [12] bypasses ASR entirely, operating on STFT-based spectrograms with SE-ResNet and bidirectional long short-term memory layers and attention to detect multi-label disfluencies across datasets, while a two-stage deep neural network classifier combines MFCC features with disfluent-segment localization [1]. Closest to the present work, Whister [13] builds on frozen Whisper encoder outputs by adding a feature-extraction block and classification heads, reporting up to 22% higher F1-scores than FluentNet and StutterNet on benchmark datasets. These results demonstrate the value of reusing robust pre-trained ASR encoders for disfluency classification, an idea we extend to Arabic, where phonetic and dialectal variation calls for dedicated treatment.

In [14], Al-Banna et al. conducted a thorough analysis of the application of various machine learning classifiers, focusing on the automated detection of stuttering disfluencies. The analysis relied on two publicly accessible benchmark datasets: FluencyBank and SEP-28k. The classifiers were evaluated for their proficiency in detecting speech disfluency. Classifiers such as Random Forest and Support Vector Machine were assessed across a comprehensive set of metrics, including accuracy, precision, recall, F1-score, and area under the curve. In this class, a Random Forest classifier achieves 50.3% accuracy on FluencyBank and 50.35% on SEP-28k. There is moderate performance across all other metrics, suggesting a range of issues, including technological problems with traditional machine learning approaches. The authors attributed these problems and barriers primarily to factors such as speech rate, oral cavity configuration, and age variability, which impact the cohesiveness of machine learning classification. They suggested that the use of ASR is considerably augmented by Deep Learning (DL), which they described as the counter-deep-learning paradigm that merges traditional DL approaches and would significantly enhance the underlying detection precision of profoundly skewed speech datasets. This indicates that the sophisticated machine learning techniques for future speech data remain to be unveiled.

Additionally, ref. [15] provided useful information on the use of AI and computational intelligence within the scope of automatic stuttering detection. This systematic review describes 14 publications in the journals concerning computational phonetics and stuttering, defining databases crafted in the timeframe 2019–2023. The review suggested the increasing use of AI, machine learning, and DL in feature classification of stuttering disfluency. This study highlights the challenges of manual assessment and practical limitations, as well as the possibility of human error, and for that reason, we emphasize the importance of automated systems. In addition, the review highlighted several challenges in this domain, including the scarcity of stuttering datasets, imbalance in data distribution, variation in disfluency types, and the difficulty of distinguishing stuttering-related disorders from normal speech disfluencies. Most existing studies focus on English datasets, while Arabic stuttering data remains very limited. This lack of data makes it difficult to conduct analyses and identify the most suitable methods for studying and detecting stuttering in Arabic speech.

In addition, the authors of [16] developed a pipelined Deep Learner-Dual Classifier (PDL-DC) to evaluate speech fluency. The approach merges a DL architecture comprising convolutional auto encoders with super learner models to detect and classify disfluencies and fluency disorders like stuttering, specific language impairment, and normal speech. Using the UCLASS dataset [17] for evaluation, the model produced satisfactory results with an average accuracy, precision, and recall of 97% and above. The researchers also addressed the difficulty in distinguishing similar fluency disorders through detailed acoustic and glottal feature analysis, showing the model’s significant improvement over traditional methods. This framework thus equips clinicians with another automated and trustworthy mechanism for swift and accurate speech impairment evaluations that can be added to existing methods to improve patient outcomes.

Use cases for speech disfluency detection are drawn from the research in [18]. To address the lack of annotated datasets, the authors apply a set of methods such as balancing the class distributions, filtering for higher inter-annotation agreement, and using Wav2Vec2.0 embeddings extracted from a contextual representation. They designed a DL architecture called DisfluencyNet that takes advantage of these contextual embeddings for classification.

These findings proved that DisfluencyNet is able to surpass the previous baseline results for different sets of speech disfluency detection techniques with as low as 25% of the SEP-28k training dataset in terms of delta. Most of the disfluency types reached 0.7 for the F1 scoring threshold. Also, the model is robust with respect to generalization for another dataset, FluencyBank, which showed the model is still able to retain its main functionality. The research highlights the problem of low-quality data annotations and suggests that powerful classifiers can be trained on meticulously assembled datasets. With such classifiers, the technology can support users with speech disfluencies.

Alongside speech-centric stuttering detection systems, new studies have looked into how AI-integrated writing aids can help People Who Stutter (PWS) cope with stuttering-induced communication anxiety. For instance, Fluent is an intelligent script-assistance tool [19], that helps users eliminate words that they may have challenges pronouncing. It features an active learning-based classifier that learns through user active engagement, which highlights words that may pose challenges, along with easier phonetic, yet synonymous alternative words. The aim of Fluent is to reduce the emotional and mental effort caused by word substitution. PWS often use this strategy to avoid difficult words, but it can become tiring, especially in stressful situations. In the first 20 interactions, Fluent reached a mean accuracy of 80% in identifying difficult words for simulated users. Although the system focuses on written preparation rather than real-time speech, it still shows how AI can support PWS when preparing for presentations, meetings, or other stressful communication tasks.

In various languages and contexts, multiple datasets have been constructed to facilitate research in speech processing and disfluency detection. For instance, LibriStutter provides audiobook speech recorded by fluent English speakers, which is primarily used for ASR tasks [12]. There are also the UCLASS and FluencyBank datasets which include conversational transcription data of English-speaking adults and children who stutter. The SEP-28k dataset consists of English podcast recordings used for stuttering event detection and lacks transcription [20]. The KSoF dataset is composed of spontaneous and reading speech samples in German recorded from speakers who stutter and is also devoid of transcriptions [21]. There is also the AS-70 dataset [22], which comprises adults who stutter in Mandarin and is complete with conversational and voice command tasks prescribed with detailed verbatim transcription.

There are notable gaps in the current literature. LibriStutter, UCLASS, FluencyBank, and SEP-28k target English, KSoF targets German, and AS-70 targets Mandarin, while only DisCoTAT [23] covers Arabic, and only the Tunisian dialect. However, our work covers a 20-hour, clinically curated, multi-dialect Arabic stuttering corpus covering Modern Standard Arabic plus the Gulf, Egyptian, and Maghrebi dialect families, and including children, teens, and adults. In addition, Whister uses a frozen Whisper encoder with a classification head but is evaluated only on English, and FluentNet bypasses ASR entirely. The Random Forest/SVM study, the PDL-DC pipeline (UCLASS), and DisfluencyNet (SEP-28k) all rely on traditional or non-ASR classifiers on English data. This paper adapts the encoder-freezing plus lightweight classification-head paradigm specifically to Arabic, with Arabic-tuned preprocessing and a Whisper-assisted, human- and clinician-verified annotation pipeline to handle low-resource label scarcity. Moreover, no prior systematic comparison of the full Whisper family against the full Wav2Vec2.0 family on Arabic stuttering. Our study presents the first controlled Whisper (Tiny–Large) vs. Wav2Vec2.0 (Base–XLarge) benchmark for Arabic stuttering classification under identical preprocessing, data splits, and evaluation metrics.

3. The Proposed Method

The targeted AI-based system for classifying stuttering in Arabic speech is accomplished using the Whisper ASR model built by OpenAI (large-v2; available online: https://github.com/openai/whisper, accessed on 7 March 2025). The system classifies each Arabic speech segment as fluent or disfluent, where disfluent segments may contain stuttering intervals, filler words, or prolongations. The pipeline is illustrated as a linear process from the collection of raw audio files containing both stuttering and fluent speech to final model building. In the next phase, under data preparation, audio files containing stuttering are processed using fluent-speech components. This includes cleaning, normalizing and shaping the audio files. The next step is called processing with noise and resampling. The prepared data is employed for fine-tuning the Whisper model for stuttering detection and transcription. Finally, model performance is evaluated with performance metrics F1-score and Word Error Rate (WER), respectively, while the confusion matrix was also considered for misclassification analysis.

Whisper has not been explicitly trained on the nuances of stuttering speech. Thus, fine-tuning is essential to adapt it for this task. This can be done using the ‘freeze and unfreeze’ technique for layers during training, as depicted in Figure 1. During the first pass, the essential ASR layers are frozen to maintain the model’s overall transcription functionality while the other layers are trained on Arabic stuttering speech. This way, overfitting is minimized, and the model learns to identify stuttered disfluencies without failing to recognize fluent speech [8]. The early encoder layers, which capture general acoustic and phonetic representations, are frozen to preserve Whisper’s transcription capability, while the later layers are unfrozen and adapted to Arabic disfluency detection.

Furthermore, by changing Whisper using progressive layer freezing and unfreezing, a lightweight classification head is added on top of Whisper’s final hidden representations, as shown in Figure 2. The head enables the system to make fluent–disfluent decisions for each audio chunk. Concretely, the encoder’s final hidden states are aggregated by temporal mean-pooling into a single utterance-level embedding, which is passed through a dropout layer (rate 0.1) and a single fully connected linear layer that projects to two output units (fluent, disfluent) followed by a softmax. The model is trained with cross-entropy loss. The head is deliberately lightweight, adding only on the order of $d\times 2$ parameters (where d is the encoder hidden width) on top of the pre-trained encoder, so that the fluent–disfluent decision reuses Whisper’s pre-trained representations rather than replacing them.

The layer, width, head, and parameter counts for each Whisper variant are summarized in

Table 1. Whisper architecture details.

Model	Tiny	Base	Small	Medium	Large
Layer	4	6	12	24	32
Width	384	512	768	1024	1280
Heads	6	8	12	16	20
Parameter	39 M	74 M	244 M	769 M	1550 M
Trainable (≈)	∼4 M	∼10 M	∼44 M	∼155 M	∼320 M

[2]. Because only the unfrozen upper layers and the classification head are updated, the number of trainable parameters is substantially smaller than each model’s nominal size reported in Table 1. The classification head itself adds only $2d+2$ parameters per model (for example, 2562 parameters for Whisper-Large, whose hidden width is $d=1280$), so the added capacity is negligible relative to the pre-trained backbone.

We monitored the results to see whether the model was starting to overfit. This was important because the model needed to work on recordings made under different conditions, not only on the training data. The same validation checks also gave us an indication of training stability, as shown in Figure 3.

During training, additional layers are thawed, enabling the model to grasp better patterns related to stuttering. This incremental modification guarantees that Whisper continues to accurately recognize overlapping Arabic speech while enhancing its efficiency in disfluency categorization. However, the Whisper architecture specifics depend on the model family. Whisper’s shard architecture enables the model to be optimized for classifying Arabic stuttering by performing coarse fine-tuning. Substantial supervised training is sidestepped while precision is still enhanced.

For the stuttering detection, the current model is set in place with a single-label binary classification framework, where each audio clip is tagged as either fluent or disfluent. This embodies the first phase in the development pipeline, as presented in Figure 2, and thus the model is able to interface with and recognize normal speech and stuttered speech at the segment level.

Since there was an inadequate subset of the dataset class imbalance (e.g., in the different types of disfluencies such as blocks), the dataset was initially categorized using single-label binary classification, fluent vs disfluent. Speech feature authenticity was maintained to ensure the evaluation was stuttering-specific. Therefore, the application of the Synthetic Minority Over-sampling Technique (SMOTE) was avoided [24]. Moreover, we omitted cluttering from the model with full awareness of the fact that this type of disfluency is stuttering and cluttering, which intersects in behavior modification techniques like slowing down speech. Cluttering is an example that is unconditioned by the classifier techniques. This range of factors determined the approach to the initial implementation of binary classification, while keeping in mind that the model would soon be expanded to multi-class classification once the additional data gathered was more balanced.

For comparison, we also applied the same methodology to Wav2Vec2.0 and adapted it to the fluent/disfluent classification task. Wav2Vec2.0 processes the raw audio waveform through a convolutional feature encoder, which produces latent speech representations. These representations are then passed to a multi-layer transformer network to capture contextual information. During self-supervised pre-training, the model uses quantization and a contrastive learning objective to learn useful speech representations from unlabeled audio [4]. Unlike Whisper, which takes log-Mel spectrograms as input and uses an encoder–decoder architecture, Wav2Vec2.0 works directly with raw audio waveforms and does not include an autoregressive text decoder. We used the same adaptation setup for both backbones as for Whisper, so that the two models could be compared under similar conditions. We froze the lower convolutional and transformer layers to keep the pre-trained speech features stable, and fine-tuned the upper layers for the stuttering classification task. A small classification head was then added to the output of each model, using temporal mean pooling, dropout, and a final linear softmax layer. This setup helps ensure that the comparison mainly reflects the difference between the backbones, rather than differences in the adaptation method.

4. Experimental Evaluation

4.1. Data

Data collection involved a total of 20 h of stuttering and non-stuttering speech samples obtained from the Medical Charitable Society in Saudi Arabia, as shown in

Table 2. Quantitative dataset summary.

Attribute	Details
Speakers	48 (22 Male, 26 Female)
Audio Clips	7200
Fluent Clips	3600
Disfluent Clips	3600
Average Clip Length (sec)	10
Age Groups	Children, Teens, Adults
Dialect Coverage	Gulf, Egyptian, Maghrebi
File Formats	wav, mp3, m4a

. The audio samples were obtained from native speakers of Arabic with varying degrees of speech stuttering. The dataset prepared for this study follows the SEP-28k framework to keep consistency during partitioning and assessment of the dataset [20]. This study also simulates the AS-70 dataset’s construction, which proved to be more effective for stuttering tasks in order to improve precision [22].

To describe the diversity of the dataset, the speakers were grouped according to dialect and age, as summarized in

Table 3. Dataset composition by dialect and age group.

Category	Speakers	% of Speakers
Dialect
Gulf	20	42%
Egyptian	18	37%
Maghrebi	10	21%
Age group
Children (6–12)	14	29%
Teens (13–17)	10	21%
Adults (18+)	24	50%
Total	48	100%

. To ensure diversity in the dataset, age and dialect were considered during data collection. Speaker counts were recorded for each group. However, clip-level counts for each age or dialect group were not separately collected.

Audio preprocessing is carried out to provide the clean, uniform input required for stuttering detection that is compatible with Whisper’s inputs. Noisy recordings were first denoised with a Wiener filter and resampled to 16 kHz. Multi-speaker recordings were excluded to retain single-speaker data, which improves model quality. Long files were split into overlapping 10 s segments and normalized to a uniform volume level. Clips shorter than three seconds were discarded, and the remaining shorter clips were zero-padded to the fixed 10 s window before feature extraction. The small, medium, and large model variants were subsequently evaluated to characterize the effect of model size on performance.

Each 10 s segment was labeled as fluent or disfluent by three native Arabic-speaking annotators who independently listened to the audio, so labels were based on the acoustic signal rather than on any automatic transcription. A clinician verified the annotations and adjudicated all disagreements to produce the final labels, and inter-annotator agreement was substantial (≈80–90%). The Whisper-Large model (the multilingual openai/whisper-large-v2 checkpoint) [2] was used only to generate a first-pass Arabic transcription, which annotators corrected to retain all disfluencies (repetitions, fillers, prolongations). It did not assign the fluent/disfluent labels. Each entry was stored in JSON format associating every audio path with its transcription and label.

To prevent speaker leakage, partitioning was performed at the speaker level. All clips from a given speaker were assigned to exactly one of the training, validation, or test sets (38/5/5 speakers, ≈80%/10%/10%), following the SEP-28k partitioning strategy, as detailed in

Table 4. Speaker-independent train/validation/test split by dialect.

Dialect	Total	Train	Val	Test
Gulf	20	16	2	2
Egyptian	18	14	2	2
Maghrebi	10	8	1	1
Total	48	38	5	5

. Quality-control filtering was then applied at the clip level (removing corrupted, silence-only, or low-quality segments), retaining 680 clips for the final evaluation. After this filtering, the retained clips are no longer evenly balanced and contain a higher proportion of fluent speech, which is why F1, rather than accuracy, is used as the primary evaluation metric. This speaker-independent protocol guarantees that no speaker appears in more than one split. Additional configuration and data-processing steps were applied consistently across all Whisper models (Tiny, Small, Medium, and Large) to allow comparative assessment of accuracy, convergence, and computational efficiency.

4.2. Evaluation Criteria

The F1-score was the primary evaluation metric for the model as it best captures the extent of class imbalance for stuttering datasets characterized by the dominance of fluent speech. Unlike accuracy, which is sensitive to the most numerous class, F1 captures the balance between precision and recall, making it more useful for evaluating performance on the stuttering events, which are seldom. It captures the model’s capability for disfluency detection without regard to the class of error and without bias. The WER metric was also included to assess the quality of transcription as explained in [2], which showed that decreasing WER improves the real-world usability of ASR by enhancing intelligibility and reducing errors. Both metrics were calculated from the same fine-tuned model. For each variant, fine-tuning was done once, so the speech representations used for fluent/disfluent classification were also the ones used when reporting transcription performance.

In order to enhance the reliability of the results and the effects contributed by random initialization and data shuffling, each model configuration was subjected to five independent training runs, and the F1-scores and WER that are reported for the model are the averages across these runs. This way the results have increased statistical validity, and this is a more conservative estimate of the model’s generalizability.

4.3. Competing Methods

The fine-tuned comparison of Whisper of OpenAI (from Tiny to Large) and Wav2Vec2.0 (from Base to XLarge) ASR models is the focus of this paper. These models were selected for their prowess in low-resource multilingual speech recognition and provide a deep comparison of architecture efficiency and scalability, along with precision in Arabic speech disfluent detection.

4.4. Implementation

To make the training process more efficient and maintain pre-existing linguistic competencies, the first half layers of the Whisper variants were “frozen” and not updated during fine-tuning. This layer-freezing strategy reduced training time while mitigating catastrophic forgetting, enabling the upper layers to focus on mastering disfluency detection. Audio was split into 10 s segments and a length was chosen to keep enough temporal context to capture disfluent speech. To simplify batch processing and improve model convergence, spectrogram features were fixed to (80, 3000) by padding or truncation.

After feature extraction, a custom collation method was developed to handle the variable-length inputs used in training. This method used Whisper’s built-in tokenizer and feature extractor, masking padded label tokens (assigned the value −100) to exclude them from the loss computation and reinforce precise autoregressive decoding. Whisper variants were fine-tuned with Hugging Face’s Seq2SeqTrainer, following the fine-tuning recipe of [5]. Whisper uses a sequence-to-sequence transformer architecture. The encoder converts the log-Mel spectrograms to embedding, while the decoder transcribes the embedding into text. This encoder-centric design, illustrated in Figure 4, is the structural backbone reused for all Whisper variants evaluated in this study. For the first round of iteration, Whisper-Tiny was used due to its 39 M parameter count and 4 encoder and decoder layers. This thin model is suitable for preliminary tests, as it achieves high efficiency in resource-constrained environments. Higher accuracy was achieved with the more complex models, though at a higher computational cost. Training combined progressive freezing of encoder layers, dataset-specific preprocessing, mixed-precision training, early stopping, and periodic checkpointing [24].

As a comparison baseline for the Whisper experiments, Wav2Vec2.0 was also fine-tuned, starting from an Arabic-optimized pre-trained checkpoint. Its architecture and the matching freeze-and-head adaptation are described in Section 3. The Base-to-XLarge variants were trained under identical preprocessing, data splits, and evaluation metrics as the Whisper variants, enabling a controlled comparison of model appropriateness for Arabic stuttering detection.

All models were implemented in the PyTorch (version 2.12.1) framework and trained on a single NVIDIA A100 GPU within the providing institution’s secure in-house environment. The 20-epoch figure is an upper bound. Because early stopping on validation loss was applied, individual runs typically halted well before this ceiling, and the use of frozen lower layers further reduced the per-run cost. Each configuration was trained five times so that results could be reported as mean ± standard deviation, isolating the variance due to random initialization and data shuffling rather than implying a large compute budget. The complete set of runs was carried out within the available A100 allocation. Checkpointing was enabled to retain the best-performing configuration throughout training. The complete pipeline of our proposed method, from initial format standardization and audio preprocessing through transcription, manual annotation, fine-tuning, dataset construction, and comprehensive evaluation, is illustrated clearly in Figure 5.

4.5. Parameters Settings and Tuning

To optimize GPU memory usage, the Whisper model was trained with cross-entropy loss and the base penalty function for incorrectly identified stuttered speech segments. The learning rate for Whisper fine-tuning was set to $2.5\times {10}^{-5}$, as recommended in the model fine-tuning research. Overfitting was taken care of with early stopping of the training, which was done through the monitoring of validation loss and stopping once the performance gains stopped improving [5].

Fine tuning Wav2Vec2.0 in

Table 5. Comparison parameters.

Parameter	Whisper	Wav2Vec2.0	Notes/Remarks
Batch Size	32	16	Whisper allowed larger batch size due to lower GPU memory consumption compared to Wav2Vec2.0.
Learning Rate	$2.5\times {10}^{-5}$	$3.0\times {10}^{-5}$	Slight difference reflecting model-specific recommendations.
Loss Function	Cross-Entropy Loss	Cross-Entropy Loss	Identical choice for both models.
Optimizer	Not explicitly stated	AdamW (Adam + Weight Decay)	Wav2Vec2.0 explicitly used AdamW, while Whisper optimizer choice was not detailed explicitly.
Learning Rate Scheduler	Not explicitly stated	Linear with warm-up (10%)	Wav2Vec2.0 explicitly mentioned scheduler. Whisper did not explicitly specify a scheduler.
Dropout Rate	0.1	0.1	Dropout (rate 0.1) applied in the classification head of both models.
Early Stopping	Yes (Validation Loss)	Yes (Validation Loss)	Identical early stopping criteria.

was done with the set of parameters configured based on the recommendations of the past research tailored to the speech recognition of Arabic. Complex—in relation to Whisper and higher GPU memory needs—was fine-tuned with a batch size of 16 [25]. Based on Wav2Vec2.0’s fine tuning recommendations, the learning rate of $3.0\times {10}^{-5}$ as set according to Adam Weight decayed the optimizer [7]. Subsequently, with the designated proportions of linear decay for the total training steps, the total training steps were warmed with stable convergence set to 10 for the total training steps [26]. The end-to-end training and tuning pipeline, covering data preparation, audio processing, representation learning, transfer learning, fine-tuning, and evaluation, is depicted in Figure 6.

To mitigate potential overfitting, a dropout layer (rate 0.1) was applied in the classification head, and early stopping was activated by continuously monitoring validation loss, halting training when performance gains ceased. Mixed-precision training (FP16) [27] was also employed to optimize training efficiency and reduce GPU resource demands, enabling effective management of the model’s substantial computational load.

4.6. Experimental Results

In the initial experiment, we manually analyzed the confusion matrix generated by the largest variant of each model, as shown in Figure 7 for Whisper and Figure 8 for Wav2Vec2.0. The results demonstrate that Whisper-Large consistently outperforms Wav2Vec2.0-XLarge across all categories by correctly identifying both fluent and disfluent speech while also minimizing false positives and false negatives, highlighting its superior overall classification performance.

As shown in

Table 6. Performance of Whisper-Large vs. Wav2Vec2.0-XLarge.

Model	Precision	Recall	F1-Score	WER Approx.
Whisper	0.80	0.81	0.81	14.96%
Wav2Vec2.0	0.76	0.78	0.77	19.95%

, Whisper-Large outperformed Wav2Vec2.0-XLarge across all evaluation metrics, achieving higher precision (0.80 vs. 0.76), recall (0.81 vs. 0.78), and F1-score (0.81 vs. 0.77), along with a significantly lower WER (14.96% vs. 19.95%). These results highlight Whisper-Large’s superior balance between classification accuracy and transcription quality, which is especially critical in detecting disfluencies within Arabic speech. Its encoder–decoder transformer architecture demonstrated greater robustness in managing the acoustic and linguistic variability of Arabic, making it more suitable for practical, real-world deployment scenarios.

Because real-world clinical data are often class-imbalanced, we additionally report two threshold-independent metrics in

Table 7. Threshold-independent performance (ROC-AUC and PR-AUC) on the test set.

Model	ROC-AUC	PR-AUC
Whisper-Large	0.90	0.86
Wav2Vec2.0-XLarge	0.85	0.81

, the Area Under the Receiver Operating Characteristic Curve (ROC-AUC) and the Area Under the Precision–Recall Curve (PR-AUC). These values preserve the ranking observed for F1: Whisper-Large attains a ROC-AUC of 0.90 and a PR-AUC of 0.86, compared with 0.85 and 0.81 for Wav2Vec2.0-XLarge. This indicates that Whisper maintains better separability between fluent and disfluent speech across decision thresholds, which is desirable for deployment scenarios where the operating point may be tuned to clinical needs.

To assess performance, we calculated the average F1-score and WER, along with standard deviations, across all variants of Whisper and Wav2Vec2.0, as presented in

Table 8. Performance metrics comparison.

Model	Avg. WER (%) ± StdDev	Avg. F1-Score ± StdDev
Whisper-Tiny	23.85 ± 1.91	0.63 ± 0.05
Whisper-Base	21.02 ± 1.75	0.67 ± 0.04
Whisper-Small	18.34 ± 1.43	0.70 ± 0.03
Whisper-Medium	16.41 ± 1.12	0.74 ± 0.03
Whisper-Large	14.96 ± 0.95	0.81 ± 0.02
Wav2Vec2.0-Base	25.45 ± 2.10	0.62 ± 0.04
Wav2Vec2.0-Large	22.18 ± 1.70	0.76 ± 0.03
Wav2Vec2.0-XLarge	19.95 ± 1.39	0.77 ± 0.03

. The results show a clear upward trend in performance with increasing model size for both architectures. However, Whisper consistently outperformed Wav2Vec2.0 at every scale. In addition to evaluating overall performance, we also examined the smallest variants of each architecture, Whisper-Tiny and Wav2Vec2.0-Base, to assess their effectiveness under limited computational resources. Whisper-Tiny achieved an average WER of 23.85% ± 1.91 and an F1-score of 0.63 ± 0.05, while Wav2Vec2.0-Base recorded a slightly higher WER of 25.45% ± 2.10 and a lower F1-score of 0.62 ± 0.04. Although both models showed reduced performance compared to their larger counterparts, Whisper-Tiny still maintained a consistent edge in both transcription accuracy and stuttering detection. This suggests that even at the smallest scale, Whisper provides a more efficient and reliable solution for Arabic speech disfluency classification, making it a more suitable choice for low-resource deployment scenarios.

To assess whether performance is uniform across Arabic dialects, we partitioned the test set by each speaker’s dialect and recomputed the metrics, as reported in

Table 9. Per-dialect classification performance (precision/recall/F1) on the test set.

Dialect	Test Spk.	Whisper-Large	Wav2Vec2.0-XLarge
Gulf	2	0.82/0.83/0.82	0.78/0.79/0.78
Egyptian	2	0.81/0.81/0.81	0.77/0.78/0.77
Maghrebi	1	0.76/0.77/0.76	0.72/0.73/0.72
Overall	5	0.80/081/0.81	0.76/0.78/0.77

. Whisper-Large outperformed Wav2Vec2.0-XLarge within every dialect family. Both models achieved their strongest results on Gulf and Egyptian speech and showed a consistent decline on Maghrebi, the lowest-resourced dialect in our corpus. We emphasize that the speaker-independent test partition contains only five speakers (two Gulf, two Egyptian, one Maghrebi). The Maghrebi figures in particular reflect a single speaker and should be interpreted as indicative rather than conclusive. A larger, dialect-stratified evaluation is part of our planned future work.

Each disfluency type presents its own challenges, but silent blocks were the most difficult to detect. In some cases, blocks appeared as silent intervals that were hard to distinguish from natural pauses or hesitation. This often led the model to classify them as fluent speech, resulting in false negatives. This problem comes from relying only on the audio signal. When a stuttering block appears as a short silence, Whisper has little information to separate it from a normal pause within a 10 s clip. To handle this better, future work should label blocks in more detail, including where the silence starts and where speech returns. It should also examine acoustic cues around the silence, such as voice-onset time and tension-related measures [17], to help distinguish real blocks from ordinary pauses. This will be addressed in future work on disfluency-type classification.

5. Discussion

We tested Whisper for Arabic fluent/disfluent stuttering detection by freezing the encoder and training only a lightweight classification head. To make the comparison fair, Wav2Vec2.0 was evaluated on the same corpus, with the same preprocessing, speaker-independent split, and evaluation metrics. On the speaker-independent test set, Whisper-Large achieved an F1-score of 0.81 and a ROC-AUC of 0.90, as reported in Table 6 and Table 7. The results also show that the difference was not only a matter of model size. Whisper-Tiny still performed better than Wav2Vec2.0-Base, which suggests that Whisper’s pre-trained encoder was better suited to this task. At the dialect level, Gulf and Egyptian Arabic showed similar performance, whereas Maghrebi Arabic performed worse. This result is expected given the smaller number of Maghrebi samples in the training data, and it should therefore be interpreted with some caution, as shown in Table 9.

5.1. Comparison with Prior Work

Researchers often reused a pre-trained ASR model and added a small classifier for the final fluent/disfluent decision. Whister reported strong results with this approach on English data [13], and our Arabic results follow a similar pattern. FluentNet takes a different route and works on spectrograms without ASR [12]. Random Forest/SVM [14], PDL-DC [16], and DisfluencyNet [18] were mainly tested on English corpora. Here we fine-tune a multilingual ASR encoder for Arabic speech. Many published systems target English or German, and the main Arabic-related corpus in this line of work, DisCoTAT, is limited to Tunisian speech [23]. We ran Whisper from Tiny to Large and Wav2Vec2.0 from Base to XLarge on the same multi-dialect Arabic data using the same pipeline. This side-by-side comparison has not been reported in this form before, and it can support follow-up work on Arabic stuttering.

5.2. Clinical Implications

Clinically, we do not see this system as a replacement for the speech-language pathologist. Its value is in screening and triage. It can flag the parts of a recording that are more likely to contain disfluency, so the clinician can review those segments first instead of listening to the full session from beginning to end. This can save time while keeping the final judgment in human hands. The ROC-AUC score of 0.90 supports this use case, indicating that fluent and disfluent clips are still separated fairly well, even when the decision threshold changes. This gives clinics some flexibility. For early screening, the threshold can be adjusted to catch more possible disfluent cases. For confirmation, it can be adjusted to reduce false positives. Whisper-Tiny is a good practical choice for clinics that do not have access to high-end GPUs such as the A100. The system was designed to support clinicians during assessment, not to make decisions on their behalf.

5.3. Limitations

This study has several limitations that should be considered when interpreting the results. First, the clips were labeled only as fluent or disfluent. We did not label repetitions, prolongations, and blocks separately. For this reason, the model can recognize that a clip is disfluent, but it cannot tell which type of disfluency occurred. Another limitation is the size of the speaker-independent test set. It included only five speakers, and only one of them was Maghrebi. For this reason, the dialect-level results, especially for Maghrebi Arabic, should be treated as indicative rather than conclusive. Another limitation concerns data access. The recordings were collected by a single institution under a data-use agreement, so the dataset cannot be made publicly available. Although the code can be shared, other researchers will not be able to train or reproduce the experiments using the same audio files. In addition, all experiments were run on a single NVIDIA A100 GPU, and we did not measure inference speed, memory use, or power consumption on phones or edge devices. The training setup was also kept fixed: we used an 80/10/10 split, 10 s clips, and early stopping with a capped number of epochs. We did not try other split ratios, clip lengths, or epoch limits, nor did we run ablation experiments to measure the effect of each choice separately. Because of this, the reported results should be viewed as the outcome of this specific setup rather than as a general estimate for all possible configurations.

5.4. Future Work

The next stage of this work should focus on four areas. First, the task needs to move beyond the current fluent/disfluent label. The model should be extended to recognize specific types of disfluency, such as repetitions, prolongations, interjections, and blocks. This can start with the subset of the data that already has partial type labels. Blocks should be given priority, since they were one of the main sources of error in the current results. Second, the annotation process should be made more systematic. The Whisper-assisted transcription and clinician review steps should be written as a fixed protocol, with clear rules for each disfluency type and regular checks for inter-annotator agreement. Active learning could also help here by pointing annotators to the clips where the model is least confident, so their time is spent on the most informative examples. Third, the dataset should be expanded. More speakers are needed, especially Maghrebi speakers, and broader age-group coverage is needed. This would make the dialect-level evaluation more reliable and help show how well the system performs across different Arabic-speaking populations. Fourth, the system should be prepared for practical deployment. Small Whisper models can be quantized for low-power devices, the classifier can be integrated into a therapy or screening application, and the full pipeline can be tested with clinicians in real use. These steps are needed before the system can move from a lab benchmark to a routine assessment of Arabic stuttering.

6. Conclusions

This paper has developed a specialized AI-based tool for Arabic stuttering classification in response to the absence of automated tools for detecting speech disfluency in Arabic. Through customization and extensive testing of the leading ASR systems, Whisper and Wav2Vec2.0, across numerous parameters and settings, we learned that Whisper, and more specifically, the large model, substantially outperformed the rest with regard to both transcription and disfluency classification. The system outperformed Wav2Vec2.0 models at all scales, including the most compressed, on F1-score and WER. With the polished and assembled 20 h dataset consisting of fluent and stuttered speech, and with the ASR system’s support, the study advanced speech therapy methods to more advanced systems. This work further confirms the applicability of the ASR approaches and, more particularly, the Whisper encoder–decoder architecture, for clinical, preliminary, and investigational workflows regarding speech disorders in Arabic. This study attempts to create what is possibly the most extensive and clinically relevant Arabic stuttering dataset, given the severe paucity of Arabic disability speech corpora and of the most appropriate of its kind. The speech-language research community has made available a crucial resource to address the multitude of problems and serve as a point of departure for future work in the area, whether in clinical, linguistic, or computational research.

A concrete, staged plan for future work, spanning multi-class disfluency recognition, annotation optimization, dataset expansion and balancing, and edge and clinical deployment, is detailed in Section 5. We stress that the proposed system is designed to assist, not replace, speech-language pathologists, and that any clinical deployment must first be prospectively validated for accuracy, ethical compliance, and user acceptance.