Arab-DF: An Arabic-Media Visual Deepfake Dataset and a Reproducibility-Controlled Cross-Dataset Benchmark

Abstract

Deepfake detectors are rarely evaluated on Arabic-media content. We introduce Arab-DF, an Arabic-media visual deepfake dataset of 1066 authentic and 11,240 manipulated videos covering four manipulation families. Thirteen public detector checkpoints are evaluated zero-shot under a common protocol, and each must first reproduce its DeepfakeBench Celeb-DF-v2 frame-level AUC within ±0.02; eight meet this criterion. On the fixed 824-video subset, their mean video-level AUC falls from 0.820 on Celeb-DF-v2 to 0.552 on Arab-DF (ΔAUC = −0.267), and five have bootstrap 95% confidence intervals above 0.5. When the authentic sample is enlarged from 75 to 392 while retaining 749 manipulated videos, the mean is 0.508 (95% CI [0.478, 0.537]); only EfficientNet-B4 retains a 95% confidence interval entirely above 0.5. Lip synchronization has the lowest family-level point estimate, although intervals overlap. Seven file-level variables separate the classes (out-of-fold AUC 0.895), yet normalization on a 94-video subset leaves mean detector AUC essentially unchanged (ΔAUC = −0.0008; 95% CI [−0.054, 0.042]). Effort reaches 0.695 on the fixed subset but fails the control and is reported separately. Overall, reproducible public checkpoints transfer poorly to this fixed Arabic-media evaluation domain.

1. Introduction

Recent advances in face synthesis have made realistic video manipulation much easier to create using both open-source tools and consumer applications. Face swapping, facial reenactment, and lip synchronization no longer require the specialized infrastructure once needed for high-quality results. This growing accessibility makes it harder to assess the authenticity of visual information in interviews, public communication, television, news, and online media [1,2,3].

Deepfake detection research relies heavily on benchmarks such as FaceForensics++ [4], Celeb-DF-v2 [5], and the DeepFake Detection Challenge dataset [6]. These datasets are valuable, but they were not designed to represent Arabic-media visual domains. Their manipulation methods, subjects, acquisition conditions, and processing pipelines also cover only part of the variation found in real-world media. As a result, strong performance on one benchmark may not transfer to another domain.

Recent cross-dataset and in-the-wild studies confirm this problem: detector performance often drops when the manipulation method, source, or acquisition conditions differ from those seen during development [7,8]. At the same time, Arabic-focused visual deepfake datasets remain relatively limited, especially resources that combine several facial-manipulation families. Existing Arabic or multilingual datasets often focus more on synthetic speech, audiovisual manipulation, lip synchronization, or code-switching [9].

To address this gap, we introduce Arab-DF, an Arabic-media visual deepfake dataset with 1066 authentic and 11,240 manipulated videos. The authentic material comes from publicly accessible Arabic-language media collected under a historical design intended to cover the 22 Arab countries and approximately 450 public figures. The surviving metadata do not contain a complete per-video country field or a deduplicated person-level registry, so both figures are treated as historical collection-design descriptors rather than independently reconstructed counts. The manipulated videos cover four operational families: face swapping, facial reenactment, lip synchronization, and consumer mobile-application manipulation. We do not claim that Arabic language itself causes detector failure. Here, “Arabic-media” describes the source domain from which the videos were collected, not a linguistic input to the detectors: the evaluated models operate only on visual face crops and receive no speech, transcript, or language information. Arab-DF therefore provides a complementary visual test domain that differs from established benchmarks in subjects, visual content, manipulation methods, acquisition conditions, and processing pipelines.

A key challenge in cross-dataset evaluation is separating genuine transfer failure from problems caused by the checkpoint, preprocessing, or evaluation code. To reduce this ambiguity, we combine Arab-DF with a checkpoint-reproducibility control. Each public detector is first evaluated on Celeb-DF-v2, and the resulting frame-level AUC is compared with the corresponding Celeb-DF-v2 (CDFv2) frame-level value reported by DeepfakeBench [10]. Only checkpoints within the predefined ±0.02 tolerance are used for the main cross-domain conclusions.

The main contributions are:

• Arabic-media visual dataset. Arab-DF contains 12,306 videos across four operational manipulation families. It is intended as a complementary benchmark for Arabic-media visual content rather than a claim of priority over all existing Arabic deepfake resources.
• Reproducibility-controlled zero-shot benchmark. We evaluate thirteen public detectors through a common DeepfakeBench-based pipeline. A checkpoint contributes to the main Arab-DF analysis only if it reproduces its Celeb-DF-v2 frame-level reference AUC within ±0.02. Eight of the thirteen checkpoints meet this criterion. To add recent-method coverage without changing the predefined primary benchmark, we also evaluate Effort [11] as a separate exploratory extension under the same control-first principle.
• Cross-domain and manipulation-family analysis. For the eight checkpoint-reproducible detectors, mean video-level AUC drops from 0.820 on Celeb-DF-v2 to 0.552 on the fixed Arab-DF subset. We quantify uncertainty with a class-stratified individual-video bootstrap for both the overall and manipulation-family analyses. Lip synchronization has the lowest mean family-level point estimate, but the family-level bootstrap intervals overlap substantially. We additionally perform an expanded-authentic sensitivity analysis that retains the same 749 manipulated videos and increases the authentic sample from 75 to 392 (1141 videos total).
• Controls for alternative explanations. We perform a fresh ffprobe-based technical-metadata audit on all 824 evaluation videos and run a controlled re-encoding sensitivity analysis on 94 videos arranged into 47 recorded real/fake groups. The recorded pair_id is treated only as an experimental grouping and not as verified authentic-source provenance.

Our goal is not to propose a new detector. Instead, we provide an Arabic-focused evaluation domain and a more transparent way to test how well existing public checkpoints transfer. The results show that strong, reproducible performance on an established benchmark does not guarantee similar performance on the fixed Arab-DF subset.

2. Related Work

2.1. Deepfake Datasets

Deepfake benchmarks have become larger and more diverse over time. Early resources such as DF-TIMIT [12] were relatively small proof-of-concept datasets. FaceForensics++ [4] later became a widely used benchmark with 1000 authentic videos and several manipulation methods. Celeb-DF-v2 [5] focused on higher-quality celebrity deepfakes, while the DeepFake Detection Challenge [6] greatly increased dataset scale. DeeperForensics-1.0 [13] added controlled perturbations and varied acquisition conditions, and KoDF [14] expanded geographic coverage beyond many early benchmarks.

More recent benchmarks focus more directly on cross-domain and real-world generalization. DF40 [8] includes forty deepfake techniques and studies factors that affect detector transfer. Deepfake-Eval-2024 [7] evaluates in-the-wild material collected from 88 websites and 52 languages and reports a marked drop in performance for open-source detectors compared with standard academic benchmarks. These findings show why detectors need to be tested beyond the datasets on which they were originally developed.

The field has also expanded in scale, modality, and task design. Omni-Fake [15], FVBench [16], CoCoVideo [17], ActivityForensics [18], and the MMDF/X-AVDT work [19] explore areas such as multimodal media, commercial generators, large multimodal models, temporal localization, and seen-versus-unseen synthesis. Their goals differ from Arab-DF, which focuses on Arabic-media visual content across several facial-manipulation families.

Deepfake research involving Arabic-language or Arabic-media data is also growing. ArEnAV [9] is a large Arabic-English audiovisual benchmark built around code-switching, dialect variation, synthetic speech, and lip-synchronized video, with monolingual Arabic material as well. Arab-DF targets a different but complementary setting: Arabic-media visual content built from Arabic-language source material, with face swapping, facial reenactment, lip synchronization, and varied consumer mobile-application manipulations. We therefore present Arab-DF as a complementary Arabic-media visual benchmark, not a replacement for existing multilingual or multimodal resources.

2.2. Deepfake Detection Methods

Deepfake detectors rely on different types of visual evidence. Common baselines use convolutional classifiers such as Xception [20] and EfficientNet [21] to distinguish authentic from manipulated face crops. These models do not use an explicit forensic prior, but they can learn spatial textures and appearance cues linked to manipulation.

Other methods use more specialized forensic cues. F3Net [22] combines spatial and frequency-aware information, while SPSL [23] adds phase-spectrum features. SRM-based detection [24] focuses on high-frequency information through constrained filters. RECCE [25] combines reconstruction and classification. CORE [26] promotes consistent representations across augmented views. FFD [27] focuses on informative facial regions. Capsule-Forensics [28] uses capsule representations, while MesoNet [29] offers lightweight baselines.

Several approaches were designed to improve generalization across manipulations or datasets. Face X-ray [30] uses blending-boundary cues, Self-Blended Images [31] creates synthetic examples that expose blending inconsistencies, and DCL [32] uses contrastive learning. UCF [33] separates forgery-irrelevant, method-specific, and common forgery features, while LSDA [34] expands the latent space to reduce dependence on method-specific artifacts. UnivFD [35] and later CLIP-based approaches [36] reflect the broader shift toward large pretrained representations that may transfer better across generators and domains.

This study does not aim to cover every recent detector family. The main benchmark uses a varied set of public checkpoints available through DeepfakeBench. To provide a limited recent-method check, we add Effort [11] as a separate exploratory extension; broader foundation-model, CLIP-based, temporal, and multimodal coverage remains future work and should follow the same reproducibility-control principle.

2.3. Evaluation and Reproducibility

Cross-dataset comparisons are difficult because results can change with the checkpoint, face preprocessing, frame sampling, score aggregation, or evaluation code. A reported performance difference may therefore reflect both genuine domain shift and differences in the evaluation pipeline.

DeepfakeBench [10] was created to standardize major parts of detector training and evaluation and provides a common implementation framework for many widely used methods. We use its preprocessing and evaluation infrastructure and add an explicit checkpoint-reproducibility check before interpreting cross-domain results.

Each public checkpoint is first tested on Celeb-DF-v2 with the adopted pipeline. We compare the reproduced frame-level AUC with an external frame-level reference. A checkpoint is considered reproducible under our protocol when the absolute difference is no more than 0.02 AUC. Only checkpoints that meet this rule are used for the main Arab-DF generalization conclusions. This check does not prove statistical equivalence between implementations or show that every remaining difference is caused by domain shift. Its purpose is simpler: to avoid drawing conclusions from checkpoints that already behave very differently from their reference on the control benchmark.

3. The Arab-DF Dataset

3.1. Overview

Figure 1 summarizes the Arab-DF construction and evaluation workflow, from Arabic-media source material through manipulation generation and manual usability quality control to the final collection and fixed evaluation subset. No identifiable dataset subjects are shown.

Figure 1. Arab-DF construction and evaluation workflow. Counts refer to retained videos; historical generation-attempt and rejection totals were not preserved.

Arab-DF contains 12,306 videos: 1066 authentic and 11,240 manipulated. The authentic material was collected from publicly accessible Arabic-language media, mainly online platforms hosting television programs, interviews, news broadcasts, films, and series. The historical collection design targeted media associated with the 22 Arab countries and approximately 450 public figures. However, the surviving metadata do not preserve a complete per-video country field or a deduplicated person-level identity registry; these figures should therefore be read as historical design estimates rather than independently reconstructed coverage counts. The current release can substantiate the video totals, manipulation-family labels, and dialect-group field described below, but not a verified person-level census of the collection.

The manipulated videos were created from the authentic collection using the methods described in Section 3.3. They cover four operational families: face swapping, facial reenactment, lip synchronization, and consumer mobile applications. Because complete authentic-source linkage cannot be independently verified across the historical collection, the final benchmark does not treat historical source-linkage fields as validated provenance for uncertainty estimation. Figure 2 compares the class and manipulation-family composition of the complete collection with the fixed evaluation subset.

Figure 2. Composition of the full Arab-DF collection (n = 12,306) and fixed evaluation subset (n = 824) across authentic videos and the four manipulation families. Bars show within-set percentages; labels give counts and percentages.

3.2. Data Collection

Authentic videos were selected when they were suitable for facial-manipulation research. Clips needed a clearly visible face for a sufficiently long period and enough visual quality for face detection and alignment. We intentionally kept a range of subjects, recording conditions, and media settings so that the collection reflects varied Arabic-language visual content rather than a single controlled environment.

Candidate clips were reviewed manually. We excluded material with severely degraded quality, no sufficiently visible face, or other problems that prevented reliable facial analysis. Historical collection notes include longer authentic source segments and shorter manipulated exports, but this description does not characterize the fixed 824-video benchmark. In that benchmark, median duration is nearly identical across classes: 15.024 s for authentic videos and 15.000 s for manipulated videos, and duration alone has a direction-free univariate AUC of only 0.543 (Section 5.6). The fixed subset’s measured frame-rate, resolution, frame-count, file-size, and bitrate distributions are reported explicitly in Section 5.6. Videos are stored in MP4 format, and the collection was assembled over several years using heterogeneous acquisition, generation, and export pipelines.

3.3. Manipulation Generation

Manipulated videos were created from the authentic collection with several public research tools and consumer applications. We group them into four operational families according to the main type of facial change:

• Face swapping: FaceSwap, DeepFaceLab [37], and FSGAN [38], which replace the target facial identity with a source identity.
• Facial reenactment: First Order Motion Model (FOMM) [39], which transfers facial motion and expression while keeping the target identity.
• Lip synchronization: Wav2Lip [40], which changes the mouth region to match a driving audio signal while leaving most of the face unchanged.
• Consumer mobile applications: Reface, DiffSnap, WOMBO, MyHeritage, and FaceApp. This group includes proprietary applications that apply face replacement, animation, stylization, or related consumer effects.

This grouping is practical rather than strictly based on underlying model architecture. The mobile-application category, in particular, does not represent one generation method because the internal models are proprietary, not fully observable, and may have changed over time. We therefore treat it as a consumer-tool category rather than a single algorithmic family.

Manipulations were generated between 2022 and 2025. For FaceSwap and DeepFaceLab, models were trained for selected source-target identity pairs. Wav2Lip used driving audio from multiple sources, while consumer applications used the templates, filters, or animation modes available at the time. The surviving traceable information consists of generator/application name, manipulation-family assignment, retained output counts, and limited high-level workflow notes where available. Permanently unavailable historical details include, for many runs, exact software or application versions, repository commits, checkpoints, complete parameter settings, iteration histories, and run-level rejection/regeneration logs. Consequently, the retained collection can be audited at the generator/family level but cannot be regenerated bit for bit. Appendix A and Table A1 explicitly separate retained workflow information from configuration details that were not systematically archived.

3.4. Quality Control

All videos retained in the final Arab-DF collection were manually checked for usability, including all 1066 authentic and 11,240 manipulated videos. We checked that the face was visible enough for analysis, that manipulated regions were reasonably aligned with the underlying face, that temporal behavior remained interpretable, and that severe corruption or rendering failures did not make the video unusable. For Wav2Lip, visible lip-speech synchronization was also checked.

The goal was not to keep only visually perfect deepfakes. We intentionally retained some videos with blending artifacts, temporal instability, or other visible imperfections so that the dataset reflects a realistic range of outputs from both research tools and consumer applications rather than only highly curated examples.

The historical quality-control process has two main limitations. First, the full-collection review was performed by one annotator. Second, per-video checklists, the total number of generation attempts, and complete rejection and regeneration logs were not preserved. Consequently, the original historical rejection rate cannot be reconstructed, and generator counts in Appendix A refer only to videos retained in the final collection. The authentic/manipulated labels themselves are provenance-defined, but the historical usability review may still introduce selection effects.

To provide a retrospective two-rater verification, Rater A and Rater B separately reviewed a stratified sample of 100 retained videos: 20 authentic, 20 face-swapping, 20 facial-reenactment, 20 lip-synchronization, and 20 mobile-application videos. The archived audit files identify the reviewers only as Rater A and Rater B; their relationship to the original historical annotator and their blinding status were not recorded, so we do not claim independent or blinded reassessment. Using the same usability criteria, the raters agreed on the overall retain decision for 97 of 100 videos (97%; Cohen’s κ = 0.653); Rater A retained 97 videos and Rater B retained 94. Exact agreement was 97% for face visibility, 95% for temporal interpretability, and 95% for severe-corruption assessment, with quadratic-weighted κ values of 0.879, 0.525, and 0.593, respectively. The overall retain/non-retain field was a separate holistic usability judgment rather than a deterministic threshold on the three 0–2 component ratings; therefore, a video could be marked non-retain even when no component received a score of 0. Three videos were rated non-retain by both raters, and three additional videos were retain/non-retain disagreements. All six flagged items were manipulated videos, and none belong to the fixed 824-video benchmark, so the reported benchmark results are unchanged. These six items are flagged for curation/adjudication. This retrospective audit verifies a retained sample and does not reconstruct the historical rejection rate or establish agreement for all 12,306 videos.

Table 1 summarizes the four manipulation families and their representation in the fixed 824-video evaluation subset.

Table 1. Manipulation families and composition of the fixed 824-video Arab-DF evaluation subset. The subset contains 75 authentic and 749 manipulated videos; counts refer to evaluation videos, not to the full dataset distribution.

Family	Generators	Manipulation Level	Eval. Videos
Face swapping	FaceSwap, DeepFaceLab, FSGAN	Identity	432
Facial reenactment	First Order Motion Model (FOMM)	Expression	71
Lip synchronization	Wav2Lip	Mouth region	66
Mobile applications	Reface, DiffSnap, WOMBO, MyHeritage, FaceApp	Consumer-oriented	180
Authentic	Arabic-language source videos	-	75

3.5. Comparison with Existing Deepfake Datasets

Table 2 places Arab-DF alongside representative deepfake datasets that differ in scale, modality, language coverage, and manipulation type. Arab-DF focuses on Arabic-media visual content and combines four operational manipulation families. Because the compared datasets were built for different purposes and use different units, the scale values are shown for context rather than as a direct ranking.

Table 2. Arab-DF compared with representative deepfake datasets. Reported scale follows each source and may use different units or construction protocols; the table is intended for qualitative positioning rather than direct scale ranking.

Dataset	Reported Scale	Manipulation Scope	Modality	Language/Domain
FaceForensics++ [4]	1000 source videos + 4000 manipulated videos	Face swap, reenactment	Visual	General face video
Celeb-DF-v2 [5]	590 real + 5639 fake videos	Face swap	Visual	Celebrity videos
DFDC [6]	>100 k clips from 3426 actors	Face swapping/facial manipulation	Video	General
ArEnAV [9]	~387 k audiovisual clips	TTS + lip-sync	Audiovisual	Arabic/Arabic-English
DeepFakeX [41]	200 real + 800 synthetic videos	Multiple audiovisual manipulations	Audiovisual	English, Urdu
Arab-DF (ours)	1066 real + 11,240 fake videos	4 visual manipulation families	Visual	Arabic-language media

3.6. Dialect and Geographic Metadata

The current Arab-DF metadata assign each of the 12,306 videos to one of six coarse dialect groups: Egyptian Arabic, Gulf Arabic, Maghrebi Arabic, Levantine Arabic, Sudanese Arabic, or Yemeni Arabic. No dialect_group values are missing in the current metadata, either in the full collection or in the fixed 824-video evaluation subset. Table 3 reports the distribution in both cohorts.

Table 3. Dialect-group distribution in the full Arab-DF collection and the fixed 824-video evaluation subset. Percentages are within each cohort; no dialect_group values are missing.

Dialect Group	Full Arab-DF, n (%)	Fixed 824 Subset, n (%)
Egyptian Arabic	2850 (23.2%)	137 (16.6%)
Gulf Arabic	2244 (18.2%)	72 (8.7%)
Maghrebi Arabic	2238 (18.2%)	133 (16.1%)
Levantine Arabic	2158 (17.5%)	277 (33.6%)
Sudanese Arabic	1497 (12.2%)	11 (1.3%)
Yemeni Arabic	1319 (10.7%)	194 (23.5%)
Total	12,306 (100%)	824 (100%)

These labels provide descriptive linguistic context only. They were not independently validated as speaker-level linguistic annotations, and the fixed evaluation subset is not balanced across dialect groups. Levantine and Yemeni Arabic are more prominent in the evaluation subset, whereas Sudanese and Gulf Arabic are less represented. We therefore do not perform dialect-stratified detector comparisons or make claims about robustness or fairness across Arabic dialects.

The collection was designed to cover publicly accessible Arabic-language media associated with the 22 Arab countries. However, the current metadata do not preserve a per-video country or source_country field. Reliable country-level frequencies and a per-video country missingness rate therefore cannot be reconstructed. We report the 22-country coverage as a collection-design descriptor and do not retrospectively assign country labels without supporting records. Likewise, the historical estimate of approximately 450 public figures cannot be independently reconstructed from a complete deduplicated person-level registry in the surviving metadata.

3.7. Dataset Partitions and Fixed Evaluation Subset

The surviving records contain historical train/validation/test labels, but these labels are not stable across the archived artifacts. The current metadata release labels all 824 benchmark identifiers as train, whereas an older 5484-video experimental feature manifest records an 824-video test cohort with the same class counts (75 authentic and 749 manipulated). Because no preserved construction log reconciles these historical versions, we do not treat either split label as a stable provenance claim or as evidence of source-video or person-identity disjointness.

Selection audit. The released Arab-DF collection contains 12,306 videos (1066 authentic and 11,240 manipulated). A preserved historical experimental manifest contains 5484 videos (500 authentic and 4984 manipulated), split as 3838 train, 822 validation, and 824 test videos. The 824-video cohort contains 75 authentic and 749 manipulated videos, matching the class counts of the fixed benchmark. However, no preserved selection script or stage-wise audit log explains how the 12,306-video release was reduced to the 5484-video experimental manifest, so the 6822 videos outside that historical manifest cannot be assigned reliable exclusion reasons. Likewise, the 4660 videos outside the 824-video test cohort in that manifest were train/validation members, not quality-control rejections. We therefore do not describe the 11,482 videos outside the fixed benchmark as ‘rejected’ and do not invent stage-specific exclusion counts that were not preserved. The fixed 824 identifiers are treated as a historical evaluation manifest whose selection path is only partially reconstructible; Appendix A.8 summarizes the surviving audit trail.

For this reason, uncertainty on the fixed 824-video subset is estimated with a class-stratified individual-video bootstrap that separately resamples the 75 authentic and 749 manipulated videos. This avoids treating an unverified provenance map as ground truth, but it cannot model possible dependence between manipulated videos that may come from the same authentic source.

Historical split names are therefore not used to define the benchmark semantically. For the thirteen public detectors, the distinction is operationally irrelevant because they use external pretrained checkpoints and receive no Arab-DF training, fine-tuning, threshold calibration, or adaptation before evaluation. The same fixed 824 Arab-DF videos are evaluation data only for every detector in the primary benchmark.

The preserved experimental notes indicate that the fixed cohort was retained for computational feasibility and the availability of valid aligned face crops, but the exact historical ranking or sampling rule was not preserved. The subset was not created by documented probability sampling from the full Arab-DF collection. Reported AUCs and confidence intervals therefore describe this fixed cohort rather than the full set of 12,306 videos. Requiring valid aligned face crops may also favor videos in which the face is easier to detect.

Although the subset is non-random and cannot be considered statistically representative of the full collection, its class and manipulation-family proportions are broadly similar to the corresponding full-collection proportions. Authentic videos account for 9.1% of the evaluation subset versus 8.66% of the full dataset. Among manipulated videos, the evaluation subset contains 57.7% face swapping, 9.5% facial reenactment, 8.8% lip synchronization, and 24.0% mobile-application videos, compared with 55.4%, 9.6%, 9.7%, and 25.3%, respectively, in the full manipulated collection. This similarity does not extend to every attribute: dialect composition differs materially (Table 3), for example, Levantine Arabic is 33.6% of the fixed subset versus 17.5% of the full collection, while Sudanese Arabic is 1.3% versus 12.2%. The fixed subset is therefore not claimed to be representative with respect to dialect, subjects, media sources, technical quality, or other unmeasured factors.

All thirteen detectors use exactly the same 824 video identifiers and the same preprocessed inputs. When UCF and SRM were added during the later checkpoint audit, the existing subset was retained unchanged rather than reselected. Frame-level preprocessing exclusions are reported once in Section 4.3: all 824 videos retained at least one valid crop and therefore received a video-level score.

Expanded-authentic sensitivity cohort. To examine sensitivity to the limited number of authentic videos in the predefined benchmark, we retain the same 749 manipulated videos and add 317 additional authentic Arab-DF videos from a preserved preprocessing archive containing usable aligned face-crop folders. These 317 videos were the additional recoverable preprocessed authentic material available for revision-time evaluation; they were not selected by probability sampling and do not exhaust the remaining authentic videos in the 1066-video collection. The expanded cohort therefore contains 1141 videos (392 authentic and 749 manipulated). Additional authentic videos are used only for evaluation: no detector is retrained, fine-tuned, recalibrated, or selected using these videos. The eight checkpoint-reproducible primary detectors are evaluated on the recovered crops with the same score orientation, up-to-32-frame sampling logic, and mean video-level aggregation used for the primary benchmark. The recovered archive contains aligned face crops in the benchmark input representation, but it does not preserve a complete preprocessing command history that would independently re-audit Dlib extraction for every added video; we therefore do not overstate that provenance. The archive also does not preserve a complete ffprobe technical-metadata audit for the 317 original files. This sensitivity analysis is not a replacement for the predefined fixed 824-video benchmark and is not claimed to be a random, technically matched, or statistically representative sample of the full collection.

Historical-field sensitivity analysis. As an extra robustness check, we tested how the uncertainty estimates change when the historical source-linkage field is used to define bootstrap clusters. This field could not be independently validated as true authentic-source provenance, so it is not used in the main statistical analysis. However, it may still capture some dependence between manipulated videos. We therefore report the earlier clustered-bootstrap results only as a sensitivity analysis. The procedure resampled the 75 historical groups with replacement for 10,000 repetitions (seed 20260721). These intervals are not treated as provenance-validated source-cluster confidence intervals and do not replace the class-stratified individual-video bootstrap used for the main inference.

3.8. Data Governance, Licensing, and Controlled Access

The openly released non-video reproducibility artifacts authored for this study are made available under the Creative Commons Attribution 4.0 International (CC BY 4.0) license unless a specific file states otherwise. This license applies only to original metadata, statistical outputs, documentation, and other non-video research artifacts created by the authors. It does not relicense third-party software, model checkpoints, source media, or any other third-party material, which remain subject to their original terms and rights.

The raw audiovisual files and extracted face crops are not distributed under an open license. They contain identifiable individuals and include third-party audiovisual material for which the authors cannot transfer copyright, publicity, portrait, or other underlying rights. Controlled access, when legally and ethically feasible, is therefore governed by a project-specific Arab-DF Research-Only Data Use Agreement (DUA). The DUA permits approved non-commercial research in deepfake detection, media forensics, robustness, and closely related scientific topics; it does not authorize onward redistribution, public reposting of raw media, commercial exploitation, deceptive impersonation, harassment, or use of the material to develop or deploy identity-targeted synthesis systems. Publication of identifiable frames requires an independent legal basis or permission and is not granted by access to Arab-DF.

Qualified researchers may request controlled access by contacting the corresponding author at sa.bouchareb@univ-bouira.dz from an institutional email address. A request should identify the applicant, institution, and responsible principal investigator; describe the research question and requested subset; explain the intended analyses, storage location, and access controls; and provide ethics approval, exemption, or an institutional statement when such review is applicable. Requests are assessed case by case for scientific purpose, scope, legal feasibility, data minimization, and protection of the identifiable subjects. Approval is not automatic. Before any raw files are supplied, the named research team must accept the DUA, including restrictions on redistribution, access by unlisted personnel, insecure storage, and uses outside the approved project. Where the authors cannot lawfully or responsibly provide a particular source file, access to that file will be denied or limited even if the overall request is otherwise approved.

To improve verifiability of the historical dataset contribution, future provenance recovery will use a versioned source-mapping registry rather than retrospective inference. The reconstruction procedure will audit surviving source URLs, archived filenames, download records, generation directories, historical metadata, and available generation artifacts. For each recoverable item, the registry will record a stable Arab-DF identifier, source-page or source-record identifier when available, manipulation family and generator, parent authentic-source identifier when independently supported, file checksum, and a verification status such as verified, partially verified, or unresolved. Only mappings supported by preserved evidence will be labeled verified; unresolved links will remain explicitly missing rather than being inferred from similarity or naming conventions. Sanitized provenance fields will be released with the non-video reproducibility materials where legally permissible, while rights-sensitive source information will remain under the same controlled-access process.

3.9. Ethical and Legal Considerations and Misuse Prevention

Arab-DF contains identifiable public figures whose source appearances were collected from publicly accessible media. Public accessibility is not treated as equivalent to research consent or as a blanket waiver of copyright, privacy, portrait/personality, publicity, or related rights. Research-specific consent was not obtained from the individuals depicted in the source media for the creation or redistribution of manipulated derivatives. For this reason, the raw videos and extracted face crops are not openly released, and Arab-DF is treated as a controlled research resource rather than as unrestricted public media.

The principal foreseeable harms include deceptive impersonation, reputational or privacy harm, harassment, unauthorized republication of third-party media, and reuse of the manipulated material to improve identity-targeted synthesis. The intended use is defensive research on deepfake detection, media forensics, robustness, and reproducibility. The safeguards in Section 3.8 therefore combine data minimization with project-specific access: non-video benchmark artifacts are openly available, while identifiable media require a named research team, an institutional request, a research-only DUA, secure storage, and project-bounded use. The DUA prohibits onward redistribution, public reposting of raw media, commercial exploitation, deceptive or harassing use, and training or deployment of identity-targeted synthesis systems.

The authors do not claim source-by-source copyright or portrait/personality-right clearance for the entire historical collection. The source media were gathered over several years from multiple public platforms, and a complete rights ledger was not preserved. Accordingly, access to Arab-DF does not transfer or sublicense any third-party rights, and individual files may be withheld when the legal or ethical basis for further disclosure is uncertain. Researchers remain responsible for complying with applicable law, institutional requirements, and the original terms governing source material in the jurisdictions relevant to their use.

The corresponding-author contact used for access requests also provides a route for depicted individuals, rightsholders, or other affected parties to raise privacy, copyright, or misuse concerns. Substantiated concerns are reviewed and can lead to suspension of future access to an affected file, removal of public provenance pointers where appropriate, or additional restrictions in later dataset versions. Future provenance reconstruction will therefore track, where recoverable, not only source-verification status but also rights or availability status and access restrictions. These measures reduce foreseeable misuse and legal uncertainty but do not eliminate them; this is why the raw identifiable media are not released under an open dataset license.

4. Benchmark Protocol

The benchmark protocol is designed to separate genuine cross-domain degradation from obvious checkpoint or evaluation-pipeline mismatch. It relies on two main choices: the same face-preprocessing pipeline for all detectors and a reproducibility check on Celeb-DF-v2 before Arab-DF results are used for the main interpretation.

4.1. Detectors

We evaluate thirteen detectors available in DeepfakeBench [10]: Xception [20], EfficientNet-B4 [21], CNN-Aug/ResNet-34 [42], F3Net [22], SPSL [23], SRM [24], RECCE [25], CORE [26], FFD [27], Capsule-Forensics [28], Meso4 and Meso4-Inception [29], and UCF [33]. These methods were published between 2017 and 2023 and were selected based on the availability of public pretrained checkpoints, compatible DeepfakeBench configurations, and a common Celeb-DF-v2 reference from the DeepfakeBench cross-dataset benchmark. The reference values are therefore benchmark-generated DeepfakeBench measurements rather than a mixture of values copied from the original detector papers. The goal is not to claim state-of-the-art coverage, but to evaluate a varied public set under one consistent pipeline. All thirteen checkpoints are applied strictly zero-shot to Arab-DF, with no Arab-DF retraining, fine-tuning, recalibration, or adaptation before the main evaluation. To provide recent-method coverage without altering this predefined primary set, we separately evaluate the public Effort checkpoint [11]. Effort is subjected to the same control-first logic; because its Celeb-DF-v2 frame-level reproduction falls outside the ±0.02 tolerance, its Arab-DF results are reported only as an exploratory extension and are excluded from all primary checkpoint-reproducible aggregates.

Effort is evaluated as an image-level detector applied frame by frame and aggregated at video level with the same mean-of-valid-frame-scores rule used for the primary benchmark. The audited configuration uses model_name = effort, a ViT backbone with OpenAI CLIP ViT-L/14, 224 × 224 model input resolution, up to 32 sampled frames per video, test batch size 32, with_mask = False, and with_landmark = False. The public FaceForensics-trained Effort checkpoint was loaded strictly with no missing or unexpected keys. Exact checkpoint, repository, CLIP-snapshot, software-version, and integrity identifiers are reported in Appendix A.3. Because Effort was added after the thirteen-detector benchmark had been fixed, and because its Celeb-DF-v2 control result does not meet the prespecified ±0.02 reproduction criterion, it is kept as a separate recent-method sensitivity check rather than promoted into the primary detector set.

4.2. Preprocessing

All detectors use the same pre-extracted and aligned face inputs. Face detection, landmark extraction, alignment, and cropping follow the Dlib-based DeepfakeBench pipeline [10,43]. We keep the largest detected face, derive five alignment points from the 81-point Dlib landmark predictor, use a crop margin factor of 1.3, and store the shared crops at 256 × 256 pixels. Using the same input preparation reduces preprocessing differences between detectors, while model-specific normalization and internal input handling remain those of each detector implementation.

4.3. Video-Level Aggregation

Each detector produces frame-level manipulation scores. Before computing metrics, we align all outputs so that larger values always mean stronger evidence of manipulation. We uniformly sample up to 32 frames per video. Across the 824 benchmark videos, this produced 26,368 candidate frames; 25,127 yielded valid aligned face crops, and 1241 (4.71%) were excluded after failed face detection/alignment. Rejection was 6.96% for authentic frames and 4.48% for manipulated frames (face swap 4.56%, mobile applications 4.83%, reenactment 3.13%, lip sync 4.50%). The higher frame-exclusion rate for authentic videos may introduce a preprocessing-selection effect and is therefore not assumed to be class-neutral. All 824 videos retained at least one valid crop. A video’s score is the mean of its valid frame-level scores. Unless stated otherwise, AUC values are reported at video level. Frame-level AUC is used only for the checkpoint-reproduction check because the DeepfakeBench CDFv2 references used for certification are frame-level measurements [10].

4.4. Metrics and Statistical Uncertainty

Video-level Area Under the ROC Curve (AUC) [44] is our main performance metric. We report both overall and manipulation-family AUCs on Arab-DF. For the global Arab-DF AUC, uncertainty is estimated with a class-stratified individual-video bootstrap using 10,000 replicates (seed 20260907). The 75 authentic and 749 manipulated videos are resampled separately with replacement, and the same bootstrap indices are used for every detector. These intervals describe sampling variability within the fixed evaluation subset. They do not account for possible dependence between manipulated videos that may share an authentic source.

For the family-level analysis, we apply the same class-stratified individual-video bootstrap separately within each manipulation family. In each of 10,000 replicates (seed 20260907), the 75 authentic videos and the manipulated videos belonging to that family are resampled separately with replacement, and the same bootstrap resamples are used across the eight checkpoint-reproducible detectors. Detector-specific family AUC intervals are reported in Appendix A.2. We also summarize the mean AUC across the eight fixed detectors in each bootstrap replicate, yielding a conditional interval for the eight-detector family mean. Because family sizes differ substantially, the evaluation subset is fixed and non-random, and possible authentic-source dependence among manipulated videos is not modeled, these intervals are used as conditional uncertainty estimates rather than evidence of statistically significant ordering between families. The fixed subset is highly imbalanced, with 75 authentic and 749 manipulated videos (9.1% versus 90.9%), closely matching the full Arab-DF composition (8.66% versus 91.34%). We therefore preserve the observed evaluation composition rather than artificially rebalance it. Fixed-threshold accuracy is not used for the main conclusions because it is strongly affected by class prevalence and score calibration. AUC remains the headline metric because it is threshold-independent. As complementary prevalence-sensitive metrics, we report average precision in both score orientations: AP-authentic, obtained after reversing the manipulation score and treating authentic videos as positive, and AP-manipulated, using the original manipulation score. Their no-skill baselines are 75/824 = 0.091 and 749/824 = 0.909, respectively. AP-authentic is emphasized because authentic videos are the minority class and its low baseline makes failure to retrieve authentic samples under the observed imbalance easier to see; AP-manipulated is also reported for completeness, but its high baseline means that large absolute values should not be interpreted without reference to prevalence. Neither AP metric is used for checkpoint certification or the primary cross-dataset comparison.

For the expanded-authentic sensitivity analysis, we use the same class-stratified individual-video bootstrap with 10,000 replicates and seed 20260907, separately resampling the 392 authentic and 749 manipulated videos with replacement. The same bootstrap indices are reused across detectors. This analysis is restricted to the eight checkpoint-reproducible primary detectors evaluated on the 317 added authentic videos; the five non-certified primary checkpoints and Effort were not included in the expanded-authentic analysis.

4.5. Checkpoint-Reproducibility Criterion

All thirteen reference AUCs used for checkpoint certification are taken from the Celeb-DF-v2 (CDFv2) column of the cross-dataset results reported by DeepfakeBench [10]. DeepfakeBench reports these values as frame-level AUCs for detectors trained on FaceForensics++ c23 and evaluated under its unified preprocessing/evaluation framework. The DeepfakeBench release preprocesses datasets to 32 sampled face frames per video for testing; the exact four-decimal CDFv2 values and their three-decimal manuscript display values are listed in Appendix A.6. These reference values are not taken separately from the original detector papers.

Our reproduction control uses the official Celeb-DF-v2 testing split defined by List_of_testing_videos.txt (518 videos: 178 authentic and 340 manipulated) [5], with up to 32 uniformly sampled frames per video, the same Dlib-based detection/alignment and 256 × 256 shared face crops described above, and model-specific normalization from the corresponding DeepfakeBench detector configuration. Frame-level ROC AUC is computed from the valid frame predictions pooled across the control set. Video-level scores are obtained by mean aggregation and are reported descriptively, but no video-level metric enters the certification decision.

Before using a detector’s Arab-DF result in the main cross-domain conclusions, the reproduced frame-level AUC must be within an absolute tolerance of 0.02 of the corresponding DeepfakeBench CDFv2 reference. This ±0.02 window is a practical quality-control rule meant to separate clear checkpoint or pipeline mismatches from smaller implementation, numerical, and preprocessing differences; it is not a confidence interval or a universal statistical threshold. A video-level AUC cannot be substituted for the frame-level control value. Checkpoints that meet the criterion are termed checkpoint-reproducible and form the primary analysis set; the remaining technically valid checkpoints are retained for transparency but interpreted descriptively. Because UCF and Capsule lie near the boundary, we also report stricter ±0.015 and ±0.010 sensitivity checks. Certification status is determined from the highest-precision values available before rounding.

4.6. Technical-Metadata Audit and Controlled Re-Encoding

This section audits whether low-level file properties could confound interpretation of the fixed 824-video benchmark and then evaluates a narrower controlled re-encoding sensitivity subset. The separate historical Xception fine-tuning experiment is not part of this analysis and is reported only in Appendix A.7.

We performed a fresh file-level technical audit of all 824 videos in the fixed evaluation subset using seven numeric variables extracted from the stored files: duration, frame rate, frame count, width, height, file size, and bitrate. All seven variables were available for all 824 videos (75 authentic and 749 manipulated), so no imputation was required for the descriptive summaries. For the multivariable shortcut diagnostic, features were standardized within each training fold, and a class-balanced logistic-regression classifier was evaluated by five-fold stratified cross-validation. The primary diagnostic value is the pooled out-of-fold (OOF) AUC across all 824 predictions; fold-wise AUCs are also summarized. This analysis measures label separability from technical file properties and does not establish that the face-crop detectors themselves directly use container metadata.

The controlled analysis uses 94 videos arranged into 47 recorded pair_id groups, with one authentic and one manipulated video in each group. The manipulated side contains 21 face-swapping videos, 12 consumer mobile-application manipulations, seven facial-reenactment videos, and seven lip-synchronization videos (47 manipulated videos in total). The historical rule used to create the 47 groups is no longer available, so pair_id is treated only as the recorded experimental grouping, not as proof that the two videos share an authentic source. The preserved record supports this family-level composition but does not provide a fully auditable reconstruction of the exact per-generator/application selection within the 47 manipulated videos; we therefore do not claim per-tool counts for this controlled subset. All 94 videos were standardized to 10 s, 640 × 360 pixels, 25 fps, H.264 encoding at a target bitrate of about 1200 kbps, with audio removed. Six checkpoint-reproducible detectors—EfficientNet-B4, Xception, CORE, F3Net, RECCE, and Capsule—were evaluated on both the original and controlled versions of the same 94 video identities. UCF and SRM were added later and are not part of this secondary analysis. After invalid crops were removed, 2846 face crops remained for evaluation.

We measured changes in video-level AUC with a paired bootstrap over the 47 recorded pair_id groups (10,000 replicates; seed 20260726). The same resampled group indices were used for the original and controlled conditions and for all six detectors. The main quantity of interest is the 95% percentile interval for ΔAUC = AUC_controlled − AUC_original.

5. Results

5.1. Checkpoint Reproduction and Benchmark Composition

All thirteen checkpoints completed the technical evaluation: each loaded correctly, all 824 videos received frame- and video-level scores, and no predictions were missing. Eight detectors meet the predefined ±0.02 frame-level reproducibility criterion: EfficientNet-B4, Xception, UCF, CORE, F3Net, RECCE, Capsule, and SRM. Five—SPSL, FFD, Meso4-Inception, Meso4, and CNN-Aug/ResNet-34—do not meet the criterion and are excluded from the main cross-domain conclusions. Using the highest-precision archived control values, UCF has |Δ| = 0.0189 (0.771579 reproduced versus 0.7527 reference) and remains inside the 0.02 boundary; Capsule is also close to the boundary. Xception provides an additional consistency check: its reproduced Celeb-DF-v2 frame AUC is 0.740 versus a 0.7365 reference, and its Celeb-DF-v2 video-level AUC is 0.816. Table 4 summarizes the complete thirteen-checkpoint benchmark and identifies the eight checkpoint-reproducible detectors used for the primary interpretation.

Table 4. Main benchmark. The reference frame AUCs are the Celeb-DF-v2 (CDFv2) frame-level values reported by DeepfakeBench [10], rounded here to three decimals; exact four-decimal source values are given in Appendix A.6. Arab-DF video-level AUC is the primary metric. AP-authentic and AP-manipulated are complementary prevalence-sensitive metrics with no-skill baselines of 0.091 and 0.909, respectively; the last column reports AP-authentic on the first line and AP-manipulated on the second. AUC 95% confidence intervals use a class-stratified individual-video bootstrap with 10,000 replicates (seed 20260907), separately resampling the 75 authentic and 749 manipulated videos. Primary interpretation is restricted to the eight checkpoints whose Celeb-DF-v2 frame-level AUC is reproduced within ±0.02. Because authentic-source linkage for manipulated videos could not be independently verified, the intervals do not model possible dependence among manipulated videos derived from a shared authentic source.

Detector	Ref. Frame AUC [10]	Repro. Frame AUC	Celeb-DF-v2 Video AUC	Arab-DF Video AUC [95% CI]	Checkpoint Reproducible	AP-Auth. AP-Manip.
EfficientNet-B4	0.749	0.751	0.808	0.655 [0.612, 0.698]	Yes	0.133 0.959
Xception	0.737	0.740	0.816	0.608 [0.559, 0.658]	Yes	0.146 0.950
UCF	0.753	0.772	0.837	0.598 [0.556, 0.640]	Yes	0.101 0.951
CORE	0.743	0.741	0.809	0.589 [0.537, 0.640]	Yes	0.139 0.946
F3Net	0.735	0.729	0.789	0.550 [0.503, 0.599]	Yes	0.094 0.942
RECCE	0.732	0.741	0.823	0.515 [0.469, 0.563]	Yes	0.124 0.937
Capsule	0.747	0.765	0.834	0.469 [0.399, 0.540]	Yes	0.101 0.911
SRM	0.755	0.760	0.840	0.435 [0.388, 0.485]	Yes	0.113 0.921
SPSL	0.765	0.729	0.799	0.572 [0.529, 0.616]	No	0.134 0.947
FFD	0.744	0.687	0.742	0.620 [0.574, 0.666]	No	0.114 0.953
Meso4-Inception	0.697	0.650	0.702	0.507 [0.456, 0.560]	No	0.104 0.931
Meso4	0.609	0.531	0.532	0.430 [0.367, 0.495]	No	0.084 0.910
CNN-Aug/ResNet-34	0.703	0.675	0.742	0.341 [0.256, 0.429]	No	0.081 0.837

The retained-sample composition must also be interpreted in light of the incomplete historical generation logs. The total numbers of generation attempts, rejected outputs, and regeneration events were not preserved. Accordingly, the 11,240 manipulated videos describe the distribution of outputs retained after historical curation, not the complete distribution of outputs originally produced by each generator or application. The retained counts cannot therefore be converted into generator success rates, rejection rates, or comparative quality/yield estimates. Selective rejection or regeneration may have changed the observed quality distribution—for example by removing some unusable failures while retaining other visibly imperfect outputs—but the direction and magnitude of that selection cannot be quantified from the surviving records. Family-level detector results should therefore be interpreted as performance on the retained Arab-DF material rather than on the full latent output distribution of the generation tools.

5.2. Cross-Domain Degradation

For the eight checkpoint-reproducible detectors, mean video-level AUC drops from 0.820 on Celeb-DF-v2 to 0.552 on the fixed Arab-DF subset, giving ΔAUC = −0.267 (Arab-DF − Celeb-DF-v2). The highest precision preserved across the archived control outputs gives an eight-detector Celeb-DF-v2 mean of approximately 0.819750; the saved Arab-DF score matrix gives an eight-detector mean of 0.552312. Their difference is approximately −0.267438, which rounds to −0.267. The apparent −0.268 obtained from 0.552 to 0.820 results from subtracting already rounded summary means. Every detector performs worse on the fixed subset, with decreases ranging from about 0.153 for EfficientNet-B4 to 0.405 for SRM. This consistent drop across different architectures is the main cross-dataset result. Figure 3 visualizes this cross-dataset drop for the eight checkpoint-reproducible detectors.

Figure 3. Video-level AUC of the eight checkpoint-reproducible detectors on Celeb-DF-v2 and the fixed Arab-DF evaluation subset. The dashed line indicates chance-level AUC (0.5).

The finding is not tied to the exact ±0.02 reproducibility threshold. With stricter limits of ±0.015 or ±0.010, UCF and Capsule are excluded. The remaining six detectors still drop from a mean Celeb-DF-v2 video-level AUC of about 0.814 to 0.559 on Arab-DF (ΔAUC ≈ −0.256). The overall conclusion of substantial cross-dataset degradation therefore stays the same.

Detector rankings also change across the two datasets. SRM, UCF, Capsule, and RECCE perform relatively well on Celeb-DF-v2, while EfficientNet-B4 has the highest Arab-DF AUC among the checkpoint-reproducible detectors. In practice, a detector’s ranking on Celeb-DF-v2 is therefore a poor guide to its relative performance on the fixed Arab-DF subset. This reflects limited transfer of the evaluated checkpoints, not proof that the underlying architectures are generally ineffective. Figure 4 shows the corresponding Celeb-DF-v2 versus Arab-DF comparison for all thirteen evaluated checkpoints.

Figure 4. Celeb-DF-v2 versus Arab-DF video-level AUC for all thirteen evaluated checkpoints. Marker type distinguishes checkpoint-reproducible models from checkpoints that do not satisfy the reproduction criterion. The diagonal dashed line represents equal performance on the two evaluation domains, while the horizontal dotted line indicates chance-level AUC (0.5).

As complementary prevalence-sensitive views, AP-authentic among the eight checkpoint-reproducible detectors ranges from 0.094 to 0.146, compared with the no-skill baseline of 0.091, whereas AP-manipulated ranges from 0.911 to 0.959 against a much higher no-skill baseline of 0.909. The high absolute AP-manipulated values therefore partly reflect the 90.9% manipulated prevalence and should not be read as evidence of uniformly strong practical discrimination. AP-authentic provides the more stringent minority-class view in this fixed subset: even the best value, 0.146 for Xception, is only modestly above its 0.091 baseline. This narrow margin above the prevalence-defined no-skill baseline indicates weak precision–recall discrimination for authentic videos under the observed 9.1% authentic prevalence; in other words, the evaluated checkpoints provide little practical gain over prevalence alone when authentic videos are treated as the positive minority class. Both AP orientations are descriptive and do not replace the AUC-based cross-dataset conclusions.

5.3. Statistical Uncertainty and Relation to Chance

We estimate uncertainty in Arab-DF video-level AUC with the class-stratified individual-video bootstrap described in Section 4.4. In each of 10,000 replicates (seed 20260907), the 75 authentic and 749 manipulated videos are resampled separately with replacement. The same bootstrap indices are used for all detectors.

Among the eight checkpoint-reproducible detectors, EfficientNet-B4, Xception, UCF, CORE, and F3Net have 95% bootstrap intervals entirely above 0.5 on the fixed subset. RECCE and Capsule have intervals that cross 0.5. SRM has a point estimate below 0.5, and its individual-video bootstrap interval also remains below 0.5. These observations apply only under the chosen individual-video resampling assumption.

Because the historical authentic-source links could not be independently verified, the individual-video bootstrap does not model possible dependence between manipulated videos that may share the same source. The intervals should therefore be read as uncertainty estimates for this fixed subset, not as population-level confidence intervals for the full Arab-DF collection.

Sensitivity to the resampling unit. The historical-field clustered analysis gives wider intervals than the main individual-video bootstrap (Table A2). Under this alternative grouping, only EfficientNet-B4 and Xception keep intervals entirely above 0.5. UCF includes 0.5 at full archived precision ([0.495, 0.688]), and CORE has an archived lower bound of exactly 0.50. F3Net, RECCE, Capsule, and SRM also have intervals that are not fully above 0.5. This shows that the number of detectors clearly above chance depends on the assumed dependence structure. The point estimates do not change, and all eight checkpoint-reproducible detectors still have lower AUC on Arab-DF than on Celeb-DF-v2. Because the historical grouping field is not verified authentic-source provenance, these clustered intervals are reported only as a robustness check and are not used for the main inference. Figure 5 visualizes the primary individual-video-bootstrap AUC intervals.

Figure 5. Video-level AUC and 95% confidence intervals for the eight checkpoint-reproducible detectors on the fixed Arab-DF evaluation subset. Error bars are obtained using a class-stratified individual-video bootstrap with 10,000 replicates (seed 20260907), independently resampling the 75 authentic and 749 manipulated videos with replacement. The dashed line indicates chance-level AUC (0.5). The intervals do not account for possible dependence among manipulated videos derived from a shared authentic source.

5.4. Per-Family Analysis

For each checkpoint-reproducible detector, we compute family-specific AUC by comparing the manipulated videos from one family with the same 75 authentic videos used in the overall evaluation. Across the eight detectors, the mean point estimates are 0.5933 for consumer mobile applications, 0.5460 for facial reenactment, 0.5417 for face swapping, and 0.5167 for lip synchronization. Under the family-level class-stratified individual-video bootstrap, the corresponding 95% intervals for the eight-detector mean are [0.5335, 0.6535], [0.4646, 0.6280], [0.4945, 0.5889], and [0.4247, 0.6066], respectively. The intervals overlap substantially; therefore, lip synchronization is described as having the lowest mean point estimate in this fixed subset, not as being statistically established as more difficult than the other families.

At the detector level, lip synchronization has the lowest point-estimate AUC for seven of the eight checkpoint-reproducible models. SRM is the exception, for which facial reenactment has a lower point estimate (0.392) than lip synchronization (0.439). The detector-specific 95% intervals are reported in Table A3. In particular, the smaller reenactment (n = 71) and lip-synchronization (n = 66) groups produce relatively wide intervals, reinforcing that the observed family ordering should be interpreted descriptively rather than as a universal or statistically resolved ranking.

One possible reason for the lower lip-synchronization point estimates is that Wav2Lip changes mainly the mouth region while leaving most of the face visually unchanged. A whole-face detector may therefore receive a weaker or more localized manipulation signal than it does for broader face swaps or consumer-application effects. This remains a hypothesis because manipulation extent and generator-specific artifacts were not independently controlled. The bootstrap intervals also show that the family-level uncertainty is substantial, especially for the two smallest manipulation groups, so this mechanistic explanation should not be read as a causal result. Figure 6 summarizes the family-specific point estimates, with detector-specific intervals reported in Table A3.

Figure 6. Video-level AUC by manipulation family for the eight checkpoint-reproducible detectors on the fixed Arab-DF evaluation subset. Each family is evaluated against the same 75 authentic videos. Manipulated-video counts are 432 for face swapping, 180 for mobile applications, 71 for facial reenactment, and 66 for lip synchronization. The heatmap shows point estimates; detector-specific class-stratified individual-video bootstrap 95% confidence intervals (10,000 replicates; seed 20260907) are reported in Table A3.

5.5. Checkpoints Not Satisfying the Reproduction Criterion

For completeness, Table 4 also includes the five checkpoints that do not reproduce the Celeb-DF-v2 reference within ±0.02: SPSL, FFD, Meso4-Inception, Meso4, and CNN-Aug/ResNet-34. Their predictions are technically valid and are kept as a record of the experiments, but they are not used for the main cross-dataset conclusions.

This distinction matters because an Arab-DF score from a non-reproducible checkpoint is hard to interpret. A low AUC could reflect true cross-dataset degradation, imperfect checkpoint reproduction, differences in the evaluation pipeline, or a mixture of these factors. A relatively high Arab-DF score does not solve the problem either. For example, FFD reaches 0.620 on Arab-DF but reproduces only 0.687 frame AUC on Celeb-DF-v2 compared with a 0.744 reference, well outside the ±0.02 tolerance. We therefore report its Arab-DF score descriptively rather than as evidence of stronger cross-dataset robustness.

5.6. Technical-Metadata Audit and Controlled Re-Encoding Sensitivity Analysis

The fresh ffprobe-based audit of the fixed 824-video subset shows measurable class-linked technical differences, although they are not uniform across variables. Table 5 reports median [interquartile range] values separately for the 75 authentic and 749 manipulated videos; none of the seven variables has missing values. Importantly, the reviewer’s collection-level concern that authentic clips are 30–120 s while manipulated clips are approximately 15 s does not describe this fixed benchmark: median duration is 15.024 s for authentic videos and 15.000 s for manipulated videos. A class-balanced logistic-regression diagnostic using the seven variables yields a pooled five-fold out-of-fold AUC of 0.8949 (mean fold AUC 0.8957, SD 0.0240), confirming substantial multivariable technical separability in the unnormalized evaluation files. This establishes the potential for technical confounding but does not show that the evaluated deepfake detectors exploit these metadata fields. Because the detectors operate on aligned RGB face crops, any technical shortcut would have to act indirectly through visual consequences such as resolution, compression, or frame quality rather than through direct access to container metadata.

Table 5. Technical-metadata distributions in the fixed 824-video Arab-DF evaluation subset. Values are median [IQR]. The audit includes all 75 authentic and 749 manipulated videos, with zero missing values for all seven variables.

Variable	Authentic (n = 75)	Manipulated (n = 749)
Duration (s)	15.024 [13.024–15.024]	15.000 [13.024–15.000]
Frame rate (fps)	23.976 [23.976–25.000]	25.000 [25.000–30.000]
Frame count	360 [312–375]	375 [325–375]
Width (px)	1280 [640–1280]	1280 [1280–1920]
Height (px)	720 [480–720]	720 [720–1080]
File size (MB)	1.521 [0.711–2.198]	1.688 [0.990–2.442]
Bitrate (kbps)	718.1 [351.8–1275.8]	948.0 [610.0–1194.8]

Frame rate is the strongest individual separator among the seven audited variables (direction-free univariate AUC 0.804), followed by width and height (0.688 and 0.686); duration alone is only weakly separating (0.543). The direction of the technical imbalance is also not trivially aligned with a ‘lower-quality fake’ shortcut: authentic videos have lower typical bitrate (median 718.1 versus 948.0 kbps) and lower resolution over much of the interquartile range, while both classes have median resolution 1280 × 720. We therefore do not infer that the measured differences must inflate manipulation scores; their effect may differ by detector. In the separate 94-video controlled re-encoding analysis, all videos are standardized to 10 s, 640 × 360 pixels, 25 fps, H.264 at approximately 1200 kbps, with audio removed. Under this common specification, the metadata-only diagnostic is 0.4894, close to chance, before the same six detector checkpoints are compared before and after re-encoding. Table 6 reports the corresponding detector-level AUCs before and after controlled re-encoding.

Table 6. Video-level AUC of six checkpoint-reproducible detectors on 94 videos (47 recorded pair_id groups) before and after controlled re-encoding. ΔAUC = AUC_controlled − AUC_original. The 95% confidence intervals use a paired bootstrap over the 47 groups (10,000 replicates; seed 20260726). pair_id is an experimental grouping, not verified authentic-source correspondence; UCF and SRM were not part of this sensitivity analysis.

Detector	AUC Before	AUC After	ΔAUC	95% CI for ΔAUC
EfficientNet-B4	0.6811	0.6718	−0.0093	[−0.0758, 0.0462]
Xception	0.6277	0.6247	−0.0029	[−0.0987, 0.0708]
CORE	0.5059	0.5179	+0.0120	[−0.0781, 0.0903]
F3Net	0.5611	0.5849	+0.0238	[−0.0521, 0.0917]
RECCE	0.4135	0.4052	−0.0084	[−0.0620, 0.0360]
Capsule	0.3950	0.3748	−0.0201	[−0.0446, 0.0052]
Mean (six)	0.5307	0.5299	−0.0008	[−0.0536, 0.0416]

Across the same 94 video identities, the six checkpoint-reproducible detectors show only small detector-specific changes after re-encoding. ΔAUC ranges from −0.0201 for Capsule to +0.0238 for F3Net, and none of the six paired-bootstrap confidence intervals exclude zero.

Across the six detectors, mean video-level AUC changes from 0.5307 before re-encoding to 0.5299 after re-encoding (ΔAUC = −0.0008). A paired bootstrap over the 47 recorded pair_id groups gives a 95% confidence interval of [−0.0536, 0.0416] for the mean ΔAUC. In this subset, the tested re-encoding therefore produces no meaningful average change. This does not mean that encoding or compression is generally irrelevant to Arab-DF performance.

5.7. Exploratory Recent-Method Extension: Effort

To address recent-method coverage without changing the predefined thirteen-detector primary benchmark, we evaluated the public Effort checkpoint [11] as a separate exploratory extension using the exact configuration documented in Section 4.1. This choice provides one recent CLIP-based image-detector check while preserving the prospectively fixed primary benchmark and applying the same control-first rule. On the Celeb-DF-v2 control, Effort reproduced a frame-level AUC of 0.867 compared with the 0.901 reference (Δ = −0.034), which is outside the prespecified ±0.02 reproduction tolerance. Effort therefore remains non-certified and is excluded from Table 4, the eight-detector mean, and all primary cross-dataset conclusions.

We nevertheless evaluated the checkpoint on the same fixed 824 Arab-DF video identifiers. Each valid face crop was processed by the Effort configuration at 224 × 224, scored independently, and up to 32 frame scores per video were averaged to obtain the video-level score, matching the aggregation logic of the main benchmark. During this run, one previously listed preprocessed crop (MANIP_000000672/043.png) was found to be genuinely unreadable by both OpenCV and PIL; it was excluded from Effort aggregation while the affected video remained in the evaluation with its other 31 valid crops. The Effort run therefore contains 25,126 valid crops. Its Arab-DF video-level AUC is 0.6951 (95% CI [0.6538, 0.7352]) under the same class-stratified individual-video bootstrap with 10,000 replicates (seed 20260907). Frame-level AUC is 0.6835. AP-authentic is 0.1796 (95% CI [0.1321, 0.2461]), compared with the no-skill baseline of 0.0910.

Family-level AUC is 0.7408 for mobile applications, 0.7012 for face swapping, 0.6413 for facial reenactment, and 0.5882 for lip synchronization. Thus, lip synchronization remains the lowest-AUC family for this recent method as well. Although Effort’s Arab-DF point estimate is numerically higher than the individual point estimates of the eight checkpoint-reproducible detectors, the comparison is descriptive only: its control checkpoint did not satisfy the reproduction criterion, so the score is not treated as evidence of superior cross-dataset generalization and is not used to revise the primary benchmark conclusions.

5.8. Expanded-Authentic Sensitivity Analysis

To assess how the limited number of authentic videos affects the primary point estimates, we retain the same 749 manipulated videos and increase the authentic sample from 75 to 392 by adding the 317 additional authentic videos available in the recovered preprocessed archive described in Section 3.7. No detector is retrained, fine-tuned, or recalibrated for this analysis. For the eight checkpoint-reproducible detectors, mean video-level AUC changes from 0.5523 on the predefined 824-video subset to 0.5075 on the 1141-video expanded cohort (mean ΔAUC = −0.0448). A class-stratified individual-video bootstrap with 10,000 replicates (seed 20260907) gives a 95% confidence interval of [0.4776, 0.5366] for the expanded eight-detector mean. Table 7 reports the detector-level results for the expanded-authentic cohort.

Table 7. Expanded-authentic sensitivity analysis for the eight checkpoint-reproducible detectors. The predefined benchmark contains 75 authentic and 749 manipulated videos (824 total); the expanded cohort contains 392 authentic and the same 749 manipulated videos (1141 total). ΔAUC = AUC_expanded − AUC_historical. Expanded 95% confidence intervals use a class-stratified individual-video bootstrap with 10,000 replicates (seed 20260907).

Detector	AUC, 824	AUC, 1141	ΔAUC	Expanded 95% CI
EfficientNet-B4	0.6549	0.5379	−0.1170	[0.5048, 0.5715]
Xception	0.6083	0.5016	−0.1066	[0.4690, 0.5341]
UCF	0.5979	0.5118	−0.0861	[0.4786, 0.5441]
CORE	0.5885	0.5074	−0.0811	[0.4744, 0.5398]
F3Net	0.5505	0.4914	−0.0590	[0.4582, 0.5241]
RECCE	0.5149	0.5204	+0.0055	[0.4872, 0.5529]
Capsule	0.4686	0.5135	+0.0449	[0.4794, 0.5469]
SRM	0.4350	0.4759	+0.0409	[0.4425, 0.5086]
Mean	0.5523	0.5075	−0.0448	[0.4776, 0.5366]

Five of the eight checkpoint-reproducible detectors decrease on the expanded cohort, whereas RECCE, Capsule, and SRM increase modestly, so the change is detector-dependent rather than a uniform shift. Only EfficientNet-B4 has an expanded-cohort 95% confidence interval entirely above 0.5 ([0.5048, 0.5715]); the intervals of the other seven detectors include 0.5. The expanded analysis therefore shows that absolute AUC estimates are sensitive to authentic-set composition and do not reveal stronger zero-shot discrimination hidden by the smaller authentic subset. The 317 additional authentic videos were included because usable preprocessed crop material was recoverable, not because they were a probability sample of the remaining authentic collection; the recovered archive does not independently preserve the complete crop-generation command history or complete file-level technical metadata for the original videos. The result is therefore a sensitivity analysis and does not replace the predefined 824-video benchmark.

6. Discussion

6.1. Interpretation of the Main Findings

The main finding is straightforward: the eight checkpoint-reproducible detectors transfer much less well to the fixed Arab-DF subset than to Celeb-DF-v2. Their mean video-level AUC falls from 0.820 to 0.552, and every detector shows a decrease, although the size of the drop varies across architectures.

The expanded-authentic sensitivity analysis qualifies the absolute value of the 0.552 estimate. When the authentic sample is increased from 75 to 392 while the same 749 manipulated videos are retained, the eight-detector mean AUC is 0.5075 (95% CI [0.4776, 0.5366]), and only EfficientNet-B4 retains an individual 95% confidence interval entirely above 0.5. Detector-specific changes are mixed rather than uniformly negative, but the expanded result does not strengthen evidence for robust zero-shot transfer. Instead, it shows that the point estimate is sensitive to authentic-set composition and reinforces the need to interpret the 824-video result as a property of the fixed benchmark subset rather than of Arab-DF as a whole.

The precision–recall view is even more restrictive for the minority authentic class. On the fixed subset, AP-authentic is only 0.094–0.146 against a no-skill baseline of 0.091. Values this close to the prevalence baseline indicate weak practical retrieval of authentic videos under the observed class imbalance, even for detectors whose ROC AUC exceeds chance. Conversely, AP-manipulated is numerically high partly because manipulated videos constitute 90.9% of the subset. We therefore treat the two AP orientations as prevalence-sensitive complements to AUC rather than as evidence of strong deployment-ready discrimination.

The reproducibility check makes this result easier to interpret. Every retained checkpoint reproduces its Celeb-DF-v2 frame-level reference within the predefined ±0.02 tolerance, which reduces the chance that the cross-domain drop is mainly caused by a badly configured or poorly reproduced checkpoint. The ±0.02 value is a practical rule, not a statistically special boundary. Even with stricter thresholds of ±0.015 and ±0.010, the main pattern remains: the six retained detectors still fall from a mean video-level AUC of 0.814 to 0.559.

The eight retained detectors use different designs, so the cross-domain drop is not limited to one detector family. However, the size of the degradation varies considerably, showing that model design and learned representations still affect transfer. The manipulation-family analysis likewise shows heterogeneous point estimates: lip synchronization has the lowest mean AUC (0.5167) and is the lowest-AUC family for seven of the eight checkpoint-reproducible models. However, its eight-detector mean bootstrap interval [0.4247, 0.6066] overlaps those of face swapping, mobile applications, and facial reenactment. We therefore interpret the family ordering as a descriptive pattern within the fixed subset rather than as evidence that one manipulation family is statistically more difficult than another. In addition, because historical generation-attempt, rejection, and regeneration logs are unavailable, differences among manipulation families may also reflect unknown curation intensity; the family ordering should not be read as an intrinsic generator-quality ranking. The mobile-application group also remains internally heterogeneous because it combines several proprietary consumer products whose underlying models and versions are not fully observable; its family-level AUC should therefore not be interpreted as the behavior of one algorithmic generator family.

The technical-metadata audit provides an important caution: seven file-level properties jointly separate authentic from manipulated videos with pooled OOF AUC 0.8949, so technical confounding cannot be ignored in the unnormalized subset. The controlled re-encoding experiment then provides a narrower sensitivity check: after the tested normalization removes the measured metadata separability (metadata-only AUC 0.4894), mean AUC across six detectors changes only by −0.0008, and the paired-bootstrap interval includes zero. Taken together, these results argue against treating the measured encoding variables as a complete explanation of detector performance, but they do not prove that technical acquisition or compression effects are irrelevant outside the 94-video controlled subset.

Taken together, the experiments support a focused conclusion: public checkpoints that reproduce their control-benchmark performance still transfer poorly to the fixed Arab-DF evaluation domain. Because the detectors receive only visual face crops, the observed degradation cannot be attributed specifically to the Arabic language. Instead, Arab-DF should be interpreted as an Arabic-media visual domain in which subjects, visual content, manipulation methods, acquisition conditions, and processing pipelines differ jointly from Celeb-DF-v2. Isolating any language-specific effect would require a matched Arabic/non-Arabic evaluation that controls these factors, which is outside the scope of this study.

6.2. Detector Ranking Across Datasets

Detector rankings also change between Celeb-DF-v2 and Arab-DF. Models near the top on the control benchmark do not necessarily keep the same position on the fixed Arab-DF subset. For the eight checkpoint-reproducible detectors, the Spearman rank correlation is about −0.52 (p ≈ 0.18). With only eight models, this is not statistically conclusive, so we treat it as a descriptive pattern rather than formal evidence of rank reversal.

Even so, the change in ordering highlights a practical problem with choosing detectors from a single benchmark. A high AUC on Celeb-DF-v2 shows strong performance on that dataset, but it does not guarantee the same relative advantage on Arabic-media visual material. Celeb-DF-v2 remains a useful common reference, but its ranking should not be treated as a universal measure of robustness.

6.3. Toward More Transferable Deepfake Detection

The results suggest that robustness cannot be judged from one benchmark alone. Better cross-domain performance will likely require training and evaluation across a wider range of generators, source domains, compression settings, resolutions, and acquisition pipelines. Another direction is to reduce reliance on generator-specific artifacts and focus more on transferable cues such as facial appearance, temporal consistency, physiological signals, or cross-frame coherence, while carefully controlling for shortcuts related to camera, codec, identity, content, and acquisition conditions.

Related work in adjacent computer-vision tasks also points to potentially useful design directions. Liu et al. developed domain-transformed robust feature representations for human parsing under complex and corrupted conditions [45], while Wang et al. proposed mask-guided state-space fusion to integrate complementary visible and infrared features for vehicle detection [46]. Although these tasks differ from deepfake detection, they motivate future investigation of domain-robust representations and principled multimodal fusion for Arab-DF.

The separate Effort [11] extension provides a first check on a more recent public method without changing the primary benchmark. On the fixed 824-video Arab-DF subset, its video-level AUC is 0.6951 (95% CI [0.6538, 0.7352]), with the lowest family-level AUC again observed for lip synchronization (0.5882). However, its Celeb-DF-v2 frame-level reproduction is 0.867 versus the 0.901 reference, outside the ±0.02 tolerance. We therefore keep Effort non-certified and separate from Table 4 and the primary aggregate. The result illustrates why recent methods should be evaluated under the same checkpoint-control discipline rather than folded into the primary benchmark solely on the basis of their target-domain score.

Beyond this single Effort extension, broader evaluation of foundation-model, CLIP-based, temporal, and multimodal detectors remains needed. Such methods should follow the same principle used here: first reproduce a public checkpoint on a control benchmark, then interpret its Arab-DF performance under the corresponding reproducibility status.

6.4. Limitations

This study has several limitations. First, the historical selection path of the fixed 824-video benchmark is only partially reconstructible. A preserved 5484-video experimental manifest provides one intermediate count, but no retained construction script explains how the 12,306-video release was reduced to that manifest or assigns stage-specific reasons to the 6822 videos outside it. The 824-video benchmark is therefore treated as a fixed, non-random historical cohort rather than a probability sample. Its class and manipulation-family proportions are broadly similar to those of the full collection, but dialect proportions differ materially (Table 3), and this does not establish statistical representativeness on other attributes. The reported results describe this fixed subset and should not be treated as population-level estimates for the full dataset. We partially probe this limitation with an expanded-authentic sensitivity cohort containing 392 authentic and 749 manipulated videos; however, the 317 additional authentic videos were those for which usable preprocessed crop material was recoverable, not a probability sample of all remaining authentic videos, and their recovered score archive does not include a complete file-level technical audit of the originals. The expanded analysis likewise does not establish population representativeness.

Second, we could not independently verify the authentic-source links of the manipulated videos from the surviving records. Historical metadata contain source-linkage fields, but we could not confirm them against an independent generation or provenance archive. We therefore do not use them as verified source groups in the main statistical analysis. Instead, the main confidence intervals use a class-stratified individual-video bootstrap. This avoids treating an uncertain mapping as ground truth, but it cannot account for dependence between manipulated videos that may share the same authentic source. Table A2 shows how the intervals change when the historical field is used only as a sensitivity check.

Third, an auxiliary historical fine-tuned Xception experiment is reported only in Appendix A.7 for transparency. The exact training split cannot be reconstructed at the identity or source-video level, so this experiment is excluded from the primary results and is not used as evidence of leakage-free supervised generalization or of a causal fine-tuning benefit.

Fourth, the full-subset technical audit quantifies seven measured file properties for all 824 evaluation videos and shows substantial joint class separability, but it cannot determine whether the face-crop detectors exploit those properties or their visual consequences. The controlled re-encoding analysis addresses this question only on 94 videos arranged into 47 recorded real/fake groups. The manipulated side is recoverable at the family level (21 face swaps, 12 mobile-application manipulations, seven reenactments, and seven lip-sync videos), but the original rule used to create the groups and a fully auditable per-generator/application selection record are no longer available. Accordingly, pair_id is not interpreted as verified authentic-source correspondence, and the re-encoding result should not be generalized beyond the tested subset and normalization settings.

Fifth, the historical full-collection quality control was performed by one annotator, and complete rejection logs were not preserved. A retrospective two-rater audit of 100 stratified retained videos found 97% agreement on the overall retain decision (Cohen’s κ = 0.653), but the archived audit files do not record whether either rater was the original annotator or whether the reassessment was blinded. We therefore do not claim independent or blinded validation. The overall retain decision was a separate holistic judgment rather than a deterministic threshold on the three ordinal component scores, which explains why non-retain decisions can occur even though no component score of 0 was used in the sample. This sample-based verification cannot reconstruct the historical rejection rate or establish agreement for all 12,306 videos. Six sampled videos received at least one non-retain rating and are flagged for curation; none is part of the fixed 824-video benchmark. The historical estimates of geographic/person coverage should not be interpreted as a verified demographic census. We also did not systematically preserve exact software versions, repository commits, generator checkpoints, application versions, or complete run-level settings for every historical manipulation, so we do not claim bit-for-bit regeneration of the full manipulated collection. Because selective rejection or regeneration may have altered which quality levels are represented among the retained manipulations, the final collection cannot be used to estimate the original output-quality distribution or failure rate of any generator/application.

Sixth, the historical collection does not have a complete source-by-source rights ledger, and research-specific consent was not obtained from the identifiable individuals depicted in the public-source media. Controlled access, research-only DUA terms, data minimization, and the rights-concern review process reduce exposure but do not convert the underlying third-party media into openly licensed research data. Because copyright, privacy, portrait/personality, publicity, and related rules can vary by source and jurisdiction, the manuscript does not claim a universal legal basis for unrestricted redistribution of the raw collection, and some files may remain unavailable for future requests.

Seventh, the primary benchmark is limited to thirteen public checkpoints published between 2017 and 2023 and does not cover the full range of current foundation-model, CLIP-based, temporal, or multimodal approaches. We add Effort as one exploratory recent-method extension, but its control reproduction falls outside the prespecified tolerance, and it therefore does not provide certified coverage of the broader recent-method literature. A further limitation is that the ±0.02 checkpoint-reproduction tolerance is a practical quality-control rule rather than a statistically privileged boundary, although the main degradation pattern remains under stricter thresholds.

Future work should therefore use verified source mappings, identity-disjoint supervised splits, independently replicated preprocessing controls, and broader evaluation across datasets, detector families, and demographic or media subgroups. Section 3.8 defines a staged provenance-reconstruction procedure in which recovered links are versioned and labeled verified, partially verified, or unresolved rather than retrospectively inferred.

7. Conclusions

We introduced Arab-DF, an Arabic-media visual deepfake dataset covering face swapping, facial reenactment, lip synchronization, and consumer mobile-application manipulations. We then evaluated public deepfake detectors with a reproducibility-controlled zero-shot protocol. Thirteen public checkpoints were tested, and eight reproduced their Celeb-DF-v2 frame-level reference AUC within the predefined ±0.02 tolerance and were retained for the main analysis.

For these eight detectors, mean video-level AUC fell from 0.820 on Celeb-DF-v2 to 0.552 on the fixed Arab-DF subset (ΔAUC = −0.267). Under the class-stratified individual-video bootstrap, five detectors had 95% confidence intervals entirely above 0.5 on the fixed subset. The complementary AP-authentic values are only 0.094–0.146 against a no-skill baseline of 0.091, indicating weak minority-class precision–recall discrimination under the fixed subset’s class imbalance. Lip synchronization had the lowest mean family-level point estimate (0.5167), but its 95% bootstrap interval for the eight-detector mean [0.4247, 0.6066] overlaps the corresponding intervals of the other manipulation families, so the family ordering is interpreted descriptively. In the expanded-authentic sensitivity analysis, which retains the same 749 manipulated videos and increases the authentic sample from 75 to 392, the eight-detector mean AUC is 0.5075 (95% CI [0.4776, 0.5366]); only EfficientNet-B4 has an individual expanded-cohort 95% interval entirely above 0.5. This reinforces that the 0.552 primary estimate is specific to the predefined fixed subset rather than a population estimate for all Arab-DF videos.

The technical-metadata audit shows that seven file properties are jointly class-separable in the fixed 824-video subset (pooled OOF AUC 0.8949), which motivates caution about technical confounding. In the controlled re-encoding analysis on 94 videos, the metadata-only diagnostic falls to 0.4894 while the mean AUC of six checkpoint-reproducible detectors changes only from 0.5307 to 0.5299 (ΔAUC = −0.0008; paired-bootstrap 95% CI [−0.0536, 0.0416]). This sensitivity result suggests that the measured encoding differences are not, by themselves, sufficient to explain the detector behavior observed on that controlled subset.

As a separate exploratory recent-method extension, Effort reaches 0.6951 video-level AUC on the fixed 824-video Arab-DF subset (95% CI [0.6538, 0.7352]) and 0.1796 AP-authentic. Its Celeb-DF-v2 frame-level reproduction, however, is 0.867 versus a 0.901 reference, outside the ±0.02 tolerance; Effort is therefore not included in the primary eight-detector aggregates. Lip synchronization is again its lowest-AUC family (0.5882).

Overall, the study shows substantial cross-dataset performance degradation under the zero-shot protocol and reinforces the need to test deepfake detectors beyond standard benchmarks. The findings should still be read in light of the fixed non-random evaluation subset, uncertain authentic-source provenance, and the limits of the re-encoding analysis. The auxiliary historical supervised experiment in Appendix A.7 is not used in these conclusions. Future work should prioritize verified source mappings, identity-disjoint supervised evaluation, broader cross-dataset replication, and analysis across additional linguistic, demographic, and media groups.

Appendix A.1. Output Properties and Quality Range

Technical properties vary across generation tools, application exports, and later file-handling steps. The historical metadata do not provide reliable generator-specific distributions for resolution, frame rate, codec, or bitrate, so we do not report these as fixed generation parameters. The metadata audit in Section 5.6 describes the evaluated files directly and should not be read as a complete generator-level description of all 11,240 manipulated videos.

The dataset intentionally keeps a range of manipulation quality. Some outputs with visible artifacts, imperfect blending, temporal instability, or other quality problems were retained as low-quality examples rather than systematically removed. Because generation attempts and rejection decisions were not preserved as numerical logs, we cannot report the total number of attempts or a quantitative rejection/regeneration rate.

Table A2. Sensitivity of Arab-DF AUC uncertainty to the resampling unit. The main intervals use the class-stratified individual-video bootstrap from the primary analysis. As a sensitivity check only, the earlier procedure uses the historical source-linkage field to define bootstrap clusters, even though this field could not be independently verified as authentic-source provenance (10,000 repetitions; seed 20260721). The six previously archived detector intervals are shown to two decimal places; UCF and SRM retain three-decimal precision because their relation to chance is sensitive to rounding.

Detector	Arab-DF AUC	Primary Individual-Video 95% CI	Historical-Field Clustered 95% CI
EfficientNet-B4	0.655	[0.612, 0.698]	[0.56, 0.75]
Xception	0.608	[0.559, 0.658]	[0.53, 0.68]
UCF	0.598	[0.556, 0.640]	[0.495, 0.688]
CORE	0.589	[0.537, 0.640]	[0.50, 0.67]
F3Net	0.550	[0.503, 0.599]	[0.45, 0.64]
RECCE	0.515	[0.469, 0.563]	[0.42, 0.60]
Capsule	0.469	[0.399, 0.540]	[0.37, 0.56]
SRM	0.435	[0.388, 0.485]	[0.325, 0.532]

Table A2. Sensitivity of Arab-DF AUC uncertainty to the resampling unit. The main intervals use the class-stratified individual-video bootstrap from the primary analysis. As a sensitivity check only, the earlier procedure uses the historical source-linkage field to define bootstrap clusters, even though this field could not be independently verified as authentic-source provenance (10,000 repetitions; seed 20260721). The six previously archived detector intervals are shown to two decimal places; UCF and SRM retain three-decimal precision because their relation to chance is sensitive to rounding.

Detector	Arab-DF AUC	Primary Individual-Video 95% CI	Historical-Field Clustered 95% CI
EfficientNet-B4	0.655	[0.612, 0.698]	[0.56, 0.75]
Xception	0.608	[0.559, 0.658]	[0.53, 0.68]
UCF	0.598	[0.556, 0.640]	[0.495, 0.688]
CORE	0.589	[0.537, 0.640]	[0.50, 0.67]
F3Net	0.550	[0.503, 0.599]	[0.45, 0.64]
RECCE	0.515	[0.469, 0.563]	[0.42, 0.60]
Capsule	0.469	[0.399, 0.540]	[0.37, 0.56]
SRM	0.435	[0.388, 0.485]	[0.325, 0.532]

Appendix A.2. Family-Level Bootstrap Intervals

Table A3. Family-specific video-level AUC and 95% confidence intervals for the eight checkpoint-reproducible detectors on the fixed Arab-DF evaluation subset. Each manipulation family is compared with the same 75 authentic videos. Confidence intervals use a class-stratified individual-video bootstrap with 10,000 replicates (seed 20260907), separately resampling authentic and family-specific manipulated videos with replacement and using the same bootstrap resamples across detectors. These intervals are conditional on the fixed subset and do not account for possible dependence among manipulated videos derived from a shared authentic source.

Detector	Manipulation Family	n	AUC	95% CI
EfficientNet-B4	Face swapping	432	0.651	[0.602, 0.700]
Xception	Face swapping	432	0.596	[0.540, 0.650]
UCF	Face swapping	432	0.582	[0.532, 0.630]
CORE	Face swapping	432	0.584	[0.525, 0.641]
F3Net	Face swapping	432	0.542	[0.488, 0.596]
RECCE	Face swapping	432	0.503	[0.449, 0.556]
Capsule	Face swapping	432	0.452	[0.376, 0.527]
SRM	Face swapping	432	0.424	[0.369, 0.479]
EfficientNet-B4	Mobile applications	180	0.688	[0.625, 0.749]
Xception	Mobile applications	180	0.642	[0.576, 0.707]
UCF	Mobile applications	180	0.650	[0.586, 0.713]
CORE	Mobile applications	180	0.617	[0.550, 0.684]
F3Net	Mobile applications	180	0.583	[0.514, 0.650]
RECCE	Mobile applications	180	0.563	[0.495, 0.631]
Capsule	Mobile applications	180	0.525	[0.447, 0.604]
SRM	Mobile applications	180	0.478	[0.407, 0.548]
EfficientNet-B4	Facial reenactment	71	0.651	[0.555, 0.741]
Xception	Facial reenactment	71	0.621	[0.529, 0.713]
UCF	Facial reenactment	71	0.606	[0.510, 0.704]
CORE	Facial reenactment	71	0.603	[0.509, 0.698]
F3Net	Facial reenactment	71	0.552	[0.455, 0.649]
RECCE	Facial reenactment	71	0.495	[0.398, 0.593]
Capsule	Facial reenactment	71	0.448	[0.353, 0.546]
SRM	Facial reenactment	71	0.392	[0.296, 0.490]
EfficientNet-B4	Lip synchronization	66	0.592	[0.489, 0.691]
Xception	Lip synchronization	66	0.584	[0.483, 0.680]
UCF	Lip synchronization	66	0.552	[0.448, 0.655]
CORE	Lip synchronization	66	0.527	[0.424, 0.627]
F3Net	Lip synchronization	66	0.514	[0.409, 0.617]
RECCE	Lip synchronization	66	0.484	[0.380, 0.586]
Capsule	Lip synchronization	66	0.441	[0.344, 0.537]
SRM	Lip synchronization	66	0.439	[0.335, 0.544]

Table A3. Family-specific video-level AUC and 95% confidence intervals for the eight checkpoint-reproducible detectors on the fixed Arab-DF evaluation subset. Each manipulation family is compared with the same 75 authentic videos. Confidence intervals use a class-stratified individual-video bootstrap with 10,000 replicates (seed 20260907), separately resampling authentic and family-specific manipulated videos with replacement and using the same bootstrap resamples across detectors. These intervals are conditional on the fixed subset and do not account for possible dependence among manipulated videos derived from a shared authentic source.

Detector	Manipulation Family	n	AUC	95% CI
EfficientNet-B4	Face swapping	432	0.651	[0.602, 0.700]
Xception	Face swapping	432	0.596	[0.540, 0.650]
UCF	Face swapping	432	0.582	[0.532, 0.630]
CORE	Face swapping	432	0.584	[0.525, 0.641]
F3Net	Face swapping	432	0.542	[0.488, 0.596]
RECCE	Face swapping	432	0.503	[0.449, 0.556]
Capsule	Face swapping	432	0.452	[0.376, 0.527]
SRM	Face swapping	432	0.424	[0.369, 0.479]
EfficientNet-B4	Mobile applications	180	0.688	[0.625, 0.749]
Xception	Mobile applications	180	0.642	[0.576, 0.707]
UCF	Mobile applications	180	0.650	[0.586, 0.713]
CORE	Mobile applications	180	0.617	[0.550, 0.684]
F3Net	Mobile applications	180	0.583	[0.514, 0.650]
RECCE	Mobile applications	180	0.563	[0.495, 0.631]
Capsule	Mobile applications	180	0.525	[0.447, 0.604]
SRM	Mobile applications	180	0.478	[0.407, 0.548]
EfficientNet-B4	Facial reenactment	71	0.651	[0.555, 0.741]
Xception	Facial reenactment	71	0.621	[0.529, 0.713]
UCF	Facial reenactment	71	0.606	[0.510, 0.704]
CORE	Facial reenactment	71	0.603	[0.509, 0.698]
F3Net	Facial reenactment	71	0.552	[0.455, 0.649]
RECCE	Facial reenactment	71	0.495	[0.398, 0.593]
Capsule	Facial reenactment	71	0.448	[0.353, 0.546]
SRM	Facial reenactment	71	0.392	[0.296, 0.490]
EfficientNet-B4	Lip synchronization	66	0.592	[0.489, 0.691]
Xception	Lip synchronization	66	0.584	[0.483, 0.680]
UCF	Lip synchronization	66	0.552	[0.448, 0.655]
CORE	Lip synchronization	66	0.527	[0.424, 0.627]
F3Net	Lip synchronization	66	0.514	[0.409, 0.617]
RECCE	Lip synchronization	66	0.484	[0.380, 0.586]
Capsule	Lip synchronization	66	0.441	[0.344, 0.537]
SRM	Lip synchronization	66	0.439	[0.335, 0.544]

Appendix A.3. Effort Reproducibility Configuration

Table A4 reports the audited checkpoint, repository, software, and model-dependency identifiers used for the Effort extension.

Table A4. Audited configuration used for the separate Effort exploratory extension. These identifiers document the exact checkpoint and software/model dependencies used in the reported control and Arab-DF evaluations.

Item	Audited Value
Detector role	Separate exploratory recent-method extension; image-level detector scored frame by frame and mean-aggregated per video
Model configuration	model_name = effort; backbone_name = vit; OpenAI CLIP ViT-L/14; resolution = 224; frame_num[test] = 32; test_batchSize = 32; with_mask = False; with_landmark = False
Checkpoint	effort_clip_L14_trainOn_FaceForensic.pth
Checkpoint repository	boom2511/effort
Repository revision	033bbdbf d5aec5f3 266e6c2d 356ac245 2c79461e
Checkpoint SHA-256	8d86711f 098d16b4 9c048962 fc3e16a9 06380f7b b17b8a0e 89bd545b 926943ee
CLIP dependency	openai/clip-vit-large-patch14
CLIP snapshot revision	32bd6428 8804d66e efd0ccbe 215aa642 df71cc41
Checkpoint load	Strict after DataParallel-prefix removal: 681/681 keys; 0 missing; 0 unexpected
DeepfakeBench commit	f188b1c1 05465e2e 5377eb53 6a95022a e0e4522d
Albumentations	1.1.0
Video aggregation	Mean of valid frame-level manipulation scores; up to 32 frames per video

Table A4. Audited configuration used for the separate Effort exploratory extension. These identifiers document the exact checkpoint and software/model dependencies used in the reported control and Arab-DF evaluations.

Item	Audited Value
Detector role	Separate exploratory recent-method extension; image-level detector scored frame by frame and mean-aggregated per video
Model configuration	model_name = effort; backbone_name = vit; OpenAI CLIP ViT-L/14; resolution = 224; frame_num[test] = 32; test_batchSize = 32; with_mask = False; with_landmark = False
Checkpoint	effort_clip_L14_trainOn_FaceForensic.pth
Checkpoint repository	boom2511/effort
Repository revision	033bbdbf d5aec5f3 266e6c2d 356ac245 2c79461e
Checkpoint SHA-256	8d86711f 098d16b4 9c048962 fc3e16a9 06380f7b b17b8a0e 89bd545b 926943ee
CLIP dependency	openai/clip-vit-large-patch14
CLIP snapshot revision	32bd6428 8804d66e efd0ccbe 215aa642 df71cc41
Checkpoint load	Strict after DataParallel-prefix removal: 681/681 keys; 0 missing; 0 unexpected
DeepfakeBench commit	f188b1c1 05465e2e 5377eb53 6a95022a e0e4522d
Albumentations	1.1.0
Video aggregation	Mean of valid frame-level manipulation scores; up to 32 frames per video

Appendix A.4. Controlled-Access and Licensing Protocol

Table A5 summarizes the operational access conditions for the identifiable audiovisual component of Arab-DF. These conditions govern only copies supplied by the authors and do not transfer third-party intellectual-property or personality rights.

Table A5. Controlled-access request and use conditions for raw Arab-DF audiovisual material and extracted face crops.

Component	Policy
Eligibility	Bona fide non-commercial researchers affiliated with a recognized university, research institute, or public-interest research organization; access is project-specific and not automatic.
Request contents	Institutional email request stating applicant, institution, and responsible PI; scientific purpose; requested subset; intended analyses; storage/security plan; and ethics approval, exemption, or institutional statement where applicable.
Review criteria	Scientific relevance, data minimization, legal/ethical feasibility, rights status, and safeguards for identifiable subjects. Specific files may be withheld when lawful redistribution or responsible disclosure cannot be established.
Agreement before access	Named users must accept the Arab-DF Research-Only Data Use Agreement. Access does not confer copyright, portrait, publicity, or other third-party rights.
Permitted use	Approved research in deepfake detection, media forensics, robustness evaluation, and closely related scientific analysis within the accepted project scope.
Prohibited use and dissemination	No onward redistribution or public reposting of raw files/crops; no unlisted users; no commercial exploitation; no deceptive impersonation, harassment, or identity-targeted synthesis use; no publication of identifiable frames without a separate lawful basis or permission.
Security and closure	Store data in access-controlled research storage, limit access to the approved team, report material security incidents, and delete raw copies when the approved project ends, or access is terminated, subject to any overriding institutional retention requirement.

Table A5. Controlled-access request and use conditions for raw Arab-DF audiovisual material and extracted face crops.

Component	Policy
Eligibility	Bona fide non-commercial researchers affiliated with a recognized university, research institute, or public-interest research organization; access is project-specific and not automatic.
Request contents	Institutional email request stating applicant, institution, and responsible PI; scientific purpose; requested subset; intended analyses; storage/security plan; and ethics approval, exemption, or institutional statement where applicable.
Review criteria	Scientific relevance, data minimization, legal/ethical feasibility, rights status, and safeguards for identifiable subjects. Specific files may be withheld when lawful redistribution or responsible disclosure cannot be established.
Agreement before access	Named users must accept the Arab-DF Research-Only Data Use Agreement. Access does not confer copyright, portrait, publicity, or other third-party rights.
Permitted use	Approved research in deepfake detection, media forensics, robustness evaluation, and closely related scientific analysis within the accepted project scope.
Prohibited use and dissemination	No onward redistribution or public reposting of raw files/crops; no unlisted users; no commercial exploitation; no deceptive impersonation, harassment, or identity-targeted synthesis use; no publication of identifiable frames without a separate lawful basis or permission.
Security and closure	Store data in access-controlled research storage, limit access to the approved team, report material security incidents, and delete raw copies when the approved project ends, or access is terminated, subject to any overriding institutional retention requirement.

Appendix A.5. Retrospective Two-Rater Quality-Control Verification

A stratified sample of 100 retained videos (20 per group: authentic, face swapping, facial reenactment, lip synchronization, and mobile applications) was re-reviewed by two raters using the published usability criteria. The archived audit files identify them only as Rater A and Rater B; they do not record whether either rater was the original historical annotator or whether the reassessment was blinded, so this audit is not described as independent or blinded validation. Table A6 reports agreement between the two retrospective raters. The overall retain/non-retain field was recorded as a separate holistic usability decision, not as a deterministic threshold on the three component scores. The three component criteria use archived 0–2 ordinal ratings; no score of 0 was used by either rater in this sample, so the weighted κ values reflect agreement on the observed 1-versus-2 distinctions.

Table A6. Retrospective two-rater quality-control agreement on 100 retained Arab-DF videos. Exact agreement is the proportion of identical ratings. Cohen’s κ is reported for the binary retain decision; quadratic-weighted κ is reported for the ordinal 0–2 criteria.

Criterion	Rater A	Rater B	Exact Agreement	κ
Overall retain decision	97 retain/3 non-retain	94 retain/6 non-retain	97%	0.653
Face visibility	87 score 2/13 score 1	84 score 2/16 score 1	97%	0.879 *
Temporal interpretability	97 score 2/3 score 1	92 score 2/8 score 1	95%	0.525 *
Severe-corruption assessment	96 score 2/4 score 1	91 score 2/9 score 1	95%	0.593 *

* Quadratic-weighted Cohen’s κ. Three sampled videos were jointly rated non-retain, and three additional videos were disagreements. None of the six flagged videos is included in the fixed 824-video benchmark.

Table A6. Retrospective two-rater quality-control agreement on 100 retained Arab-DF videos. Exact agreement is the proportion of identical ratings. Cohen’s κ is reported for the binary retain decision; quadratic-weighted κ is reported for the ordinal 0–2 criteria.

Criterion	Rater A	Rater B	Exact Agreement	κ
Overall retain decision	97 retain/3 non-retain	94 retain/6 non-retain	97%	0.653
Face visibility	87 score 2/13 score 1	84 score 2/16 score 1	97%	0.879 *
Temporal interpretability	97 score 2/3 score 1	92 score 2/8 score 1	95%	0.525 *
Severe-corruption assessment	96 score 2/4 score 1	91 score 2/9 score 1	95%	0.593 *

* Quadratic-weighted Cohen’s κ. Three sampled videos were jointly rated non-retain, and three additional videos were disagreements. None of the six flagged videos is included in the fixed 824-video benchmark.

Appendix A.6. Provenance of Celeb-DF-v2 Reference AUCs

Table A7 records the exact DeepfakeBench CDFv2 frame-level AUC values used as the external control for the thirteen-detector benchmark. All values come from the same DeepfakeBench cross-dataset result table [10], which reports frame-level AUC for models trained on FaceForensics++ c23. The main benchmark table rounds these source values to three decimals for compact presentation; the certification decision uses the highest-precision values available. DeepfakeBench names Meso4-Inception as MesoIncep, EfficientNet-B4 as EfficientB4, and CNN-Aug/ResNet-34 as CNN-Aug with a ResNet backbone.

Table A7. Provenance and precision of the Celeb-DF-v2 frame-level reference AUCs used for checkpoint-reproducibility certification. All exact source values are from the CDFv2 column of DeepfakeBench [10].

Manuscript Detector	DeepfakeBench Label	CDFv2 Source AUC	Table 4 Display
EfficientNet-B4	EfficientB4	0.7487	0.749
Xception	Xception	0.7365	0.737
UCF	UCF	0.7527	0.753
CORE	CORE	0.7428	0.743
F3Net	F3Net	0.7352	0.735
RECCE	Recce	0.7319	0.732
Capsule	Capsule	0.7472	0.747
SRM	SRM	0.7552	0.755
SPSL	SPSL	0.7650	0.765
FFD	FFD	0.7435	0.744
Meso4-Inception	MesoIncep	0.6966	0.697
Meso4	Meso4	0.6091	0.609
CNN-Aug/ResNet-34	CNN-Aug	0.7027	0.703

Table A7. Provenance and precision of the Celeb-DF-v2 frame-level reference AUCs used for checkpoint-reproducibility certification. All exact source values are from the CDFv2 column of DeepfakeBench [10].

Manuscript Detector	DeepfakeBench Label	CDFv2 Source AUC	Table 4 Display
EfficientNet-B4	EfficientB4	0.7487	0.749
Xception	Xception	0.7365	0.737
UCF	UCF	0.7527	0.753
CORE	CORE	0.7428	0.743
F3Net	F3Net	0.7352	0.735
RECCE	Recce	0.7319	0.732
Capsule	Capsule	0.7472	0.747
SRM	SRM	0.7552	0.755
SPSL	SPSL	0.7650	0.765
FFD	FFD	0.7435	0.744
Meso4-Inception	MesoIncep	0.6966	0.697
Meso4	Meso4	0.6091	0.609
CNN-Aug/ResNet-34	CNN-Aug	0.7027	0.703

The reference-source table is distinct from the reproduced Celeb-DF-v2 frame AUC and video AUC columns in Table 4, which are measurements from our own control reruns.

Appendix A.7. Exploratory Historical Fine-Tuned Xception Positive Control

For transparency, we retain an auxiliary historical Xception fine-tuning experiment in this appendix rather than the main Results. The preserved record describes 1000 Arab-DF videos (500 authentic and 500 manipulated) producing 9140 face crops resized to 299 × 299 pixels. The recorded configuration used seed 42, 865 training videos (7864 crops), 135 validation videos (1276 crops), AdamW optimization (learning rate 2 × 10⁻⁵, weight decay 10⁻³), label smoothing 0.1, cosine annealing over eight epochs, batch size 32, mixed-precision training, and light image augmentation. The best preserved validation video-level AUC was 0.727.

The saved checkpoint reaches 0.6915 video-level AUC on the fixed 824-video evaluation subset, compared with 0.6083 for the zero-shot Xception checkpoint. This +0.083 difference is descriptive only. The saved checkpoint does not retain sufficient metadata to recover the exact historical training-video identities, and the original fine-tuning notebook is unavailable; source-video and person-identity disjointness therefore cannot be independently verified. The result may be affected by leakage and must not be interpreted as a leakage-free estimate of supervised generalization or as the causal benefit of fine-tuning. It is excluded from all primary aggregates and conclusions. Table A8 gives the corresponding per-family zero-shot and historical fine-tuned Xception point estimates.

Table A8. Exploratory per-family video-level AUC for zero-shot and historical fine-tuned Xception on the fixed Arab-DF evaluation subset. Gain is the difference between the two point estimates; n is the number of manipulated evaluation videos in each family. The table is descriptive only because the historical fine-tuning split cannot be verified as source-video- or person-identity-disjoint.

Manipulation Family	Zero-Shot Xception	Fine-Tuned Xception	Gain	Eval. Manipulated Videos (n)
Face swapping	0.596	0.701	+0.105	432
Mobile application	0.642	0.719	+0.077	180
Facial reenactment	0.621	0.652	+0.031	71
Lip synchronization	0.584	0.596	+0.013	66

Table A8. Exploratory per-family video-level AUC for zero-shot and historical fine-tuned Xception on the fixed Arab-DF evaluation subset. Gain is the difference between the two point estimates; n is the number of manipulated evaluation videos in each family. The table is descriptive only because the historical fine-tuning split cannot be verified as source-video- or person-identity-disjoint.

Manipulation Family	Zero-Shot Xception	Fine-Tuned Xception	Gain	Eval. Manipulated Videos (n)
Face swapping	0.596	0.701	+0.105	432
Mobile application	0.642	0.719	+0.077	180
Facial reenactment	0.621	0.652	+0.031	71
Lip synchronization	0.584	0.596	+0.013	66

The point-estimate gains vary from +0.105 for face swapping to +0.013 for lip synchronization. Because the composition and identity/source structure of the historical fine-tuning material cannot be reconstructed sufficiently, these differences are not attributed to training-set representation and are not used to support the primary Arab-DF conclusions.

Appendix A.8. Historical Selection Audit of the Fixed 824-Video Benchmark

Table A9 records the subset-selection information that can be reconstructed from surviving artifacts. The preserved historical experimental artifacts include ArabDF_metadata_large_5484_preprocessed.csv and videomae_large_5484_features.npz. The table deliberately distinguishes membership in historical experimental manifests from documented rejection. The historical files do not contain a complete stage-wise selection log from the 12,306-video release to the fixed 824-video benchmark, so unavailable reasons are reported as unavailable rather than retrospectively inferred.

Table A9. Reconstructible selection audit for the fixed Arab-DF benchmark. Counts describe surviving artifacts; membership outside a later stage is not interpreted as a quality-control rejection unless a preserved record supports that interpretation.

Audit Stage/Artifact	Documented Count	Interpretation/Limitation
Released Arab-DF collection	12,306 = 1066 authentic + 11,240 manipulated	Current released collection total.
Historical 5484-video experimental manifest	5484 = 500 authentic + 4984 manipulated	6822 release videos are not represented. No preserved selection script assigns reasons to them; they are not described as rejected.
Historical split within that manifest	Train 3838 (350/3488); validation 822 (75/747); test 824 (75/749)	The 4660 videos outside the test cohort are train/validation members, not documented QC rejections. Historical split labels changed across later artifacts.
Fixed benchmark manifest	824 = 75 authentic + 749 manipulated	Same fixed cohort used for all 13 public detector evaluations. The historical path selecting these identifiers is only partially reconstructible.
Frame sampling and face alignment	26,368 candidate frames → 25,127 valid crops	1241 frames excluded (4.71%); 0 whole videos excluded. All 824 videos retained at least one valid crop.

Table A9. Reconstructible selection audit for the fixed Arab-DF benchmark. Counts describe surviving artifacts; membership outside a later stage is not interpreted as a quality-control rejection unless a preserved record supports that interpretation.

Audit Stage/Artifact	Documented Count	Interpretation/Limitation
Released Arab-DF collection	12,306 = 1066 authentic + 11,240 manipulated	Current released collection total.
Historical 5484-video experimental manifest	5484 = 500 authentic + 4984 manipulated	6822 release videos are not represented. No preserved selection script assigns reasons to them; they are not described as rejected.
Historical split within that manifest	Train 3838 (350/3488); validation 822 (75/747); test 824 (75/749)	The 4660 videos outside the test cohort are train/validation members, not documented QC rejections. Historical split labels changed across later artifacts.
Fixed benchmark manifest	824 = 75 authentic + 749 manipulated	Same fixed cohort used for all 13 public detector evaluations. The historical path selecting these identifiers is only partially reconstructible.
Frame sampling and face alignment	26,368 candidate frames → 25,127 valid crops	1241 frames excluded (4.71%); 0 whole videos excluded. All 824 videos retained at least one valid crop.

The current metadata release labels the fixed 824 identifiers differently from the older 5484-video feature manifest. Because no preserved audit trail reconciles these changing split labels, the benchmark is identified by its fixed video-ID manifest and class/family composition rather than by a historical split name.