DeepFakeX: A Comprehensive Multimodal Deepfake Dataset for Research and Analysis

Abstract

The expanding capabilities of deep learning-based media synthesis have intensified concerns regarding the authenticity of digital content and the reliability of forensic analysis tools. In response to these challenges, this work introduces DeepFakeX, a collection of 800 synthetically generated videos available under controlled access for research purposes. The dataset encompasses four distinct categories of AI-driven synthesis: facial identity replacement, audio track substitution, neural voice cloning, and combined audiovisual alteration. Unlike existing deepfake datasets that predominantly focus on facial synthesis, DeepFakeX covers a broader range of manipulation modalities, reflecting the diversity of synthetic media encountered in real-world settings. All deepfakes were generated using state-of-the-art, publicly available tools. Standardized post-processing procedures were applied to each video to ensure uniformity in terms of quality, duration and encoding format. DeepFakeX also emphasizes diversity in gender, age, ethnicity, and language. Video contexts span speeches, informational videos, movie clips, news broadcasts, and interviews that reflect content scenarios commonly encountered in real-world online environments. The dataset includes videos in both English and Urdu. The dataset’s quality and structural variability were assessed through visual and audio analyses using the Structural Similarity Index Measure (SSIM), Mel-Frequency Cepstral Coefficients (MFCCs), and Principal Component Analysis (PCA). The evaluation results revealed substantial variability within each manipulation category, along with clearly distinguishable patterns specific to each modality. DeepFakeX has been developed to facilitate rigorous and transparent research in deepfake detection, cross-modal forensic analysis, and AI-driven media forensics. It is hosted on Zenodo under controlled access for research use.

1. Summary

The development of AI-generated media content, and deepfakes in particular, poses a multifaceted challenge concerning media authenticity, disinformation mitigation, and digital forensics [1]. Most datasets curated for deepfake detection have significant limitations in terms of modality coverage, manipulation type diversity, and demographic representation. DeepFakeX was initially developed during the evaluation of a deepfake detection model. During early testing, an initial set of 200 deepfake videos was generated to assess model performance across four manipulation modalities, and this subset was subsequently documented in a peer-reviewed article [2].

Building on this foundation, the dataset was expanded to 800 videos, offering a resource that the research community can utilize for the development and benchmarking of deepfake detection models. It consists of four deepfake types: face-swapped, audio-swapped, voice-cloned, and combined audiovisual. The deepfake content was created using a range of publicly accessible software frameworks and mobile applications, enabling realistic reproduction of diverse real-world media scenarios. DeepFakeX covers a wide range of video scenarios, genders, languages, ethnicities, and resolutions, making it broadly applicable to deepfake research. This data article outlines the dataset structure and describes how the data were obtained, providing the foundation for rigorous academic use and ensuring reusability across deepfake detection, multimedia forensics, and related domains.

The first step involved exploratory evaluation using structural and spectral measures, including SSIM [3], while acoustic characteristics [4] were assessed using extracted audio features. In addition, PCA was applied to MFCCs [5] to analyze clustering behavior within the feature space. These analyses confirmed the clear separability of the investigated categories and revealed the existence of modality-specific traits in the synthesized videos. DeepFakeX serves as a resource for scholarly research in the areas of deepfake detection, synthetic content assessment, and audiovisual forensic investigation. The dataset is released under the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) license and hosted on the Zenodo platform. The dataset is intended to foster open science practices, research transparency, and responsible progress in artificial intelligence.

2. Data Description

The design and structure of DeepFakeX are described in the subsequent subsections. This section presents a detailed account of the dataset’s organization, storage format, file naming conventions, and the diversity of deepfake modalities included. While the dataset is broadly applicable across multiple research domains, it is particularly suited to audiovisual forensic evaluation.

A machine-readable metadata file in CSV format is also provided with the dataset. This file contains key attributes for each video sample, including the video identifier, filename, deepfake category, generation method, language, contextual scenario, resolution, and duration, enabling easier dataset exploration and reproducibility of experimental studies.

2.1. Dataset Overview

The DeepFakeX dataset contains a total of 800 synthetically manipulated video samples. These samples are systematically organized into four primary categories based on the type of artificial alteration applied:

• Identity replacement through facial manipulation.
• Audio track substitution while preserving the original visuals.
• Artificially generated speech imitating the original speaker.
• Simultaneous manipulation of both facial identity and audio content.

Each category represents a distinct form of synthetic modification, enabling comprehensive evaluation across different deepfake modalities. All samples were created using commonly accessible, state-of-the-art desktop software, mobile applications, and AI-based tools. All media files were encoded in MP4 format using the H.264 codec, with video resolutions of either 720p or 1080p, durations ranging from 4 to 12 s, and frame rates of 25 or 30 frames per second. The audio signals were normalized across the dataset to ensure consistent loudness levels and perceptual audio quality.

The dataset follows a structured directory hierarchy, where compressed archives are grouped by deepfake category. Each folder contains video files that follow a standardized filename structure, where the assigned label clearly identifies the corresponding category of deepfake.

2.2. Directory Organization and File Naming Convention

DeepFakeX is organized into four primary directories (provided as compressed archives), with each directory dedicated to a specific deepfake type:

• deepfake_face_swapped/—facial identity altered while preserving the original audio track.
• deepfake_audio_swapped/—audio content modified while retaining the original visual stream.
• deepfake_voice_cloned/—synthetic voice generation applied to the original visual content.
• deepfake_combined_face_audio_swapped/—facial identity and audio content are both synthetically altered.

Table 1. Video prefixes according to the corresponding deepfake category.

Prefix	Type of Deepfake
ff_	Face-swapped
fa_	Audio-swapped
vc_	Voice-cloned
ffa_	Face- + audio-swapped

lists the prefixes used to identify each deepfake category in the filename structure.

The name of every video file has an appended prefix that represents the category of deepfake, followed by an identifier. For example, ff_047.mp4 denotes the forty-seventh face-swapped video. Because deepfake types are already encoded in the filenames, separate annotation files are not required.

2.3. Modal Diversity

DeepFakeX reflects the diversity of user-generated media encountered in real-world online environments. The videos feature subjects diversified across the following dimensions:

• Ethnicities: South Asian, African, East Asian, and Caucasian.
• Gender Identity: Male, female, and transgender individuals are represented.
• Age groups: Young, middle-aged, and older adults.
• Languages: Videos are primarily in English, with a subset in Urdu.
• Contexts: Video contexts include speeches, informational videos, interviews, news coverage, vlogs, and film excerpts.

Each sample was carefully selected to exclude sensitive content, including material involving minors, political leaders, religious figures, election-related events, and sexually explicit material. The deepfake videos are predominantly characterized by a single speaker in a frontal, camera-facing orientation with clear, intelligible speech.

Table 2. Demographic and contextual attribute distribution in DeepFakeX.

Attribute	Demographic Category	Proportion (%)
Ethnicity	South Asian	37.8
	Caucasian	32.7
	East Asian	15.3
	African	14.3
Gender Identity	Male	51.0
	Female	47.0
	Transgender	2.0
Age Group	Young-aged	19.2
	Middle-aged	60.6
	Older Adults	20.2
Video Context	News segments	12.7
	Speeches	22.4
	Informational/Video clips	34.7
	Interviews	23.5
	Film Excerpts	7.1

summarizes the ethnicity, gender, age, and content-context distributions within the dataset.

2.4. Data Specifications and Availability

• Video format: All files are provided in .mp4.
• Audio configuration: Mono or stereo tracks with normalized volume levels.
• Spatial resolution: Videos are available in 720p or 1080p formats.
• Video duration: Each video ranges between 4 and 12 s.
• Repository link: https://doi.org/10.5281/zenodo.14202454
• Usage License: Distributed under the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) license.

3. Methods

3.1. Deepfake Video Generation and Processing Pipeline

The pipeline used to construct DeepFakeX consists of several sequential stages, each designed to ensure consistency, diversity, and quality of the generated samples. DeepFakeX was developed through the following steps:

(1)

Source content selection;

(2)

Use of deepfake creation applications;

(3)

Post-processing operations for uniformity;

(4)

Dataset organization and file structuring.

The synthetic videos in DeepFakeX were generated using automated processes from publicly available source content featuring public figures (celebrities, content creators, and public-domain media), in line with established practice in the deepfake research literature (e.g., FakeAVCeleb, Celeb-DF, FaceForensics++). The original source videos are not redistributed; only the synthetic derivatives are included in the public release. The research does not involve human subjects in an experimental capacity, does not collect personal data from private individuals, and follows established ethical frameworks (IEEE Code of Ethics, ACM Code of Ethics, and the core principles of the EU General Data Protection Regulation), particularly the legitimate interests basis (GDPR Article 6(1)(f)) for research use of publicly available material.

3.2. Deepfake Generation Tools and Software Platforms

Table 3. Deepfake generation tools and corresponding video distribution across the four deepfake types.

Deepfake Category	Generation Tools/Platforms	Number of Videos
Face-swapped	DeepFaceLab [6], SimSwap [7], First Order Motion Model [8], SwapFace [9], Reface [10]	500
Audio-swapped	Wav2Lip [11], Pika Labs [12], Lalamu Studio [13]	100
Voice-cloned	ElevenLabs [14], Play.ht [15], Gooey.ai [16], Lalamu Studio [13]	100
Combined (Face + Audio) swapped	Combination of face-swap and audio-swap tools listed above	100

summarizes the desktop-based software, mobile applications, and tools that were utilized to generate each deepfake category.

All generation frameworks and software tools were executed on a single local machine without relying on cloud services or distributed computing infrastructure. This setup ensured full control over the synthesis process, supporting consistency and replicability throughout dataset construction.

3.3. Post-Processing and Dataset Curation

To improve the quality and consistency of the created deepfake videos, the following post-processing were applied:

• Format Standardization: The videos were encoded in MP4 format by utilizing the H.264 codec.
• Resolution Adjustment: The video files were scaled to either 720p or 1080p resolution.
• Frame Rate Normalization: The frame rate was standardized to 25 or 30 frames per second to ensure temporal consistency.
• Audio Level Normalization: Audio levels were calibrated to achieve balanced and uniform audio output across the dataset.
• File Naming Protocol: Each file was assigned a standardized prefix (e.g., ff_, fa_, vc_, ffa_) identifying its deepfake category.

In addition, every generated sample was manually reviewed to identify and remove those exhibiting visible artifacts, generation failures, or insufficient synthesis quality. Only samples demonstrating both visual and acoustic coherence were retained in the final dataset.

3.4. Dataset Validation and Quality Assessment

To analyze the structural diversity and audiovisual characteristics of the DeepFakeX dataset, a set of exploratory analytical procedures was employed. For comparative analysis, a set of original video sources served as reference material for computing visual and audio feature statistics. In particular, the Structural Similarity Index Measure (SSIM) [3] was used to quantify visual similarity between original and manipulated frames. SSIM measures perceptual similarity based on luminance, contrast, and structural coherence, producing values between 0 (no similarity) and 1 (perfect match). SSIM has been widely adopted in image and video quality assessment tasks, including deepfake detection and image manipulation analysis, and is used in this work to characterize the visual variability of synthetic content across manipulation categories.

These reference clips were used only to provide a baseline comparison between authentic and manipulated content. The original videos are not included in the released DeepFakeX dataset, as they were used solely during the validation stage and are excluded due to licensing and ethical considerations.

3.4.1. Visual Integrity Evaluation by Structural Similarity Index Measure (SSIM)

To characterize the visual integrity and diversity of the DeepFakeX dataset, we conducted two complementary SSIM analyses: (i) an inter-category analysis measuring structural deviation between real source videos and their corresponding manipulated counterparts, and (ii) an intra-category analysis measuring within-category visual diversity. SSIM produces values between 0 (no similarity) and 1 (perfect match) based on perceptual differences in luminance, contrast, and structural coherence; lower SSIM values indicate greater visual dissimilarity.

Inter-Category SSIM Analysis: For each manipulation category, ten real source videos were selected and compared against ten corresponding manipulated videos, yielding 100 pairwise comparisons per category. SSIM was computed on the first 100 frames of each video, with frames aligned by temporal index. The average inter-category SSIM scores were approximately 0.41 for real vs. face-swapped (range 0.19–0.90), 0.35 for real vs. audio-swapped, 0.39 for real vs. voice-cloned (range 0.18–0.66), and 0.33 for real vs. face-and-audio-swapped (range 0.15–0.61). These results indicate that all four manipulation types introduce substantial structural deviation from authentic source content, with combined face-and-audio-swapped videos showing the greatest deviation which is consistent with the compounded nature of dual-modality manipulation.

Intra-Category SSIM Analysis: To assess natural visual diversity within each category, all pairwise comparisons among ten randomly selected videos per category were computed (45 comparisons per category). The intra-category mean SSIM scores were:

-

Real: 0.37 (range 0.15–0.61).

-

Face-swapped: 0.32 (range 0.16–0.66).

-

Audio-swapped: 0.35 (range 0.04–0.63).

-

Voice-cloned: 0.36 (range 0.19–0.98).

-

Face-and-audio-swapped: 0.44 (range 0.33–0.77).

These intra-category values capture the inherent visual diversity of each category, reflecting variation in scenes, subjects, lighting, and manipulation techniques, rather than near-identity between a video and itself.

Interpretation: Figure 1 reports the intra-category mean SSIM values for each of the five categories. The “real” category value (~0.37) represents intra-category SSIM (real videos compared to other real videos in the dataset), not real-vs-itself (which would trivially yield ~1.0). The moderate value reflects the natural visual diversity across different real-world scenes and subjects and serves as a diversity baseline against which the intra-category values for manipulated categories can be interpreted. Notably, the face-and-audio-swapped category shows the highest intra-category SSIM (0.44), suggesting more uniform manipulation patterns within this category compared with single-modality manipulations.

The wide range of SSIM values across both inter- and intra-category comparisons confirms that DeepFakeX captures a realistic spectrum of synthesis complexity and visual diversity, supporting its use as a benchmark for evaluating detection performance across varying manipulation severities. In addition to SSIM-based characterization, all synthetic videos were subjected to manual visual inspection by the research team during dataset construction to verify acceptable visual quality and to exclude generation failures (e.g., severely distorted faces, audio synchronization errors) from the final release.

3.4.2. Audio Feature Analysis

To analyze the audio content of voice-cloned and audio-swapped deepfakes systematically, three major audio features were extracted:

• Spectral centroid and Spectral Rolloff [17] capture spectral brightness distribution and overall energy concentration.
• Zero-crossing rate (ZCR) [5] measures waveform complexity and the rate of signal sign changes, reflecting transient characteristics.
• Mel-Frequency Cepstral Coefficients (MFCCs) [5] encode phonetic patterns and vocal-tract properties, and are widely used in speech and speaker recognition tasks.

Figure 2 presents boxplots comparing these features between original and synthetic samples. The empirical findings demonstrated that:

• Audio generated through audio-swapping and voice-cloning techniques exhibited reduced spectral centroid and zero-crossing rate values, indicating more uniform and less complex waveform characteristics.
• The MFCCs extracted from synthetic clips were more uniform compared to real audio; however, inter-class differences remained sufficiently distinct to support effective classification.
• Real audio recordings exhibited a wider range of feature values, reflecting natural variation in pitch, timbre, and articulation.

3.4.3. Audio Feature Correlation Heatmap

Figure 3 presents a correlation heatmap [18] computed to examine the relationships among spectral audio features. The heatmap revealed the following:

• Strong correlations among the spectral attributes of synthetic audio reflect the structured, algorithm-driven nature of the generation pipeline.
• Real audio exhibited weaker and more diffuse inter-feature correlations, consistent with the natural variability inherent in human speech.

3.4.4. Spectrogram Comparison

Mel spectrograms [17] were computed for representative samples from each category and are presented in Figure 4. These representations reveal differences in spectral complexity and temporal dynamics across the deepfake categories.

• Real speech exhibited complex harmonic transitions and greater diversity in energy distribution across the time-frequency domain.
• Synthetic audio displayed flatter spectral profiles, characterized by smoother frequency bands and reduced temporal variation, consistent with the constrained output of generative synthesis pipelines.

3.4.5. Principal Component Analysis of MFCC Features

Principal Component Analysis (PCA) was applied to MFCC feature vectors to examine the distribution and clustering of samples across deepfake categories in the feature space.

• Synthetic samples exhibited tight clustering, suggesting relative homogeneity introduced by the synthesis pipeline.
• Real samples were more dispersed, reflecting the natural variability present in human speech.

3.4.6. Data Quality Assurance and Noise Mitigation

All audiovisual recordings were subjected to a manual inspection procedure designed to omit samples that exhibit the following:

• Visible artifacts such as facial distortions and lip-synchronization mismatches.
• Misalignment between vocal and facial expressions.
• High levels of background noise or extended silent segments.

The dataset was curated to exclude duplicates and to ensure that every retained sample contributes distinct structural or acoustic information.

These analyses are intended to provide exploratory insights into the structural and acoustic characteristics of the dataset rather than a definitive validation of its quality.

Figure 5 presents the PCA projection of MFCC features across the real, audio-swapped, and voice-cloned categories, illustrating the clustering behavior described above.

3.4.7. Benchmark Evaluation Using State-of-the-Art Detection Models

To further assess the practical utility and research relevance of the DeepFakeX dataset, benchmarking experiments were conducted using a diverse set of state-of-the-art (SOTA) deepfake detection models. These models span visual-only, audio-only, lip-synchronization, and multimodal detection approaches, enabling assessment of model behavior across different manipulation categories. All detectors were evaluated in an inference-only setting, without additional training or fine-tuning on DeepFakeX.

Evaluation Subset: Each baseline was applied only to the DeepFakeX subset corresponding to the manipulation type the baseline was designed to detect, paired with real reference videos. Specifically:

-

Visual-only detectors targeting face-swap manipulations (MesoNet, GenConViT, AltFreezing, FreqNet, CNN-based Ensemble) were evaluated on the 500 face-swapped videos plus the 200 real reference videos.

-

Audio-only detectors (Synthetic Speech Detection, FakeSound, RawNet2Vocoder) were evaluated on the 100 audio-swapped or 100 voice-cloned videos (as appropriate to each model’s published scope) plus the 200 real reference videos.

-

Audiovisual and lip-synchronization detectors (SyncNet, SyncNet with Frequency, LIPINC) were evaluated on the audio-swapped, voice-cloned, and combined face-and-audio-swapped subsets plus the 200 real reference videos.

-

Multimodal detectors (FakeAVCeleb Ensemble, REFEREE, AVT2-DWF, Not Made for Each Other) were evaluated on all 800 synthetic videos plus the 200 real reference videos.

Train/Test Protocol: As DeepFakeX is positioned as an evaluation benchmark rather than a training resource in this work, all baseline models were applied in an inference-only configuration using their publicly released pretrained weights. No fine-tuning or re-training on DeepFakeX was performed for any baseline.

Preprocessing: Each baseline was evaluated using its published preprocessing pipeline to preserve its native performance characteristics. Common preprocessing steps included frame extraction at the model’s native frame rate, face detection and cropping (typically 224 × 224 for visual-only and multimodal models), and audio normalization with model-specific spectrogram parameters (e.g., Mel spectrograms with model-specified channel counts and analysis windows). Where a baseline required a specific input format, that baseline’s published preprocessing was applied; otherwise standard frame and audio extraction was used.

Decision Threshold: All baselines used the standard 0.5 sigmoid decision threshold to classify a video as authentic or manipulated. No per-model threshold tuning was performed, to avoid introducing evaluator bias.

Metric Computation: The F1-score reported in

Table 4. Benchmark performance of state-of-the-art deepfake detection models evaluated on DeepFakeX.

Model	Detection Modality	Applied Deepfake Types	F1-Score
FakeAVCeleb Ensemble [19]	Audio–Visual	All deepfake types	0.81
FakeSound [20]	Audio-only	Audio-swapped, voice-cloned	0.69
REFEREE [21]	Audio–Visual	All deepfake types	0.64
RawNet2Vocoder [22]	Audio-only	Voice-cloned	0.76
LIPINC [23]	Visual-only	Audio-swapped	0.75
AltFreezing [24]	Visual-only	Face-swapped	0.75
AVT2-DWF [25]	Audio–Visual	All deepfake types	0.80
FreqNet [26]	Visual-only	Face-swapped	0.70
SyncNet [11]	Audio–Visual	Audio-swapped	0.20
SyncNet with Frequency [2]	Audio–Visual	All deepfake types	0.69
MesoNet [27]	Visual-only	Face-swapped	0.54
CNN-based Ensemble (Face Manipulation) [28]	Visual-only	Face-swapped	0.35
Not Made for Each Other [29]	Audio–Visual	All deepfake types	0.62
Synthetic Speech Detection [30]	Audio-only	Audio-swapped	0.54
GenConViT (Hybrid CNN-Transformer) [31]	Visual-only	Face-swapped	0.68

was computed separately for each baseline on its applicable DeepFakeX subset (as listed under “Applied Deepfake Types”), rather than as a pooled or averaged value across all manipulation categories. This per-category reporting provides a more informative comparison of each baseline’s detection capability within the manipulation type it was designed for, and it avoids artificially deflating unimodal baselines on manipulation types outside their intended scope.

Table 4 summarizes the benchmarking results across different detection modalities and applicable deepfake types.

The benchmarking results demonstrate that multimodal approaches generally achieve stronger performance across combined manipulation types, whereas unimodal detectors show varying effectiveness depending on the manipulation modality. We note that “FakeAVCeleb Ensemble” in Table 4 refers to the ensemble detection baseline from the FakeAVCeleb paper [19] (combining Xception, MesoInception4, and Meso4), distinct from the FakeAVCeleb dataset discussed in the dataset comparison (Table 5).

Visual-only models perform competitively on face-swapped videos, while audio-based models show improved performance on voice-cloned and audio-swapped samples. These results underscore the importance of multimodal detection approaches and confirm the suitability of DeepFakeX as a challenging benchmark for evaluating cross-modal robustness and generalization.

3.5. Ethical Considerations

The DeepFakeX dataset was developed using generative AI techniques applied to publicly accessible source material featuring public figures (celebrities, content creators, and public-domain media), consistent with the established practice of comparable deepfake datasets including FakeAVCeleb [19], Celeb-DF, and FaceForensics++. We acknowledge that public availability does not, in itself, eliminate the need for explicit ethical and legal grounding. Accordingly, the ethical positioning of DeepFakeX rests on the following:

Legal basis: The use of publicly available source content is grounded in the legitimate-interests basis under Article 6(1)(f) of the EU General Data Protection Regulation [32], supplemented by the data-minimization principle of GDPR Article 5(1)(c), operationalized by the non-redistribution of source videos—a stronger safeguard than is typical of comparable datasets.

Licensing: The synthetic content is released under the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) license and hosted on Zenodo (DOI: 10.5281/zenodo.14202454) under a controlled-access mechanism by which researchers may request access subject to creator review.

Ethical review: The research does not involve human subjects in an experimental capacity, does not collect personal data from private individuals, and does not involve interaction with the people whose public-figure likenesses appear in the source content. Under these conditions, formal institutional ethics committee review was not required at our institution. The project nonetheless adhered throughout to the principles of established ethical frameworks: the IEEE Code of Ethics [33] (Principle I.5: respect for privacy; Principle I.7: avoid injury), the ACM Code of Ethics [34] (Principle 1.2: avoid harm), and the GDPR principles cited above.

Operational safeguards: Specific exclusion criteria were applied during dataset construction: content involving minors, political leaders, religious figures, election-related material, sexually explicit content, and private (non-public) individuals was strictly excluded.

All videos in the dataset are synthetically created and are intended solely for academic and research purposes, consistent with MDPI’s ethical principles for research publication and with established principles of responsible AI development.

4. Dataset Novelty and Research Relevance

DeepFakeX seeks to overcome limitations in current deepfake datasets by offering carefully structured and systematically curated resources that will enable future research in the field of deepfake detection. Although popular databases like FaceForensics++ [35], DFDC [36], and Celeb-DF [37] have made significant progress in advancing the field, they are subject to certain limitations, including the following:

Modality Imbalance: Most existing datasets focus predominantly on face-swap artifacts, with limited coverage of audio-based manipulations such as voice cloning or audio swapping. DeepFakeX addresses this by incorporating four deepfake types, i.e., face-swapped, audio-swapped, voice-cloned, and combined face-and-audio-swapped—enabling comprehensive evaluation across both unimodal and multimodal detection frameworks. The higher proportion of face-swapped samples reflects the greater prevalence of facial identity manipulation in real-world deepfake media. The dataset is used exclusively as an evaluation benchmark in this work; detection models were not trained on DeepFakeX but applied in an inference-only configuration using their publicly released pretrained weights (Section 3.4.7) so the asymmetric sample distribution does not introduce class-imbalance bias during model training. For benchmarking, per-category metrics are reported separately for each manipulation type in Table 4, ensuring fair evaluation regardless of sample size. Users intending to fine-tune or train detection models on DeepFakeX should consider stratified sampling or class-weighted loss to ensure balanced representation across categories.

Demographic Homogeneity: Existing datasets frequently exhibit imbalances in demographic representation across gender, age group, ethnicity, and language, which limits the generalizability of trained detection models. DeepFakeX was curated to achieve balanced representation across genders (male, female, and transgender) and includes speakers from different ethnicities, with content in both English and Urdu. This demographic and linguistic diversity is expected to enhance the generalizability of models trained on the dataset.

Contextual Limitations: Most existing datasets feature paid actors or scripted, studio-created content. In comparison, DeepFakeX consists of real-life scenarios, such as news-style broadcasts, public addresses, and educational content, which reflect the type of media regularly seen on digital platforms.

To further characterize the dataset, analytical evaluations were conducted using SSIM, MFCC, and PCA, demonstrating structural variability and modality-specific patterns across deepfake categories.

By addressing these limitations, DeepFakeX is designed to support:

• Evaluation of deepfake detection performance across many modalities (visual, audio, and multimodal).
• Generalization testing across diverse languages, demographic groups, and real-world scenarios.
• Development of bias-aware and explainable deepfake detection models.
• Enhancement of audiovisual coherence analysis for digital forensic investigation.

By enabling the examination of manipulated synthetic media under diverse, real-world conditions, DeepFakeX contributes to ethical artificial intelligence, integrated audiovisual forensic research, and the design of equitable detection frameworks.

To further position DeepFakeX within the existing landscape of deepfake datasets, a comparative analysis is presented in Table 5. The comparison highlights key characteristics, including manipulation types, modality coverage, and diversity-related attributes. Unlike many existing datasets that primarily focus on visual manipulations, DeepFakeX incorporates multiple forms of synthetic media generation and supports both audio and visual modalities, enabling comprehensive multimodal analysis.

Table 5. Comparative analysis of DeepFakeX with existing deepfake datasets based on manipulation types, modality coverage, and diversity attributes.

Dataset	Face Swap	Audio Swap	Voice Cloning	Multimodal	Demographic Diversity	Language Diversity	Realistic Scenarios
FaceForensics++ [35]	✓	X	X	X	Medium	Low	Partial
Celeb-DF v2 [37]	✓	X	X	X	Medium	Low	Partial
DFDC [36]	✓	Δ	Δ	✓	High	Low	Partial
DeepFake-TIMIT [38]	✓	X	X	X	Low	Low	No
FakeAVCeleb [19]	✓	X	✓	✓	High	Medium	Partial
DeepFakeX (Proposed)	✓	✓	✓	✓	High	Medium	Full

Note: Diversity ratings are defined by objective criteria applied uniformly across all entries: (1) Demographic Diversity—High: ≥3 ethnicities with balanced gender and varied age coverage; Medium: 2 ethnicities or partial balance; Low: predominantly homogeneous. (2) Language Diversity—High: ≥3 languages with balanced representation; Medium: 2 languages; Low: single-language/English-dominated. (3) Realistic Scenarios—(Full): multiple real-world contexts (e.g., news, interviews, tutorials, speeches); Partial: limited or indirect real-world support; No: controlled or laboratory settings only. Symbols: ✓ supports manipulation type; X absent; Δ partial support.

Table 5. Comparative analysis of DeepFakeX with existing deepfake datasets based on manipulation types, modality coverage, and diversity attributes.

Dataset	Face Swap	Audio Swap	Voice Cloning	Multimodal	Demographic Diversity	Language Diversity	Realistic Scenarios
FaceForensics++ [35]	✓	X	X	X	Medium	Low	Partial
Celeb-DF v2 [37]	✓	X	X	X	Medium	Low	Partial
DFDC [36]	✓	Δ	Δ	✓	High	Low	Partial
DeepFake-TIMIT [38]	✓	X	X	X	Low	Low	No
FakeAVCeleb [19]	✓	X	✓	✓	High	Medium	Partial
DeepFakeX (Proposed)	✓	✓	✓	✓	High	Medium	Full

Note: Diversity ratings are defined by objective criteria applied uniformly across all entries: (1) Demographic Diversity—High: ≥3 ethnicities with balanced gender and varied age coverage; Medium: 2 ethnicities or partial balance; Low: predominantly homogeneous. (2) Language Diversity—High: ≥3 languages with balanced representation; Medium: 2 languages; Low: single-language/English-dominated. (3) Realistic Scenarios—(Full): multiple real-world contexts (e.g., news, interviews, tutorials, speeches); Partial: limited or indirect real-world support; No: controlled or laboratory settings only. Symbols: ✓ supports manipulation type; X absent; Δ partial support.

As shown in Table 5, most existing datasets predominantly focus on face-swapped visual manipulations and provide limited or no support for audio-based or multimodal deepfake scenarios. In contrast, DeepFakeX integrates multiple manipulation types, including face swapping, audio swapping, and voice cloning, within a unified framework. This enables more comprehensive evaluation of both unimodal and multimodal deepfake detection approaches.

Furthermore, DeepFakeX emphasizes diversity in terms of demographic attributes and linguistic variation, which are often underrepresented in existing datasets. The inclusion of multiple languages and varied real-world contexts enhances its applicability for realistic and robust deepfake detection research. These characteristics collectively distinguish DeepFakeX as a more versatile and application-oriented resource compared to existing datasets.

5. Limitations

While DeepFakeX provides a valuable multimodal resource for deepfake detection research, several limitations should be acknowledged.

Dataset Scale and Language Coverage: DeepFakeX comprises 800 synthetic videos across four manipulation categories, supplemented with 200 real reference videos. While this scale is comparable to several established deepfake datasets, larger collections remain valuable for training data-intensive detection models. Similarly, the current release covers two languages, English and Urdu, which represents an advance over predominantly English language existing datasets, but broader multilingual expansion is a priority for future releases.

Generative Model Coverage: The synthetic content was produced using a selected set of state-of-the-art generation tools current at the time of dataset construction (2024). Deepfakes generated by emerging or future generative architectures may exhibit different artifact signatures, and detection models trained exclusively on DeepFakeX may require periodic re-training to maintain performance against evolving manipulation techniques.

Demographic Representation: While DeepFakeX offers broader demographic representation than several predecessor datasets covering multiple ethnicities, gender distributions, and content contexts—demographic coverage remains finite. Subtle demographic biases may persist, and ongoing efforts to expand representation are planned.

Operational Deployment: DeepFakeX is intended as a research dataset for the development and benchmarking of deepfake detection methods. Real-world operational deployment of detectors trained on DeepFakeX, including testing under live-streaming conditions, adversarial perturbation scenarios, and integration with content-moderation pipelines is beyond the scope of the current work and represents a direction for applied research.

These limitations identify natural directions for future work and underscore the importance of ongoing dataset development in this fast-moving research area.

6. User Notes

The DeepFakeX dataset is hosted on Zenodo and can be accessed via the DOI: https://doi.org/10.5281/zenodo.14202454. It is released under the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) license. Due to ethical and licensing considerations associated with the distribution of synthetic media containing identifiable human faces and voices, the dataset is provided under controlled access. Researchers interested in using the dataset for academic and research purposes may request access through the Zenodo platform. All access requests are reviewed by the dataset creators to support responsible use of the data.

The source media used to generate synthetic samples were obtained from publicly accessible online content and were used solely for research and dataset construction purposes. These original materials are not redistributed as part of DeepFakeX.

To improve transparency and reproducibility, a machine-readable metadata file in CSV format is also provided with the dataset. This file contains descriptive information for each video sample, including identifiers, deepfake category, generation method, language, context, resolution, and duration. The metadata file allows researchers to systematically analyze, filter, and organize the dataset for experimental studies.

No additional annotation files are included, as the dataset organization, folder hierarchy, filename prefixes, and accompanying metadata file clearly indicate the corresponding deepfake category and associated attributes for each sample. The dataset can be utilized by researchers to:

• Train and evaluate deepfake detection systems across visual, audio, and multimodal frameworks.
• Examine model robustness and generalization across diverse synthesis techniques.
• Conduct research in digital media forensics, synthetic media detection, and content authenticity verification.

The majority of videos feature frontal facial views and clearly audible speech, with demographic diversity across gender, age, and ethnicity, enhancing suitability for algorithm development and experimental studies involving audiovisual data processing. In addition, the inclusion of face-swapped, audio-swapped, voice-cloned, and combined audiovisual manipulations makes the dataset particularly valuable for multimodal deepfake detection research.

Researchers using DeepFakeX are requested to cite this manuscript in their publications and may also use the provided metadata attributes and extracted feature statistics (e.g., SSIM trends and MFCC characteristics) as a foundation for further validation, benchmarking, or modeling efforts.