Search Results (108)

The expanding capabilities of deep learning-based media synthesis have intensified concerns regarding the authenticity of digital content and the reliability of forensic analysis tools. In response to these challenges, this work introduces DeepFakeX, a collection of 800 synthetically generated videos available under controlled access for research purposes. The dataset encompasses four distinct categories of AI-driven synthesis: facial identity replacement, audio track substitution, neural voice cloning, and combined audiovisual alteration. Unlike existing deepfake datasets that predominantly focus on facial synthesis, DeepFakeX covers a broader range of manipulation modalities, reflecting the diversity of synthetic media encountered in real-world settings. All deepfakes were generated using state-of-the-art, publicly available tools. Standardized post-processing procedures were applied to each video to ensure uniformity in terms of quality, duration and encoding format. DeepFakeX also emphasizes diversity in gender, age, ethnicity, and language. Video contexts span speeches, informational videos, movie clips, news broadcasts, and interviews that reflect content scenarios commonly encountered in real-world online environments. The dataset includes videos in both English and Urdu. The dataset’s quality and structural variability were assessed through visual and audio analyses using the Structural Similarity Index Measure (SSIM), Mel-Frequency Cepstral Coefficients (MFCCs), and Principal Component Analysis (PCA). The evaluation results revealed substantial variability within each manipulation category, along with clearly distinguishable patterns specific to each modality. DeepFakeX has been developed to facilitate rigorous and transparent research in deepfake detection, cross-modal forensic analysis, and AI-driven media forensics. It is hosted on Zenodo under controlled access for research use. Full article

Deepfakes threaten sensor-based authentication systems, including biometric sensors, surveillance cameras, and IoT edge devices. Unimodal detectors remain vulnerable to modality-specific attacks. We propose a multimodal deepfake detection framework optimized for resource-constrained edge devices, featuring a novel cross-modal attention fusion mechanism with adaptive gating. The architecture combines enhanced Res2Net for audio, temporal 3D CNN with SE attention for video, and bidirectional cross-modal attention with quality-based gates. On our benchmark (5472 audio + 1842 video samples), the fusion model achieves 96.7% accuracy, 96.6% F1-score, 0.988 AUC-ROC, and 3.3% EER. Adversarial testing shows 92.3% accuracy under the Fast Gradient Sign Method (FGSM) attack. The model has a 30.3 MB footprint and runs at 20 FPS on edge hardware. Modality contribution analysis reveals adaptive weighting (72% audio for TTS forgery, 78% video for lip-synced attacks). Cross-dataset evaluation on FakeAVCeleb achieves 92.3% overall accuracy, confirming generalization. Full article

This study examines how learning input quality shapes students’ trust, perceived learning value, and post-exposure behavioural intentions, and whether AI-supported instructional content contributes to conceptual learning. Grounded in the Technology Acceptance Model, trust theory, and the Cognitive Theory of Multimedia Learning, the study proposes the Learning Input Quality, Belief, User Learning Experience, and Continuance (LBUC) model. Data were collected from 320 university students in Türkiye via an online survey. To evaluate the proposed framework in an authentic instructional setting, participants watched NotebookLM-generated instructional videos in Turkish and completed pre-test and post-test knowledge measures together with Likert-type scales assessing Audio and Narration Quality, Perceived Visual Design Quality, AI Trust and Persuasion, Instructional Design Effectiveness, Perceived Learning Value, Using Intention, and Recommendation Intention. Learning gains were assessed using paired-samples t-tests, and the proposed LBUC model was evaluated using Partial Least Squares Structural Equation Modelling. The findings showed a significant within-group increase in post-test scores, suggesting short-term conceptual gains after exposure to the videos. In the structural model, Audio and Narration Quality strongly predicted AI Trust and Persuasion, whereas Perceived Visual Design Quality significantly predicted Instructional Design Effectiveness but did not directly predict trust. Both AI Trust and Persuasion and Instructional Design Effectiveness positively influenced Perceived Learning Value, which in turn strongly predicted Using Intention and Recommendation Intention. The results suggest that students’ immediate post-exposure Using Intention and Recommendation Intention are associated less with visual appeal alone than with pedagogically coherent narration, AI Trust and Persuasion, and Perceived Learning Value in the context of NotebookLM-generated instructional videos. Full article

Objective: To develop and evaluate an edge-hosted Large Language Model (LLM)-assisted system for automated Neonatal Intensive Care Unit (NICU) discharge summary generation using an evidence-grounded, field-level evaluation framework. Methods: This implementation and evaluation study was conducted in a Level III NICU in India. Longitudinal patient records were constructed from integrated bedside physiologic data (ARCHITECT) and a structured electronic medical record (EMR) platform Although an embedded audio–video module was present, it was not used in this study. Automated discharge summaries were generated by MORPHEUS, an edge-hosted orchestration pipeline running on NVIDIA Jetson AGX Orin hardware with JetPack 6.2. Local orchestration, preprocessing, and workflow execution were performed on the edge device, while language generation inference was performed using the OpenAI gpt-4o-mini API. Documentation quality was assessed with an LLM-based evaluator guided by a clinician-defined rubric comprising 72 fields organized across 14 section contexts and scored on five dimensions: clinical accuracy, completeness, actionability, coherence, and non-hallucination. Paired, field-level comparisons were performed against clinician-authored summaries. Of 549 NICU admissions screened between 1 October 2024 and 3 November 2025, 401 met the inclusion criteria for evaluation. Prompt refinement was performed iteratively using omission-derived feedback without model weight updates. Results: Across 401 evaluated admissions, MORPHEUS-generated summaries demonstrated higher rubric-based scores and lower omission burden than clinician-authored summaries within the structured evaluation framework used in this study, with mean scores of 0.93 versus 0.75 for accuracy, 0.91 versus 0.67 for completeness, 0.93 versus 0.72 for actionability, 0.94 versus 0.74 for coherence, and 0.95 versus 0.78 for non-hallucination, with the largest absolute advantage observed for completeness. Error taxonomy analysis demonstrated fewer omissions, unsupported assertions, and contradictions in AI-generated summaries than in clinician-authored summaries. Iterative prompt refinement was associated with directional improvement across quality dimensions and reduced omission burden, with omission rate per patient decreasing from 2.484 to 1.807 in the later iteration. Conclusions: An edge-hosted LLM-assisted pipeline can generate NICU discharge summaries that meet or exceed clinician-authored documentation quality under a reproducible, clinician-grounded evaluation framework. These findings support the feasibility of deploying edge-orchestrated generative AI systems for high-stakes neonatal clinical documentation using a clinician-grounded field-level evaluation framework. Full article

Background/Objectives: Access to specialists and diagnostic resources continues to limit differential diagnosis of cognitive impairment in primary care. This pilot study examined the feasibility, usability, and clinical integration of a digitally supported Remote Cognitive Assessment (RCA) model embedded in general practice settings. Methods: A mixed-method design was used, combining structured quantitative surveys from patients (n = 10; mean age = 77.03; SD = 14.1) and neuropsychologists (10 RCAs completed by three neuropsychologists) with qualitative interviews from general practitioners (GP; n = 4). Patients were assessed remotely via a secure videoconference system operated by trained neuropsychologists. Assessments were conducted in the GP’s office, supported by local staff, to facilitate the process. Results: Patients reported high satisfaction with audio (M = 8; SD = 2.28) and video quality (M = 9.17; SD = 1.17) and expressed a strong willingness to recommend RCA (M = 8.83; SD = 1.17) on a 10-point Likert scale. Despite moderate scores for perceived simplicity (M = 5; SD = 3.41) and effectiveness (M = 5.83; SD = 2.14), overall acceptance (M = 8.33; SD = 0.82) was favorable, especially given the older age of participants. Neuropsychologists rated technical functionality positively (audio quality M = 8.17; SD = 1.18; video quality M = 8; SD = 1.67) but raised concerns about clinical utility and diagnostic depth (effectiveness M = 2.83; SD = 2.71). GPs highlighted the benefits of local facilitation, early screening, and improved access to specialist input while also noting space limitations, communication gaps, and the need for sustainable infrastructure. Conclusions: The RCA model was well accepted by patients and GPs, and technically feasible for neuropsychologists. However, neuropsychologists reported important reservations regarding usability and effectiveness. The results suggest an important mismatch between patient satisfaction and clinical confidence and RCA cannot yet be recommended for routine clinical implementations based on patient acceptability alone. This model holds promises for hybrid cognitive care, particularly in underserved or rural areas, but future development must prioritize diagnostic confidence and clinician workflow usability before scalable integration into rural cognitive care pathways can be established. Full article

Whilst advance care planning (ACP) is important to ensure person-centred end of life care, there is sparse evidence about factors contributing towards engagement for people from Mandarin and Cantonese speaking backgrounds (MCSB) affected by cancer. This study aimed to establish barriers and facilitators for quality ACP among people from MCSB with cancer and carers. A qualitative study utilising semi-structured interviews and focus groups was conducted. Participants included adult community members from MCSB in New South Wales who had accessed cancer care services in Australia as a support person or a patient in the last five years with recruitment done purposefully. Data collected from eligible consenting participants were audio/video recorded, transcribed verbatim and analysed using the Framework Method applying the Theoretical Domains Framework. Eighteen people participated (11 in two focus groups, seven individual interviews). Key barriers to engagement with ACP were unclear understanding of process and conduct, poor quality communication by healthcare staff, resource constraints and cultural misalignment of ACP concepts. The main facilitators were openness of participants to discussions, culturally informed community resources and dedicated ACP services. Co-design provides a useful approach to address varied identified factors. At the system and service level, co-design with these communities and healthcare providers could potentially develop resources to assist these communities in engaging with ACP, including preparing for ACP communication. Understanding and acknowledging cultural factors that impact ACP and integrating this knowledge in ACP communication may enhance engagement. Full article

Background: The incidence of thyroid cancer has increased globally. In recent years, robotic surgical systems have been applied in thyroid surgery, and the rapid development of fifth-generation (5G) communication technology has laid a solid foundation for the smooth implementation of remote surgery. Objective: The aim was to explore the feasibility and safety of telerobotic radical thyroidectomy using 5G communication technology to treat thyroid cancer. Methods: From August 2024 to October 2024, telerobotic radical thyroidectomy was performed on seven female patients using a 5G wireless network and a dedicated line network (or ordinary wired broadband) spanning 22–2200 km. The patients’ clinical and information transmission data were analyzed. Results: All patients (papillary thyroid carcinoma, female, with an average age of 44.0 ± 4.6 years) underwent uneventful surgical procedures without any transfer to open surgery or complications. The average surgical duration was 91.3 ± 11.8 min, the average blood loss was 11.4 ± 4.8 mL, and the average postoperative hospital stay was 3.6 ± 0.8 days. All subjects were successfully discharged within 5 days after surgery. The average total latency time of the intraoperative network was 137.5 (range, 121–159) ms, and there were no adverse events, such as network disconnection, frame loss, or network attacks. The operator worked smoothly without any obvious delay or lag, and the recorded audio and video are clear. Conclusions: Telerobotic radical thyroidectomy for thyroid cancer over a 5G network demonstrates promising feasibility and safety. With stable network transmission and a clear surgical field, the precise operations required in thyroid surgery can be performed reliably. These findings suggest that this technology can facilitate high-quality surgical care in remote areas, contributing to a more balanced distribution of medical resources. Full article

Speech-driven lip synchronization is an important technique for talking-face video generation, with broad application potential in virtual humans, video dubbing, digital media, and human–computer interaction. However, existing methods still face challenges in achieving reliable lip synchronization while maintaining stable identity preservation, high visual fidelity, and efficient inference, especially in Chinese-language scenarios where related research remains relatively limited. To address these issues, we propose FastTalk, a speech-driven lip synchronization method for Chinese-language scenarios. The proposed framework performs latent-space restoration for efficient video synthesis, uses a fixed-mask strategy to suppress shortcut visual cues and strengthen audio-driven lip-shape prediction, and adopts a two-stage training scheme to reduce the gap between training and inference. This design improves generation stability while preserving efficiency. Experimental results show that FastTalk achieves competitive lip synchronization performance while improving visual quality and identity preservation. These results indicate that FastTalk provides an effective solution for Chinese speech-driven lip synchronization video generation. Full article

Ambient Assisted Living (AAL) systems have become increasingly relevant as aging populations intensify the demand for technologies that promote autonomy, safety, and quality of life. However, the widespread adoption of audiovisual sensing in smart homes raises critical concerns regarding data protection, privacy, and user trust. Ensuring secure processing while maintaining accurate activity recognition remains a key challenge. This work introduces DistSense, a distributed Peer-to-Peer (P2P) system designed to enhance activity detection in domestic environments through collaborative inference among intelligent audiovisual sensors. DistSense prioritizes privacy by performing local processing, sharing only high-level events, and leveraging distributed ledger mechanisms to ensure data integrity and auditability and support cross-device validation. This collaborative strategy reduces false positives caused by occlusions, illumination variability, and acoustic noise. To assess the system, functional tests were conducted for each module, followed by two use cases evaluated in both simulated and real edge hardware environments. The trained models achieved 88% accuracy for audio and 80% for video, and the system demonstrated effective performance in detecting daily activities and domestic hazards under varying noise conditions. Results indicate that DistSense successfully balances security, user acceptance, and inference robustness, positioning it as a viable solution for privacy-preserving activity monitoring in smart home contexts. Full article

The utilization of Artificial Intelligence-Generated Content (AIGC) has attracted widespread attention in video content creation. To generate high-quality videos, this paper presents a controllable multimodal fusion architecture for AIGC-driven short-video production. This architecture employs hierarchical constraint mechanisms and a multimodal attention fusion mechanism to enhance video content coherence and user controllability. Specifically, a scene coherence scheme is first designed to construct graph-based global and transition-level constraints by integrating text descriptions, reference images, and audio features. By leveraging the extracted style vector data, preliminary video clips are then generated through a combination of the cross-modal fusion unit and the spatio-temporal consistency unit. Finally, a fine-grained adjustment mechanism is implemented to ensure logical consistency and stylistic uniformity in the AIGC-generated videos. Experimental results indicate that the proposed architecture improves generation quality, controllability, and cross-segment coherence under the adopted evaluation settings. Full article

Fine-grained bird identification is crucial for ecosystem monitoring, species conservation, and habitat assessment. However, in real-world environments, there are challenges such as imbalances in modality quality and interference from background noise. To improve fine-grained audio-visual bird classification under heterogeneous modality conditions, we propose an audio-visual feature fusion framework named CABIF-Net. This framework introduces a confidence-based Top-K mean pooling module to select key frames to optimize the visual representations at the video level. Through a Confidence Calibration module, it dynamically assesses the reliability of the visual and audio modalities and integrates a Bidirectional Inter-modulation Fusion module to achieve controllable cross-modal information interaction. Experiments were conducted on the publicly available SSW60 dataset, characterized by severe noise and imbalance in modality quality, and the self-built Birds21 dataset with balanced modality quality. The experimental results show that the classification accuracies were 85.76% and 96.67%, respectively, outperforming existing unimodal methods and several mainstream fusion strategies. Weight distribution and visualization analyses further indicate that the proposed method can adaptively adjust the modality contributions based on discriminative evidence at the sample level. This study provides an effective framework for fine-grained audio-visual bird species recognition. Full article

Current 360-degree video streaming systems consider viewport prediction, adaptive bitrate allocation, tile selection, and quality-of-experience (QoE) estimation as independent activities, yielding fragmented pipelines that do not scale well across content type and network conditions and do not scale well to individual users. We propose ImmerseFM-3D, a foundation model that jointly solves all four sub-tasks through a single shared representation. Seven input modalities, namely video frames, network traces, head-motion trajectories, ambisonics audio, depth maps, eye-tracking signals, and CLIP scene semantics, are fused by four-layer cross-modal attention and compressed into a 256-dimensional bottleneck latent via a variational information bottleneck. Four task-specific decoders operate on this shared latent simultaneously. A model-agnostic meta-learning adapter augmented with episodic memory and a hypernetwork personalizes the model from as little as 1 s of user interaction data. An extended branch supports six-degrees-of-freedom volumetric content through spherical harmonic viewport decoding and depth-aware tile importance weighting. Trained and evaluated on the IMMERSE-1M combined dataset (1000 h of 360° and volumetric video, 524 users, and over 50,000 mean opinion scores), ImmerseFM-3D reduces the mean angular viewport error by 34%, lowers the bandwidth violation rate from 8.3% to 3.1%, and achieves a QoE Pearson correlation of 0.891. The personalization adapter reaches 90% of peak performance in 22 s, while zero-shot cross-format transfer attains 72% of full in-domain accuracy. Full article

Talking Head Generation (THG) is a rapidly advancing field at the intersection of computer vision, deep learning, and speech synthesis, enabling the creation of animated human-like heads that can produce speech and express emotions with high visual realism. The core objective of THG systems is to synthesize coherent and natural audio–visual outputs by modeling the intricate relationship between speech signals, facial dynamics, and emotional cues. These systems find widespread applications in virtual assistants, interactive avatars, video dubbing for multilingual content, educational technologies, and immersive virtual and augmented reality environments. Moreover, the development of THG has significant implications for accessibility technologies, cultural preservation, and remote healthcare interfaces. This survey paper presents a comprehensive and systematic overview of the technological landscape of Talking Head Generation. We begin by outlining the foundational methodologies that underpin the synthesis process, including generative adversarial networks (GANs), motion-aware recurrent architectures, and attention-based models. A taxonomy is introduced to organize the diverse approaches based on the nature of input modalities and generation goals. We further examine the contributions of various domains such as computer vision, speech processing, and human–robot interaction, each of which plays a critical role in advancing the capabilities of THG systems. The paper also provides a detailed review of datasets used for training and evaluating THG models, highlighting their coverage, structure, and relevance. In parallel, we analyze widely adopted evaluation metrics, categorized by their focus on image quality, motion accuracy, synchronization, and semantic fidelity. Operating parameters such as latency, frame rate, resolution, and real-time capability are also discussed to assess deployment feasibility. Special emphasis is placed on the integration of generative artificial intelligence (GenAI), which has significantly enhanced the adaptability and realism of talking head systems through more powerful and generalizable learning frameworks. Full article

Diffusion models have emerged as the state-of-the-art generative paradigm, surpassing GANs in synthesizing high-fidelity images, videos, and audio. However, their reliance on iterative denoising processes imposes substantial computational burdens and memory overheads, creating a significant barrier to their deployment on resource-constrained edge devices. Unlike existing surveys that broadly cover general methodologies, this paper provides a focused review with a specific emphasis on efficient and lightweight diffusion models. We systematically analyze the trade-offs between generation quality and computational cost, categorizing acceleration techniques into sampling optimization, architectural compression, and knowledge distillation. Furthermore, we explore the integration of diffusion models with emerging architectures (e.g., Mamba) and their evolution towards general-purpose world simulators. This survey aims to provide a roadmap for “Green AI,” bridging the gap between high-end academic research and practical, real-world applications. Full article

Recent technologies such as music retrieval, soundtrack generation, and video understanding have developed rapidly. Consequently, the aesthetic evaluation of video soundtracks has become an important research topic in academia. Soundtracks are key elements in shaping the emotional atmosphere and driving the narrative rhythm. Therefore, they require systematic methods to assess their artistic coordination with visual content. However, existing approaches mostly focus on evaluating the quality of the music itself. They often lack the ability to model the deeper aesthetic synergy between audio and visuals. To address this gap, we propose MEMA, a new soundtrack aesthetic evaluation model. MEMA employs a two-stage training strategy. The first stage builds a crossmodal imagination mechanism using a Conditional Variational Autoencoder. This method achieves bidirectional semantic reconstruction between audio and visuals. The second stage introduces a Guided Cross-Attention Alignment Module. This module enhances the model’s focus on key narrative moments in video. To facilitate this research, we also construct VMAE-Sets. It is the first large-scale dataset dedicated to soundtrack aesthetic evaluation. Finally, MEMA performs scoring and textual evaluation along three core aesthetic dimensions. Experimental results demonstrate that MEMA outperforms existing methods, achieving average improvements of 18.137% in LCC and 17.866% in SRCC compared to the strongest baseline. These findings confirm its superior audio–visual narrative alignment, demonstrating high consistency with human judgments. Full article