Search Results (17)

High-fidelity motion cueing in driving simulators is essential for delivering a realistic and immersive user experience. However, the trade-off between motion accuracy and computational efficiency often hinders achieving this. Fixed-horizon Model Predictive Control (MPC)-based Motion Cueing Algorithm (MCA) frameworks frequently struggle to adapt to rapid dynamic changes in vehicle behaviour, resulting in suboptimal simulator responses. Their reliance on worst-case horizon tuning can result in inefficient platform usage and increased computational load, limiting computational efficiency and practical deployment. This study presents an adaptive MPC-based MCA designed to enhance the fidelity of motion platforms used in vehicle dynamic simulations. The proposed method dynamically adjusts the MPC prediction horizon to improve overall simulation performance while minimising motion sensation error. Within the simulation environment, the prediction horizon is adaptively updated at each simulated control step according to recent tracking-performance metrics, enabling responsiveness to varying vehicle dynamic models and driving scenarios. The system was developed and implemented using Python and MATLAB environments, with Long Short-Term Memory (LSTM) networks employed to enhance the adaptability and precision of prediction horizon adjustments. Due to safety constraints, the proposed framework was evaluated exclusively within a simulation environment and compared against both classical MPC-based MCA and RL MPC-based MCA. Experimental results demonstrate that the proposed adaptive framework improves workspace utilisation and substantially reduces computational load compared with the classical and RL-based MPC-based MCA approaches, while maintaining competitive motion cueing tracking performance. The adaptive system effectively enhances linear displacement (LD), ensuring better alignment of motion cues with platform constraints. While minor trade-offs were observed in root mean square error (RMSE) and correlation coefficients (CCs) for sensed angular velocity (SAV) and sensed specific force (SSF), the framework improves workspace utilisation and computational efficiency while maintaining competitive motion cueing performance. Furthermore, the adaptive LSTM-MPC framework substantially reduces computational load, achieving approximately 44.26 times faster execution compared with the classical MPC-based MCA and approximately 30.03 times faster execution compared with the RL MPC-based MCA. These findings highlight the potential of integrating deep learning (DL) with MPC to optimise the trade-off between motion cueing performance, platform utilisation, and computational efficiency in driving simulators. Full article

Talking Head Generation (THG) is a rapidly advancing field at the intersection of computer vision, deep learning, and speech synthesis, enabling the creation of animated human-like heads that can produce speech and express emotions with high visual realism. The core objective of THG systems is to synthesize coherent and natural audio–visual outputs by modeling the intricate relationship between speech signals, facial dynamics, and emotional cues. These systems find widespread applications in virtual assistants, interactive avatars, video dubbing for multilingual content, educational technologies, and immersive virtual and augmented reality environments. Moreover, the development of THG has significant implications for accessibility technologies, cultural preservation, and remote healthcare interfaces. This survey paper presents a comprehensive and systematic overview of the technological landscape of Talking Head Generation. We begin by outlining the foundational methodologies that underpin the synthesis process, including generative adversarial networks (GANs), motion-aware recurrent architectures, and attention-based models. A taxonomy is introduced to organize the diverse approaches based on the nature of input modalities and generation goals. We further examine the contributions of various domains such as computer vision, speech processing, and human–robot interaction, each of which plays a critical role in advancing the capabilities of THG systems. The paper also provides a detailed review of datasets used for training and evaluating THG models, highlighting their coverage, structure, and relevance. In parallel, we analyze widely adopted evaluation metrics, categorized by their focus on image quality, motion accuracy, synchronization, and semantic fidelity. Operating parameters such as latency, frame rate, resolution, and real-time capability are also discussed to assess deployment feasibility. Special emphasis is placed on the integration of generative artificial intelligence (GenAI), which has significantly enhanced the adaptability and realism of talking head systems through more powerful and generalizable learning frameworks. Full article

Effective collective motion hinges on the seamless transfer of local information, yet vision-based mechanisms, while potent for generating rapid consensus, are inherently fragile. Visual links can be severed instantly by occlusions, leading to a phenomenon characterized as “sensory amnesia.” Seeking to fortify this vulnerability, Pheromone-Modulated Body Orientation Change (PM-BOC) is introduced as a dual-channel framework that fuses transient visual cues with a persistent environmental memory. Rather than treating these inputs in isolation, motion salience is quantified via BOC and mapped onto a decaying virtual pheromone field, dynamically modulating interaction weights by coupling instantaneous visual projections with local pheromone concentrations. This strategy effectively constructs a temporal buffer, bridging the informational voids left by blind spots. Validation, spanning from systematic physics simulations to high-fidelity simulations with a swarm of 50 UUVs, reveals that PM-BOC sustains superior cohesion in obstacle-laden environments where baseline visual models falter. Notably, this coupling suppresses high-frequency sensory noise while inducing resilient, scale-free velocity correlations that scale linearly with system size. By reconciling the trade-off between the immediacy of visual responsiveness and the robustness of environmental memory, this study offers a scalable paradigm for engineering resilient swarm systems capable of navigating the uncertainties of perception-limited environments. Full article

In modern high-demand applications, such as real-time video communication, cloud gaming, and high-definition live streaming, achieving both superior transmission speed and high visual fidelity is paramount. However, unstable networks and packet loss remain major bottlenecks, making accurate and low-latency video error concealment a critical challenge. Traditional error control strategies, such as Forward Error Correction (FEC) and Automatic Repeat Request (ARQ), often introduce excessive latency or bandwidth overhead. Meanwhile, receiver-side concealment methods struggle under high motion or significant packet loss, motivating the exploration of predictive models. SimVP-v2, with its efficient convolutional architecture and Gated Spatiotemporal Attention (GSTA) mechanism, provides a strong baseline by reducing complexity and achieving competitive prediction performance. Despite its merits, SimVP-v2’s reliance on 2D convolutions for implicit temporal aggregation limits its capacity to capture complex motion trajectories and long-term dependencies. This often results in artifacts such as motion blur, detail loss, and accumulated errors. Furthermore, its single-modality design ignores the complementary contextual cues embedded in the audio stream. To overcome these issues, we propose A3DSimVP (Audio- and 3D-Enhanced SimVP-v2), which integrates explicit spatio-temporal modeling with multimodal feature fusion. Architecturally, we replace the 2D depthwise separable convolutions within the GSTA module with their 3D counterparts, introducing a redesigned GSTA-3D module that significantly improves motion coherence across frames. Additionally, an efficient audio–visual fusion strategy supplements visual features with contextual audio guidance, thereby enhancing the model’s robustness and perceptual realism. We validate the effectiveness of A3DSimVP’s improvements through extensive experiments on the KTH dataset. Our model achieves a PSNR of 27.35 dB, surpassing the 27.04 of the SimVP-v2 baseline. Concurrently, our improved A3DSimVP model reduces the loss metrics on the KTH dataset, achieving an MSE of 43.82 and an MAE of 385.73, both lower than the baseline. Crucially, our LPIPS metric is substantially lowered to 0.22. These data tangibly confirm that A3DSimVP significantly enhances both structural fidelity and perceptual quality while maintaining high predictive accuracy. Notably, A3DSimVP attains faster inference speeds than the baseline with only a marginal increase in computational overhead. These results establish A3DSimVP as an efficient and robust solution for latency-critical video applications. Full article

Training-free video summarization tackles the challenge of selecting the most informative keyframes from a video without relying on costly training or complex deep models. This work introduces C2FVS-DPP (Contextual Feature Fusion Video Summarization with Determinantal Point Process), a lightweight framework that generates concise video summaries by jointly modeling semantic importance, visual diversity, temporal structure, and symmetry. The design centers on a symmetry-aware fusion strategy, where appearance, motion, and semantic cues are aligned in a unified embedding space, and on a reward-guided optimization logic that balances representativeness and diversity. Specifically, appearance features from ResNet-50, motion cues from optical flow, and semantic representations from BERT-encoded BLIP captions are fused into a contextual embedding. A Transformer encoder assigns importance scores, followed by shot boundary detection and K-Medoids clustering to identify candidate keyframes. These candidates are refined through a reward-based re-ranking mechanism that integrates semantic relevance, representativeness, and visual uniqueness, while a Determinantal Point Process (DPP) enforces globally diverse selection under a keyframe budget. To enable reliable evaluation, enhanced versions of the SumMe and TVSum50 datasets were curated to reduce redundancy and increase semantic density. On these curated benchmarks, C2FVS-DPP achieves F1-scores of 0.22 and 0.43 and fidelity scores of 0.16 and 0.40 on SumMe and TVSum50, respectively, surpassing existing models on both metrics. In terms of compression ratio, the framework records 0.9959 on SumMe and 0.9940 on TVSum50, remaining highly competitive with the best-reported values of 0.9981 and 0.9983. These results highlight the strength of C2FVS-DPP as an inference-driven, symmetry-aware, and resource-efficient solution for video summarization. Full article

Micro-expressions, characterized by brief and subtle facial muscle movements, are essential for conveying nuanced emotions in digital humans, yet existing rendering techniques often produce rigid or emotionally monotonous animations due to the inadequate modeling of temporal dynamics and action unit interdependencies. This paper proposes a graph-driven framework for micro-expression rendering that generates emotionally diverse and lifelike expressions. We employ a 3D-ResNet-18 backbone network to perform joint spatio-temporal feature extraction from facial video sequences, enhancing sensitivity to transient motion cues. Action units (AUs) are modeled as nodes in a symmetric graph, with edge weights derived from empirical co-occurrence probabilities and processed via a graph convolutional network to capture structural dependencies and symmetric interactions. This symmetry is justified by the inherent bilateral nature of human facial anatomy, where AU relationships are based on co-occurrence and facial anatomy analysis (as per the FACS), which are typically undirected and symmetric. Human faces are symmetric, and such relationships align with the design of classic spectral GCNs for undirected graphs, assuming that adjacency matrices are symmetric to model non-directional co-occurrences effectively. Predicted AU activations and timestamps are interpolated into continuous motion curves using B-spline functions and mapped to skeletal controls within a real-time animation pipeline (Unreal Engine). Experiments on the CASME II dataset demonstrate superior performance, achieving an F1-score of 77.93% and an accuracy of 84.80% (k-fold cross-validation, k = 5), outperforming baselines in temporal segmentation. Subjective evaluations confirm that the rendered digital human exhibits improvements in perceptual clarity, naturalness, and realism. This approach bridges micro-expression recognition and high-fidelity facial animation, enabling more expressive virtual interactions through curve extraction from AU values and timestamps. Full article

This study presents the development of an open-source Driver-in-the-Loop simulation platform, specifically designed to test and analyze advanced automated driving functions. We emphasize the creation of a versatile system architecture that ensures seamless integration and interchangeability of components, supporting diverse research needs. Central to the simulator’s configuration is a hexapod motion platform with six degrees of freedom, chosen through a detailed benchmarking process to ensure dynamic accuracy and fidelity. The simulator employs a half-vehicle cabin, providing an immersive environment where drivers can interact with authentic human–machine interfaces such as pedals, steering, and gear shifters. By projecting complex driving scenarios onto a curved screen, drivers engage with critical maneuvers in a controlled virtual environment. Key innovations include the integration of a motion cueing algorithm and an adaptable, cost-effective open-source framework, facilitating collaboration among researchers and industry experts. The platform enables standardized testing and offers a robust solution for the iterative development and validation of automated driving technologies. Functionality and effectiveness were validated through testing with the ISO lane change maneuver, affirming the simulator’s capabilities. Full article

Recent breakthroughs in AI-generated content (AIGC) have transformed video creation, empowering systems to translate text, images, or audio into visually compelling stories. Yet reliable evaluation of these machine-crafted videos remains elusive because quality is governed not only by spatial fidelity within individual frames but also by temporal coherence across frames and precise semantic alignment with the intended message. The foundational role of sensor technologies is critical, as they determine the physical plausibility of AIGC outputs. In this perspective, we argue that multimodal large language models (MLLMs) are poised to become the cornerstone of next-generation video quality assessment (VQA). By jointly encoding cues from multiple modalities such as vision, language, sound, and even depth, the MLLM can leverage its powerful language understanding capabilities to assess the quality of scene composition, motion dynamics, and narrative consistency, overcoming the fragmentation of hand-engineered metrics and the poor generalization ability of CNN-based methods. Furthermore, we provide a comprehensive analysis of current methodologies for assessing AIGC video quality, including the evolution of generation models, dataset design, quality dimensions, and evaluation frameworks. We argue that advances in sensor fusion enable MLLMs to combine low-level physical constraints with high-level semantic interpretations, further enhancing the accuracy of visual quality assessment. Full article

The paper presents the innovative approach to a high-fidelity motorcycle riding simulator based on VR (Virtual Reality)-visualization, equipped with a Gough-Stewart 6-DOF (Degrees of Freedom) motion platform. Such a solution integrates a real-time tension sensor system as a source for highly realistic motion cueing control as well as the servomotor integrated into the steering system. Tension forces are measured at four points on the mock-up chassis, allowing a comprehensive analysis of rider interaction during various maneuvers. The simulator is developed to simulate realistic riding scenarios with immersive motion and visual feedback, enhanced with the simulation of external influences—headwind. This paper presents results of a validation study—pilot experiments conducted to evaluate selected riding scenarios and validate the innovative simulator setup, focusing on force distribution and system responsiveness to support further research in motorcycle HMI (Human–Machine Interaction), rider behavior, and training. Full article

Sign Language Recognition (SLR) has made substantial progress through advances in deep learning and video-based action recognition. Conventional SLR systems typically segment input videos into a fixed number of clips (e.g., five clips per video), regardless of the video’s actual length, to meet the fixed-length input requirements of deep learning models. While this approach simplifies model design and training, it fails to account for temporal variations inherent in sign language videos. Specifically, applying a fixed number of clips to videos of varying lengths can lead to significant information loss: longer videos suffer from excessive frame skipping, causing the model to miss critical gestural cues, whereas shorter videos require frame duplication, introducing temporal redundancy that distorts motion dynamics. To address these limitations, we propose a dynamic clip generation method that adaptively adjusts the number of clips during inference based on a novel coverage metric. This metric quantifies how effectively a clip selection captures the temporal information in a given video, enabling the system to maintain both temporal fidelity and computational efficiency. Experimental results on benchmark SLR datasets using multiple models-including 3D CNNs, R(2+1)D, Video Swin Transformer, and Multiscale Vision Transformers demonstrate that our method consistently outperforms fixed clip generation methods. Notably, our approach achieves 98.67% accuracy with the Video Swin Transformer while reducing inference time by 28.57%. These findings highlight the effectiveness of coverage-based dynamic clip generation in improving both accuracy and efficiency, particularly for videos with high temporal variability. Full article

Holographic displays are an upcoming technology for AR and VR applications, with the ability to show 3D content with accurate depth cues, including accommodation and motion parallax. Recent research reveals that only a fraction of holographic pixels are needed to display images with high fidelity, improving energy efficiency in future holographic displays. However, the existing iterative method for computing sparse amplitude and phase layouts does not run in real time; instead, it takes hundreds of milliseconds to render an image into a sparse hologram. In this paper, we present a non-iterative amplitude and phase computation for sparse Fourier holograms that uses Perlin noise in the image–plane phase. We conduct simulated and optical experiments. Compared to the Gaussian-weighted Gerchberg–Saxton method, our method achieves a run time improvement of over 600 times while producing a nearly equal PSNR and SSIM quality. The real-time performance of our method enables the presentation of dynamic content crucial to AR and VR applications, such as video streaming and interactive visualization, on holographic displays. Full article

Perception-based fidelity evaluation metrics are crucial in driving simulators, as they play a key role in the automatic tuning, assessment, and comparison of motion cueing algorithms. Nevertheless, there is presently no unified and effective evaluation framework for these algorithms. To tackle this challenge, our study initially establishes a model rooted in visual–vestibular interaction and head tilt angle perception systems. We then employ metrics like the Normalized Average Absolute Difference (NAAD), Normalized Pearson Correlation (NPC), and Estimated Delay (ED) to devise an evaluation index system. Furthermore, we use a combined approach incorporating CRITIC and gray relational analysis to ascertain the weights of these indicators. This allows us to consolidate them into a comprehensive evaluation metric that reflects the overall fidelity of motion cueing algorithms. Subjective evaluation experiments validate the reasonableness and efficacy of our proposed Perception Fidelity Evaluation (PFE) method. Full article

To migrate Loss of Control In-flight, the number one cause of aviation fatalities, pilots need to undergo upset prevention and recovery training with flight simulators. The fidelity of a moving base flight simulator is greatly dependent on the washout algorithm of the Stewart platform, which may reach the workspace limits when simulating the aircraft recovery from upset conditions. In this paper, a washout algorithm optimal design method based on the model predictive control technique is proposed for flight simulator upset prevention and recovery training. The parameters of the washout algorithm are calculated directly based on the platform model, and the system limits are explicitly taken into account. The human perception model is incorporated into the optimization problem, for which the objective is to minimize the pilot’s perceived motion mismatch between the real flight and the simulator training. Simulations are conducted and compared with the classical filter-based washout algorithm. Responses of the flight simulator model show that the proposed method can improve the motion cueing effect when the aircraft is in upset conditions. Full article

The present paper focuses on vehicle simulator fidelity, particularly the effect of motion cues intensity on driver performance. The 6-DOF motion platform was used in the experiment; however, we mainly focused on one characteristic of driving behavior. The braking performance of 24 participants in a car simulator was recorded and analyzed. The experiment scenario was composed of acceleration to 120 km/h followed by smooth deceleration to a stop line with prior warning signs at distances of 240, 160, and 80 m to the finish line. To assess the effect of the motion cues, each driver performed the run three times with different motion platform settings–no motion, moderate motion, and maximal possible response and range. The results from the driving simulator were compared with data acquired in an equivalent driving scenario performed in real conditions on a polygon track and taken as reference data. The driving simulator and real car accelerations were recorded using the Xsens MTi-G sensor. The outcomes confirmed the hypothesis that driving with a higher level of motion cues in the driving simulator brought more natural braking behavior of the experimental drivers, better correlated with the real car driving test data, although exceptions were found. Full article

Driving simulators are increasingly being incorporated by driving schools into a training process for a variety of vehicles. The motion platform is a major component integrated into simulators to enhance the sense of presence and fidelity of the driving simulator. However, less effort has been devoted to assessing the motion cues feedback on trainee performance in simulators. To address this gap, we thoroughly study the impact of motion cues on braking at a target point as an elementary behavior that reflects the overall driver’s performance. In this paper, we use an eye-tracking device to evaluate driver behavior in addition to evaluating data from a driving simulator and considering participants’ feedback. Furthermore, we compare the effect of different motion levels (“No motion”, “Mild motion”, and “Full motion”) in two road scenarios: with and without the pre-braking warning signs with the speed feedback given by the speedometer. The results showed that a full level of motion cues had a positive effect on braking smoothness and gaze fixation on the track. In particular, the presence of full motion cues helped the participants to gradually decelerate from 5 to 0 ms⁻¹ in the last 240 m before the stop line in both scenarios, without and with warning signs, compared to the hardest braking from 25 to 0 ms⁻¹ produced under the no motion cues conditions. Moreover, the results showed that a combination of the mild motion conditions and warning signs led to an underestimation of the actual speed and a greater fixation of the gaze on the speedometer. Questionnaire data revealed that 95% of the participants did not suffer from motion sickness symptoms, yet participants’ preferences did not indicate that they were aware of the impact of simulator conditions on their driving behavior. Full article