Search Results (4)

In today’s digital era, the realms of virtual reality (VR), augmented reality (AR), and mixed reality (MR) collectively referred to as extended reality (XR) are reshaping human–computer interactions. XR technologies are poised to overcome geographical barriers, offering innovative solutions for enhancing emotional and social engagement in telecommunications and remote collaboration. This paper delves into the integration of (AI)-powered 3D talking heads within XR-based telecommunication systems. These avatars replicate human expressions, gestures, and speech, effectively minimizing physical constraints in remote communication. The contributions of this research encompass an extensive examination of audio-driven 3D head generation methods and the establishment of comprehensive evaluation criteria for 3D talking head algorithms within Shared Virtual Environments (SVEs). As XR technology evolves, AI-driven 3D talking heads promise to revolutionize remote collaboration and communication. Full article

Recent world events have caused a dramatic rise in the use of video conferencing solutions such as Zoom and FaceTime. Although 3D capture and display technologies are becoming common in consumer products (e.g., Apple iPhone TrueDepth sensors, Microsoft Kinect devices, and Meta Quest VR headsets), 3D telecommunication has not yet seen any appreciable adoption. Researchers have made great progress in developing advanced 3D telepresence systems, but often with burdensome hardware and network requirements. In this work, we present HoloKinect, an open-source, user-friendly, and GPU-accelerated platform for enabling live, two-way 3D video conferencing on commodity hardware and a standard broadband internet connection. A Microsoft Azure Kinect serves as the capture device and a Looking Glass Portrait multiscopically displays the final reconstructed 3D mesh for a hologram-like effect. HoloKinect packs color and depth information into a single video stream, leveraging multiwavelength depth (MWD) encoding to store depth maps in standard RGB video frames. The video stream is compressed with highly optimized and hardware-accelerated video codecs such as H.264. A search of the depth and video encoding parameter space was performed to analyze the quantitative and qualitative losses resulting from HoloKinect’s lossy compression scheme. Visual results were acceptable at all tested bitrates (3–30 Mbps), while the best results were achieved with higher video bitrates and full 4:4:4 chroma sampling. RMSE values of the recovered depth measurements were low across all settings permutations. Full article

One of the most significant challenges of telepresence distance education is to bring the professor and the students closer together in a synchronistic educational experience where the professor is perceived as anatomically proportionate. Telepresence, an educational technology ecosystem using holograms, offers a way to solve this technological challenge. Our mixed exploratory research investigating this methodology had two purposes: (1) propose the key elements to teach distance courses synchronously in an educational technology ecosystem, and (2) demonstrate the technological, didactic practices that result in positive student learning outcomes in several specified courses. This methodology included applying a student questionnaire to collect their perceptions of the educational experience. The scores and written comments from the questionnaire were analyzed using Grounded Theory. On a Likert scale from 1 to 5, the students scored their educational experience, attaining a mean of 4.05. The positive perception affirmed that they valued: (a) recreating the natural dynamics of face-to-face classes, where the students perceived their professors as being physically present in the classroom; (b) professors renowned in their disciplines; (c) professor–student and campus and intercampus learning community interactions, and, finally, (d) class design and content. The main conclusions of this research were that students positively perceived the “wow” effect of the technology, feeling comfort, amazement, interest, and engagement. In addition, we found that professors and keynote speakers with excellent pedagogical skills and experts in their disciplines were well appreciated. Key elements for the success of the experience were professor-student, campus, and intercampus interactions and the quality of the technological and communication infrastructure. Full article

Advancements in the study of the human sense of touch are fueling the field of haptics. This is paving the way for augmenting sensory perception during object palpation in tele-surgery and reproducing the sensed information through tactile feedback. Here, we present a novel tele-palpation apparatus that enables the user to detect nodules with various distinct stiffness buried in an ad-hoc polymeric phantom. The contact force measured by the platform was encoded using a neuromorphic model and reproduced on the index fingertip of a remote user through a haptic glove embedding a piezoelectric disk. We assessed the effectiveness of this feedback in allowing nodule identification under two experimental conditions of real-time telepresence: In Line of Sight (ILS), where the platform was placed in the visible range of a user; and the more demanding Not In Line of Sight (NILS), with the platform and the user being 50 km apart. We found that the entailed percentage of identification was higher for stiffer inclusions with respect to the softer ones (average of 74% within the duration of the task), in both telepresence conditions evaluated. These promising results call for further exploration of tactile augmentation technology for telepresence in medical interventions. Full article