Search Results (12)

Oral presentations for investor relations, government funding, and technical review are typically prepared without objective, repeatable feedback on delivery. We present an integrated web-based pre-screening system for Korean oral presentations that combines automatic speech recognition (ASR), prosodic and disfluency analytics, presentation–material matching, and large language model (LLM)-based qualitative evaluation into a single automated pipeline. To ground the pipeline in a reliable transcription engine, we benchmark five commercial and open-source speech-to-text (STT) engines (Naver CLOVA Speech, Azure Speech, OpenAI Whisper, Google Speech-to-Text v2, and AssemblyAI) on a Korean presentation-domain evaluation set using character error rate (CER), timestamp quality, filler-word preservation, and speaker diarization. On 20 spontaneous Korean speech recordings with corrected reference normalization, CLOVA achieved the lowest mean CER (4.65%, SD 3.75), but CER differences among the five engines were not statistically significant (Friedman p = 0.219, Kendall’s W = 0.07). In contrast, filler preservation separated the engines into two sharply distinct clusters: preserving engines (CLOVA, Azure, Google: 84–93%) versus deleting engines (Whisper, AssemblyAI: 2–19%), with a large and highly significant effect (Friedman p < 10⁻⁶, Kendall’s W = 0.87). A complementary benchmark on 20 scripted professional voice-actor recordings reverses the CER ranking (Whisper best at 2.8%), confirming that engine rankings do not transfer across speech domains. CLOVA was adopted as the system’s default engine on the basis of this fitness profile: competitive CER combined with high filler preservation, fine-grained timestamps, and native diarization, rather than transcription accuracy alone. The deployed system converts an uploaded or live-recorded presentation into a sentence-level timeline; computes speaking rate, silence, filler-word, and per-speaker statistics; measures keyword coverage against uploaded slides; and generates prioritized improvement points and anticipated reviewer questions via an LLM. Three independent runs of the deployed system on the same recording produced identical deterministic outputs, including under an outage of the LLM service. A three-rater expert evaluation of nine generated reports rated evidence accuracy and anticipated-question quality highest and specificity lowest (grand mean 3.88/5), with low inter-rater agreement reflecting the small panel. The results provide a reproducible, domain-specific Korean ASR benchmark and a practical reference architecture for automated presentation assessment. Full article

This study presents a multi-tier framework for deploying multi-modal Conversational AI on edge devices, spanning from constrained ultra-low-power systems to high-performance edge workstations. Utilizing an automated model discovery process and a modular benchmarking testbed, the research demonstrates that real-time, fully edge AI execution is feasible through strategic model selection and hardware acceleration. Key outcomes from the performance analysis are as follows. Speech-to-Text: Fun-ASR-Nano achieved the highest transcription accuracy with a Word Error Rate of 0.026, while Moonshine Tiny was the most efficient, recording a Real-Time Factor of 0.036 on the CPU. Scaling up to the high-performance tier, Whisper Large-V3 Turbo demonstrated high speed and robustness on a dedicated GPU, achieving an RTF of 0.093. Language Modeling: The Qwen 2.5 (1.5B Instruct) model, optimized for the Intel edge NPU, delivered robust constrained edge performance with an average generation speed of 20.15 tokens per second and a high semantic accuracy score of 0.86. The non-transformer Liquid LFM-24B model showcased server-level reasoning capabilities on the high-performance edge, reaching an impressive 39.2 tokens per second when fully offloaded to a dedicated GPU, despite its massive VRAM requirements. Text-to-Speech: Piper TTS emerged as the most efficient model for constrained environments (RTF of 0.034). However, Kokoro TTS redefined high-fidelity zero-shot synthesis on the GPU tier, achieving a groundbreaking RTF of 0.024 and far outperforming larger autoregressive audio models like OuteTTS, which remained too slow for real-time use without significant acceleration. Hardware Acceleration and Energy Efficiency: The use of Intel OpenVINO 2026.0 for hardware offloading significantly reduced energy consumption; for example, Whisper Large-V3 Turbo’s energy per audio second dropped from 52.68 Joules on the CPU to just 3.24 Joules on the integrated GPU. Furthermore, dedicated GPU acceleration revealed a critical “race-to-sleep” paradigm, where higher peak wattage is offset by drastically reduced processing times. The study concludes by identifying two optimal cascaded pipelines: a constrained edge tier (Moonshine, Qwen 1.5B, Piper) running on a Khadas NUC (Khadas Technology, Shenzhen, China powered by an Intel processor (Intel Corporation, Santa Clara, CA, USA) maximizing energy efficiency, and a high-performance tier (Whisper V3 Turbo, Liquid LFM-24B, Kokoro) running on an NVIDIA 5060ti, delivering uncompromising accuracy and subsecond latency for privacy-preserving, advanced edge AI. Full article

Large language models (LLMs) exhibit strong general conversational capabilities, yet their deployment in domain-specific service environments such as hospitality remains limited by the absence of emotionally grounded datasets and validated end-to-end system architectures. This paper presents HOSPIT-LLM, an EU-funded euROBIN Technology Exchange Program pilot, as a complete, integrated pilot pipeline for human-centered conversational AI in hotel reception scenarios. We deploy a multimodal hotel-terminal assistant in a real hotel reception, capturing synchronized dual-camera video and audio to collect authentic guest–staff interactions. Speech is transcribed using Whisper, and emotion is extracted from the corresponding video segments via DeepFace, producing 582 real Greek guest–receptionist exchange examples. The resulting data are classified into eight Standard Operating Procedure (SOP) categories. To address data scarcity, we augment the corpus with 1269 synthetic dialogues generated by eight diverse LLMs through the OpenRouter API, yielding a total of 1851 dialogue records with explicit emotion-token annotation. We fine-tune Qwen3.5-35B-A3B using Low-Rank Adaptation (LoRA) through a two-stage process: supervised fine-tuning (SFT) on an 888-example conversation pool and Simple Preference Optimization (SimPO) on a 1899-pair preference pool, each split 80/10/10 into training, validation, and test. The resulting model is integrated into an interactive hotel-terminal system combining YOLO-based person detection, face-recognition-driven guest personalization, Kokoro neural text-to-speech (TTS), and a multi-service orchestration layer connected to the hotel Property Management System (PMS). Evaluation combines standard text metrics, emotion-aware scoring, and a large-model judge. The results indicate targeted improvements in the rule-based contextual emotion-policy match and staff-emotion policy compliance compared to the base model, while general response-quality gains remain more modest. In particular, the rule-based contextual policy-match score improves from 0.614 to 0.901, while forbidden staff-emotion outputs decrease from 0.142 to 0.018. The deployed pilot demonstrates the practical integration of a personalized, emotion-aware LLM assistant in an interactive hotel-terminal setting; end-to-end latency and fully hotel-side edge deployment were not evaluated and are left for future work. HOSPIT-LLM provides a reproducible framework for multimodal dataset creation, preference-based fine-tuning, and deployment of human-centered AI systems. A mobile robotic embodiment is planned as future work. Full article

Stuttering and related speech disorders can interrupt the natural flow of speech through repetitions, prolonged sounds, and pauses or delays, affecting millions of people worldwide. Although considerable progress has been made in artificial intelligence-based Automatic Speech Recognition (ASR) technology, most of the current models remain mainly designed for high resource and dominant lingua franca languages, e.g., English, and underperform for Arabic. This paper presents insights into stuttering classification in Arabic using Whisper ASR from OpenAI, trained to classify speech segments as fluent or disfluent. As a necessary first step, we frame the task as a binary (fluent vs. disfluent) classification. Fine-grained recognition of specific disfluency types (such as repetitions, prolongations, and blocks) is left to future work. The most salient aspect of this study is the construction of a marked stuttering speech corpus in which speech segments of fluent and disfluent speech were collected from real clinical cases. A systematic comparative framework is built between the full Whisper family (Tiny–Large) and the Wav2Vec2.0 family (Base–XLarge) under identical conditions, providing the first benchmark for Arabic stuttering classification. We find that Whisper outperforms Wav2Vec2.0 at every scale, including the smallest variants, and remains reliable even for low-resource deployment. This confirms that the Whisper encoder is suitable for clinical Arabic speech-disorder workflows. Full article

With the rapid development of artificial intelligence and machine learning technologies, automatic speech recognition (ASR) and text-to-speech (TTS) have become key components of the digital transformation of society. The Kazakh language, as a representative of the Turkic language family, remains a low-resource language with limited audio corpora, language models, and high-quality speech synthesis systems. This study provides a comprehensive analysis of existing speech recognition and synthesis models, emphasizing their applicability and adaptation to the Kazakh language. Special attention is given to linguistic and technical barriers, including the agglutinative structure, rich vowel system, and phonemic variability. Both open-source and commercial solutions were evaluated, including Whisper, GPT-4 Transcribe, ElevenLabs, OpenAI TTS, Voiser, KazakhTTS2, and TurkicTTS. Speech recognition systems were assessed using BLEU, WER, TER, chrF, and COMET, while speech synthesis was evaluated with MCD, PESQ, STOI, and DNSMOS, thus covering both lexical–semantic and acoustic–perceptual characteristics. The results demonstrate that, for speech-to-text (STT), the strongest performance was achieved by Soyle on domain-specific data (BLEU 74.93, WER 18.61), while Voiser showed balanced accuracy (WER 40.65–37.11, chrF 80.88–84.51) and GPT-4 Transcribe achieved robust semantic preservation (COMET up to 1.02). In contrast, Whisper performed weakest (WER 77.10, BLEU 13.22), requiring further adaptation for Kazakh. For text-to-speech (TTS), KazakhTTS2 delivered the most natural perceptual quality (DNSMOS 8.79–8.96), while OpenAI TTS achieved the best spectral accuracy (MCD 123.44–117.11, PESQ 1.14). TurkicTTS offered reliable intelligibility (STOI 0.15, PESQ 1.16), and ElevenLabs produced natural but less spectrally accurate speech. Full article

Background/Objectives: Accurate assessment of speech intelligibility is necessary for individuals with motor speech disorders. Transcription or scaled rating methods by naïve listeners are the most reliable tasks for these purposes; however, they are often resource-intensive and time-consuming within clinical contexts. Automatic speech recognition (ASR) systems, which transcribe speech into text, have been increasingly utilized for assessing speech intelligibility. This study investigates the feasibility of using an open-source ASR system to assess speech intelligibility in Hebrew and English speakers with Down syndrome (DS). Methods: Recordings from 65 Hebrew- and English-speaking participants were included: 33 speakers with DS and 32 typically developing (TD) peers. Speech samples (words, sentences) were transcribed using Whisper (OpenAI) and by naïve listeners. The proportion of agreement between ASR transcriptions and those of naïve listeners was compared across speaker groups (TD, DS) and languages (Hebrew, English) for word-level data. Further comparisons for Hebrew speakers were conducted across speaker groups and stimuli (words, sentences). Results: The strength of the correlation between listener and ASR transcription scores varied across languages, and was higher for English (r = 0.98) than for Hebrew (r = 0.81) for speakers with DS. A higher proportion of listener–ASR agreement was demonstrated for TD speakers, as compared to those with DS (0.94 vs. 0.74, respectively), and for English, in comparison to Hebrew speakers (0.91 for English DS speakers vs. 0.74 for Hebrew DS speakers). Listener–ASR agreement for single words was consistently higher than for sentences among Hebrew speakers. Speakers’ intelligibility influenced word-level agreement among Hebrew- but not English-speaking participants with DS. Conclusions: ASR performance for English closely approximated that of naïve listeners, suggesting potential near-future clinical applicability within single-word intelligibility assessment. In contrast, a lower proportion of agreement between human listeners and ASR for Hebrew speech indicates that broader clinical implementation may require further training of ASR models in this language. Full article

This study investigates the energy consumption and carbon footprint of two prominent automatic speech recognition (ASR) systems: OpenAI’s Whisper and Google’s Speech-to-Text API. We evaluate both local and cloud-based speech recognition approaches using a public Kaggle dataset of 20,000 short audio clips in Urdu, utilizing CodeCarbon, PyJoule, and PowerAPI for comprehensive energy profiling. As a result of our analysis, we expose some substantial differences between the two systems in terms of energy efficiency and carbon emissions, with the cloud-based solution showing substantially lower environmental impact despite comparable accuracy. We discuss the implications of these findings for sustainable AI deployment and minimizing the ecological footprint of speech recognition technologies. Full article

Natural Language Processing (NLP) and advanced acoustic analysis have opened new avenues in animal welfare research by decoding the vocal signals of farm animals. This study explored the feasibility of adapting a large-scale Transformer-based model, OpenAI’s Whisper, originally developed for human speech recognition, to decode chicken vocalizations. Our primary objective was to determine whether Whisper could effectively identify acoustic patterns associated with emotional and physiological states in poultry, thereby enabling real-time, non-invasive welfare assessments. To achieve this, chicken vocal data were recorded under diverse experimental conditions, including healthy versus unhealthy birds, pre-stress versus post-stress scenarios, and quiet versus noisy environments. The audio recordings were processed through Whisper, producing text-like outputs. Although these outputs did not represent literal translations of chicken vocalizations into human language, they exhibited consistent patterns in token sequences and sentiment indicators strongly correlated with recognized poultry stressors and welfare conditions. Sentiment analysis using standard NLP tools (e.g., polarity scoring) identified notable shifts in “negative” and “positive” scores that corresponded closely with documented changes in vocal intensity associated with stress events and altered physiological states. Despite the inherent domain mismatch—given Whisper’s original training on human speech—the findings clearly demonstrate the model’s capability to reliably capture acoustic features significant to poultry welfare. Recognizing the limitations associated with applying English-oriented sentiment tools, this study proposes future multimodal validation frameworks incorporating physiological sensors and behavioral observations to further strengthen biological interpretability. To our knowledge, this work provides the first demonstration that Transformer-based architectures, even without species-specific fine-tuning, can effectively encode meaningful acoustic patterns from animal vocalizations, highlighting their transformative potential for advancing productivity, sustainability, and welfare practices in precision poultry farming. Full article

This paper explores advancements in real-time talking-head generation, focusing on overcoming challenges in Audio Feature Extraction (AFE), which often introduces latency and limits responsiveness in real-time applications. To address these issues, we propose and implement a fully integrated system that replaces conventional AFE models with OpenAI’s Whisper, leveraging its encoder to optimize processing and improve overall system efficiency. Our evaluation of two open-source real-time models across three different datasets shows that Whisper not only accelerates processing but also improves specific aspects of rendering quality, resulting in more realistic and responsive talking-head interactions. Although interviewer training systems are considered a potential application, the primary contribution of this work is the improvement of the technical foundations necessary for creating responsive AI avatars. These advancements enable more immersive interactions and expand the scope of AI-driven applications, including educational tools and simulated training environments. Full article

Social media has revolutionized the dissemination of information, enabling the rapid and widespread sharing of news, concepts, technologies, and ideas. YouTube is one of the most important online video sharing platforms of our time. In this research, we investigate the trace of separation through membrane distillation (MD) on YouTube using statistical methods and natural language processing. The dataset collected on 04.01.2024 included 212 videos with key characteristics such as durations, views, subscribers, number of comments, likes, etc. The results show that the number of videos is not sufficient, but there is an increasing trend, especially since 2019. The high number of channels offering information about MD technology in countries such as the USA, India, and Canada indicates that these countries recognized the practical benefits of this technology, especially in areas such as water treatment, desalination, and industrial applications. This suggests that MD could play a pivotal role in finding solutions to global water challenges. Word cloud analysis showed that terms such as “water”, “treatment”, “desalination”, and “separation” were prominent, indicating that the videos focused mainly on the principles and applications of MD. The sentiment of the comments is mostly positive, and the dominant emotion is neutral, revealing that viewers generally have a positive attitude towards MD. The narrative intensity metric evaluates the information transfer efficiency of the videos and provides a guide for effective content creation strategies. The results of the analyses revealed that social media awareness about MD technology is still not sufficient and that content development and sharing strategies should focus on bringing the technology to a wider audience. Full article

In an era where dietary habits significantly impact health, technological interventions can offer personalized and accessible food choices. This paper introduces Chef Dalle, a recipe recommendation system that leverages multi-model and multimodal human-computer interaction (HCI) techniques to provide personalized cooking guidance. The application integrates voice-to-text conversion via Whisper and ingredient image recognition through GPT-Vision. It employs an advanced recipe filtering system that utilizes user-provided ingredients to fetch recipes, which are then evaluated through multi-model AI through integrations of OpenAI, Google Gemini, Claude, and/or Anthropic APIs to deliver highly personalized recommendations. These methods enable users to interact with the system using voice, text, or images, accommodating various dietary restrictions and preferences. Furthermore, the utilization of DALL-E 3 for generating recipe images enhances user engagement. User feedback mechanisms allow for the refinement of future recommendations, demonstrating the system’s adaptability. Chef Dalle showcases potential applications ranging from home kitchens to grocery stores and restaurant menu customization, addressing accessibility and promoting healthier eating habits. This paper underscores the significance of multimodal HCI in enhancing culinary experiences, setting a precedent for future developments in the field. Full article

Nowadays, fine-tuning has emerged as a powerful technique in machine learning, enabling models to adapt to a specific domain by leveraging pre-trained knowledge. One such application domain is automatic speech recognition (ASR), where fine-tuning plays a crucial role in addressing data scarcity, especially for languages with limited resources. In this study, we applied fine-tuning in the context of atypical speech recognition, focusing on Italian speakers with speech impairments, e.g., dysarthria. Our objective was to build a speaker-dependent voice user interface (VUI) tailored to their unique needs. To achieve this, we harnessed a pre-trained OpenAI’s Whisper model, which has been exposed to vast amounts of general speech data. However, to adapt it specifically for disordered speech, we fine-tuned it using our private corpus including 65 K voice recordings contributed by 208 speech-impaired individuals globally. We exploited three variants of the Whisper model (small, base, tiny), and by evaluating their relative performance, we aimed to identify the most accurate configuration for handling disordered speech patterns. Furthermore, our study dealt with the local deployment of the trained models on edge computing nodes, with the aim to realize custom VUIs for persons with impaired speech. Full article