Search Results (19)

Using personal smart devices such as mobile phones to perform precise distal pointing in intelligent cockpits is a developing trend. The present study investigated the effects of different control display gains (CD gains) and wrist movement modalities on performance and comfort for precise distal pointing interaction. Twenty healthy participants performed a precise distant pointing task with four constant CD gains (0.6, 0.8, 0.84, and 1.0), two dynamic CD gains, and two wrist movement modalities (wrist extension and rotation) by using a mobile phone as the input device. Physiological electromyographic data, task performance, and subjective questionnaire data were collected. Comparative results show that constant CD gain is superior to dynamic CD gain and that 0.8 to 1.0 is the optimum range of values. The data showed a clear and consistent trend in performance and comfort as the CD gain increased from 0.6 to 1.0, with performance and comfort becoming progressively better, reaching an optimum at 0.84. In terms of the wrist control method, the rotation mode had smaller task completion time than the extension mode. The results of this study provide a basis for the design of remote interaction using mobile phones in an intelligent cockpit. Full article

Driver factors are the main cause of road traffic accidents. For the research of automotive active safety, an identification method for road hypnosis of a driver of a car with dynamic human–vehicle heterogeneous data fusion calculation is proposed. Road hypnosis is an unconscious driving state formed by the combination of external environmental factors and the psychological state of the car driver. When drivers fall into a state of road hypnosis, they cannot clearly perceive the surrounding environment and make various reactions in time to complete the driving task. The safety of humans and cars is greatly affected. Therefore, the study of the identification of drivers’ road hypnosis is of great significance. Vehicle and virtual driving experiments are designed and carried out to collect human and vehicle data. Eye movement data and EEG data of human data are collected with eye movement sensors and EEG sensors. Vehicle speed and acceleration data are collected by a mobile phone with AutoNavi navigation, which serves as an onboard sensor. In order to screen the characteristics of human and vehicles related to the road hypnosis state, the characteristic parameters of the road hypnosis in the preprocessed data are selected by the method of independent sample T-test, the hidden Markov model (HMM) is constructed, and the identification of the road hypnosis of the Ridge Regression model is combined. In order to evaluate the identification performance of the model, six evaluation indicators are used and compared with multiple regression models. The results show that the hidden Markov-Ridge Regression model is the most superior in the identification accuracy and effect of the road hypnosis state. A new technical scheme reference for the development of intelligent driving assistance systems is provided by the proposed comprehensive road hypnosis state identification model based on human–vehicle data can provide, which can effectively improve the life recognition ability of automobile intelligent cockpits, enhance the active safety performance of automobiles, and further improve traffic safety. Full article

The advent of Artificial Intelligence in the cockpit and the air traffic control centre in the coming decade could mark a step-change improvement in aviation safety, or else could usher in a flush of ‘AI-induced’ accidents. Given that contemporary AI has well-known weaknesses, from data biases and edge or corner effects, to outright ‘hallucinations’, in the mid-term AI will almost certainly be partnered with human expertise, its outputs monitored and tempered by human judgement. This is already enshrined in the EU Act on AI, with adherence to principles of human agency and oversight required in safety-critical domains such as aviation. However, such sound policies and principles are unlikely to be enough. Human interactions with current automation in the cockpit or air traffic control tower require extensive requirements, methods, and validations to ensure a robust (accident-free) partnership. Since AI will inevitably push the boundaries of traditional human-automation interaction, there is a need to revisit Human Factors to meet the challenges of future human-AI interaction design. This paper briefly reviews the types of AI and ‘Intelligent Agents’ along with their associated levels of AI autonomy being considered for future aviation applications. It then reviews the evolution of Human Factors to identify the critical areas where Human Factors can aid future human-AI teaming performance and safety, to generate a detailed requirements set organised for Human AI Teaming design. The resultant requirements set comprises eight Human Factors areas, from Human-Centred Design to Organisational Readiness, and 165 detailed requirements, and has been applied to three AI-based Intelligent Agent prototypes (two cockpit, one air traffic control tower). These early applications suggest that the new requirements set is scalable to different design maturity levels and different levels of AI autonomy, and acceptable as an approach to Human-AI Teaming design teams. Full article

Human factors are the most important factor in road traffic crashes. Human-caused traffic crashes can be reduced through the active safety system of vehicles. Road hypnosis is an unconscious driving state caused by the combination of external environmental factors and the driver’s psychological state. When drivers fall into a state of road hypnosis, they cannot clearly perceive the surrounding environment and make various reactions in time to complete the driving task, and driving safety is greatly affected. Therefore, road hypnosis identification is of great significance for the active safety of vehicles. A road hypnosis identification model based on XGBoost—Hidden Markov is proposed in this study. Driver data and vehicle data related to road hypnosis are collected through the design and conduct of vehicle driving experiments. Driver data, including eye movement data and EEG data, are collected with eye movement sensors and EEG sensors. A mobile phone with AutoNavi navigation is used as an on-board sensor to collect vehicle speed, acceleration, and other information. Power spectrum density analysis, the sliding window method, and the point-by-point calculation method are used to extract the dynamic characteristics of road hypnosis, respectively. Through normalization and standardization, the key features of the three types of data are integrated into unified feature vectors. Based on XGBoost and the Hidden Markov algorithm, a road hypnotic identification model is constructed. The model is verified and evaluated through visual analysis. The results show that the road hypnosis state can be effectively identified by the model. The extraction of road hypnosis-related features is realized in non-fixed driving routes in this study. A new research idea for road hypnosis and a technical scheme reference for the development of intelligent driving assistance systems are provided, and the life identification ability of the vehicle intelligent cockpit is also improved. It is of great significance for the active safety of vehicles. Full article

The cockpit is evolving from passive, reactive interaction toward proactive, cognitive interaction, making precise predictions of driver intent a key factor in enhancing proactive interaction experiences. This paper introduces Cockpit-Llama, a novel language model specifically designed for predicting driver behavior intent. Cockpit-Llama predicts driver intent based on the relationship between current driver actions, historical interactions, and the states of the driver and cockpit environment, thereby supporting further proactive interaction decisions. To improve the accuracy and rationality of Cockpit-Llama’s predictions, we construct a new multi-attribute cockpit dataset that includes extensive historical interactions and multi-attribute states, such as driver emotional states, driving activity scenarios, vehicle motion states, body states and external environment, to support the fine-tuning of Cockpit-Llama. During fine-tuning, we adopt the Low-Rank Adaptation (LoRA) method to efficiently optimize the parameters of the Llama3-8b-Instruct model, significantly reducing training costs. Extensive experiments on the multi-attribute cockpit dataset demonstrate that Cockpit-Llama’s prediction performance surpasses other advanced methods, achieving BLEU-4, ROUGE-1, ROUGE-2, and ROUGE-L scores of 71.32, 80.01, 76.89, and 81.42, respectively, with relative improvements of 92.34%, 183.61%, 95.54%, and 201.27% compared to ChatGPT-4. This significantly enhances the reasoning and interpretative capabilities of intelligent cockpits. Full article

Situation awareness (SA) discrimination is significant, allowing for the pilot to maintain task performance and ensure flight safety, especially during high-stress flight tasks. Although previous research has attempted to identify and classify SA, existing SA discrimination models are predominantly binary and rely on traditional machine learning methods with limited physiological modalities. The current study aimed to construct a triple-class SA discrimination model for pilots facing high-stress tasks. To achieve this, a flight simulation experiment under typical high-stress tasks was carried out and deep learning algorithms (multilayer perceptron (MLP) and the attention mechanism) were utilized. Specifically, eye-tracking (ET), heart rate variability (HRV), and electroencephalograph (EEG) modalities were chosen as the model’s input features. Comparing the unimodal models, the results indicate that EEG modality surpasses ET and HRV modalities, and the attention mechanism structure has advantageous implications for processing the EEG modalities. The most superior model fused the three modalities at the decision level, with two MLP backbones and an attention mechanism backbone, achieving an accuracy of 83.41% and proving that the model performance would benefit from multimodal fusion. Thus, the current research established a triple-class SA discrimination model for pilots, laying the foundation for the real-time evaluation of SA under high-stress aerial operating conditions and providing a reference for intelligent cockpit design and dynamic human–machine function allocation. Full article

The development of large language models (LLMs) has promoted a transformation of human–computer interaction (HCI) models and has attracted the attention of scholars to the evaluation of personality traits of LLMs. As an important interface for the HCI and human–machine interface (HMI) in the future, the intelligent cockpit has become one of LLM’s most important application scenarios. When in-vehicle intelligent systems based on in-vehicle LLMs begin to become human assistants or even partners, it has become important to study the “personality” of in-vehicle LLMs. Referring to the relevant research on personality traits of LLMs, this study selected the psychological scales Big Five Inventory-2 (BFI-2), Myers–Briggs Type Indicator (MBTI), and Short Dark Triad (SD-3) to establish a personality traits evaluation framework for in-vehicle LLMs. Then, we used this framework to evaluate the personality of three in-vehicle LLMs. The results showed that psychological scales can be used to measure the personality traits of in-vehicle LLMs. In-vehicle LLMs showed commonalities in extroversion, agreeableness, conscientiousness, and action patterns, yet differences in openness, perception, decision-making, information acquisition methods, and psychopathy. According to the results, we established anthropomorphic personality personas of different in-vehicle LLMs. This study represents a novel attempt to evaluate the personalities of in-vehicle LLMs. The experimental results deepen our understanding of in-vehicle LLMs and contribute to the further exploration of personalized fine-tuning of in-vehicle LLMs and the improvement in the user experience of the automobile in the future. Full article

Lane changing is a frequent traffic accident scenario. To improve the driving safety in lane changing scenarios, the blind spot display of lane changing is increased through human–machine interaction (HMI) interfaces in intelligent vehicles to improve the driver’s rate of risk perception with regard to the driving environment. However, blind spot information will increase the cognitive load of drivers and lead to driving distraction. To quantify the coupling relationship between blind spot display and drivers’ cognitive load, we proposed a method to quantify the cognitive load of the driver’s interaction by improving the AttenD algorithm, collecting feature data by carrying out a variety of real-vehicle road-testing experiments on three kinds of intelligent vehicles, and then establishing a model blind spot design and driver cognitive load correlation model using Bayesian Logistic Ordinal Regression (BLOR) and Categorical Boosting (CatBoost). The results show that the blind spot image display can reduce the driver’s cognitive load more effectively as it is closer to the driver, has a larger area, and occupies a higher proportion of the center control screen, especially when it is located in the middle and upper regions of the center control screen. The improved AttenD algorithm is able to quantify the cognitive load of the driver, which can be widely used in vehicle testing, HMI interface development and evaluation. In addition, the analytical framework constructed in this paper can help us to understand the complex impact of HMI in intelligent vehicles and provide optimization criteria for lane change blind spot design. Full article

Highly integrated information sharing among people, vehicles, roads, and cloud systems, along with the rapid development of autonomous driving technologies, has spurred the evolution of automobiles from simple “transportation tools” to interconnected “intelligent systems”. The intelligent cockpit is a comprehensive application space for various new technologies in intelligent vehicles, encompassing the domains of driving control, riding comfort, and infotainment. It provides drivers and passengers with safety, comfort, and pleasant driving experiences, serving as the gateway for traditional automobile manufacturing to upgrade towards an intelligent automotive industry ecosystem. This is the optimal convergence point for the intelligence, connectivity, electrification, and sharing of automobiles. Currently, the form, functions, and interaction methods of the intelligent cockpit are gradually changing, transitioning from the traditional “human adapts to the vehicle” viewpoint to the “vehicle adapts to human”, and evolving towards a future of natural interactive services where “humans and vehicles mutually adapt”. This article reviews the definitions, intelligence levels, functional domains, and technical frameworks of intelligent automotive cockpits. Additionally, combining the core mechanisms of human–machine interactions in intelligent cockpits, this article proposes an intelligent-cockpit human–machine interaction process and summarizes the current state of key technologies in intelligent-cockpit human–machine interactions. Lastly, this article analyzes the current challenges faced in the field of intelligent cockpits and forecasts future trends in intelligent cockpit technologies. Full article

The OHIO (Odin Hospital Indoor cOmpass) project received funding from the European Union’s Horizon 2020 research and innovation action program, via ODIN–Open Call, which is issued and executed under the ODIN project and focuses on enhancing hospital safety, productivity, and quality by introducing digital solutions, such as the Internet of Things (IoT), robotics, and artificial intelligence (AI). OHIO aims to enhance the productivity and quality of medical equipment maintenance activities within the pilot hospital, “Le Scotte” in Siena (Italy), by leveraging internal informational resources. OHIO will also be completely integrated with the ODIN platform, taking advantage of the available services and functionalities. OHIO exploits Bluetooth Low Energy (BLE) tags and antennas together with the resources provided by the ODIN platform to develop a complex ontology-based IoT framework, which acts as a central cockpit for the maintenance of medical equipment through a central management web application and an indoor real-time location system (RTLS) for mobile devices. The application programmable interfaces (APIs) are based on REST architecture for seamless data exchange and integration with the hospital’s existing computer-aided facility management (CAFM) and computerized maintenance management system (CMMS) software. The outcomes of the project are assessed both with quantitative and qualitative methods, by evaluating key performance indicators (KPIs) extracted from the literature and performing a preliminary usability test on both the whole system and the graphic user interfaces (GUIs) of the developed applications. The test implementation demonstrates improvements in maintenance timings, including a reduction in maintenance operation delays, duration of maintenance tasks, and equipment downtime. Usability post-test questionnaires show positive feedback regarding the usability and effectiveness of the applications. The OHIO framework enhanced the effectiveness of medical equipment maintenance by integrating existing software with newly designed, enhanced interfaces. The research also indicates possibilities for scaling up the developed methods and applications to additional large-scale pilot hospitals within the ODIN network. Full article

Artificial intelligence is developing at a rapid pace, with examples of machine learning already being used in aviation to improve efficiency. In the coming decade, it is likely that intelligent assistants (IAs) will be deployed to assist aviation personnel in the cockpit, the air traffic control center, and in airports. This will be a game-changer and may herald the way forward for single-pilot operations and AI-based air traffic management. Yet in aviation there is a core underlying tenet that ‘people create safety’ and keep the skies and passengers safe, based on a robust industry-wide safety culture. Introducing IAs into aviation might therefore undermine aviation’s hard-won track record in this area. Three experts in safety culture and human-AI teaming used a validated safety culture tool to explore the potential impacts of introducing IAs into aviation. The results suggest that there are indeed potential negative outcomes, but also possible safety affordances wherein AI could strengthen safety culture. Safeguards and mitigations are suggested for the key risk owners in aviation organizations, from CEOs to middle managers, to safety departments and frontline staff. Such safeguards will help ensure safety remains a priority across the industry. Full article

Detecting the factors affecting drivers’ safe driving and taking early warning measures can effectively reduce the probability of automobile safety accidents and improve vehicle driving safety. Considering the two factors of driver fatigue and distraction state, their influences on driver behavior are elaborated from both experimental data and an accident library analysis. Starting from three modes and six types, intelligent detection methods for driver fatigue and distraction detection from the past five years are reviewed in detail. Considering its wide range of applications, the research on machine vision detection based on facial features in the past five years is analyzed, and the methods are carefully classified and compared according to their innovation points. Further, three safety warning and response schemes are proposed in light of the development of autonomous driving and intelligent cockpit technology. Finally, the paper summarizes the current state of research in the field, presents five conclusions, and discusses future trends. Full article

Considering the current limitations of intelligent interactive in electronic integration and luminescent modes, this paper proposes a novel fabrication method for pressure-sensing POF fabrics with a non-direct-contact conductive system. In this system, conductive materials are concealed in the fabric structure to avoid direct contact with the human body. It was enabled by integrating layered weave structure, POFs, conductive yarns, and fabric patches within the fabric. Laser engraving was also applied on the fabric surface to achieve intricate pattern design. Experimental tests were conducted on sensing and luminescent properties of this POF fabric. The circuit module and software were developed to support the interactive function. The potential application of this fabric in the interior components of intelligent cockpits was envisioned. The research results show that the POF fabric integrated with conductive yarns and conductive fabric patches has good pressure sensitivity, enabling control of the fabric’s luminescent color by pressing the fabric surface. The non-direct-contact conductive system developed in this study offers the advantage of electrical signal stability by avoiding interference from human body resistance and grounding conditions. The development of this type of interactive luminescent textile holds promising prospects for application and development in various fields, including intelligent cockpits. Full article

Recently, artificial intelligence and data science have witnessed dramatic progress and rapid growth, especially Automatic Speech Recognition (ASR) technology based on Hidden Markov Models (HMMs) and Deep Neural Networks (DNNs). Consequently, new end-to-end Recurrent Neural Network (RNN) toolkits were developed with higher speed and accuracy that can often achieve a Word Error Rate (WER) below 10%. These toolkits can nowadays be deployed, for instance, within aircraft cockpits and Air Traffic Control (ATC) systems in order to identify aircraft and display recognized voice messages related to flight data, especially for airports not equipped with radar. Hence, the performance of air traffic controllers and pilots can ultimately be improved by reducing workload and stress and enforcing safety standards. Our experiment conducted at Tangier’s International Airport ATC aimed to build an ASR model that is able to recognize aircraft call signs in a fast and accurate way. The acoustic and linguistic models were trained on the Ibn Battouta Speech Corpus (IBSC), resulting in an unprecedented speech dataset with approved transcription that includes real weather aerodrome observation data and flight information with a call sign captured by an ADS-B receiver. All of these data were synchronized with voice recordings in a structured format. We calculated the WER to evaluate the model’s accuracy and compared different methods of dataset training for model building and adaptation. Despite the high interference in the VHF radio communication channel and fast-speaking conditions that increased the WER level to 20%, our standalone and low-cost ASR system with a trained RNN model, supported by the Deep Speech toolkit, was able to achieve call sign detection rate scores up to 96% in air traffic controller messages and 90% in pilot messages while displaying related flight information from ADS-B data using the Fuzzy string-matching algorithm. Full article

Production scheduling management is crucial for optimizing mine productivity. With the trend towards intelligent mines, a lean scheduling management mode is required to align with intelligent conditions. This paper proposes a lean scheduling framework, based on short interval control as an effective tool to adapt intelligent scheduling in underground mines. The framework shortens the production monitoring and adjustment cycle to near-real-time, enabling timely corrective measures to minimize schedule deviations and improve overall production efficiency. An intelligent scheduling platform is implemented by adopting the digital twin platform framework, the intelligent scheduling mobile terminal module, and the integrated scheduling control cockpit module. The results indicate that the platform is effective in promoting mine intelligence by providing benefits in information transparency, flexible scheduling, lean production, and scientific decision-making. The proposed framework provides a practical solution for implementing intelligent scheduling in underground mines, contributing to the overall improvement of mine productivity. Overall, this paper provides insights for implementing intelligent scheduling in underground mines. Full article