Multimodal learning analytics (MMLA), which has become increasingly popular, can help provide an accurate understanding of learning processes. However, it is still unclear how multimodal data is integrated into MMLA. By following the Preferred Report...
In the digital era, multimodal behavioral research has emerged as a pivotal discipline, integrating diverse data sources to comprehensively understand human behavior. This paper defines and distinguishes data governance from mere data management with...
Multimodal clustering achieves outstanding performance in various applications by aggregating information from heterogeneous devices. However, previous methods rely on strong-notion distances to fuse crossmodal complementary knowledge, established on...
Data processing in robotics is currently challenged by the effective building of multimodal and common representations. Tremendous volumes of raw data are available and their smart management is the core concept of multimodal learning in a new paradi...
Efficient multimodal instruction tuning requires reducing both training data volume and computational cost without substantially sacrificing downstream performance. Existing data selection methods can remove redundancy but cannot repair imperfect sup...
One of the most significant challenges in cities concerns urban mobility. Urban mobility involves the use of different modes of transport, which can be individual or collective, and different organizations can produce their respective datasets that,...
In recent years, the focus of healthcare and wellness technologies has shown a significant shift towards personal vital signs devices. The technology has evolved from smartphone-based wellness applications to fitness bands and smartwatches. The novel...
Multimodal Learning Analytics (MMLA) is an extension of Learning Analytics that combines multiple data streams such as audio, video, physiological signals, logs, and spatial trails to analyze learning processes that cannot be easily captured through...
Pupillometry is commonly used to evaluate cognitive effort, attention, and facial expression response, offering valuable insights into human performance. The combination of eye tracking and facial expression data under the iMotions platform provides...
Different mixtures of multimodal training data significantly impact the performance of multimodal large language models, and manually tuning data mixtures is inefficient, computationally expensive, and frequently suboptimal because of complex, nonlin...
There have been multiple technological advancements that promise to gradually enable devices to measure and record signals with high resolution and accuracy in the domain of brain–computer interfaces (BCIs). Multimodal BCIs have been able to ga...
Ship trajectory prediction is essential for ensuring safe route planning and to have advanced warning of the dangers at sea. With the development of deep learning, most of the current research has explored advanced prediction methods based on histori...
Multimodal remote sensing data classification can enhance a model’s ability to distinguish land features through multimodal data fusion. In this context, how to help models understand the relationship between multimodal data and target tasks ha...
Multimodal medical data provides a wide and real basis for disease diagnosis. Computer-aided diagnosis (CAD) powered by artificial intelligence (AI) is becoming increasingly prominent in disease diagnosis. CAD for multimodal medical data requires add...
Today, data are no longer confined to numerical values arranged in row-by-column matrices or stored neatly within relational databases. One of the defining characteristics of big data is its high variety, encompassing unstructured and multimodal form...
This study investigated to what extent multimodal data can be used to detect mistakes during Cardiopulmonary Resuscitation (CPR) training. We complemented the Laerdal QCPR ResusciAnne manikin with the Multimodal Tutor for CPR, a multi-sensor system c...
With the advent of big data science, the field of geoscience has undergone a paradigm shift toward data-driven scientific discovery. However, the abundance of geoscience data distributed across multiple sources poses significant challenges to researc...
The effective association of multimodal data is the basis of massive multi-source heterogeneous data sharing in the era of big data. How to realize data autonomous association between massive multimodal databases and the automatic intelligent screeni...
In many tasks related to an object’s observation or real-time monitoring, the gathering of temporal multimodal data is required. Such data sets are semantically connected as they reflect different aspects of the same object. However, data sets...
In social interactions, people who are perceived as competent win more chances, tend to have more opportunities, and perform better in both personal and professional aspects of their lives. However, the process of evaluating competence is still poorl...
Lung disease is one of the leading causes of death worldwide. This emphasizes the need for early diagnosis in order to provide appropriate treatment and save lives. Physicians typically require information about patients’ clinical symptoms, var...
This study aims to develop an emotion evaluation method for second language learners, utilizing multimodal information to comprehensively evaluate students’ emotional expressions. Addressing the limitations of existing emotion evaluation method...
Moving traces are essential data for target detection and associated behavior recognition. Previous studies have used time–location sequences, route maps, or tracking videos to establish mathematical recognition models for behavior recognition....
This study deeply integrates multimodal data analysis and big data technology, proposing a multimodal learning framework that consolidates various information sources, such as user geographic location, behavior data, and product attributes, to achiev...
The latest satellite infrastructure for data processing, transmission and reception can certainly be improved by upgrading tools used to deal with very large amounts of data from every different sensor incorporated within the space missions. In order...
Autopilot technology in the field of aviation has developed over many years. However, it is difficult for an autopilot system to autonomously operate a civil aircraft under bad weather conditions. In this paper, we present a reinforcement learning (R...
Personalised modelling has become dominant in personalised medicine and precision health. It creates a computational model for an individual based on large data repositories of existing personalised data, aiming to achieve the best possible personal...
During natural disasters, social media can provide real time or rapid disaster, perception information to help government managers carry out disaster response efforts efficiently. Therefore, it is of great significance to mine social media informatio...
Although abundant spatiotemporal data are collected before and after landslides, the volume, variety, intercorrelation, and heterogeneity of multimodal data complicates disaster assessments, so it is challenging to select information from multimodal...
Tomato leaf diseases cause substantial yield and quality losses worldwide, yet reliable detection in real fields remains challenging. Two practical bottlenecks dominate current research: (i) limited data, including small samples for rare diseases, cl...
With the development of social media, the internet, and sensing technologies, multimodal data are becoming increasingly common. Integrating these data into knowledge graphs can help models to better understand and utilize these rich sources of inform...
Audio-visual emotion recognition is the research of identifying human emotional states by combining the audio modality and the visual modality simultaneously, which plays an important role in intelligent human-machine interactions. With the help of d...
This study introduces a multimodal framework integrating retrieval-augmented generation (RAG) with multimodal large language models (MLLMs) to enhance the accessibility, interpretability, and analysis of Eurobarometer survey data. Traditional approac...
Accurate geospatial data are essential for intelligent transportation systems and automated reporting applications, as location precision directly impacts safety analysis and decision-making. GPS devices are now routinely employed by law enforcement...
Identifying sedimentary facies represents a fundamental aspect of oil and gas exploration. In recent years, geologists have employed deep learning methods to develop comprehensive predictions of sedimentary facies. However, their methods are often co...
The paper aims to develop an information system for human emotion recognition in streaming data obtained from a PC or smartphone camera, using different methods of modality merging (image, sound and text). The objects of research are the facial expre...
With the development of network technology, the number of gambling websites has grown dramatically, causing a threat to social stability. There are many machine learning-based methods are proposed to identify gambling websites by analyzing the URL, t...
Background: Transcatheter aortic valve replacement (TAVR) has emerged as a pivotal minimally invasive interventional therapy for aortic valve disease and has seen increasingly widespread clinical adoption in recent years. Despite its overall safety,...
Industrial control systems (ICSs) are cyber–physical environments in which physical process data and network communication data are generated simultaneously. Existing studies have mainly focused on either sensor-based or network-based anomaly d...
Although various studies on monitoring dog behavior have been conducted, methods that can minimize or compensate data noise are required. This paper proposes multimodal data-based dog behavior recognition that fuses video and sensor data using a came...
This study investigates the efficacy of early multimodal data fusion for identifying academic performance profiles and predicting academic risk in specialized postgraduate education under small-sample constraints (
Recently, the research community has shown significant interest in the continuous temporal data obtained from motion sensors in wearable devices. These data are useful for classifying and analysing different human activities in many application areas...
Conventional DC control and protection (C&P) systems rely on single-modal electrical data and are susceptible to false tripping and missed detection under complex operating conditions or novel fault types. In high-voltage direct current (HVDC) tr...
One of the top concerns for travelers when choosing public transportation is whether they can reach their destination in limited time and monetary cost on the basis of ensured reliability. However, the existing literature shows no studies on how to e...
Urban mobility systems generate large volumes of heterogeneous geospatial data that differ in temporal resolution, spatial coverage, update frequency, and semantic structure, making integrated near-real-time analysis difficult. This paper presents a...
To address the challenges in existing multi-sensor data fusion methods for rail surface defect diagnosis, particularly their limitations in fully exploiting potential synergistic information among multimodal data and effectively bridging the semantic...
This article presents the implementation and technical feasibility evaluation of the multimodal sensing and feature-level fusion layer of BLEXER v3, a broader serious-game ecosystem under development for upper-limb rehabilitation. The implemented fra...
Precision livestock farming (PLF), by integrating multimodal sensor data, provides opportunities to enhance welfare monitoring and management in small ruminants. This study evaluated whether environmental, physiological, and behavioral measurements&m...
Background/Objectives: In recent years, people have been facing a lot of difficulties in handling stress, emotions, social, and behavioral issues, which have led to severe mental disorders. Schizophrenia is one disorder that requires more attention....
A novel eggplant disease detection method based on multimodal data fusion and attention mechanisms is proposed in this study, aimed at improving both the accuracy and robustness of disease detection. The method integrates image and sensor data, optim...
of 120