Search Results (19)

This paper presents a Self-Learning Robotic System (SLRS) for healthcare assistance using Deep Imitation Learning (DIL). The proposed SLRS solution can observe and replicate human demonstrations, thereby acquiring complex skills without the need for explicit task-specific programming. It incorporates modular components for perception (i.e., advanced computer vision methodologies), actuation (i.e., dynamic interaction with patients and healthcare professionals in real time), and learning. The innovative approach of implementing a hybrid model approach (i.e., deep imitation learning and pose estimation algorithms) facilitates autonomous learning and adaptive task execution. The environmental awareness and responsiveness were also enhanced using both a Convolutional Neural Network (CNN)-based object detection mechanism using YOLOv8 (i.e., with 94.3% accuracy and 18.7 ms latency) and pose estimation algorithms, alongside a MediaPipe and Long Short-Term Memory (LSTM) framework for human action recognition. The developed solution was tested and validated in healthcare, with the aim to overcome some of the current challenges, such as workforce shortages, ageing populations, and the rising prevalence of chronic diseases. The CAD simulation, validation, and verification tested functions (i.e., assistive functions, interactive scenarios, and object manipulation) of the system demonstrated the robot’s adaptability and operational efficiency, achieving an 87.3% task completion success rate and over 85% grasp success rate. This approach highlights the potential use of an SLRS for healthcare assistance. Further work will be undertaken in hospitals, care homes, and rehabilitation centre environments to generate complete holistic datasets to confirm the system’s reliability and efficiency. Full article

Artificial intelligence (AI) and assistive robotics can transform older-person care by offering new, personalised solutions for an ageing population. This paper outlines recent advances in AI-driven applications and robotic assistance in silver care, emphasising their role in improved healthcare services, quality of life and ageing-in-place and alleviating pressure on healthcare systems. Advances in machine learning, natural language processing and computer vision have enabled more accurate early diagnosis, targeted treatment plans and robust remote monitoring for elderly patients. These innovations support continuous health tracking and timely interventions to improve patient outcomes and extend home-based care. In addition, AI-powered assistive robots with advanced motion control and adaptive response mechanisms are studied to support physical and cognitive health. Among these, companion robots, often enhanced with emotional AI, have shown potential in reducing loneliness and increasing connectedness. The combined goal of these technologies is to offer holistic patient-centred care, which preserves the autonomy and dignity of our seniors. This paper also touches on the technical and ethical challenges of integrating AI/robotics into eldercare, like privacy and accessibility, and alludes to future directions on optimising AI-human interaction, expanding preventive healthcare applications and creating an effective, ethical framework for eldercare in the digital age. Full article

Effective detection techniques are important for wildlife monitoring and conservation applications and are especially helpful for species that live in complex environments, such as arboreal animals like koalas (Phascolarctos cinereus). The implementation of infrared cameras and drones has demonstrated encouraging outcomes, regardless of whether the detection was performed by human observers or automated algorithms. In the case of koala detection in eucalyptus plantations, there is a risk to spotters during forestry operations. In addition, fatigue and tedium associated with the difficult and repetitive task of checking every tree means automated detection options are particularly desirable. However, obtaining high detection rates with minimal false alarms remains a challenging task, particularly when there is low contrast between the animals and their surroundings. Koalas are also small and often partially or fully occluded by canopy, tree stems, or branches, or the background is highly complex. Biologically inspired vision systems are known for their superior ability in suppressing clutter and enhancing the contrast of dim objects of interest against their surroundings. This paper introduces a biologically inspired detection algorithm to locate koalas in eucalyptus plantations and evaluates its performance against ten other detection techniques, including both image processing and neural-network-based approaches. The nature of koala occlusion by canopy cover in these plantations was also examined using a combination of simulated and real data. The results show that the biologically inspired approach significantly outperformed the competing neural-network- and computer-vision-based approaches by over 27%. The analysis of simulated and real data shows that koala occlusion by tree stems and canopy can have a significant impact on the potential detection of koalas, with koalas being fully occluded in up to 40% of images in which koalas were known to be present. Our analysis shows the koala’s heat signature is more likely to be occluded when it is close to the centre of the image (i.e., it is directly under a drone) and less likely to be occluded off the zenith. This has implications for flight considerations. This paper also describes a new accurate ground-truth dataset of aerial high-dynamic-range infrared imagery containing instances of koala heat signatures. This dataset is made publicly available to support the research community. Full article

Human action recognition (HAR) as the most representative human-centred computer vision task is critical in human resource management (HRM), especially in human resource recruitment, performance appraisal, and employee training. Currently, prevailing approaches to human action recognition primarily emphasize either temporal or spatial features while overlooking the intricate interplay between these two dimensions. This oversight leads to less precise and robust action classification within complex human resource recruitment environments. In this paper, we propose a novel human action recognition methodology for human resource recruitment environments, which aims at symmetrically harnessing temporal and spatial information to enhance the performance of human action recognition. Specifically, we compute Depth Motion Maps (DMM) and Depth Temporal Maps (DTM) from depth video sequences as space and time descriptors, respectively. Subsequently, a novel feature fusion technique named Center Boundary Collaborative Canonical Correlation Analysis (CBCCCA) is designed to enhance the fusion of space and time features by collaboratively learning the center and boundary information of feature class space. We then introduce a spatio-temporal information filtration module to remove redundant information introduced by spatio-temporal fusion and retain discriminative details. Finally, a Support Vector Machine (SVM) is employed for human action recognition. Extensive experiments demonstrate that the proposed method has the ability to significantly improve human action recognition performance. Full article

In the context of Industry 5.0, the concept of the Metaverse aligns with the vision of Web 4.0, representing a digital ecosystem where individuals and organizations collaborate in a human-centric approach to create personalized value. This virtual universe connects multiple interconnected worlds, enabling real-time interactions between users and computer-generated environments. By integrating technologies like artificial intelligence (AI), virtual reality (VR), and the Internet of Things (IoT), the Metaverse within Industry 5.0 aims to foster innovation and enhance productivity, efficiency, and overall well-being through tailored and value-driven solutions. Therefore, this entry explores the concept of the Metaverse in the context of Industry 5.0, highlighting its definition, evolution, advantages, and disadvantages. It also discusses the pillars of technological advancement, challenges, and opportunities, including its integration into manufacturing. The entry concludes with a proposal for a conceptual framework for integrating the human-centric Metaverse into manufacturing. Full article

With the rapid development of enabling technologies like VR and AR, we human beings are on the threshold of the ubiquitous human-centric intelligence era. 6G is believed to be an indispensable cornerstone for efficient interaction between humans and computers in this promising vision. 6G is supposed to boost many human-centric applications due to its unprecedented performance improvements compared to 5G and before. However, challenges are still to be addressed, including but not limited to the following six aspects: Terahertz and millimeter-wave communication, low latency and high reliability, energy efficiency, security, efficient edge computing and heterogeneity of services. It is a daunting job to fit traditional analytical methods into these problems due to the complex architecture and highly dynamic features of ubiquitous interactive 6G systems. Fortunately, deep learning can circumvent the interpretability issue and train tremendous neural network parameters, which build mapping relationships from neural network input (status and specific requirements of a 6G application) to neural network output (settings to satisfy the requirements). Deep learning methods can be an efficient alternative to traditional analytical methods or even conquer unresolvable predicaments of analytical methods. We review representative deep learning solutions to the aforementioned six aspects separately and focus on the principles of fitting a deep learning method into specific 6G issues. Based on this review, our main contributions are highlighted as follows. (i) We investigate the representative works in a systematic view and find out some important issues like the vital role of deep reinforcement learning in the 6G context. (ii) We point out solutions to the lack of training data in 6G communication context. (iii) We reveal the relationship between traditional analytical methods and deep learning, in terms of 6G applications. (iv) We identify some frequently used efficient techniques in deep-learning-based 6G solutions. Finally, we point out open problems and future directions. Full article

The recognition of human activities using vision-based techniques has become a crucial research field in video analytics. Over the last decade, there have been numerous advancements in deep learning algorithms aimed at accurately detecting complex human actions in video streams. While these algorithms have demonstrated impressive performance in activity recognition, they often exhibit a bias towards either model performance or computational efficiency. This biased trade-off between robustness and efficiency poses challenges when addressing complex human activity recognition problems. To address this issue, this paper presents a computationally efficient yet robust approach, exploiting saliency-aware spatial and temporal features for human action recognition in videos. To achieve effective representation of human actions, we propose an efficient approach called the dual-attentional Residual 3D Convolutional Neural Network (DA-R3DCNN). Our proposed method utilizes a unified channel-spatial attention mechanism, allowing it to efficiently extract significant human-centric features from video frames. By combining dual channel-spatial attention layers with residual 3D convolution layers, the network becomes more discerning in capturing spatial receptive fields containing objects within the feature maps. To assess the effectiveness and robustness of our proposed method, we have conducted extensive experiments on four well-established benchmark datasets for human action recognition. The quantitative results obtained validate the efficiency of our method, showcasing significant improvements in accuracy of up to 11% as compared to state-of-the-art human action recognition methods. Additionally, our evaluation of inference time reveals that the proposed method achieves up to a 74× improvement in frames per second (FPS) compared to existing approaches, thus showing the suitability and effectiveness of the proposed DA-R3DCNN for real-time human activity recognition. Full article

Vision-based human activity recognition (HAR) has emerged as one of the essential research areas in video analytics. Over the last decade, numerous advanced deep learning algorithms have been introduced to recognize complex human actions from video streams. These deep learning algorithms have shown impressive performance for the video analytics task. However, these newly introduced methods either exclusively focus on model performance or the effectiveness of these models in terms of computational efficiency, resulting in a biased trade-off between robustness and computational efficiency in their proposed methods to deal with challenging HAR problem. To enhance both the accuracy and computational efficiency, this paper presents a computationally efficient yet generic spatial–temporal cascaded framework that exploits the deep discriminative spatial and temporal features for HAR. For efficient representation of human actions, we propose an efficient dual attentional convolutional neural network (DA-CNN) architecture that leverages a unified channel–spatial attention mechanism to extract human-centric salient features in video frames. The dual channel–spatial attention layers together with the convolutional layers learn to be more selective in the spatial receptive fields having objects within the feature maps. The extracted discriminative salient features are then forwarded to a stacked bi-directional gated recurrent unit (Bi-GRU) for long-term temporal modeling and recognition of human actions using both forward and backward pass gradient learning. Extensive experiments are conducted on three publicly available human action datasets, where the obtained results verify the effectiveness of our proposed framework (DA-CNN+Bi-GRU) over the state-of-the-art methods in terms of model accuracy and inference runtime across each dataset. Experimental results show that the DA-CNN+Bi-GRU framework attains an improvement in execution time up to 167× in terms of frames per second as compared to most of the contemporary action-recognition methods. Full article

Computing has undergone a significant transformation over the past two decades, shifting from a machine-based approach to a human-centric, virtually invisible service known as ubiquitous or pervasive computing. This change has been achieved by incorporating small embedded devices into a larger computational system, connected through networking and referred to as edge devices. When these devices are also connected to the Internet, they are generally named Internet-of-Thing (IoT) devices. Developing Machine Learning (ML) algorithms on these types of devices allows them to provide Artificial Intelligence (AI) inference functions such as computer vision, pattern recognition, etc. However, this capability is severely limited by the device’s resource scarcity. Embedded devices have limited computational and power resources available while they must maintain a high degree of autonomy. While there are several published studies that address the computational weakness of these small systems-mostly through optimization and compression of neural networks- they often neglect the power consumption and efficiency implications of these techniques. This study presents power efficiency experimental results from the application of well-known and proven optimization methods using a set of well-known ML models. The results are presented in a meaningful manner considering the “real world” functionality of devices and the provided results are compared with the basic “idle” power consumption of each of the selected systems. Two different systems with completely different architectures and capabilities were used providing us with results that led to interesting conclusions related to the power efficiency of each architecture. Full article

Metallographic analyses of nodular iron casting methods are based on visual comparisons according to measuring standards. Specifically, the microstructure is analyzed in a subjective manner by comparing the extracted image from the microscope to pre-defined image templates. The achieved classifications can be confused, due to the fact that the features extracted by a human being could be interpreted differently depending on many variables, such as the conditions of the observer. In particular, this kind of problem represents an uncertainty when classifying metallic properties, which can influence the integrity of castings that play critical roles in safety devices or structures. Although there are existing solutions working with extracted images and applying some computer vision techniques to manage the measurements of the microstructure, those results are not too accurate. In fact, they are not able to characterize all specific features of the image and, they cannot be adapted to several characterization methods depending on the specific regulation or customer. Hence, in order to solve this problem, we propose a framework to improve and automatize the evaluations by combining classical machine vision techniques for feature extraction and deep learning technologies, to objectively make classifications. To adapt to the real analysis environments, all included inputs in our models were gathered directly from the historical repository of metallurgy from the Azterlan Research Centre (labeled using expert knowledge from engineers). The proposed approach concludes that these techniques (a classification under a pipeline of deep neural networks and the quality classification using an ANN classifier) are viable to carry out the extraction and classification of metallographic features with great accuracy and time, and it is possible to deploy software with the models to work on real-time situations. Moreover, this method provides a direct way to classify the metallurgical quality of the molten metal, allowing us to determine the possible behaviors of the final produced parts. Full article

The detection of crowd density levels and anomalies is a hot topic in video surveillance. Especially in human-centric action and activity-based movements. In some respects, the density level variation is considered an anomaly in the event. Crowd behaviour identification relies on a computer-vision-based approach and basically deals with spatial information of foreground video information. In this work, we focused on a deep-learning-based attention-oriented classification system for identifying several basic movements in public places, especially, human flock movement, sudden motion changes and panic events in several indoor and outdoor places. The important spatial features were extracted from a bilinear CNN and a multicolumn multistage CNN with preprocessed morphological video frames from videos. Finally, the abnormal and crowd density estimation was distinguished by using an attention feature combined with a multilayer CNN feature by modifying the fully connected layer for several categories (binary and multiclass). We validate the proposed method on several video surveillance datasets including PETS2009, UMN and UCSD. The proposed method achieved an accuracy of 98.62, 98.95, 96.97, 99.10 and 98.38 on the UCSD Ped1, UCSD Ped2, PETS2009, UMN Plaza1 and UMN Plaza2 datasets, respectively, with the different pretrained models. We compared the performance between recent modern approaches and the proposed method (MCMS-BCNN-Attention) and achieved the highest accuracy. The anomaly detection performance on the UMN and PETS2009 datasets was compared with that of a state-of-the-art method and achieved the best AUC results as 0.9953 and 1.00 for both scenarios, respectively, with a binary classification. Full article

The adoption of computer vision pose estimation approaches, used to identify keypoint locations which are intended to reflect the necessary anatomical landmarks relied upon by biomechanists for musculoskeletal modelling, has gained increasing traction in recent years. This uptake has been further accelerated by keypoint use as inputs into machine learning models used to estimate biomechanical parameters such as ground reaction forces (GRFs) in the absence of instrumentation required for direct measurement. This study first aimed to investigate the keypoint detection rate of three open-source pose estimation models (AlphaPose, BlazePose, and OpenPose) across varying movements, camera views, and trial lengths. Second, this study aimed to assess the suitability and interchangeability of keypoints detected by each pose estimation model when used as inputs into machine learning models for the estimation of GRFs. The keypoint detection rate of BlazePose was distinctly lower than that of AlphaPose and OpenPose. All pose estimation models achieved a high keypoint detection rate at the centre of an image frame and a lower detection rate in the true sagittal plane camera field of view, compared with slightly anteriorly or posteriorly located quasi-sagittal plane camera views. The three-dimensional ground reaction force, instantaneous loading rate, and peak force for running could be estimated using the keypoints of all three pose estimation models. However, only AlphaPose and OpenPose keypoints could be used interchangeably with a machine learning model trained to estimate GRFs based on AlphaPose keypoints resulting in a high estimation accuracy when OpenPose keypoints were used as inputs and vice versa. The findings of this study highlight the need for further evaluation of computer vision-based pose estimation models for application in biomechanical human modelling, and the limitations of machine learning-based GRF estimation models that rely on 2D keypoints. This is of particular relevance given that machine learning models informing athlete monitoring guidelines are being developed for application related to athlete well-being. Full article

Research and technology developments on autonomous agents and autonomic computing promote a vision of artificial systems that are able to resiliently manage themselves and autonomously deal with issues at runtime in dynamic environments. Indeed, autonomy can be leveraged to unburden humans from mundane tasks (cf. driving and autonomous vehicles), from the risk of operating in unknown or perilous environments (cf. rescue scenarios), or to support timely decision-making in complex settings (cf. data-centre operations). Beyond the results that individual autonomous agents can carry out, a further opportunity lies in the collaboration of multiple agents or robots. Emerging macro-paradigms provide an approach to programming whole collectives towards global goals. Aggregate computing is one such paradigm, formally grounded in a calculus of computational fields enabling functional composition of collective behaviours that could be proved, under certain technical conditions, to be self-stabilising. In this work, we address the concept of collective autonomy, i.e., the form of autonomy that applies at the level of a group of individuals. As a contribution, we define an agent control architecture for aggregate multi-agent systems, discuss how the aggregate computing framework relates to both individual and collective autonomy, and show how it can be used to program collective autonomous behaviour. We exemplify the concepts through a simulated case study, and outline a research roadmap towards reliable aggregate autonomy. Full article

The study of the human body and its movements is still a matter of great interest today. Most of these issues have as their fulcrum the study of the balance characteristics of the human body and the determination of its Centre of Mass. In sports, a lot of attention is paid to improving and analysing the athlete’s performance. Almost all the techniques for determining the Centre of Mass make use of special sensors, which allow determining the physical magnitudes related to the different movements made by athletes. In this paper, a markerless method for determining the Centre of Mass of a subject has been studied, comparing it with a direct widely validated equipment such as the Wii Balance Board, which allows determining the coordinates of the Centre of Pressure. The Motion Capture technique was applied with the OpenPose software, a Computer Vision method boosted with the use of Convolution Neural Networks. Ten quasi-static analyses have been carried out. The results have shown an error of the Centre of Mass position, compared to that obtained from the Wii Balance Board, which has been considered acceptable given the complexity of the analysis. Furthermore, this method, despite the traditional methods based on the use of balances, can be used also for prediction of the vertical position of the Centre of Mass. Full article

The integration of exponential technologies in the traditional manufacturing processes constitutes a noteworthy trend of the past two decades, aiming to reshape the industrial environment. This kind of digital transformation, which is driven by the Industry 4.0 initiative, not only affects the individual manufacturing assets, but the involved human workforce, as well. Since human operators should be placed in the centre of this revolution, they ought to be endowed with new tools and through-engineering solutions that improve their efficiency. In addition, vivid visualization techniques must be utilized, in order to support them during their daily operations in an auxiliary and comprehensive way. Towards this end, we describe a user-centered methodology, which utilizes augmented reality (AR) and computer vision (CV) techniques, supporting low-skilled operators in the maintenance procedures. The described mobile augmented reality maintenance assistant (MARMA) makes use of the handheld’s camera and locates the asset on the shop floor and generates AR maintenance instructions. We evaluate the performance of MARMA in a real use case scenario, using an automotive industrial asset provided by a collaborative manufacturer. During the evaluation procedure, manufacturer experts confirmed its contribution as an application that can effectively support the maintenance engineers. Full article