Search Results (27)

This paper presents the theoretical foundation, practical implementation, and empirical evaluation of a glove for interaction with 3-D virtual environments. At the dawn of the “Spatial Computing Era”, where users continuously interact with 3-D Virtual and Augmented Reality environments, the need for a practical and intuitive interaction system that can efficiently engage 3-D elements is becoming pressing. Over the last few decades, there have been attempts to provide such an interaction mechanism using a glove. However, glove systems are currently not in widespread use due to their high cost and, we propose, due to their inability to sustain high levels of performance under certain situations. Performance deterioration has been observed due to the distortion of the local magnetic field caused by ordinary ferromagnetic objects present near the glove’s operating space. There are several areas where reliable hand-tracking gloves could provide a next generation of improved solutions, such as American Sign Language training and automatic translation to text and training and evaluation for activities that require high motor skills in the hands (e.g., playing some musical instruments, training of surgeons, etc.). While the use of a hand-tracking glove toward these goals seems intuitive, some of the currently available glove systems may not meet the accuracy and reliability levels required for those use cases. This paper describes our concept of an interaction glove instrumented with miniature magnetic, angular rate, and gravity (MARG) sensors and aided by a single camera. The camera used is an off-the-shelf red, green, and blue–depth (RGB-D) camera. We describe a proof-of-concept implementation of the system using our custom “GMVDK” orientation estimation algorithm. This paper also describes the glove’s empirical evaluation with human-subject performance tests. The results show that the prototype glove, using the GMVDK algorithm, is able to operate without performance losses, even in magnetically distorted environments. Full article

While the availability of low-cost micro electro-mechanical systems (MEMS) accelerometers, gyroscopes, and magnetometers initially seemed to promise the possibility of using them to easily track the position and orientation of virtually any object that they could be attached to, this promise has not yet been fulfilled. Navigation-grade accelerometers and gyroscopes have long been the basis for tracking ships and aircraft, but the signals from low-cost MEMS accelerometers and gyroscopes are still orders of magnitude poorer in quality (e.g., bias stability). Therefore, the applications of MEMS inertial measurement units (IMUs), containing tri-axial accelerometers and gyroscopes, are currently not as extensive as they were expected to be. Even the addition of MEMS tri-axial magnetometers, to conform magnetic, angular rate, and gravity (MARG) sensor modules, has not fully overcome the challenges involved in using these modules for long-term orientation estimation, which would be of great benefit for the tracking of human–computer hand-held controllers or tracking of Internet-Of-Things (IoT) devices. Here, we present an algorithm, GMVDμK (or simply GMVDK), that aims at taking full advantage of all the signals available from a MARG module to robustly estimate its orientation, while preventing damaging overcorrections, within the context of a human–computer interaction application. Through experimental comparison, we show that GMVDK is more robust to magnetic disturbances than three other MARG orientation estimation algorithms in representative trials. Full article

Motion capture systems have enormously benefited the research into human–computer interaction in the aerospace field. Given the high cost and susceptibility to lighting conditions of optical motion capture systems, as well as considering the drift in IMU sensors, this paper utilizes a fusion approach with low-cost wearable sensors for hybrid upper limb motion tracking. We propose a novel algorithm that combines the fourth-order Runge–Kutta (RK4) Madgwick complementary orientation filter and the Kalman filter for motion estimation through the data fusion of an inertial measurement unit (IMU) and an ultrawideband (UWB). The Madgwick RK4 orientation filter is used to compensate gyroscope drift through the optimal fusion of a magnetic, angular rate, and gravity (MARG) system, without requiring knowledge of noise distribution for implementation. Then, considering the error distribution provided by the UWB system, we employ a Kalman filter to estimate and fuse the UWB measurements to further reduce the drift error. Adopting the cube distribution of four anchors, the drift-free position obtained by the UWB localization Kalman filter is used to fuse the position calculated by IMU. The proposed algorithm has been tested by various movements and has demonstrated an average decrease in the RMSE of 1.2 cm from the IMU method to IMU/UWB fusion method. The experimental results represent the high feasibility and stability of our proposed algorithm for accurately tracking the movements of human upper limbs. Full article

The care of cystic fibrosis (CF) traditionally consists of regular visits to the clinic where a multidisciplinary team can visit the patient, adjust treatments and monitor the disease. During the COVID-19 pandemic when access to hospitals and medical environments was very limited, the role of telemedicine was crucial to keep in touch with patients with chronic diseases such as CF. Increasing evidence demonstrates that electronic health can successfully support healthcare professionals in the management of people with CF. The use of devices connected to digital platforms or smartphones results in a continuous flow of data that can be shared with the clinician and the team in order to improve the knowledge of patients’ diseases and the level of care needed. This narrative review aims to describe the application of telemedicine in CF disease with pros and cons. A literature analysis showed that telemedicine has several advantages in the management of patients with CF. With the evolving support of digital technology, telemedicine can promote clinical visits, adherence to daily treatment, including respiratory physiotherapy and physical exercise, early identification of pulmonary exacerbations and management of psychological issues. The main disadvantages are missed physical exam findings, lack of physical contact that can prevent conversation on sensitive topics, lack of access to technology and lack of technological skills. Furthermore, healthcare operators need appropriate training for telemedicine systems and need time to organise and analyse data generated remotely, which may increase the burden of daily work. Hybrid personalised care models that marge telemedicine and traditional care can be an ideal solution. Full article

In this paper, we present the FIU MARG Dataset (FIUMARGDB) of signals from the tri-axial accelerometer, gyroscope, and magnetometer contained in a low-cost miniature magnetic–angular rate–gravity (MARG) sensor module (also known as magnetic inertial measurement unit, MIMU) for the evaluation of MARG orientation estimation algorithms. The dataset contains 30 files resulting from different volunteer subjects executing manipulations of the MARG in areas with and without magnetic distortion. Each file also contains reference (“ground truth”) MARG orientations (as quaternions) determined by an optical motion capture system during the recording of the MARG signals. The creation of FIUMARGDB responds to the increasing need for the objective comparison of the performance of MARG orientation estimation algorithms, using the same inputs (accelerometer, gyroscope, and magnetometer signals) recorded under varied circumstances, as MARG modules hold great promise for human motion tracking applications. This dataset specifically addresses the need to study and manage the degradation of orientation estimates that occur when MARGs operate in regions with known magnetic field distortions. To our knowledge, no other dataset with these characteristics is currently available. FIUMARGDB can be accessed through the URL indicated in the conclusions section. It is our hope that the availability of this dataset will lead to the development of orientation estimation algorithms that are more resilient to magnetic distortions, for the benefit of fields as diverse as human–computer interaction, kinesiology, motor rehabilitation, etc. Full article

Precise pedestrian positioning based on smartphone-grade sensors has been a research hotspot for several years. Due to the poor performance of the mass-market Micro-Electro-Mechanical Systems (MEMS) Magnetic, Angular Rate, and Gravity (MARG) sensors, the standalone pedestrian dead reckoning (PDR) module cannot avoid long-time heading drift, which leads to the failure of the entire positioning system. In outdoor scenes, the Global Navigation Satellite System (GNSS) is one of the most popular positioning systems, and smartphone users can use it to acquire absolute coordinates. However, the smartphone’s ultra-low-cost GNSS module is limited by some components such as the antenna, and so it is susceptible to serious interference from the multipath effect, which is a main error source of smartphone-based GNSS positioning. In this paper, we propose a multi-phase GNSS/PDR fusion framework to overcome the limitations of standalone modules. The first phase is to build a pseudorange double-difference based on smartphone and reference stations, the second phase proposes a novel multipath mitigation method based on multipath partial parameters estimation (MPPE) and a Double-Difference Code-Minus-Carrier (DDCMC) filter, and the third phase is to propose the joint stride lengths and heading estimations of the two standalone modules, to reduce the long-time drift and noise. The experimental results demonstrate that the proposed multipath error estimation can effectively suppress the double-difference multipath error exceeding 4 m, and compared to other methods, our fusion method achieves a minimum error RMSE of 1.63 m in positioning accuracy, and a minimum error RMSE of 4.71 m in long-time robustness for 20 min of continuous walking. Full article

Background: This article presents the system architecture and validation of the NeuroSuitUp body–machine interface (BMI). The platform consists of wearable robotics jacket and gloves in combination with a serious game application for self-paced neurorehabilitation in spinal cord injury and chronic stroke. Methods: The wearable robotics implement a sensor layer, to approximate kinematic chain segment orientation, and an actuation layer. Sensors consist of commercial magnetic, angular rate and gravity (MARG), surface electromyography (sEMG), and flex sensors, while actuation is achieved through electrical muscle stimulation (EMS) and pneumatic actuators. On-board electronics connect to a Robot Operating System environment-based parser/controller and to a Unity-based live avatar representation game. BMI subsystems validation was performed using exercises through a Stereoscopic camera Computer Vision approach for the jacket and through multiple grip activities for the glove. Ten healthy subjects participated in system validation trials, performing three arm and three hand exercises (each 10 motor task trials) and completing user experience questionnaires. Results: Acceptable correlation was observed in 23/30 arm exercises performed with the jacket. No significant differences in glove sensor data during actuation state were observed. No difficulty to use, discomfort, or negative robotics perception were reported. Conclusions: Subsequent design improvements will implement additional absolute orientation sensors, MARG/EMG based biofeedback to the game, improved immersion through Augmented Reality and improvements towards system robustness. Full article

Advances in micro-electro-mechanical systems technology have led to the emergence of compact attitude measurement sensor products that integrate acceleration, magnetometer, and gyroscope sensors on a single chip, making them important devices in the field of three-dimensional (3D) attitude measurement for unmanned aerial vehicles, smartphones, and other devices. Sensor fusion algorithms for posture measurement have become an indispensable technology in cutting-edge research, such as human posture measurement using wearable sensors, and stabilization problems in robot position and posture measurement. We have also developed wearable sensors and powered suits in our previous research. We needed a technology for the real-time measurement of a 3D human body motion. It is known that quaternions can be used to algebraically handle 3D rotations; however, sensor fusion algorithms for three sensors are presently complex. This is because these algorithms deal with the post-rotation attitude (pure quaternions) rather than rotation information (the rotor) to avoid a double covering problem involving the rotor. If we are dealing with rotation, it may be possible to make the algorithm simpler and faster by dealing directly with the rotor. In this study, to solve the double covering problem involving the rotor, we propose a stateful rotor and develop a technique for uniquely determining the time-varying states of the rotor. The proposed stateful rotor guarantees the continuity of the rotor parameters with respect to angular changes, and this paper confirms its effectiveness by simulating two rotations around an arbitrary axis. In addition, we verify experimentally that a fast sensor fusion method using stateful rotor can be used for attitude calculation. Experiments also confirm that the calculated results converge to the desired rotation angle for two spatial rotations around an arbitrary axis. Since the proposed stateful rotor extends and stabilizes the definition of the rotor, it is applicable to any algorithm that deals with time-varying quaternionic rotors. In this research, an algorithm based on a multiply–add operation is designed to reduce computational complexity as a high-speed calculation for embedded systems. This method is theoretically equivalent to other methods, while contributing to power saving and the cost reduction of products. Full article

Aiming at the problem of the weak dynamic performance of the gradient descent method in the attitude and heading reference system, the susceptibility to the interference of accelerometers and magnetometers, and the complex calculation of the nonlinear Kalman Filter method, an extended Kalman filter suitable for a low-cost magnetic, angular rate, and gravity (MARG) sensor system is proposed. The method proposed in this paper is a combination of a two-stage gradient descent algorithm and the extended Kalman filter (GDEKF). First, the accelerometer and magnetometer are used to correct the attitude angle according to the two-stage gradient descent algorithm. The obtained attitude quaternion is combined with the gyroscope measurement value as the observation vector of EKF and the calculated attitude of the gyroscope and the bias of the gyroscope are corrected. The elimination of the bias of the gyroscope can further improve the stability of the attitude observation results. Finally, the MARG sensor system was designed for mathematical model simulation and hardware-in-the-loop simulation to verify the performance of the filter. The results show that compared with the gradient descent method, it has better anti-interference performance and dynamic performance, and better measurement accuracy than the extended Kalman filter. Full article

Indoor localization based on pedestrian dead reckoning (PDR) is drawing more and more attention of researchers in location-based services (LBS). The demand for indoor localization has grown rapidly using a smartphone. This paper proposes a 3D indoor positioning method based on the micro-electro-mechanical systems (MEMS) sensors of the smartphone. A quaternion-based robust adaptive cubature Kalman filter (RACKF) algorithm is proposed to estimate the heading of pedestrians based on magnetic, angular rate, and gravity (MARG) sensors. Then, the pedestrian behavior patterns are distinguished by detecting the changes of pitch angle, total accelerometer and barometer values of the smartphone in the duration of effective step frequency. According to the geometric information of the building stairs, the step length of pedestrians and the height difference of each step can be obtained when pedestrians go up and downstairs. Combined with the differential barometric altimetry method, the optimal height can be computed by the robust adaptive Kalman filter (RAKF) algorithm. Moreover, the heading and step length of each step are optimized by the Kalman filter to reduce positioning error. In addition, based on the indoor map vector information, this paper proposes a heading calculation strategy of the 16-wind rose map to improve the pedestrian positioning accuracy and reduce the accumulation error. Pedestrian plane coordinates can be solved based on the Pedestrian Dead-Reckoning (PDR). Finally, combining pedestrian plane coordinates and height, the three-dimensional positioning coordinates of indoor pedestrians are obtained. The proposed algorithm is verified by actual measurement examples. The experimental verification was carried out in a multi-story indoor environment. The results show that the Root Mean Squared Error (RMSE) of location errors is 1.04–1.65 m by using the proposed algorithm for three participants. Furthermore, the RMSE of height estimation errors is 0.17–0.27 m for three participants, which meets the demand of personal intelligent user terminal for location service. Moreover, the height parameter enables users to perceive the floor information. Full article

This paper presents an open-source environment for development, tuning, and performance evaluation of magnetic, angular rate, and gravity-based (MARG-based) filters, such as pose estimators and classification algorithms. The environment is available in both ROS/Gazebo and MATLAB/Simulink, and it contains a six-degrees of freedom (6 DOF) test bench, which simultaneously moves and rotates an MARG unit in the three-dimensional (3D) space. As the quality of MARG-based estimation becomes crucial in dynamic situations, the proposed test platform intends to simulate different accelerating and vibrating circumstances, along with realistic magnetic perturbation events. Moreover, the simultaneous acquisition of both the real pose states (ground truth) and raw sensor data is supported during these simulated system behaviors. As a result, the test environment executes the desired mixture of static and dynamic system conditions, and the provided database fosters the effective analysis of sensor fusion algorithms. The paper systematically describes the structure of the proposed test platform, from mechanical properties, over mathematical modeling and joint controller synthesis, to implementation results. Additionally, a case study is presented of the tuning of popular attitude estimation algorithms to highlight the advantages of the developed open-source environment. Full article

The demands for indoor positioning in location-based services (LBS) and applications grow rapidly. It is beneficial for indoor positioning to combine attitude and heading information. Accurate attitude and heading estimation based on magnetic, angular rate, and gravity (MARG) sensors of micro-electro-mechanical systems (MEMS) has received increasing attention due to its high availability and independence. This paper proposes a quaternion-based adaptive cubature Kalman filter (ACKF) algorithm to estimate the attitude and heading based on smart phone-embedded MARG sensors. In this algorithm, the fading memory weighted method and the limited memory weighted method are used to adaptively correct the statistical characteristics of the nonlinear system and reduce the estimation bias of the filter. The latest step data is used as the memory window data of the limited memory weighted method. Moreover, for restraining the divergence, the filter innovation sequence is used to rectify the noise covariance measurements and system. Besides, an adaptive factor based on prediction residual construction is used to overcome the filter model error and the influence of abnormal disturbance. In the static test, compared with the Sage-Husa cubature Kalman filter (SHCKF), cubature Kalman filter (CKF), and extended Kalman filter (EKF), the mean absolute errors (MAE) of the heading pitch and roll calculated by the proposed algorithm decreased by 4–18%, 14–29%, and 61–77% respectively. In the dynamic test, compared with the above three filters, the MAE of the heading reduced by 1–8%, 2–18%, and 2–21%, and the mean of location errors decreased by 9–22%, 19–31%, and 32–54% respectively by using the proposed algorithm for three participants. Generally, the proposed algorithm can effectively improve the accuracy of heading. Moreover, it can also improve the accuracy of attitude under quasistatic conditions. Full article

Assistive robots support people with limited mobility in their everyday life activities and work. However, most of the assistive systems and technologies for supporting eating and drinking require a residual mobility in arms or hands. For people without residual mobility, different hands-free controls have been developed. For hands-free control, the combination of different modalities can lead to great advantages and improved control. The novelty of this work is a new concept to control a robot using a combination of head and eye motions. The control unit is a mobile, compact and low-cost multimodal sensor system. A Magnetic Angular Rate Gravity (MARG)-sensor is used to detect head motion and an eye tracker enables the system to capture the user’s gaze. To analyze the performance of the two modalities, an experimental evaluation with ten able-bodied subjects and one subject with tetraplegia was performed. To assess discrete control (event-based control), a button activation task was performed. To assess two-dimensional continuous cursor control, a Fitts’s Law task was performed. The usability study was related to a use-case scenario with a collaborative robot assisting a drinking action. The results of the able-bodied subjects show no significant difference between eye motions and head motions for the activation time of the buttons and the throughput, while, using the eye tracker in the Fitts’s Law task, the error rate was significantly higher. The subject with tetraplegia showed slightly better performance for button activation when using the eye tracker. In the use-case, all subjects were able to use the control unit successfully to support the drinking action. Due to the limited head motion of the subject with tetraplegia, button activation with the eye tracker was slightly faster than with the MARG-sensor. A further study with more subjects with tetraplegia is planned, in order to verify these results. Full article

This study proposes a minimal modeling magnetic, angular rate and gravity (MARG) methodology for assessing spatiotemporal and kinematic measures of functional fitness exercises. Thirteen healthy persons performed repetitions of the squat, box squat, sandbag pickup, shuffle-walk, and bear crawl. Sagittal plane hip, knee, and ankle range of motion (ROM) and stride length, stride time, and stance time measures were compared for the MARG method and an optical motion capture (OMC) system. The root mean square error (RMSE), mean absolute percentage error (MAPE), and Bland–Altman plots and limits of agreement were used to assess agreement between methods. Hip and knee ROM showed good to excellent agreement with the OMC system during the squat, box squat, and sandbag pickup (RMSE: 4.4–9.8°), while ankle ROM agreement ranged from good to unacceptable (RMSE: 2.7–7.2°). Unacceptable hip and knee ROM agreement was observed for the shuffle-walk and bear crawl (RMSE: 3.3–8.6°). The stride length, stride time, and stance time showed good to excellent agreement between methods (MAPE: (3.2 ± 2.8)%–(8.2 ± 7.9)%). Although the proposed MARG-based method is a valid means of assessing spatiotemporal and kinematic measures during various exercises, further development is required to assess the joint kinematics of small ROM, high velocity movements. Full article

This paper presents the use of eye tracking data in Magnetic AngularRate Gravity (MARG)-sensor based head orientation estimation. The approach presented here can be deployed in any motion measurement that includes MARG and eye tracking sensors (e.g., rehabilitation robotics or medical diagnostics). The challenge in these mostly indoor applications is the presence of magnetic field disturbances at the location of the MARG-sensor. In this work, eye tracking data (visual fixations) are used to enable zero orientation change updates in the MARG-sensor data fusion chain. The approach is based on a MARG-sensor data fusion filter, an online visual fixation detection algorithm as well as a dynamic angular rate threshold estimation for low latency and adaptive head motion noise parameterization. In this work we use an adaptation of Madgwicks gradient descent filter for MARG-sensor data fusion, but the approach could be used with any other data fusion process. The presented approach does not rely on additional stationary or local environmental references and is therefore self-contained. The proposed system is benchmarked against a Qualisys motion capture system, a gold standard in human motion analysis, showing improved heading accuracy for the MARG-sensor data fusion up to a factor of 0.5 while magnetic disturbance is present. Full article