The loss of a limb, e.g., caused by vascular/infectious diseases or trauma, is an event that strongly affects the quality of life under several aspects. Indeed, this event usually limits the person in performing working and daily living activities, thus having a strong impact on the social life. To date, an upper limb prosthesis able to achieve the dexterity (22 degrees of freedom–DoFs) and the grasp force (more than 500 N) of the human hand with anatomical size and weight (about 400 g) is far from being realized.
Focusing on the control strategy of such devices, commercially available artificial hands and arms exploit surface electromyographic (EMG) electrodes that record the electrical activity generated by the residual muscles during contraction. When independent agonist/antagonist muscle pairs are available each of these can be mapped to a unique function in the prosthesis, thus implementing the so called direct control [1
]. The latter is the most intuitive and robust approach [2
], albeit either not possible, due to the lack of accessible independent control sources with surface electrodes or inefficient [3
] in the case of multiple functions/DoFs.
To overcome this limit and enhance the number of naturally controllable DoFs, several alternatives driven by technological innovations and surgical procedures were proposed. Novel technologies allowed direct interfacing with the physiological structures involved in motor control, e.g., wireless implantable myoelectric sensors (IMES) [4
] or epymisial electrodes wired through osseointegrated implants [6
]. In a complimentary fashion, new surgical procedures, like the targeted muscle reinnervation [9
] or the regenerative peripheral nerve interfaces [10
] made possible the creation of novel control sites.
In this work, we focus on an alternative solution for the control of dexterous prostheses first introduced by our group, and dubbed the myokinetic approach [11
]. The idea behind this method is to implant permanent magnets (magnetic markers—MM) into the residual muscles, track their movements using magnetic sensors, and use these signals as control inputs in a prosthesis. As an example for transradial amputations, MMs would be implanted in the residual forearm muscles, i.e., the extrinsic muscles of the hand. Knowing the position of each marker implanted in a specific muscle would then allow to control the DoF of the prosthetic hand originally controlled by that muscle. Indeed, localizing the position of the magnet is equivalent to measuring the contraction/elongation of the muscle it is implanted in, as the magnet moves with it. If successful, this approach could allow to physiologically (i.e., simultaneously and proportionally) control multiple, independent DoFs of the prosthesis by exploiting simple passive implants.
In our previous work, we presented a system able to localize online up to four MMs using six magnetic sensors and a desktop PC [11
]. Building on this, we developed and validated an embedded version able to retrieve online the pose (i.e., the position and orientation) of up to five MMs using 32 sensors. We assessed its performance using a 3D positioning platform and an anatomical mockup of the human forearm with simulated muscles. In particular, the 3D positioning platform was used to evaluate the accuracy and precision of the localization with respect to the distance between the sensors and one single magnet. The mockup was instead used to simulate muscle movements along anatomical trajectories in order to compare the performance (accuracy, precision and computation time) between the embedded solution and the PC version.
The results with the 3D platform provided insights on the maximum distance between sensors and the implanted magnet. The tests with the forearm mockup demonstrated a localization error in the range of ~10% the entire trajectories, remarkable precision (1% repeatability) and high linearity (R2 = 0.99), for both setups. The embedded system, however, proved four times faster than the PC implementation, exhibiting a computation time below 4 ms for localizing up to five MMs. These outcomes demonstrate the feasibility of implementing simultaneous, independent and continuous control of individual digits on a hand prosthesis using the myokinetic approach on a portable device.
The tests with a single magnet using the 3D platform showed that the accuracy of the pose estimation has a non-monotonic trend with respect to the distance between the magnet and the sensors plane (Figure 4
a). In particular, the position error (
ranged between ~0.5 mm at a distance of 20 mm and ~1.5 mm at 40 mm, in its median value; the precision (
) proved very good and ranged between 14 μm and 165 μm (Figure 4
b). Similarly, the mean orientation errors (
ranged between ~3° at 20 mm and ~6° at 40 mm (Figure 4
c), with a precision (
) spreading between 0.04° and ~0.6° (Figure 4
The embedded implementation of the localizer proved functional also when using the forearm mockup, exhibiting accuracies and precisions comparable to those of the desktop implementation of the localizer (absolute Euclidean difference between the localized poses below 10 µm, for each datum). Hence, only the results from the embedded platform are shown (Figure 5
The localized displacements and the metrics with the five-magnet configuration are first presented as a representative outcome (Figure 5
ranged between 10 and 380 µm across magnets (Figure 5
a). The relationship between the actual and the computed displacement proved highly linear for all MMs (R2
> 0.99, p
< 0.001, Figure 5
a). The movement of one MM affected the position estimate of the other MMs (due to cross-talk); the importance of this effect varied according to which MM was moved. The maximum
(1.38 mm, corresponding to ~11% the length of the trajectory) was measured on the position estimate of MM2 while moving MM5.
(averaged across the trajectory) proved always lower than 220 µm. This corresponds to less than 5% the entire trajectory of each magnet. The mean
proved lower than 1.19 mm, i.e., less than 10% the entire stroke. The mean
ranged between 40 µm (MM4) and 120 µm (MM2). The mean
was maximum for MM2 (and equal to 430 µm), caused by the movement of MM4. In fact, MM2 proved the magnet with higher sensibility to the movements of the other magnets (i.e., higher cross-talk) and to the variability associated to repeated measures. Conversely MM5 proved the most immune to such effects. Accuracy and precision did not substantially change with a smaller number of magnets with MM2 always being the most sensible (Figure 5
Given the linear superposition of the effects in Equation (1), the computation time linearly increased (R2
~ 95%) with the number of magnets, for both the embedded and the desktop implementation, as expected (Figure 6
). The embedded platform was able to localize 5 MMs (positions and orientations) in less than 4 ms, compared to 15 ms required by the desktop implementation. On average, a ~75% decrease in computation time was observed with the embedded platform. Notably, the real-time engine of the CU allowed the embedded platform to be much more repeatable (lower variance in the CT).
In this work, a myokinetic localizer for the eventual control of a hand prosthesis was implemented on an embedded real time system and experimentally assessed using a 3D platform and a forearm mockup. The performance, in terms of accuracy and precision with up to five magnets, were comparable to those achieved on a desktop PC implementation, while the computation time proved largely reduced.
The results obtained with the 3D platform confirm that the errors increase with distance [18
] and suggest to place the magnets within 36 mm from the plane of the sensors. Indeed within such height the median position error (
is lower than 1.1 mm (Figure 4
), i.e., ~10% the range of motion of extrinsic hand muscles [20
], which can be considered as the upper bound of the localization error. In addition, the median precision (
), which is the most relevant metric to measure the performance of the myokinetic localizer [11
], is below 120 μm. In fact, the functionality of the system is not associated with the ability to retrieve the absolute displacement from the rest position, but on the repeatability of the measurement.
The non-monotonic trend of the position and orientation accuracies with distance is in contrast with the results from similar studies [18
]. Perhaps this may be caused by un-modelled ferromagnetic components on the AU board or inside the sensors that may either deflect the magnetic field (making the assumptions behind Equation (1) even more approximated) or produce a remnant magnetization that yields to an offset in the readouts.
The accuracies and precisions obtained using the forearm mockup are consistent with those achieved in our previous work, although obtained with a larger number of magnets, viz. five instead of four [11
]. The mean position accuracy
) demonstrated always lower than 0.22 (1.19) mm, while the mean precision
) proved below 120 (430) µm in all configurations, yielding to highly repeatable measurements (Figure 5
b). The high linearity found between the real and computed displacement (Figure 5
a) is important for direct control in a hand prosthesis, as it means that the system can preserve the relationship between muscle contraction and the relative movement in the prosthesis.
The higher sensibility exhibited by MM2, and for all configurations where this was used, is likely to be due to its distance to the sensor matrix. This effect was already reported in our previous works [11
]. However, here we provide additional items to the discussion. First, the present result suggests that a wrong positioning of a magnet yields to inaccurate localizations for that magnet, regardless the number of magnets involved (Figure 5
b). Second, while the wrongly positioned magnet is highly sensible to movements of the other magnets and to the variability associated to repeated measures, it is also true that its effect on the other magnets is negligible. Hence, in the case one magnet during the surgery was accidentally implanted too far from the AU, that magnet could be taken into account but simply ignored for control, without significant effects on the functioning of the remaining magnets.
The shorter computation times exhibited by the embedded platform are largely due to the code parallelization (using the DSP) and optimization (Figure 6
). The absolute result—4 ms to localize the pose of five magnets—is rather promising in the perspective of the target application, where delays up to 100 ÷ 250 ms are considered acceptable [21
]. Our result leaves an ample time margin that can be exploited to increase the number of implanted (and tracked) magnets, and thus to increase the DoFs controllable with the myokinetic approach. More specifically, if the goal is to control the five digits, including the abduction of the thumb, along with the pronation/supination and flexion/extension of the wrist, the localizer should be able to track up to 16 MMs. Theoretically, 32 three-axis sensors would be enough to retrieve their unknown position and orientation. In practice, to improve the accuracy of the LMA, more sensors, spatially distributed in proximity of the implanted MMs, would be needed. Although our results suggest the feasibility of such implementation from a computational viewpoint, timing issues deserve further analyses. Notably, an increase in the number of magnets and sensors implies not only a longer CT but also longer acquisition and data-transfer times (Figure 2
). This to say that improving the performance of the system may require to modify its architecture.
In fact, the developed embedded system showed limitations under different aspects that curtail its use in an actual prosthesis. First, the sampling could not be accurately synchronized across sensors due to hardware limitations of the AU (and the architecture of the sensors). However, this represents a drawback concerning the consistency of the data and in turn the bandwidth of the localizer. Hence this study invites the development of an AU capable of simultaneous (triggered) sampling from a number of sensors. Second, the present implementation did not consider nor compensated for environmental factors, such as external magnetic fields or the effects of relative compression/depression movements between the socket (where the localizer is) and the stump tissues. These issues are known to degrade the performance of the localizer [11
] and thus will be addressed in future studies. Third, the embedded system showed limited performance in terms of power efficiency. Overall it requires ~980 mW, which is a considerable amount of power for a (portable) prosthesis controller. Indeed, low power consumption is fundamental in a prosthesis, as larger batteries mean more weight the user has to carry, and less space available for the other components. Finally, it should be noted that while each component of the localizer represents a reasonable design choice, the architecture and the hardware of the embedded system could be optimized in order to take into account all the previous limitations.
With dedicated technical effort, a custom hardware (including both the sensing and computation apparatus) based on the CU (and, specifically, the CPU) used here could be developed and directly integrated into a prosthetic socket, making the system truly portable. This paves the way towards the development of a human-machine interface that allows simultaneous and proportional control of multiple movements of a hand prosthesis. Yet, potentially, this solution could be generalized to any type of upper or lower limb amputation, given that the muscles in the forearm represent the worst case due to their sizes and tightness.