Next Article in Journal
The Road to Accountable and Dependable Manufacturing
Previous Article in Journal
Wave Propagation and Structural Health Monitoring Application on Parts Fabricated by Additive Manufacturing
Previous Article in Special Issue
UAV Thrust Model Identification Using Spectrogram Analysis
Article

Design of Tendon-Actuated Robotic Glove Integrated with Optical Fiber Force Myography Sensor

1
Department of Computer Engineering and Industrial Automation, School of Electrical and Computing Engineering, University of Campinas, Campinas 13083-852, Brazil
2
Federal Institute of Education, Science and Technology Catarinense, Luzerna 89609-000, Brazil
3
Turing Research Laboratory, Galileo University, Guatemala City 01010, Guatemala
4
Laboratory of Photonic Materials and Devices, School of Mechanical Engineering, University of Campinas, Campinas 13083-860, Brazil
5
Laboratory of Biomedical Engineering, Department of Electrical Engineering, Centro Universitario FEI, Sao Bernardo do Campo 09859-901, Brazil
*
Author to whom correspondence should be addressed.
Academic Editor: Felipe N. Martins
Automation 2021, 2(3), 187-201; https://doi.org/10.3390/automation2030012
Received: 26 June 2021 / Revised: 30 August 2021 / Accepted: 2 September 2021 / Published: 3 September 2021
(This article belongs to the Special Issue Smart Robotics for Automation)

Abstract

People taken by upper limb disorders caused by neurological diseases suffer from grip weakening, which affects their quality of life. Researches on soft wearable robotics and advances in sensor technology emerge as promising alternatives to develop assistive and rehabilitative technologies. However, current systems rely on surface electromyography and complex machine learning classifiers to retrieve the user intentions. In addition, the grasp assistance through electromechanical or fluidic actuators is passive and does not contribute to the rehabilitation of upper-limb muscles. Therefore, this paper presents a robotic glove integrated with a force myography sensor. The glove-like orthosis features tendon-driven actuation through servo motors, working as an assistive device for people with hand disabilities. The detection of user intentions employs an optical fiber force myography sensor, simplifying the operation beyond the usual electromyography approach. Moreover, the proposed system applies functional electrical stimulation to activate the grasp collaboratively with the tendon mechanism, providing motion support and assisting rehabilitation.
Keywords: assistive technology; force myography; optical fiber sensors; user interfaces assistive technology; force myography; optical fiber sensors; user interfaces

1. Introduction

Robotic hand orthoses are breakthrough technologies to improve the quality of life of individuals with upper-limb disabilities caused by stroke, spinal cord injury, cerebral palsy, or any other neurological or musculoskeletal disorder that affects the arm or hand movements [1,2]. Recent developments toward soft robotics circumvent the typical limitations of rigid-bodied devices, yielding orthoses with improved compliance and adaptability, in addition to their lightweight and straightforward mechanical design [3,4].
However, most individuals living in low- and middle-income countries cannot afford healthcare and have limited access to medical centers [5]. This challenge motivates the pursuit of even portable, low-cost technologies for at-home rehabilitation, avoiding bulky actuators and complex control schemes [1]. For example, the authors of [6] report a soft robotic glove with hydraulic actuators, but the system lacks an intuitive interface for grasps selection. In [7,8], different groups propose instrumented gloves with Bowden cables driven by surface electromyography (sEMG); despite the simplicity, decoding the sEMG data requires intensive signal processing. As to the acquisition of intentions with brain–computer interfaces (BCIs), hand orthoses with pneumatic actuation are featured in [9,10], though the complexity and demanding calibration hinder the BCI feasibility in at-home applications. Another approach for tendon-driven gloves uses admittance control by resistive flex sensors mounted in the pulse, which may be uncomfortable due to persistent wrist extension [11]. Finally, a study [12] proposes an all-polymer soft glove activated by an analog switch, which may be incompatible with practical manipulation tasks.
In addition to a responsive joint actuation mechanism with precise grasp control, robotic gloves require intuitive approaches to detect the user commands. In this sense, force myography (FMG) emerges as a non-invasive technique for assessing the hand movements and intentions based on the radial pressures exerted by the forearm muscles. Indeed, FMG exhibits performance comparable with the widespread sEMG but requires simplified signal processing and a reduced number of transducers [13]. A pilot study of a wrist orthosis controlled by FMG is shown in [14], whereas another study [15] integrates FMG and sEMG to operate a multi-grasp bionic prosthesis. However, apart from the preliminary results presented in [16], there is no experimental demonstration of a soft robotic glove fully activated by FMG stimulus.
This paper proposes a glove-like orthosis integrated with FMG control through a microcontroller unit. Optical fiber transducers measure the forearm muscle contractions to retrieve user intentions. A tendon mechanism comprising Bowden cables and servo motors carries out the robotic glove grip and release motions. Furthermore, a functional electrical stimulation (FES) device assists the hand movements in association with the mechanical support, preventing muscle atrophy. Notably, the integrated approach allows for coupling and exchanging subsystems to simplify and optimize the final project. Thus, each part undergoes an individual design before integrating into the complete system, while supplementary moduli are admissible for granting new functionalities to the robotic glove.
The FMG provides a straightforward approach to detect user intentions based on proportional force levels rather than complex machine learning classifiers. Moreover, the hybrid strategy involving servo motors and FES assists both the manipulation task and the rehabilitation of upper-limb muscles; this is possible due to the immunity of the optical fiber FMG sensor to the FES electrical interference. To the best of our knowledge, this is the first report of a robot glove orthosis integrating tendon-driven mechanism, FMG, and FES subsystems.

2. System Design

2.1. Project Overview

Figure 1 shows the FMG-based orthosis comprising five subsystems: (i) robotic glove, (ii) force myography sensor; (iii) grip force feedback; (iv) functional electrical stimulation; (v) microcontroller unit (MCU). Optical fiber force transducers attached to the forearm detect the hand open or grasp intentions. The acquired FMG signals are processed in an embedded MCU, activating the servo motors and tendon-driven mechanisms responsible for controlling the underactuated glove. During the hybrid operation mode, the FES module collaboratively supports the orthosis and promotes muscle contraction. Lastly, the force-sensing resistors (FSRs) installed on the index fingertip provides grip force feedback during manipulation tasks.

2.2. Robotic Glove

The robotic glove assists the user’s hand during the grip/release movements for manipulating objects in typical tasks. This subsystem receives an activation command from the MCU based on the FMG and FSR sensors inputs, starting or halting the servo motors responsible for moving the hand joints.
The orthosis employs an ordinary cycling glove made of polyester and neoprene as the substrate, proving softness, comfort, and durability. Most activities of daily living (ADL) involve the thumb, index, and middle fingers [17]. Thus, instead of driving each digit separately, a differential mechanism composing a pair of servo motors (Dynamixel AX-12A+ Smart Servo) is used to provide a stable grip and optimize the required number of actuators: one motor is responsible for the thumb while the other simultaneously controls the index and finger movements.
The system has a slack prevention mechanism [18] to apply preload to the tendons, avoiding cable derailment, hamper discomfort, and finger injuries. Figure 2 depicts the cable winder with a non-back-drivable device fabricated by 3D printing. When the spool rotates, it winds or releases the tendon for hand joints flexion or extension, respectively. Spool and feeder rollers are coupled by a pair of spur gears, though each feeder roller can rotate freely only in one direction due to the assembly with the one-way clutches between the shaft and the feeder rollers. As the spool rotates, the one-way clutch blocks the feeder roller and causes it to drag the tendon out of the mechanism. The idler rollers can rotate freely in any direction, but they press the tendons on the feeder rollers, thus providing enough friction for pulling out the tendons. The diameters of the spool and feeder rollers and the transmission between them yield a linear velocity on the feeder rollers’ surface higher than the tendon unwinding speed on the spool. This design ensures a drag force enough to pull the tendons out when they are released.
Regarding the tendons, a Bowden cable system was employed to keep heavy components such as motors as far away as possible from the hand and forearm, thus minimizing the physical effort from the user. It uses a ~5 mm diameter, galvanized, flexible spiral wire sheath as housing, and a flex-rite bead string wire (21 strands nylon-coated stainless steel micro-wire, 0.5 mm, Bead Smith) as the cable. Preliminary tests indicated that it adjusted to the spool surface in the cable winder, presenting no slips or slacks.
Figure 3 shows the robotic glove assembly. One end of the sheaths is fixed to the actuator side to support the Bowden cables, whereas the other anchors to the 3D-printed armband support adapted to the shape of the hand. This armband is also necessary to prevent the sheaths from moving toward the fingers when the motors transmit force to the tendons. Stiff Velcro straps fasten the armband around the hand.

2.3. Optical Fiber FMG Sensor

This subsystem detects the hand open/close intentions required to switch the state machine in the MCU and drive the robotic glove. An optoelectronic interrogator unit sends the raw FMG signals to the microcontroller. Then, the MCU proceeds with data reduction and thresholding operations.
Force myography is the mechanical counterpart of the sEMG, in which the hand movements and intentions emerge from the radial pressures exerted by the forearm muscles. Once the FMG signals comprise static force levels instead of spiky dynamic waveforms, this approach demands simplified data processing and is suitable for proportional gesture control [13].
Typical FMG systems use arrays of FSR to assess muscle activity. However, devices such as sEMG electrodes, piezoresistive, and capacitive transducers are sensitive to variations in the skin impedance caused by sweat, for example [13,19]. Moreover, electronic sensors are vulnerable to electromagnetic interference, as occurs in the presence of FES stimuli [20]. Therefore, this project applied optical fiber sensors to measure FMG signals due to their lightweight, high sensitivity, and immunity to electromagnetic noise [21,22].
Figure 4a depicts interrogation setup [23]. LED sources (HFBR-0400, Agilent Technologies, 820 nm wavelength) excite the silica step-index multimode fibers (2 m long, 62.5/125 µm core/cladding diameters). Micro-bending transducers connected to fiber sections modulate the transmitted light in response to the applied forces. Photodetectors (HFBR-24 × 6, Agilent Technologies, Santa Clara, CA, USA) measure the intensity and deliver the electrical output to the amplification and filtering stage.
The force transducer, shown in Figure 4b,c, has L = 60 mm length (60 × 15 mm2 sensitive area) and a corrugated structure with Λ = 10 mm periodicity to mechanically deform the fiber. It causes the core-guided light modes to couple with radiation modes, yielding optical losses proportional to the input displacement or force [24]. The transducer is 3D-printed using acrylonitrile butadiene styrene (ABS) filament. Velcro straps comfortably secure the device to the forearm. The robotic glove application requires a pair of force sensors to monitor the flexor digitorum superficialis (FDS) and the extensor digitorum communis (EDC) muscles. This configuration allows for detecting grasp and release intentions, respectively; the placement was decided by palpation, according to previous studies [21,22]. Nevertheless, the transducers must remain away from the wrist to avoid overlap with the glove and the tendon mechanism.
Figure 4d depicts the calibration curve obtained by stimulating the transducer with a micrometric stage of 0.1 mm resolution; each data point indicates the average of ten measurements. A linear response range occurs for the 0.1 to 0.5 mm range, yielding ~5.4 V/mm sensitivity. Indeed, the preload applied during the transducer fixation on the forearm compensates the low sensitivity range (<0.1 mm). It is worth noting that the output voltage increases with the applied displacement because the photodetector signal passes an inverting gain stage.
As regards the FMG sensor reproducibility, a previous experiment for classifying nine postures yielded an average accuracy of 99.7% regarding 6 volunteers and 10 repetitions. Moreover, the overall accuracies for the hand open and close gestures are 98.9% and 100%, respectively [21]. Nevertheless, it is crucial to calibrate the sensor for each individual, as their particular physiological characteristics may affect the FMG response.

2.4. Grip Force Feedback

This subsystem detects the contact force between the user’s hand and manipulated object, producing a proportional signal to trigger the MCU state machine, subsequently halting the servo motor and the FES hand close stimuli.
Monitoring the grip force is crucial to prevent the glove joint from unintentionally close and avoid injury to the user or damage to the system. Instead of assessing the motor currents through sensors or state observers, this research uses an FSR (Interlink Electronics, round sensing area of ~15 mm diameter, dynamic range of 0.1 to 104 N) on the index fingertip to detect contact with the manipulated object. The transducer is attached nearer the distal phalanx using hot glue, as shown in Figure 3c.

2.5. Functional Electrical Stimulation

The FES subsystem waits for the MCU command to assist the hand in close/open movements through electrical stimulation. The microcontroller also disables this module according to the FSR input or the servo motor time-out event.
Frequently employed in rehabilitation, FES is a technique for inducing muscle contractions through electrical currents that excite the axons of the motor neurons innervating the target muscles [25]. The electrical stimulation pulses also evoke an electrophysiological response (M-waves), closely related to muscle fiber recruitment detectable with sEMG. Therefore, applying electromyography to control neuroprostheses demands several processing steps to separate the FES contribution from recorded sEMG waveforms [20]; an optical fiber FMG sensor can perform disregarding the electrical stimulation because it is immune to electromagnetic interference.
The two-channel FES subsystem is a shield built in an Arduino Uno microcontroller. The device generates signals up to 20 mA and 50 Hz with several waveform modulation parameters to produce pulsed or continuous output. Then, active rectangular electrodes (Carci Trode, 3 × 5 cm2 area) placed on the motor points of the FDS and EDC muscles apply the electrical stimulation to flex and extend the finger joints, respectively [26].

2.6. Microcontroller Unit and System Integration

The microcontroller unit reads the data from all sensors and drives the actuators based on a finite-state machine, integrating the aforementioned subsystems. This module also performs data processing and synchronization tasks. Upgrades are possible by reprogramming or coupling new peripherals to the MCU.
A high-performance MCU based on the ARM Cortex-M4F architecture with parallel processing and real-time operation capabilities carries out the FMG signals acquisition and robotic glove control. The system uses the CMSIS-RTOS RTX 5 application programming interface (API) to implement concurrent threads for sampling and control the actuators.
The signal conditioning stage scales the FMG output voltages within the 0 to 3.3 V range. These signals are acquired by the sampling thread using an on-chip analog-to-digital converter (ADC) with a 1 kHz rate. After reducing the data by evaluating the RMS values for a 10 ms time window, the output FMG signals proceed to the posture identification routine. For the sake of simplicity, the hand open and grasp intentions are distinguished with a single threshold method; the responses of FDS and EDC muscles are complementary regarding the studied set of motions. A calibration decides the threshold values necessary for detecting user intent with minimum effort. Moreover, this thread also acquires and processes the FSR signal responsible for tracking the grasp event.
Figure 5 describes the control strategy. Initially, in state S0, the hand is fully opened while the system awaits the activation signal. As the user tries to perform a grasp, the transducer detects the FDS muscle stimuli and modulates the light intensity. If the FMG RMS value F is greater than the predefined threshold F T , the system triggers the servo motors for flexing the glove joints (state S1). This movement is carried out during a time T until the FSR voltage F S R exceeds a set-point S P = α F max   , where α is a proportional gain and F max is the maximum grip force defined by calibration, indicating that the object was held (S2). Alternatively, the motor can be turned off when the user performs a hand open intention, generating an FMG signal through EDC muscle stimulus, i.e., the corresponding FMG RMS value E is greater than the threshold E T . Once the posture stabilizes, the system waits for another E > E T event that changes the servo motor direction to open the hand (S3). This process is governed by a timer t so that the glove joints are extended until t = T . Finally, the system returns to the S0 and stands by the next interaction. Regarding a hybrid control strategy, the MCU enables the FES simultaneously in S1 and S3 to assist the tendon-driven systems, deactivating the electrical stimulation as the glove reaches the grasp or idle conditions, S2 and S0, respectively.

3. Experimental Validation

3.1. Measurement Procedure

To evaluate the performance of the glove-based orthosis on practical ADL tasks, objects manipulation experiments considered either the tendon-driven assistance or the hybrid approach with FES actuation. The tests addressed three main questions: (i) Can the robotic glove effectively manipulate different objects? (ii) Is the FMG system capable of trigger the state machine and control the glove grasp based on the user intentions? (iii) Is the FES feasible for simultaneous application with the tendon mechanism and the FMG sensor?
The experiments involved healthy volunteers (34 ± 2 years old) without hand disabilities, following the Ethical Committee recommendations (CAAE 17283319.7.0000.5404). After sitting comfortably with the forearm suspended, the individuals perform sequences of grasping, holding, and releasing objects by triggering the robotic glove with their motion intents. The manipulated items include stick glue (2 cm diameter), adhesive tape roll (5 cm), bottle (8 cm), and a cylindrical toy (3.5 cm), as shown in Figure 6. These tasks feature representative examples of precision grips [27], corresponding to frequent cases in ADL [17,28].
An FMG sensor calibration preceded the experiments to set up the threshold levels F T and E T . With the transducers attached to the belly of FDS and EDC muscles through Velcro straps, the individual performs hand close and open movements. Figure 7a,b shows examples of FMG waveforms acquired in this calibration session. Based on the average peak values, F T and E T are empirically set to ~2 V to avoid misdetection due to unintentional contraction of muscles. To obtain the FSR set-point S P , a volunteer wearing the glove grasps and releases the cylindrical bottle for measuring the average grip force F max . As observed in Figure 7c, the FSR output voltage related to F max is ~0.6 V. Choosing α = 0.9 empirically yields a set-point that is S P = 0.54 V. The servo motor and FES assistances are disabled in these tests to assess the user’s natural reactions. Nevertheless, one may adjust the threshold values to tailor the grasp sensitivity after relocating the glove and transducers or changing the user.
Subsequently, the individuals performed three trials for each grasp for familiarization with the apparatus and adjusting the threshold values, followed by four repetitions to assess the system response. The dataset collected in each experiment comprised the FMG flexion and extension signals, the FSR grip force, and the servo motor commands and FES stimuli for closing and opening the hand. The measured data were further correlated to the state machine variables to verify the overall performance.

3.2. Evaluation of Grasp Events

Figure 8 shows examples of signals acquired in manipulation tasks. The waveforms are scaled and shifted for the sake of visualization; the maximum voltage values are 3 V for F and E , and 1 V for F S R . The system remains idle (S0) until it recognizes a flexion intention by the FMG sensor. This event activates the servo motor to close the glove joints (S1) while the user approaches the object to grasp it. After establishing the physical contact, the servo motor stops when the FSR force exceeds the set-point to maintain the object secured with the minimum required effort. State S2 maintains the grip until the FMG sensor detects the extension stimulus, causing the hand to open (S3) and releasing the object; subsequently, returning to the initial state. Notably, the duration of states S1 and S3 depends on the size of objects; for example, the contact occurs later for the stick glue (2 cm) and earlier for the bottle (8 cm), Figure 8a,c, respectively, due to the different diameters.
Once the system depends on the FSR signals to interrupt the servo motors, for safety reasons, it is necessary to prevent the glove from closing in case of false-negative events, for example, due to the insufficient contact between FSR and the object surface. Moreover, the system must limit the fingers’ extension range during the hand opening to avoid injuries. In this sense, a software condition records a standard time—defined as the time spent to close the hand from an open palm condition until the complete flexion—used as a restriction for activating the glove. It also guarantees that the hand returns to its resting position so that states S1 and S3 have the same duration.
Another aspect is the interlock provided by the finite states machine. For example, in Figure 8b, although the FMG sensor recognizes unintentional flexion intent at ~9 s, it does not activate the glove, as the system is not responsible for such stimulus at S2. Moreover, despite the grasp accomplishment, a problem occurred during the static force detection (~9 s), probably due to the object deformation. This fact explains the shorter grip event for the adhesive tape than the stick despite their diameters, 5 and 2 cm, respectively. Fortunately, the FSR impulse is sufficient to trigger the state S1 and disable the motor, avoiding any damage to the hardware and the user.
Regarding the hybrid control strategy, Figure 9 presents manipulation tasks encompassing a combination of FES and servo motors. The MCU enables electrical stimulation in states S1 and S3 according to the required group of muscles, assisting the flexion or extension of glove joints, respectively, with minimal muscular fatigue. Furthermore, as observed during S1 and S3, involuntary muscle stimuli induced by the FES do not affect the FMG response for the proposed configuration. The hybrid strategy behaved satisfactorily for all grasps, as observed in Figure 9.
Recalling the three addressed questions, (i) the FSR attached to the fingertip detects the physical contact and stops the actuators, holding the object regardless of its size. Moreover, (ii) the FMG sensor correctly retrieves the hand close and open intentions, activating the servo motors as the output signals exceed the threshold values. Lastly, (iii) the integrated system works adequately for both operation modes, i.e., using the tendon mechanism and the FES hybrid approach.
As regards the system response time, the assessment of the IMU input/output signals by a data logger (1 kHz sampling rate) revealed delay times of ~20 ms for the FMG fiber sensor and 20–50 ms for the FSR; the latter depends on the object size and mechanical characteristics. Given the servo motor typical response of ≤0.02 ms, a conservative estimation of the overall system delay is ~50 ms, suitable for assistive devices [10].

4. Discussion

The glove-like orthosis driven by optical fiber FMG sensor presented reliable results for both mechanical and FES-assisted actuation, allowing the manipulation of different objects. Remarkably, the integrated project opens new perspectives to upgrade or exchange parts, improving the system functionalities. For example, one may replace the servo motor actuation with a pneumatic or hydraulic mechanism oriented to soft robotics. The control strategy is also eligible for dexterous bionic prosthetics and robotic manipulators [29], as well as for implementing human–computer interfaces in virtual-reality applications [30].
As regards the manipulation task, the FSR provided a straightforward way to detect the contact force, triggering the state machine to sustain the pose and secure the object. However, as the FSR has a rigid structure, its response may vary with the glove substrate characteristics, as well as with the object shape and texture [31]. An alternative comprises estimating the grip forces from the servo motor current to immediately interrupt the mechanical or FES excitation in response to a torque peak [32]. Ultimately, one may attach a tactile sensor matrix to the glove palm. Such an approach is convenient to optimize the control accordingly to the grasp type and the geometrical and mechanical properties of the object, also compensating slip [33].
Regarding the intentions, an alternative is using complementary transducers to classify additional gestures and extend the repertory of possible grasps. Similar approaches are possible through sEMG, FSR-based FMG, and even BCI technologies, though the FES-hybrid mode still requires optical fiber transducers due to the immunity to electromagnetic interference. Indeed, it implies new cases to the finite states machine, which eventually increases the complexity for implementation and application. For instance, FMG sensors are typically faster than sEMG and BCI because the force signatures for a posture emerge as static intensity level changes rather than spiky waveform patterns, avoiding the post-processing for features extraction [13]. Moreover, the proposed system simplifies the classification to a two-channel threshold analysis, yielding a negligible input lag to the robotic glove. Nevertheless, multi-modal alternatives such as computer vision classification are available for simplifying multi-grasp systems [34].
For comparison purposes, Table 1 summarizes representative examples of current glove-like orthoses. Albeit fluidic systems enhance the grip forces [6,10], the tendon mechanism actuated through servo drives provides accurate position control and straightforward integration with the MCU. Moreover, using a commercial glove instead of a dedicated one implies an inexpensive, generalized design. Concerning the gestures input, the FMG response is proportional to the forearm muscle stimuli, thus easier to decode than the sEMG and BCI approaches [7,10]; in addition, capturing the hand intentions is more intuitive than using manual switches [6,12]. Ultimately, the immunity to electromagnetic interference exhibited by optical fiber sensors allows for implementing a unique FES hybrid strategy to assist the tendon-driven mechanism; this capability does not feature in the available robotic gloves.
Despite the integrated nature, the proposed robotic glove has drawbacks concerning the rigid mechanical structure, demanding developments toward a portable and soft design [6,12]. Regarding portability, a compact control box should accommodate the motors, microcontroller unit, and battery [6,8,11]. The proposed system is still bulky due to several connect moduli such as servo motor, FES, optical interrogator, and the MCU. This aspect requires hardware miniaturization into a compact unit. Moreover, all the examples restrict the range of possible grasps, usually comprising a single-precision grip through the activation of the thumb, index, and medium fingers, making them unable a priori for multi-gesture control. Lastly, complementary studies must consider individuals with severe motion impairments, as this certainly affects the magnitude of FMG signals and require improvements to the hybrid control architecture.
As regards the quantitative metrics, Table 2 summarizes the recommended characteristics of robotic glove devices [6]. The glove weight affects the user’s comfort and natural movements. Projects based on commercial garments attend the specifications; nevertheless, dedicated soft polymer gloves allow considerable weight reduction [11]. Evidently, the control box must be portable and carried with the user, increasing the overall weight. Concerning the number of degrees of freedom (DOF) per finger, although the proposed orthosis has nominally two DOFs, the distal interphalangeal joint moves solidary with the proximal interphalangeal joint by a natural hand constraint. Regarding the autonomy, further tests must comprise the system connected to a battery instead of the utility power, especially regarding the impact of the FES on consumption. Notably, current devices achieve entire-day autonomy through efficient power management [8]. Lastly, the developed system achieves a high control loop frequency, exceeding the expected 10 Hz.
In addition to these metrics, other authors assess the maximum grip forces and the range of finger joints as quantitative standards [6,8,11]. Albeit informative to the robotic glove design, it is hard to conduct an unbiased comparative analysis once these parameters depend on the performed task, grasp type, and manipulated object characteristics. Some works proposed systematic studies to evaluate the grasp accuracy [10]; however, there is no strict protocol for comparing the results. Indeed, most glove-based systems accomplish hand close/open movements disregarding the grip type; the fingers adapt to the object due to tactile feedback or manual switches [6,12].

5. Conclusions

We presented a glove-like orthosis integrated with an optical fiber sensor. Aside from the current devices relying on underactuated mechanisms triggered with sEMG machine learning classifiers or pneumatic soft-actuators controlled by switches, the synergy between an FMG-based posture detection system and a robotic glove prototype creates an assist-as-need device for individuals with upper-limb disabilities. Additional FES stimulation apparatus is available for operation through a tendon-driven system, electrical stimulation, or a merge of both. The optical fiber sensor ensures no competition with FES due to its immunity to electromagnetic interference. Furthermore, upgrading the system functionalities is feasible by connecting extra units, including sEMG and vision-based intention detection modules [34].
Despite the promising results, the FMG detection requires some residual level of muscle activity and may present unsatisfactory performance for individuals with severe impairments; a possible solution comprises moving the sensor to another region with greater voluntary residual control. Moreover, the instrumented glove can be slightly hard to don/doff onto the hands. Hence, future developments must focus redesign the glove to facilitate these tasks and change the fabric into a more hygienic material. Integration with other user interfaces such as sEMG, BCI, and computer vision is also under investigation to improve intentions assessment regardless of the user condition. Furthermore, improvements in software and hardware will provide an embedded solution to ease portability and enable tests with potential users.

Author Contributions

Conceptualization, A.R.N., J.F. and E.F.; methodology, A.R.N., J.F. and E.F.; software, J.F.; W.H.A.d.S., M.K.G. and M.C.F.d.C.; validation, A.R.N., J.F., W.H.A.d.S. and M.K.G.; formal analysis, A.R.N., J.F. and E.F.; investigation, A.R.N. and E.F.; resources, A.R.N., E.F. and E.R.; data curation, A.R.N., J.F. and E.F.; writing—original draft preparation, A.R.N., J.F. and E.F.; writing—review and editing, A.R.N., J.F., W.H.A.d.S., M.K.G., M.C.F.d.C., E.F. and E.R.; visualization, A.R.N., J.F. and E.F.; supervision, E.F. and E.R.; project administration, E.F. and E.R.; funding acquisition, E.F. and E.R. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by Sao Paulo Research Foundation (FAPESP), Grant Number 2017/25666-2, by FAPESP CEPID Brainn, Grant Number 2013/07559-3, by Conselho Nacional de Desenvolvimento Cientifico e Tecnologico (CNPq), and by Coordenacao de Pessoal e Nivel Superior (CAPES), Finance Code 001.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Ethics Committee of the University of Campinas (protocol code CAAE 17283319.7.0000.5404, date of approval 9 September 2019).

Informed Consent Statement

Informed consent was obtained from all subjects involved in this study.

Data Availability Statement

Datasets and original images are available from the corresponding author on request.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Chu, C.Y.; Patterson, R.M. Soft robotic devices for hand rehabilitation and assistance: A narrative review. J. Neuroeng. Rehabil. 2018, 15, 1–14. [Google Scholar] [CrossRef]
  2. Shahid, T.; Gouwanda, D.; Nurzaman, S.G.; Gopalai, A.A. Moving toward soft robotics: A decade review of the design of hand exoskeletons. Biomimetics 2018, 3, 17. [Google Scholar] [CrossRef] [PubMed]
  3. Majidi, C. Soft robotics: A perspective—Current trends and prospects for the future. Soft Robot. 2014, 1, 5–11. [Google Scholar] [CrossRef]
  4. Rus, D.; Tolley, M. Design, fabrication and control of soft robotics. Nature 2015, 521, 467–475. [Google Scholar] [CrossRef]
  5. Oliver, M.; Barnes, C. Back to the future: The World Report on Disability. Disabil. Soc. 2012, 27, 575–579. [Google Scholar] [CrossRef]
  6. Polygerinos, P.; Wang, Z.; Galloway, K.C.; Wood, R.J.; Walsh, C.J. Soft robotic glove for combined assistance and at-home rehabilitation. Robot. Auton. Syst. 2015, 73, 135–143. [Google Scholar] [CrossRef]
  7. Cao, H.; Zhang, D. Soft robotic glove with integrated sEMG sensing for disabled people with hand paralysis. In Proceedings of the 2016 IEEE International Conference on Robotics and Biomimetics (ROBIO), Qingdao, China, 3–7 December 2016; IEEE: Bellingham, WA, USA, 2017; pp. 714–718. [Google Scholar] [CrossRef]
  8. Gerez, L.; Chen, J.; Liarokapis, M. On the development of adaptive, tendon-driven, wearable exo-gloves for grasping capabilities enhancement. IEEE Robot. Autom. Let. 2019, 4, 422–429. [Google Scholar] [CrossRef]
  9. Buch, E.; Weber, C.; Cohen, L.G.; Braun, C.; Dimyan, M.A.; Ard, T.; Mellinger, J.; Caria, A.; Soekadar, S.; Fourkas, A.; et al. Think to move: A neuromagnetic brain-computer interface (BCI) system for chronic stroke. Stroke 2008, 39, 910–917. [Google Scholar] [CrossRef]
  10. McConnell, A.C.; Vallejo, M.; Moioli, R.C.; Brasil, F.L.; Secciani, N.; Nemitz, M.P.; Riquart, C.P.; Corne, D.W.; Vargas, P.A.; Stokes, A.A. SOPHIA: Soft orthotic physiotherapy hand interactive aid. Front. Mech. Eng. 2017, 3, 3. [Google Scholar] [CrossRef]
  11. In, H.; Kang, B.B.; Sin, M.; Cho, K.-J. Exo-glove: A wearable robot for the hand with a soft tendon routing system. IEEE Robot. Autom. Mag. 2015, 22, 97–105. [Google Scholar] [CrossRef]
  12. Kang, B.B.; Lee, H.; In, H.; Jeong, U.; Chung, J.; Cho, K.-J. Development of a polymer-based tendon-driven wearable robotic hand. In Proceedings of the 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden, 16–21 May 2016; IEEE: Bellingham, WA, USA, 2016; pp. 3750–3755. [Google Scholar] [CrossRef]
  13. Xiao, Z.G.; Menon, C. A review of force myography research and development. Sensors 2019, 19, 4557. [Google Scholar] [CrossRef] [PubMed]
  14. Sangha, S.; Elnady, A.M.; Menon, C. A compact robotic orthosis for wrist assistance. In Proceedings of the 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob), Singapore, 28 July 2016; IEEE: Bellingham, WA, USA, 2016; pp. 1080–1085. [Google Scholar] [CrossRef]
  15. Ahmadizadeh, C.; Merhi, L.-K.; Pousett, B.; Sangha, S.; Menon, C. Toward intuitive prosthetic control: Solving common issues using force myography, surface electromyography, and pattern recognition in a pilot case study. IEEE Robot. Autom. Mag. 2017, 24, 102–111. [Google Scholar] [CrossRef]
  16. Fajajrdo, J.; Ribas Neto, A.; Silva, W.; Gomes, M.; Fujiwara, E.; Rohmer, E. A wearable robotic glove based on optical FMG driven controller. In Proceedings of the 2019 IEEE 4th International Conference on Advanced Robotics and Mechatronics (ICARM), Toyonaka, Japan, 3–5 July 2019; IEEE: Bellingham, WA, USA, 2019; pp. 81–86. [Google Scholar] [CrossRef]
  17. Bullock, I.M.; Zheng, J.Z.; De La Rosa, S.; Guertler, C.; Dollar, A.M. Grasp frequency and usage in daily household and machine shop tasks. IEEE Trans. Haptics 2013, 6, 296–308. [Google Scholar] [CrossRef] [PubMed]
  18. In, H.; Lee, H.; Jeong, U.; Kang, B.B.; Cho, K.-J. Feasibility study of a slack enabling actuator for actuating tendon-driven soft wearable robot without pretension. In Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA, 26–30 May 2015; IEEE: Bellingham, WA, USA, 2015; pp. 1229–1234. [Google Scholar] [CrossRef]
  19. Tiwana, M.I.; Redmond, S.J.; Lovell, N.H. A review of tactile sensing technologies with applications in biomedical engineering. Sens. Actuat. A-Phys. 2012, 179, 17–31. [Google Scholar] [CrossRef]
  20. Toledo-Peral, C.L.; Gutiérrez-Martínez, J.; Mercado-Gutiérrez, J.A.; Martín-Vignon-Whaley, A.I.; Vera-Hernández, A.; Leija-Salas, L. sEMG signal acquisition strategy towards hand FES control. J. Healthc. Eng. 2018, 2018, 2350834. [Google Scholar] [CrossRef]
  21. Fujiwara, E.; Suzuki, C.K. Optical fiber force myography sensor for identification of hand postures. J. Sens. 2018, 2018, 8940373. [Google Scholar] [CrossRef]
  22. Fujiwara, E.; Wu, Y.T.; Santos, M.F.M.; Schenkel, E.A.; Suzuki, C.K. Optical fiber specklegram sensor for measurement of force myography signals. IEEE Sens. J. 2017, 17, 951–958. [Google Scholar] [CrossRef]
  23. Wu, Y.T.; Gomes, M.K.; da Silva, W.H.A.; Lazari, P.M.; Fujiwara, E. Integrated optical fiber force myography sensor as pervasive predictor of hand postures. Biomed. Eng. Comput. Biol. 2020, 11, 1179597220912825. [Google Scholar] [CrossRef]
  24. Wang, W.; Yiu, H.H.P.; Li, W.J.; Roy, V.A.L. The principle and architectures of optical stress sensors and the progress on the development of microbend optical sensors. Adv. Opt. Mater. 2021, 9, 2001693. [Google Scholar] [CrossRef]
  25. De Marchis, C.; Monteiro, T.S.; Simon-Martinez, C.; Conforto, S.; Gharabaghi, A. Multi-contact functional electrical stimulation for hand opening: Electrophysiologically driven identification of the optical stimulation site. J. Neuroeng. Rehabil. 2016, 13, 22. [Google Scholar] [CrossRef]
  26. Ribas Neto, A.; Fajardo, J.; Ferman, V.; Fujiwara, E.; Rohmer, E. A hybrid control strategy for tendon-actuated robotic glove and functional electrical stimulation—A preliminary study. In Proceedings of the 2019 IEEE 4th International Conference on Advanced Robotics and Mechatronics (ICARM), Toyonaka, Japan, 3–5 July 2019; IEEE: Bellingham, WA, USA, 2019; pp. 244–249. [Google Scholar] [CrossRef]
  27. Cutkosky, M.R. On grasp choice, grasp models, and the design of hands for manufacturing tasks. IEEE Trans. Robotic. Autom. 1989, 5, 269–279. [Google Scholar] [CrossRef]
  28. Vergara, M.; Sancho-Bru, J.L.; Garcia-Ibáñez, V.; Pérez-González, A. An introductory study of common grasps used by adults during performance of activities of daily living. J. Hand. Ther. 2014, 27, 225–234. [Google Scholar] [CrossRef]
  29. Fajardo, J.; Ferman, V.; Cardona, D.; Maldonado, G.; Lemus, A.; Rohmer, E. Galileo Hand: An anthropomorphic and affordable upper-limb prosthetics. IEEE Access 2020, 8, 81365–81377. [Google Scholar] [CrossRef]
  30. Fujiwara, E.; Wu, Y.T.; Suzuki, C.K.; de Andrade, D.T.G.; Ribas Neto, A.; Rohmer, E. Optical fiber forcer myography sensor for applications in prosthetic hand control. In Proceedings of the 2018 IEEE 15th International Workshop on Advanced Motion Control (AMC), Tokyo, Japan, 9–11 March 2018; IEEE: Bellingham, WA, USA, 2018; pp. 342–347. [Google Scholar] [CrossRef]
  31. Chi, C.; Sun, X.; Xue, N.; Li, T.; Liu, C. Recent progress in technologies for tactile sensors. Sensors 2018, 18, 948. [Google Scholar] [CrossRef] [PubMed]
  32. Esposito, D.; Savino, S.; Cosenza, C.; Andreozzi, E.; Gargiulo, G.D.; Polley, C.; Cesarelli, G.; D’Addio, G.; Bifulco, P. Evaluation of grip force and energy efficiency of the “Federica” Hand. Machines 2021, 9, 25. [Google Scholar] [CrossRef]
  33. Zhang, T.; Jiang, L.; Liu, H. Design and functional evaluation of a dexterous myoelectric hand prosthesis with biomimetic tactile sensor. IEEE Trans. Neur. Sys. Reh. 2018, 26, 1391–1399. [Google Scholar] [CrossRef]
  34. Fajardo, J.; Ferman, V.; Muñoz, A.; Andrade, D.; Ribas Neto, A.; Rohmer, E. User-prosthesis interface for upper limb prosthesis based on object classification. In Proceedings of the Latin American Robotic Symposium/Brazilian Robotics Symposium (LARS/SBR), Joao Pessoa, Brazil, 6–10 November 2018; IEEE: Bellingham, WA, USA, 2018; pp. 390–395. [Google Scholar] [CrossRef]
Figure 1. Design of the integrated FMG-driven robotic glove orthosis.
Figure 1. Design of the integrated FMG-driven robotic glove orthosis.
Automation 02 00012 g001
Figure 2. Cable winder with non-back-drivable mechanism: (a) partial front and (b) back views. 1: idler rollers; 2: feeder rollers; 3: one-way clutch; 4: spur gears; 5: spool; 6: tendons.
Figure 2. Cable winder with non-back-drivable mechanism: (a) partial front and (b) back views. 1: idler rollers; 2: feeder rollers; 3: one-way clutch; 4: spur gears; 5: spool; 6: tendons.
Automation 02 00012 g002
Figure 3. Robotic glove: (a) system parts; (b) dorsal and (c) palmar sides.
Figure 3. Robotic glove: (a) system parts; (b) dorsal and (c) palmar sides.
Automation 02 00012 g003
Figure 4. Optical fiber FMG sensor: (a) interrogation system; (b) force transducer and (c) photography of the fabricated device; (d) calibration curve, the red line is a quadratic curve fitting.
Figure 4. Optical fiber FMG sensor: (a) interrogation system; (b) force transducer and (c) photography of the fabricated device; (d) calibration curve, the red line is a quadratic curve fitting.
Automation 02 00012 g004
Figure 5. Main loop flowchart for the robotic glove control strategy.
Figure 5. Main loop flowchart for the robotic glove control strategy.
Automation 02 00012 g005
Figure 6. Objects used in the experiments: (a) glue stick, (b) adhesive tape roll, (c) bottle, and (d) toy.
Figure 6. Objects used in the experiments: (a) glue stick, (b) adhesive tape roll, (c) bottle, and (d) toy.
Automation 02 00012 g006
Figure 7. Examples of sensor waveforms for adjusting the threshold levels: (a) FMG sensor response to hand close intention; (b) FMG sensor response to hand open intention; (c) FSR response to a cylindrical grasp. The horizontal lines indicate the thresholds.
Figure 7. Examples of sensor waveforms for adjusting the threshold levels: (a) FMG sensor response to hand close intention; (b) FMG sensor response to hand open intention; (c) FSR response to a cylindrical grasp. The horizontal lines indicate the thresholds.
Automation 02 00012 g007
Figure 8. Grasping objects with the glove-like orthosis: (a) glue stick, (b) adhesive tape roll, (c) bottle, and (d) toy. F: FMG flexion stimulus; MC: motor command to close the hand; FSR: grip contact force; E: FMG extension stimulus; MO: motor command to open the hand. The measured signals are normalized and scaled for the sake of visualization. The gray rectangles indicate the system states.
Figure 8. Grasping objects with the glove-like orthosis: (a) glue stick, (b) adhesive tape roll, (c) bottle, and (d) toy. F: FMG flexion stimulus; MC: motor command to close the hand; FSR: grip contact force; E: FMG extension stimulus; MO: motor command to open the hand. The measured signals are normalized and scaled for the sake of visualization. The gray rectangles indicate the system states.
Automation 02 00012 g008
Figure 9. Grasping objects with a hybrid approach: (a) glue stick, (b) adhesive tape roll, (c) bottle, and (d) toy. F: FMG flexion stimulus; ESC, MC: FES and motor commands to close the hand; FSR: grip contact force; E: FMG extension stimulus; ESO, MO: FES and motor commands to open the hand. The measured signals are normalized and scaled for the sake of visualization. The gray rectangles indicate the system states.
Figure 9. Grasping objects with a hybrid approach: (a) glue stick, (b) adhesive tape roll, (c) bottle, and (d) toy. F: FMG flexion stimulus; ESC, MC: FES and motor commands to close the hand; FSR: grip contact force; E: FMG extension stimulus; ESO, MO: FES and motor commands to open the hand. The measured signals are normalized and scaled for the sake of visualization. The gray rectangles indicate the system states.
Automation 02 00012 g009
Table 1. Comparison of glove-like orthoses.
Table 1. Comparison of glove-like orthoses.
Ref.DescriptionProsCons
ThisTendon-driven glove, FMG controlModular, straightforward control, FES-assistanceBulky, rigid structure
[6]Hydraulic glove controlled by buttonsSoft structure, portableNonintuitive control; requires custom-made actuators
[7]Tendon-driven glove, sEMG controlPortable, low-costAmbiguities in the sEMG signals; susceptible to noise
[8]Tendon-driven glove, sEMG controlModular, portable, grasp force controlRigid structure; lacks force feedback
[10]Pneumatic glove, BCI controlSoft structure, modular, force feedbackBulk hardware, cost; low accuracy
[11]Tendon-driven glove controlled by flex sensorPortable, simple controlRequires wrist flexion/extension
[12]Tendon-drive glove controlled by analog switchSoft structure, portable, washableNonintuitive control
Table 2. Requirements of robotic gloves (adapted from [6]). NA: not available.
Table 2. Requirements of robotic gloves (adapted from [6]). NA: not available.
Ref.Glove Weight (kg)DOFs Per FingerAutonomy (h)Controller Frequency (Hz)
Rec.≤0.503≥2≥10
This0.502NA20
[6]0.2933.820
[7]NA2NA5
[8]0.55324NA
[10]NA2NA~10
[11]0.193NANA
[12]NA3NANA
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Back to TopTop