Conceptualization of a Sensory Feedback System in an Anthropomorphic Replacement Hand

Abstract

(1) Background: This paper presents a conceptual design for an anthropomorphic replacement hand made of silicone that integrates a sensory feedback system. In combination with a motorized orthosis, it allows performing movements and registering information on the flexion and the pressure of the fingers. (2) Methods: To create the replacement hand, a three-dimensional (3D) scanner was used to scan the hand of the test person. With computer-aided design (CAD), a mold was created from the hand, then 3D-printed. Bending and force sensors were attached to the mold before silicone casting to implement the sensory feedback system. To achieve a functional and anthropomorphic appearance of the replacement hand, a material analysis was carried out. In two different test series, the properties of the used silicones were analyzed regarding their mechanical properties and the manufacturing process. (3) Results: Individual fingers and an entire hand with integrated sensors were realized, which demonstrated in several tests that sensory feedback in such an anthropomorphic replacement hand can be realized. Nevertheless, the choice of silicone material remains an open challenge, as there is a trade-off between the hardness of the material and the maximum mechanical force of the orthosis. (4) Conclusion: Apart from manufacturing-related issues, it is possible to cost-effectively create a personalized, anthropomorphic replacement hand, including sensory feedback, by using 3D scanning and 3D printing techniques.

1. Introduction

There are more and more active prostheses available that allow sophisticated grip types with varying grip strengths. However, the implementation of an appropriate feedback system is an open challenge concerning manufacturing-related topics, such as nanotechnologies, as well as medical aspects, such as invasive and non-invasive approaches [1,2,3]. Vibrotactile stimulation is often used to provide feedback to the patient in non-invasive techniques, as in [4,5,6]. In [7,8] or [9], visual techniques were used, whereby in the former two works, LEDs provide visual feedback concerning the gripping process, and in the latter work, the changed body scheme is simulated and visualized. Many other studies have shown that feedback can support the grasping process with a prosthesis during training or everyday use [10,11,12].

More recently, three-dimensional (3D) scanning and 3D printing have also been used in prosthesis construction, as it offers special possibilities for personalization [13,14].

Therefore, we present an approach to a personalized replacement hand with sensory feedback that can be realized with commercially available parts [15]. The signals acquired by the sensors can subsequently be processed by visual or vibrotactile feedback.

This paper is based on the prework of Baron et al., which dealt with 3D printing of an anthropomorphic and personalized replacement hand [16]:

In this project, the test person’s hand was scanned with a 3D scanner and processed in a computer-aided design (CAD) program. Then, the hand was 3D printed from the rubber material Agilus30. The printed replacement hand had to be flexible for usage in combination with a motorized orthosis (NeoMano by Neofect) for paralyzed patients, in which the thumb is rigid and only index and middle finger are motorized. However, the study showed that the selected material was too stiff to be flexed by the orthosis. Due to the limited availability of flexible 3D printable materials, other personalized replacement hand manufacturing methods were evaluated [17].

Additionally, the study confirmed that the manufacturing process of such a 3D printed hand was time-consuming and expensive due to the costs for the 3D printable rubber material. In light of the aforementioned prestudy, the following paper presents the possibility of obtaining a personalized replacement hand using a silicone casting method. In this process, a mold is constructed from the 3D scanned model of the hand. Next, the mold is 3D printed, and a two-component room temperature vulcanizing silicone is filled in. After curing is completed, the personalized replica of the hand can be demolded. Since the replacement hand is intended to be used with a motorized orthosis as a cost-effective and personalized prosthesis, several silicone materials with different mechanical properties are tested. The selection of materials focuses on ensuring that the prosthetic hand feels natural in terms of a body-like rigidity of the hand, rather than a skin-like feeling. The second objective of this work is to integrate a sensory feedback system into the replacement hand to provide information about grip strength, consistency of the object grabbed, and the degree of grasp. This information could be displayed in a further step, for example, color-coded or by text messages through an augmented reality system as shown in [18,19] or through vibrotactile feedback. In order to cast the sensors in the proper position, thin spacers are integrated into the mold to attach bending and pressure sensors and hold them at the intended position during the casting process. As the replacement hand is designed to be used with the aforementioned orthosis, two pressure sensors and one bending sensor are integrated into the index and middle finger. The bending sensor measures the degree of flexion of the entire finger. Two force sensors are added, one at the fingertip and one at the proximal end of the finger. These are the two points that are mainly affected by the object during the gripping process. Both sensors are placed directly under the skin surface.

2. Results

2.1. Finger Test Series for Sensor Attachment

An initial test series determined how the sensors could be attached to the mold and how the sensors would behave after curing. In

Table 1. Concept of sensor mounting. Finger II does not have any sensors and serves as a reference.

Finger	Measurement	Sensor Mounting
I	Flexion	Wire
II	-	-
III	Flexion	Bar
IV	Pressure	Bar
V	Pressure and Flexion	Bar

the test set is listed to reference the manufactured fingers in Figure 1. These are examined in more detail for their softness, sensor attachment options, and sensory behavior. The test results lead to suggestions for improvement, which are adopted for the final replacement hand. With a Shore hardness of 25 ShA, the material of the fingers does not feel any softer than the Agilus30 replacement hand in Baron et al. [16], although the latter has a Shore hardness of 30 ShA. It should be mentioned that fingers with integrated sensors become stiffer, demonstrated by a reference finger without sensors. Even if the sensors themselves are more flexible than a whole finger, combining sensor and finger results in a stiffer model, as the silicone core may not deform undisturbed as before. If the orthosis flexes these test fingers, a complete flexion of the fingers is not possible yet, as the elasticity is not sufficient. Therefore, silicone material with a much lower Shore hardness is needed. Upon analyzing the sensory activity during the influence of force, it was found that all sensors in the corresponding fingers were functional.

2.2. Test Series for Combining Silicones with Different Shore Hardnesses during Casting

Since the material of the fingers to include sensors had to be much softer than the rest of the hand to be flexed by the orthosis, a second test series was driven to investigate the behavior of the silicone during vulcanization and to address the question of whether it is possible to add a second silicone so that both materials connect but do not mix.

In Figure 2, it can be seen at the first finger (1) that the materials of this finger have been mixed. The hardness of both silicones is approximately averaged. In the remaining fingers (2–5), the fingers become more transparent from about the halfway point. These parts are marked in red. The firmness of the fingers is substantially softer in the upper half, while the other half is harder. The upper half is comparable to the sensorless reference fingers of the previous test series. The lower half is comparable to a sensorless finger made of 10 ShA silicone.

Both parts were adhered permanently to each other, which confirmed that the cross-linking during the curing process of two silicone materials is possible. As a result, it can be stated that the optimum time to fill in the second material is after 10 to 20 min.

2.3. Replacement Hands

From the three hands with various silicones with shore hardnesses of 25 ShA, 10 ShA, and 00 ShA the hand with Shore hardness 10 feels anthropomorphic but cannot be bent by the orthosis. In contrast, the replacement hand with Shore hardness 00 ShA proved functional in combination with the orthosis. However, it does not feel anthropomorphic because the material is too floppy and soft. Consequently, a combination of both silicones was needed: softer for the index and middle fingers and harder for the rest of the hand, respectively. The hand in Figure 3 shows the finished hand after molding; Figure 4 presents the replacement hand in combination with the orthosis.

2.4. Evaluating the Sensory Feedback

The aim of testing the sensory feedback system was to prove that the molded sensors can measure meaningful measurements during the grasping process and also to evaluate the quality of the signals. In the individual analysis on the bending and force sensor, the flexion sensor revealed linear and the force sensor logistic behavior, as indicated in Figure 5a,b, respectively. Both sensors showed reproducible results.

The measurements of sensory feedback during the grasping process of different objects provide repeatable and consistent results. The force sensor registers a different signal when gripping objects of similar shape and dimensions but different firmness, i.e., the bottle and the paper roll, which both had the same diameter (refer Figure 6a,b,d,e). However, while grabbing the paper roll, the force sensor measures only a signal during the gripping process (see Figure 6c,f). Despite squeezing it slightly, the paper roll does not have enough stiffness to generate sufficient force to the sensor in a stationary state. This shows that due to the strength of the leather of the glove, no exact haptic resolution is feasible. Nevertheless, with increasing force application, a linear progression of the sensory value can be observed. In a static state, i.e., while holding the object, the signal of the force sensor is noisier than the signal of the bending sensor, but for most applications, the measured noise level is acceptable (see Figure 6).

3. Discussion

In this work, a personalized prosthesis hand has been created by silicone casting using a 3D-printed mold. As the costs for manufacturing the personalized 3D-printed mold are low, compared to a personalized mold, manufactured by conservative techniques, this approach can be considered cost-effective.

In addition, possible solutions to integrate sensory feedback into such a prosthesis are shown.

It emerged that two different silicone materials were required for the replacement hand, as even the 10 ShA silicone in combination with its sensors became too stiff to be driven by the motor of the orthosis.

Unfortunately, the 00 ShA silicone was so soft that the sensors were not sufficiently fixed, leaving their position in the finger during flexion. It was observed that sensor parts might abrade each other, leading to the corresponding sensors’ failure. To overcome this challenge and even simplify the manufacturing process, it may be beneficial to use a single material with larger Shore hardness and strengthen the mechanism in the orthosis. Additionally, it is essential to note that the soldering joints of the sensors suffer from mechanical stress (see Figure 7a). These rigid parts of the sensor are placed optimally outside the movable parts of the finger or in the core of the finger, where the surrounding material gives more stability.

Another issue occurred, concerning the second force sensor at the proximal end of the finger. It turned out that the cord that flexes the finger during the grasping process is spanned between the object and the sensor in a way that the sensors are not affected by the object (see Figure 7b). Therefore, no signal is measured by this sensor, as can be seen in Figure 6. A solution for this might be a new routing of the cord inside the glove.

Regarding the sensory feedback, it could be demonstrated that the degree of hand closure and the grip strength could be measured. As a result, this replacement hand allows the patient to get some sort of haptic feeling apart from the current motor function. Further research should clarify the extent to which this sensory feedback system can support the patient in learning to handle the prosthesis or also in everyday usage. For this purpose, further research should point out to which form and presentation the measured signals are optimally converted to be most helpful to the patient.

To improve the visual appearance of the replacement hand, the silicone material can be given any color by adding a color pigmentation. In doing so, matching the color of the replacement hand to the patient’s skin color can promote greater patient acceptance.

4. Materials and Methods

4.1. Test Series to Evaluate Sensor Attachment

In this test series, different mounting options for sensor attachment were defined and tested. How the embedded sensors will behave in the cured silicone was tested. In addition, the optimal diameter of the spacers was tested for. In Table 1 the configurations of four test fingers and one reference finger are listed. The central concept for mounting the sensors in the mold was adding some bars, i.e., spacers, to which the sensors are attached. These cylindrical bars protrude from the mold, whereas the sensors are attached using glue that can be detached after casting. In Finger I, another concept for fixing the sensors in the mold was tested. Here, the bending sensor is attached to a wire with reversible glue led out through the funnel opening. The Fingers III to V include force and bending sensors as also a combination of both sensor types, respectively.

SILIXON 25 material was used for this test series. The casting was conducted according to the following steps: After the sensors and cables had been securely fastened, both mold components were screwed together. Then one half of the silicone base and one-half of the catalyst were mixed until a homogeneous mass was formed. The silicone was then carefully poured into the mold. The vulcanization process took about 4 h at a temperature of approx. 25 °C. After five hours, both silicone halves were released from each other, and the finished hand was carefully removed from the mold. Thereby the spacers detached from the sensors.

4.2. Test Series for Combining Silicones with Different Shore Hardnesses during Casting

As the index and middle fingers needed lower stiffness for usage in combination with the orthosis and in regard to the test series results for material analysis, the replacement hand was realized with two different silicones. Two silicones with different Shore hardnesses can be mixed, resulting in the average of both initial hardnesses. Therefore the first material (e.g., the softer one) had to cure to a certain degree before the second material (e.g., the stiffer one) could be filled in.

To investigate the possibility of realizing a silicone hand consisting of different silicone materials, an additional test series was performed. During the test, it was investigated after which period the second casting of a more rigid silicone material can occur. The focus was on ensuring that the two materials did not mix but still adhered to each other.

Table 2. Composition of the silicone fingers for material analysis with the different Shore hardnesses.

Finger	Shore-A-Value of the First Part	Time in Minutes until the Second Material Is Filled in	Shore-A-Value of the Second Part
1	00	0	10
2	00	10	10
3	00	10	10
4	00	20	10
5	00	20	10

shows an overview of the castings for each test finger. After filling in the first material, a specific time was waited and the other silicone was poured in subsequently.

4.3. Manufacturing Process of a Silicone Replacement Hand, Including Pressure and Bending Sensors

First, the hand was scanned using a structured-light 3D scanner to create a personalized hand, shown in Figure 8a. Subsequently, the scanned mesh body was converted to a solid so that the hand could be processed in the Autodesk Inventor CAD program, which is shown in (b). Thereafter followed the construction of the mold. The mounting options for the sensors were designed into the mold of the hand and the mold was 3D printed with a Prusa i3 MK3S FDM-based 3D-printer which can print polyethylene terephthalate glycol (PETG) material (c). PETG material is easy to process, has a low risk for warping, and has high strength. The robustness is important because the mold must be fastened with screws to be tight.

In the next step, the attachment of the force and bending sensors into the fingers of both components was done (d). Finally, both parts of the mold were put together and liquid silicone was poured in.

An additional process step was introduced to manufacture the replacement hand using two different silicones. Flexible plastic tubes were inserted into the lower mold and fixed using glue. These were used for directed injection of the softer silicone material into the index and middle fingers. After filling the softer silicone into both fingers, the tubes were carefully pulled out and, after ten minutes, the harder silicone material was poured into the remaining areas of the hand. During the test, the 00 ShA silicone material was used for the soft fingers and the 25 ShA silicone material for the more rigid part of the hand.

4.4. Evaluating Sensory Feedback System

Both sensor types, i.e., force and bending sensors, used force-sensing resistor technology (Force sensors from Interlink and Flex sensor from SpectraSymbol). Thus, the resistance of these sensors varied with applied force or flexion, respectively. The positioning of the sensors is made as shown in Figure 9. A simple voltage divider circuit was implemented, to measure the variations in the resistance of the sensors. The resistance of all three sensors of one finger was measured simultaneously using a multichannel oscilloscope.

In a first test, the encapsulated sensors were tested individually. For this purpose, two fingers, both consisting of SILIXON 25 material, were mounted on a manual rotation table (see Figure 10a,b) and a floating bar to evaluate the behavior of the bending and force sensor, respectively. The rotary table was adjusted in steps of 20 degrees and the resistance variation measured the bending behavior via a voltage dividing circuit. To evaluate the force sensor, a vertical force was applied to the finger lying on its back and was measured by a force gauge, as shown in Figure 11.

4.5. Evaluating Sensory Feedback System Using Neofect Orthosis

In a further test, the replacement hand was tested with the orthosis from Neofect. This orthosis is pulled over the paralyzed or restricted hand as a glove. The thumb of the orthosis is rigid and acts as an abutment while the index and middle finger are motorized and able to enclose an object. The orthosis is controlled by an external, Bluetooth-enabled controller that is held in the patient’s healthy hand. The orthosis can hold up to 2 kg, therefore a full bottle of water can be held without effort.

During the trial, three different objects were grabbed by the replacement hand integrated into the orthosis. First, a small hard bottle was caught, shown in Figure 12a. Second, a thin and hard tool was used to measure the signals caused by the more significant degree of hand closure (Figure 12b). Last, a paper roll with the same diameter as the bottle in the first test was used to measure the capability of the force sensors to distinguish between objects of different consistency (Figure 12c). In Video S1, see Supplementary Materials, grabbing a small bottle and a paper roll during measurements are shown.