Textronic Glove Translating Polish Sign Language

Communication between people is a basic social skill used to exchange information. It is often used for self-express and to meet basic human needs, such as the need for closeness, belonging, and security. This process takes place at different levels, using different means, with specific effects. It generally means a two-way flow of information in the immediate area of contact with another person. When people are communicating using the same language, the flow of information is much easier compared to the situation when two people use two different languages from different language families. The process of social communication with the deaf is difficult as well. It is therefore essential to use modern technologies to facilitate communication with deaf and non-speaking people. This article presents the results of work on a prototype of a glove using textronic elements produced using a physical vacuum deposition process. The signal from the sensors, in the form of resistance changes, is read by the microcontroller, and then it is processed and displayed on a smartphone screen in the form of single letters. During the experiment, 520 letters were signed by each author. The correctness of interpreting the signs was 86.5%. Each letter was recognized within approximately 3 s. One of the main results of the article was also the selection of an appropriate material (Velostat, membrane) that can be used as a sensor for the proposed application solution. The proposed solution can enable communication with the deaf using the finger alphabet, which can be used to spell single words or the most important key words.


Introduction
According to the report from the Deaf Expert Group on behalf of the Commissioner for Human Rights [1], 850 thousand people in Poland have a hearing disorder, and approximately 100 thousand people use Polish sign language. It is a natural way of communication for the deaf, for whom the Polish language is foreign. According to the report, people with hearing disorders face exclusion resulting from the problem of communication.
Up until now, translating services for the deaf existed in the form of computer software that translated sign language. Such a solution is problematic. There is scarce availability of the service in public and private locations, and access to it is limited to specific ranges of time [2]. This method can also affect person's comfort and focus because their communication is recorded.
The aim of the work introduced in this article is to recognize the changes made by the user's hand when showing expressions in sign language. The communication device should not be a burden for the user, but instead, it should be as inconspicuous as possible. The glove form of the device is comfortable. It does not reduce the range of motion, nor does it have an impact on a person's well-being.
The electronic glove is used as an interface connecting people who encounter problems when communicating in different languages (sign language and spoken language). However, there are other ways to use the device. It can also be used as a keyboard, with the addition of a speaking synthesizer to help people who lack the ability to speak. information in Polish; however, it can be adapted to support any other language. The novelty of this paper is the sensors' construction on the flexible textile composite substrates as thin electroconductive layers.

Materials and Methods
Bending level can be easily specified using bend sensors. The sensor works as a potentiometer. It changes its resistance with the component's bending angle. Bending sensors are available ready-made for purchase. However, they can be easily manufactured using a conductive core. Self-made components are cheaper, flexible along the entire length, and can be easily adjusted to the project. Considering the advantages, self-made bending sensors are used in this project.
All sensors are built based using materials with good electrically conductive properties.

Materials
Among tested materials, the commercial Velostat, also known by the name Linqstat, was used as a reference material. It is a material developed by Custom Materials and is currently a registered trademark of Desco Industries. It is a polymer film (polyolefin) coated with carbon black with very good electrically conductive properties [30]. It is used, inter alia, as a shield for electrostatic discharges [31] or as an element of electrical circuits in "smart" shoes that light up while walking. The phenomenon of resistance changes under pressure or bending is also used. Its surface resistivity is less than 31 kΩ/sq. The next test materials were Cordura and a Goretex membrane. Cordura is developed by DuPont and is a brand of fabrics showing high resistance to mechanical damage. It is made of polyamide fibers reinforced with a polyurethane coating. The Goretex membrane, on the other hand, is a more flexible material than Cordura, made of foam Teflon fibers covered with a thin layer of nanofibers [32]. Due to the fact that the fibers are covered with layers, ensuring the evenness of the substrate, it is possible to apply a thin electrically conductive layer on the surface of these materials, with a surface resistance on the order of single Ω/sq in the physical vacuum deposition process. A silver layer with a high conductivity value characteristic for this type of metal (γ = 62.5 m/Ω·mm 2 ) was applied to the selected materials. The metal evaporation process from the resistive tungsten source lasted 5 min and was carried out on a Classic 250 vacuum stand of the Pfeiffer Vacuum system. The initial vacuum was 5 × 10 −5 mbar. The silver was delivered and guaranteed by Mennica Metale Ltd. (Warsaw, Poland). The surface material was placed 6 cm from the source of evaporation. The composite textile materials serving as the substrate for the electrically conductive layers were conditioned at room temperature and 55% humidity for several hours prior to the vacuum deposition process. The electrode geometry can be modified using laser ablation of the applied metallic layer [33]. Because some defects can occur during the production process [34], influencing the resistance of the sensor, hot spots can occur [35]; therefore, the validation of the created film can be achieved using optical coherence tomography [36], or optical microscopy.
Photos of the surfaces of the materials used for creating the sensors are presented in Figure 1.
A photo of a glove prototype is presented in Figure 2. The photograph shows an example of the bend sensors created from electroconductive material, placed on each finger, and also the microcontroller, multiplexer, and power supply system. The microcontroller used in the prototype is a Bluno Beetle from DFRobot (Shanghai, China).
The components of the bend sensor are presented in Figure 3. The core is placed between two conductive layers that have no contact with each other. The conductive layers are constructed by making stitches on a piece of fabric. The current flows successively through the first conductive material, the core, and the second conductive material. The resistance of the sensor changes when it is bent. A photo of a glove prototype is presented in Figure 2. The photograph shows an example of the bend sensors created from electroconductive material, placed on each finger, and also the microcontroller, multiplexer, and power supply system. The microcontroller used in the prototype is a Bluno Beetle from DFRobot (Shanghai, China). The components of the bend sensor are presented in Figure 3. The core is placed between two conductive layers that have no contact with each other. The conductive layers are constructed by making stitches on a piece of fabric. The current flows successively through the first conductive material, the core, and the second conductive material. The resistance of the sensor changes when it is bent. A photo of a glove prototype is presented in Figure 2. The photograph shows an example of the bend sensors created from electroconductive material, placed on each finger, and also the microcontroller, multiplexer, and power supply system. The microcontroller used in the prototype is a Bluno Beetle from DFRobot (Shanghai, China). The components of the bend sensor are presented in Figure 3. The core is placed between two conductive layers that have no contact with each other. The conductive layers are constructed by making stitches on a piece of fabric. The current flows successively through the first conductive material, the core, and the second conductive material. The resistance of the sensor changes when it is bent.  The resistance range depends on several aspects: the type of material the core is made of, the size of the core, and the size of the gap between the sensor's layers.
The materials mentioned above were tested as a core of the sensor in order to choose the most suitable fabric. Each sensor with a different core was placed sequentially on a forefinger, and two tests were performed. The first test was to measure the output of the digital signal from the A/D converter when the finger was straight, and the second test measured the output when the finger was maximally bent. The possible range of the signal  The resistance range depends on several aspects: the type of material the core is made of, the size of the core, and the size of the gap between the sensor's layers.
The materials mentioned above were tested as a core of the sensor in order to choose the most suitable fabric. Each sensor with a different core was placed sequentially on a forefinger, and two tests were performed. The first test was to measure the output of the digital signal from the A/D converter when the finger was straight, and the second test measured the output when the finger was maximally bent. The possible range of the signal was 0-1023.

Algorithm
Using the introduced parameter, the algorithm to recognize the finger state was developed. After reading the data from the sensors, every finger was been assigned an appropriate bent state, and the results were saved in the memory. After gathering 20 bent states, they were compared. If all of these were identical, the finger was assigned an actual bent state. The process should be run in a thread, performed every 70 ms. The amount of necessary data to be stored in the memory and the thread frequency were determined experimentally, ensuring the correct operation of the device. The described algorithm is presented in Figure 4.

Hand Movement Recognition
Another crucial parameter which must be determined in order to translate sign language is the appearance of movement and its direction. This can be easily detected with an accelerometer. An accelerometer measures accelerations. Acceleration in each direction can be detected using a 3-axis sensor. When the movement starts, the acceleration increases, and when it ends, the acceleration decreases to zero. The 3-axis accelerometer attached to the hand can recognize the movement of the hand, along with the direction of the movement. Hand movements when using sign language are low distance movements, so the probability of uniform motion and therefore, zero acceleration, is small. An accelerometer used in the project is a wearable FLORA accelerometer from Adafruit. It combines a 3-axis accelerometer and magnetometer, using a I2C interface. The working voltage is 3 V.
Recognizing the hand movement is executed according to the following algorithm. Data read from an accelerometer is stored in the memory. After storing 25 samples, the direction of the movement is determined and saved as the current hand movement direction. The algorithm ought to be performed every 100 ms. The amount of the necessary acceleration data that have to be stored in the memory and the thread frequency were determined experimentally, ensuring the correct operation of the device ( Figure 5).

Sign Identification
The letter identification is processed by software uploaded in the Bluno Beetle microcontroller. The software used to convert electrical to digital signals is the authors' own software and it is written in C++ language, based on the presented algorithms. The actual state of a hand is compared with values saved in the memory. The action is performed in a thread every 120 ms, with a higher frequency than that of the threads reading the data from the sensors. When the identification is correct, the letter is transmitted via Bluetooth or UART port. The described algorithm is presented in Figure 6.
A mobile application for Android 6.0 was developed. It uses the Bluetooth module of the phone, and after pairing the devices, it prints the letter shown using the created glove ( Figure 7).
For every letter in a Polish sign language, a set of parameters was determined. Possible combinations are presented in Figure 8. The correct arrangement of the fingers corresponds to each letter. For some letters, the state of a single finger does not have to be specified. The stress of the sensors placed on the fingers is important for distinguishing each letter.

Hand Movement Recognition
Another crucial parameter which must be determined in order to translate s guage is the appearance of movement and its direction. This can be easily detect an accelerometer. An accelerometer measures accelerations. Acceleration in each d can be detected using a 3-axis sensor. When the movement starts, the accelera creases, and when it ends, the acceleration decreases to zero. The 3-axis acceler attached to the hand can recognize the movement of the hand, along with the dire the movement. Hand movements when using sign language are low distance move so the probability of uniform motion and therefore, zero acceleration, is small. A erometer used in the project is a wearable FLORA accelerometer from Adafruit. Each finger's state can be determined by using three scale measurements. The states are: maximally bent, slightly bent (approx. 90 • ), and straight. The ranges of the sensor's data values were evaluated experimentally. The sensors were attached to the glove and pinned to the input of the A/D converter. The Min symbol means that the finger should be straightened, Mid means that the finger should be slightly bent, and Max means that the finger should be bent to the maximum. The symbols "−" and "+" in the accelerometer column indicate the need to recognize the hand movements. The position of the fingers defining a given letter is described in Table 1. ors 2022, 22, x FOR PEER REVIEW acceleration data that have to be stored in the memory and the thread frequ determined experimentally, ensuring the correct operation of the device (Figur

Sign Identification
The letter identification is processed by software uploaded in the Bluno Be controller. The software used to convert electrical to digital signals is the aut software and it is written in C++ language, based on the presented algorithms. state of a hand is compared with values saved in the memory. The action is per a thread every 120 ms, with a higher frequency than that of the threads readin    For every letter in a Polish sign language, a set of parameters was dete ble combinations are presented in Figure 8. The correct arrangement of th sponds to each letter. For some letters, the state of a single finger does pinned to the input of the A/D converter. The Min symbol means that the finger should be straightened, Mid means that the finger should be slightly bent, and Max means that the finger should be bent to the maximum. The symbols "−" and "+" in the accelerometer column indicate the need to recognize the hand movements.
The figures below show examples of letters C and L in Polish sign language ( Figure  8). The position of the fingers defining a given letter is described in Table 1.

Results
The materials presented in Figure 1 were tested as a core for the sensor in order to choose the most suitable fabric. Each sensor with a different core was placed sequentially on a forefinger and two tests were performed. The first test measured the output of the digital signal from the A/D converter when the finger was straight, and the second test measured the output when the finger was maximally bent. The finger in each position was slightly modified to reflect the natural state of the hand. Fingers recognized as straight may be slightly bent (circa 15 degrees), and fingers bent into a fist may be squeezed more or less firmly. The resolution of the A/D converter used is 10 bits, so it can encode an analog input from 1 to 1023 different levels. Figures 9-11 present charts of gathered signals for different core materials. Table 2 shows characteristic values for the measured signals.  After analysis of Velostat's behavior in the research, it can be observed that the range of the signal is wide, from 25 to 653 levels. The sensor has a resistance range from approx. 9.2 kΩ (straight sensor, lying flat on horizontal surface) to approx. 9 kΩ (sensor bent at an angle of approx. 170 degrees). Two peaks were registered when testing the bend sensor (between the 25th and 55th samples), but it can be assumed that these are individual measurement errors caused by a temporary lack of continuity in the circuit at points joining the conductive thread and the wires.   In the case of the Goretex membrane, the range of the signal is also narrow, from 956 to 983 encoded analog the signal. The sensor has a resistance range from approx. 155 Ω (straight sensor, lying flat on horizontal surface) to approx. 90 Ω (sensor bent at an angle of approx. 170 degrees). The bending resistance of the thin layer created on the membrane and Cordura is described in the previous paper [37].
Cordura is the most rigid material among all those tested. Because of this feature of the material, the signal values do not differ when the finger is bent or straight. The resistance of the sensor fluctuates between 200 Ω (straight sensor, lying flat on horizontal surface) to approx. 208 Ω (sensor bent at an angle of approx. 170 degrees). The sensor does not work as expected with this material; therefore, it cannot be used as a core for the sensor.    The best material among those tested is the Velostat. The range of the signal is wide, as is the resistance. It gives the most accurate signal values. This is caused by the additional aspect that when bending, the sensor with the Velostat membrane causes pressure on the membrane, which leads to an additional change in its resistance. The Velostat is also durable. The membrane covered with silver provides accurate signal values as well. However, the range of the signal is narrow. On the basis of this test, the Velostat membrane was chosen as a core for the bending sensor.
Every finger of the created glove was equipped with one bend sensor in order to measure the bent angle of the respective finger.
Two measurements were performed: first, when the fingers were straight, and second, when the fingers were maximally bent. During the process, the fingers were moved slightly-into a fist, tightly or slightly clenched, and then relaxed-with the fingers fully and loosely straightened. This was completed to unify the bend level of the measured fingers. It was assumed that the finger is classified as straight when it is bent in an angle up of 0-20 • (the bend angle of a fully straightened finger is 0 • ). The finger is considered to be maximally bent when the bend angle is between 160-190 • . This ensures the procurement of a wide range of sensor data from each bending state. The described test was performed once on a single person. The resulting ranges were adjusted during manual testing of the glove, in which the recognition of individual letters was checked. The possible data values were in the range of the 0 to 1023 level. Every finger has its own data values range. Figure 12 displays values gathered for the thumb when maximally bent. The minimal value from the sensor is 139. The maximum is 568. The average is 339 and the median 319. The median and average values are considered to provide perspective regarding the most probable and the most common signal values that may appear. on the membrane, which leads to an additional change in its resistance. The Velostat is also durable. The membrane covered with silver provides accurate signal values as well. However, the range of the signal is narrow. On the basis of this test, the Velostat membrane was chosen as a core for the bending sensor. Every finger of the created glove was equipped with one bend sensor in order to measure the bent angle of the respective finger.
Two measurements were performed: first, when the fingers were straight, and second, when the fingers were maximally bent. During the process, the fingers were moved slightly-into a fist, tightly or slightly clenched, and then relaxed-with the fingers fully and loosely straightened. This was completed to unify the bend level of the measured fingers. It was assumed that the finger is classified as straight when it is bent in an angle up of 0-20° (the bend angle of a fully straightened finger is 0°). The finger is considered to be maximally bent when the bend angle is between 160-190°. This ensures the procurement of a wide range of sensor data from each bending state. The described test was performed once on a single person. The resulting ranges were adjusted during manual testing of the glove, in which the recognition of individual letters was checked. The possible data values were in the range of the 0 to 1023 level. Every finger has its own data values range. Figure 12 displays values gathered for the thumb when maximally bent. The minimal value from the sensor is 139. The maximum is 568. The average is 339 and the median 319. The median and average values are considered to provide perspective regarding the most probable and the most common signal values that may appear.   The range of values between the maximum value, when the finger is straight, a the minimal value, when the finger is maximally bent, is a range of values for the th possible state of the finger-slightly bend (approx. 90°).
The additional plots of data received for other fingers and their states are found Appendix A. Table 3 presents the ranges of signal levels for every finger in the three possible be states. Tests evaluating the correctness of the interpretation of individual letters were c ried out by the authors. As a part of testing the glove, a fragment of a poem consisting 520 letters was translated, and the effectiveness was assessed (the percentage of corr ness of the translation of the text) Depending on the signing ability (in particular, speed of change of the individual hand settings), the correctness of letter recognition w 86,5%. The recognition time of each sign was within approximately 3 s.

Discussion
The testing phase was focused on recognizing a fingers' bent states. In Table 1, blue color indicates the letters that are not easily identified. For example, the letters E and S, O are shown in a similar way. They differ from each other only with a state o single finger or hand orientation; whereas, the letter G is shown as a finger moveme The range of values between the maximum value, when the finger is straight, and the minimal value, when the finger is maximally bent, is a range of values for the third possible state of the finger-slightly bend (approx. 90 • ).
The additional plots of data received for other fingers and their states are found in Appendix A. Table 3 presents the ranges of signal levels for every finger in the three possible bend states. Tests evaluating the correctness of the interpretation of individual letters were carried out by the authors. As a part of testing the glove, a fragment of a poem consisting of 520 letters was translated, and the effectiveness was assessed (the percentage of correctness of the translation of the text) Depending on the signing ability (in particular, the speed of change of the individual hand settings), the correctness of letter recognition was 86.5%. The recognition time of each sign was within approximately 3 s.

Discussion
The testing phase was focused on recognizing a fingers' bent states. In Table 1, the blue color indicates the letters that are not easily identified. For example, the letters E, M and S, O are shown in a similar way. They differ from each other only with a state of a single finger or hand orientation; whereas, the letter G is shown as a finger movement. The identification of this letter should be performed in an alternative way to the rest of letters.
Another issue is that any force acting on a bend sensor is interpreted as a finger bending. This occurs when adjacent fingers are in extreme positions, e.g., for the letter L, the forefinger is straight and the middle finger is maximally bent. The membranes inside the sensors are stretched, which provides an incorrect data value for the straight finger. Another example is the letters R and T, where the fingers have contact at the location of the sensors. Finger pressure causes a force on the sensors, resulting in incorrect letters identification.
Another issue with incorrect letter identification is connected with the glove's cut. The glove should be well fitted to the hand in order to provide correct and repeated letter identification.
Based on the tests performed, improvements to the concept of the device can be introduced. Adding an additional bending sensor on the forefinger-one sensor for measuring the bend level between the finger and the palm, and the other in the middle of the fingercan improve the repetitiveness of letter identification, defining the bend angle of the finger itself. This provides better recognition between different states of the finger.
Another modification is the introduction of a new parameter defining hand orientation. In case of the letters M, E, and U, this is a crucial aspect.
Experiments have shown that the system can be tuned to the individual characteristics of the person, and the correctness of letter detection can be increased above 86.5%, even in the case of the most difficult to recognize letters.
In the literature, for all solutions using gloves, the analogue signal is converted to a digital one with the use of various sensors. The use of thin electroconductive layers for this type of solution is a new approach not previously found in the literature. The possibility of attaching a flexible sensor to any type of glove at any stage of its use is a value added to the presented solution. The authors indicated materials that can be used for this type of application.

Conclusions
The process of manufacturing and testing a prototype of a glove for translating sign language was introduced. It can be used as to aid communication between people using sign language and those who do not know it. The application can be an alternative for a keyboard. The device lets the user input letters using a mobile device or a computer. The process of letter identification is not fully precise. The arrangement of the fingers when pointing to a specific letter is always the same; however, even slight changes in the angle of flexion may affect the identification of the information. At the present stage of research, the influence of mechanical stress on the durability of the thin conductive layers should also be taken into account. Extending the life of textronic sensors is a challenge facing current science. In order to increase the correctness of operation, improvements should be applied. One of the improvements is adding an additional bending sensor on the forefinger in order to distinguish the bend level of a bent finger itself and the bend level of the finger in relation to the palm. Another improvement is implementing an additional parameter defining the hand orientation. It is worth conducting further research to achieve the increased resistance of thin electroconductive layers to bending stress. The possibility of using materials coated with thin electrically conductive layers in a signing glove application is a difficult challenge due to the mechanical strength of such layers. The analysis of their electrical properties and the selection of substrates for the production of thin-film sensors allowed for the construction of a glove that was used to interpret over 500 signs, with a correctness at the level of 86.5%. The obtained results are very satisfactory and promising. Constructing a glove that will be a bridge between the deaf and hearing population constitutes a tool that will improve communication between these two groups.

Conflicts of Interest:
The authors declare no conflict of interest.

Appendix A
Some additional figures showing representative data acquired from the sensors placed on the fingers of the glove are presented in Figures A1-A10. Figure A1. Data from the sensor on a maximally bent thumb. Figure A1. Data from the sensor on a maximally bent thumb.