Industrial Upper-Limb Exoskeleton Characterization: Paving the Way to New Standards for Benchmarking

: Exoskeletons have been introduced in industrial environments to prevent overload or repetitive stress injuries in workers. However, due to the lack of public detailed information about most of the commercial exoskeletons, it is necessary to further assess their load capacity and evolution over time, as their performance may change with use. We present the design and construction of a controlled device to measure the torque of industrial exoskeletons, along with the results of static and dynamic testing of an exoskeleton model. A step motor in the test bench moves the exoskeleton arm in a pre-deﬁned path at a prescribed speed. The force measured with a beam load cell located at the interface between the exoskeleton arm and the test bench is used to derive the torque. The proposed test bench can be easily modiﬁed to allow different exoskeleton models to be tested under the same conditions.


Introduction
Work-related musculoskeletal disorders (WMSDs) are a major concern in working environments [1], and they are often associated with repetitive movements and prolonged unfavorable postures during working shifts. WMSDs are the most prevalent disease among Brazilians, and to make matters worse, the number of cases escalated 184% within a period of 10 years, from 2007 to 2016, according to the Health Department in Brazil, 2018 [2].
Among WMSDs, shoulder injuries are one of the most severe, accounting for lost working days three times more than those of back injuries [1].
In Brazil, almost 39,000 workers were on sick leaves in 2019 due to WMSDs [3], and shoulder injuries are again among the leading causes [2].
Several tasks in the industry leave workers vulnerable to various injuries risks, which poses threats to factory workers' health as well as to their employers, and eventually, it may become an economic burden to the society due to therapies, leave period, and lost working days [1].
Handling, supporting, and lifting heavy materials may cause overexertion injuries, therefore contributing to WMSDs [4]. Working in awkward postures and repetitive motions are among the top 10 causes of disabling injuries also in the USA [5] apart from bringing massive costs on medical care and lost-wage expenses [1].
Assembly line workers and machine operators are frequently exposed to many unfavorable positions in their daily tasks, such as in handling heavy tools, drilling, countersinking, riveting, bucking, swaging [6] and paneling, electrical work, welding, grinding, picking, pruning, painting, inspection, and overhead assembly, all of which require sustained work at chest level to overhead level [7].
Exoskeletons are wearable devices that augment power, as well as assist and enable physical activity through mechanical interaction with the body by increasing strength and endurance, amplifying and reinforcing performance. In the case of industrial exoskeletons, mainly lower back and upper limbs work frequently objects of research [8][9][10].
Exoskeleton technology is an emerging technology used in different fields, for example, for military purposes [11], in the medical field for rehabilitation or assistance [12][13][14], or for industrial applications [15][16][17][18][19][20]. Even though there is a growing trend in automation and mechanization of a variety of processes in the industry, human-robot collaboration has been targeted in order to keep human flexibility, allowing high levels of dexterity and fine handling, much needed in dynamic manufacturing, for example [9,21]. Moreover, full automation is unfeasible and costly, and a wide range of products and tasks in the modern industry require human thinking skills, such as being able to observe, decide, and take actions [9], and this variation in demand allows us to outperform robots [16].
Upper-limb exoskeletons are designed to help sustain working posture, supporting the upper limbs during overhead tasks. To this end, they generate joint torques, for instance, in the shoulder, aiming at reducing shoulder overload and preventing injury [17]. Most of the commercially available passive exoskeletons can provide different levels of assistance by adjusting springs stiffness.
Although industrial exoskeletons have been extensively tested with subjects [18,19,22], there are no standardized procedures to test exoskeletons independently of the user that provide direct quantitative measurements. In this respect, it is difficult to assess different exoskeletons under the same conditions to obtain benchmarking metrics. Moreover, there are still gaps in the literature about design requirements and assessment of the exoskeletons in terms of the level of assistance provided to the users. In addition, the lack of validation standards also hinders their adoption in the industry, thus making their comparison difficult [23].
To disseminate knowledge and to better explore outcomes from different studies on such trendy technology, standardization is necessary. It provides minimum acceptable requirements for quality and safety for manufacturing products, because of materials that were previously tested, which also helps to minimize costs due to this testing and selection processes. Furthermore, complying with standards involves meeting legal demands, which facilitates commercialization and their adoption in global markets [24].
In a previous study [25], we presented the design of a test bench to measure the actuating torque of shoulder exoskeleton models, in order to compare these values between different manufacturers under controlled conditions. In this study, we present the final design and construction of this test bench and apply it to assess the load capacity of a commercially available exoskeleton model. We propose a set of tests that involve both static and dynamic measurements, and we present the design and construction of a controlled device to measure the torque of industrial exoskeletons, along with the results of static and dynamic testing of this model. Therefore, the goal of this manuscript is to present a device and procedure involving static and dynamic tests to assess exoskeletons. The capabilities of the device and the validity of the procedure are demonstrated with one exoskeleton model. In this way, this study contributes to filling the literature gap in terms of exoskeleton assessment and paves the way to benchmarking and standardization. This is a step toward making exoskeleton technology viable for the market to ultimately contribute to reduce and prevent injuries at workstations.

Materials and Methods
The objective was to measure exoskeleton torque as a function of arm angles, under the same conditions for all exoskeletons under study. This was achieved by means of a beam load cell located at a known distance in the test bench. The transmitted force was then directly converted into torque, and angle positions were determined by the number of steps sent to the step motor. In order to demonstrate the utility of the system, we tested the commercially available exoskeleton MATE (v1.0 Comau, Turin, Italy). Figure 1 shows the exoskeleton mounted on the test bench. then directly converted into torque, and angle positions were determined by the number of steps sent to the step motor. In order to demonstrate the utility of the system, we tested the commercially available exoskeleton MATE (v1.0 Comau, Turin, Italy). Figure 1 shows the exoskeleton mounted on the test bench. The structure of the test bench was made of structural aluminum, and moving parts were built with steel. It was composed of two towers, one for the actuation and the other for the actuated exoskeleton. A step motor controlled position and a 10 kg beam load cell provided information about forces and, thus, torque. The machine was controlled by an Arduino Uno R3 board connected to a personal computer. A dedicated power source fed the motor driver, as shown in Figure 2. For more information about the test bench, see [25].
Validation of the test bench was carried out against results measured with the device. The masses listed in Section 2.1 were directly used to calculate the resulting torque at different positions of the exoskeleton actuator. The same exoskeleton was then fixed on the test bench and used as a reference for comparison between the directly measured torque and the torque calculated by the device. It is a procedure similar to those used for interlaboratory comparisons.
The following subsections describe how several conditions were tested and considered in the tests, as summarized in Table 1, in order to validate the test bench with an exoskeleton. The structure of the test bench was made of structural aluminum, and moving parts were built with steel. It was composed of two towers, one for the actuation and the other for the actuated exoskeleton. A step motor controlled position and a 10 kg beam load cell provided information about forces and, thus, torque. The machine was controlled by an Arduino Uno R3 board connected to a personal computer. A dedicated power source fed the motor driver, as shown in Figure 2. For more information about the test bench, see [25].
Validation of the test bench was carried out against results measured with the device. The masses listed in Section 2.1 were directly used to calculate the resulting torque at different positions of the exoskeleton actuator. The same exoskeleton was then fixed on the test bench and used as a reference for comparison between the directly measured torque and the torque calculated by the device. It is a procedure similar to those used for interlaboratory comparisons.
The following subsections describe how several conditions were tested and considered in the tests, as summarized in Table 1, in order to validate the test bench with an exoskeleton.

Masses
A set of already existing nine test masses were used to calibrate the beam load cell. The values were chosen so that the complete range (from 0 kg to 3 kg) could be achieved in steps of 250 g., as follows: A metal basket was also used to allow (un)load of the main masses, with weight Mb = 41 g.
All masses were weighted against a common scale with a resolution of 1 g. Figure 2. Diagram describing the actuation, control, and data acquisition elements. The Notebook has a custom-made program to select the testing procedure that would be fed to the microcontroller (Model Uno Rev. 3, Arduino Uno, Arduino SpA, Italy) to control the driver (KTC-STR3, Kalatec, São Paulo, Brazil) that drove the step motor (HT23-400, Kalatec, São Paulo, Brazil). The forces acting on the beam of the mechanism by the exoskeleton actuation were measured with a load cell. This measurement was converted to a digital signal (ADC: Analog-to-Digital Converter) and fed to the Arduino Uno microcontroller.

Masses
A set of already existing nine test masses were used to calibrate the beam load cell. The values were chosen so that the complete range (from 0 kg to 3 kg) could be achieved in steps of 250 g, as follows: A metal basket was also used to allow (un)load of the main masses, with weight Mb = 41 g.
All masses were weighted against a common scale with a resolution of 1 g.

Beam Load Cell Calibration
A simple flexion beam load cell was used in the tests, with a capacity of 10 kg. It must be calibrated before use, and as it was measured in both directions, both should be calibrated. The calibration curves were different in each case and were assumed different but close to each other. Mass M11 = 998 g was used to calibrate initially the beam load cell. After that, the maximum mass value of 3013 g was used to generate a second calibration factor. Zero was added as a third point, and the mean value between them was defined as the calibration factor. The same procedure was used for both working sides of the beam load cell. The calibration factors obtained were as follows: • Positive: 233,000; • Negative: 45,000.
The initial assumption was confirmed partially because they were indeed different but not close to each other. Their values were corrected post-acquisition.

Beam Load Cell Linearity
Although linearity was assumed, we used a set of 13 points to assess linearity, from zero to the maximum expected value of 3 kg at 158 mm from the rotation axis, in incrementing steps of~50 g. Table 2 shows the masses and values used. In both positive and negative cases, beam load cell linearity was confirmed, with R 2 of 0.99. Figure 3 shows the results for positive deformations, and Figure 4 shows the results for negative ones. Thus, the initial assessment was confirmed, and the beam load cell could be considered linear in all desired ranges.

Beam Load Cell Hysteresis
In order to assess hysteresis, we applied a sequence of load/unload/load with ea one of the following sequence of masses: M6, M4, M1, M12, M11, M10, M3, M5, and M Figure 5 shows that the hysteresis was negligible.

Beam Load Cell Hysteresis
In order to assess hysteresis, we applied a sequence of load/unload/load with each one of the following sequence of masses: M6, M4, M1, M12, M11, M10, M3, M5, and M2. Figure 5 shows that the hysteresis was negligible.
Machines 2021, 9, x FOR PEER REVIEW 7 o Figure 5. Beam load cell hysteresis behavior.

Beam Load Cell Drift
Drift was assumed to be negligible for the duration of each run. It was measu twice: first, at the "power on" of the Arduino board and at the establishment of comm nication and, second, after 3 h of uninterrupted work. In both cases, drift was measu during the subsequent 2 h period. Figure 6 shows the results obtained.

Beam Load Cell Drift
Drift was assumed to be negligible for the duration of each run. It was measured twice: first, at the "power on" of the Arduino board and at the establishment of communication and, second, after 3 h of uninterrupted work. In both cases, drift was measured during the subsequent 2 h period. Figure 6 shows the results obtained.

Beam Load Cell Drift
Drift was assumed to be negligible for the duration of each run. It was meas twice: first, at the "power on" of the Arduino board and at the establishment of com nication and, second, after 3 h of uninterrupted work. In both cases, drift was meas during the subsequent 2 h period. Figure 6 shows the results obtained. From "power on", measurements remained stable for the period of 2 h and coul considered zero. After 3 h of use, the measured values kept rising at a rate of 12 g/h.
Thus, considering that the range of the expected force values ranged from −1 k to 5 kg and that the total time of each run reached a maximum of 1 min, the initia sumption was confirmed, and drift was considered negligible for the duration of each From "power on", measurements remained stable for the period of 2 h and could be considered zero. After 3 h of use, the measured values kept rising at a rate of 12 g/h.
Thus, considering that the range of the expected force values ranged from −1 kg up to 5 kg and that the total time of each run reached a maximum of 1 min, the initial assumption was confirmed, and drift was considered negligible for the duration of each run.

Beam Load Cell Interferences
Interference due to different sources was assumed significant. The following five cases were analyzed: For each of them, a series of measurements were performed, and the maximum value was taken as the worst possible error. In all cases, the moving arms were placed in position zero, which pointed downward. Each test was carried out as follows:

1.
Effect of deflection of the table on which the workbench was laid: The reason for this test was that sometimes, it was necessary for the user to use the table as support for an activity. In this case, a mass of~70 kg was placed during the measurements on the center of the largest side of the mass of~1 kg (mass 11) was released from a height of 10 mm on the top surface of the actuated arm, precisely over the axis (not a direct hit, however). A change of 0.05 kg was detected in the measurement.

5.
Effect of horizontal vibration of the moving arm: The reason for this test was to check if vibrations due to resonance on the arm could affect the measurement. In this case, with help of a wire, a mass of~1 kg (mass 11) was released from a height of 10 mm on the side surface of the actuated arm, precisely on the side of the axis (not a direct hit, however). A change of 0.03 kg was detected in the measurement. Table 3 summarizes the results of the tests. During normal operation, the initial assumption was refuted, and no interference was expected in the tests. However, in the case in which resonance was identified, data either required additional filtering or had to be acquired under different conditions. In this particular case, speed was limited to 18 • /s due to resonance at higher speeds.

Testing Procedures
Two types of tests were performed: static and dynamic. In both tests, angular zero was defined as the position in which the arms were pointing downward, in agreement with the anatomical reference position. Angles increased as the arms were raised, corresponding to an increase in shoulder flexion up to a maximum of 180  In the static tests, a set of 10 consecutive measurements were taken from 180° up to −20°, in steps of 8°. At every step, the machine remained still for 500 ms for mechanical stabilization. To allow for compensation of the position and masses of the test bench, three tests without exoskeleton were carried out to measure the background forces. The background force at a certain angle was the average of these 30 measurements, after outlier removal. This value was subtracted from the actual measurements. In the static tests, a set of 10 consecutive measurements were taken from 180 • up to −20 • , in steps of 8 • . At every step, the machine remained still for 500 ms for mechanical stabilization. To allow for compensation of the position and masses of the test bench, three tests without exoskeleton were carried out to measure the background forces. The background force at a certain angle was the average of these 30 measurements, after outlier removal. This value was subtracted from the actual measurements.
In the exoskeleton MATE (v1.0, Comau, Turin, Italy) used in this study, similar to most of the commercially available exoskeletons, it is possible to select different values of spring stiffness that provide different supporting torques. For the exoskeleton measurements, three runs were performed for each one of the seven spring levels available in MATE. The torque adjustment in MATE was performed with a mechanical switch with seven positions that indicate the level of strength. Level 1 is the weakest (lowest stiffness), and Level 7 is the strongest (highest stiffness). The value of the point of interest was the average of these 30 measurements, after removing the outliers.
In dynamic tests, three cycles of consecutive measurements were taken from 180 • up to −20 • , in steps of 2 • . The velocity of movement was 18 • /s, to avoid vibration due to both step transitions at lower speeds and resonance at higher speeds. A single measurement was taken from each angle during a single run.
Background measurement was taken as a set of three runs without any exoskeleton attached to the test bench. The value of the point of interest was the average of the three measurements, after outlier removal.
Three runs were carried out for each level of the spring. The value of the point of interest was the average of the three measurements, after outlier removal.

Results
In the next sections, static and dynamic results are presented.

Static Tests
The torques obtained in the static measurements for each spring level are presented in Figure 8.

Static Tests
The torques obtained in the static measurements for each spring level are presented in Figure 8. All spring levels started at zero in position 180°. Actuation increased up to a maximum of around 45°, and then, they decreased again until position 0°. From 0° to −20°, the torque remained constant between −2 Nm and −3 Nm. This behavior between 0° and −20° did not correspond to the expected traction forces, which is discussed in the next section.
The maximum torque in position 45° was 5.6 Nm. All spring levels started at zero in position 180 • . Actuation increased up to a maximum of around 45 • , and then, they decreased again until position 0 • . From 0 • to −20 • , the torque remained constant between −2 Nm and −3 Nm. This behavior between 0 • and −20 • did not correspond to the expected traction forces, which is discussed in the next section.
The maximum torque in position 45 • was 5.6 Nm.

Dynamic Tests
The torques obtained during the dynamic tests are shown in Figure 9.
Machines 2021, 9, x FOR PEER REVIEW 11 of 13 Figure 9. Computed dynamic torques with the testbench configuration (at one extreme of the shoulder translation joint) for each of the seven spring levels in MATE. Level 7 is the strongest one, and level 1 is the weakest.

Discussion
The results showed that it is possible to use our test bench to measure the performance of industrial exoskeletons under repeatable conditions. The test bench is open access, it mostly requires commercial off-the-shelf components, and the parts that need some custom fabrication are simple, making it low cost (less than 800 USD), with easy implementation.
In the tests, it was found that dynamic measurements were increased in torque when compared with static measurements. That was expected since, in dynamic measurements, inertia and other forces also contribute to the actual measured values. This is relevant because it is not possible to compare two tests carried out under different dynamic conditions, while our device can provide the same testing conditions. There were small variations in the torques measured in both tests. It is possible that those variations are due to the sliding of the biceps strap during movement. This underscores one of the contributions of this study: the torques effectively applied on the user's arm were not necessarily the torques measured directly in the exoskeleton because of the relative movement of the equipment to the trunk and the biceps.
The main advantage of using a standardized test that considers effectively transmitted torques for the exoskeletons is the possibility of comparison under the same conditions. In this way, it is possible to obtain the characteristics of each exoskeleton and the specific angular positions in which it provides the higher torque levels. This information is critical to adapt the exoskeleton to the task and the user. For instance, if the task requires a certain shoulder angle, and the exoskeleton is not sufficiently assisting the user, it is possible to increase the torque level. It is also possible that the exoskeleton is providing an excessive torque in a certain position that demands extra user effort to maintain that position. Finally, the exoskeleton angle-torque curves provided by our test bench can be very useful to plan the introduction of exoskeletons in precision tasks: if the torque changes abruptly between two angular positions, this would be perceived by the user as All spring levels started at zero in position 180 • . Actuation increased up to a maximum between 20 • and 30 • , decreasing until position −20 • . The behavior between 0 • and −20 • did not correspond to the expected traction forces and, as in the case of the static measurements, it is discussed in the next section.

Discussion
The results showed that it is possible to use our test bench to measure the performance of industrial exoskeletons under repeatable conditions. The test bench is open access, it mostly requires commercial off-the-shelf components, and the parts that need some custom fabrication are simple, making it low cost (less than 800 USD), with easy implementation.
In the tests, it was found that dynamic measurements were increased in torque when compared with static measurements. That was expected since, in dynamic measurements, inertia and other forces also contribute to the actual measured values. This is relevant because it is not possible to compare two tests carried out under different dynamic conditions, while our device can provide the same testing conditions. There were small variations in the torques measured in both tests. It is possible that those variations are due to the sliding of the biceps strap during movement. This underscores one of the contributions of this study: the torques effectively applied on the user's arm were not necessarily the torques measured directly in the exoskeleton because of the relative movement of the equipment to the trunk and the biceps.
The main advantage of using a standardized test that considers effectively transmitted torques for the exoskeletons is the possibility of comparison under the same conditions. In this way, it is possible to obtain the characteristics of each exoskeleton and the specific angular positions in which it provides the higher torque levels. This information is critical to adapt the exoskeleton to the task and the user. For instance, if the task requires a certain shoulder angle, and the exoskeleton is not sufficiently assisting the user, it is possible to increase the torque level. It is also possible that the exoskeleton is providing an excessive torque in a certain position that demands extra user effort to maintain that position. Finally, the exoskeleton angle-torque curves provided by our test bench can be very useful to plan the introduction of exoskeletons in precision tasks: if the torque changes abruptly between two angular positions, this would be perceived by the user as a perturbation that may hinder precision performance.
One of the main aspects in the design of a test bench for industrial exoskeletons is the generalization of the attachment, because there are several exoskeletons in the market, and each one has its unique characteristics. In this study, we used MATE (v1.0, Comau, Turin, Italy) to validate the test bench. This exoskeleton fitted very easily for the test bench, which was designed to simulate a human arm. However, it is possible that other exoskeleton models may need further adaptations of the test bench.
It must be noted that the results for the static and dynamic tests show a compression force between 0 • and −20 • . However, when using MATE, it is possible to feel that the spring changed its actuation direction, which should have been detected by the test bench by a transition to positive values. The reason for this behavior within the machine is believed to be due to the way the exoskeleton was fixed for testing. In the human body, that range of movement is followed by a slight torsion of the trunk and slight sliding of the exoskeleton, which was not reproduced in the machine. Thus, from 0 • to −20 • , the exoskeleton started to offer resistance to the movement, which was registered as a compression force in that region, not as a traction force, as expected.
It is important to keep in mind that the goal of this research was to evaluate the test bench, and not the exoskeleton itself. Thus, there might be differences between the values presented here and other studies about the same exoskeleton, due to differences in the testing conditions. In this work, the shoulder translation joint was at one extreme of its range of motion. This underscores the need for testing devices that provide homogeneous testing conditions for benchmarking.
Actuation can be further improved to allow better dynamic response, and additional sensors, e.g., inertial or pressure sensors, could provide more information about the tests, such as the vibration during exoskeleton use or the pressure at the contact surfaces between the exoskeleton and the user.
It is also possible to include additional fixation points to measure a variety of exoskeletons with different sizes while avoiding extreme joint positions along with improving the beam load cell g remodeling of the connections with the machine.
Finally, we presented the design and validation of a test bench for industrial exoskeletons. This device allows for the implementation of standardized tests that can be useful to assess objectively one exoskeleton allowing the comparison between different exoskeletons. Moreover, the results from the test bench combined with measurements with subjects, considering physiology, biomechanics, and user's perception, would allow designing and/or choosing the best exoskeleton for a certain user and a given task.

Data Availability Statement:
The data presented in this study are available on request from the corresponding author.