Joint Reconfiguration after Failure for Performing Emblematic Gestures in Humanoid Receptionist Robot

This study proposed a strategy for a quick fault recovery response when an actuator failure problem occurred while a humanoid robot with 7-DOF anthropomorphic arms was performing a task with upper body motion. The objective of this study was to develop an algorithm for joint reconfiguration of the receptionist robot called Namo so that the robot can still perform a set of emblematic gestures if an actuator fails or is damaged. We proposed a gesture similarity measurement to be used as an objective function and used bio-inspired artificial intelligence methods, including a genetic algorithm, a bacteria foraging optimization algorithm, and an artificial bee colony, to determine good solutions for joint reconfiguration. When an actuator fails, the failed joint will be locked at the average angle calculated from all emblematic gestures. We used grid search to determine suitable parameter sets for each method before making a comparison of their performance. The results showed that bio-inspired artificial intelligence methods could successfully suggest reconfigured gestures after joint motor failure within 1 s. After 100 repetitions, BFOA and ABC returned the best-reconfigured gestures; there was no statistical difference. However, ABC yielded more reliable reconfigured gestures; there was significantly less interquartile range among the results than BFOA. The joint reconfiguration method was demonstrated for all possible joint failure conditions. The results showed that the proposed method could determine good reconfigured gestures under given time constraints; hence, it could be used for joint failure recovery in real applications.


Introduction
Recently, a greater number of social robots have been introduced to the service industry.Anthropomorphic or humanoid social robots were designed and developed to attract special attention from human users.An anthropomorphic robot arm allows design flexible movement that can be easily adapted to various situations.A human arm can be modeled as a 7-DOF kinematic model [1]; thus, an anthropomorphic robot arm should be at least 7-DOF to be able to mimic human-like motion.An anthropomorphic robot arm that has 7-DOF would have redundancy to perform many tasks similar to humans.However, since high numbers of actuators are required for an anthropomorphic 7-DOF arm design, most humanoid robots [2] have only 6-DOF or less-than-6-DOF arms.
Since 2010, the humanoid receptionist robot called Namo, which was developed by the Institute of Field Robotics, KMUTT, has provided services for different social functions such as opening ceremonies, fashion shows, etc. [3].Namo has two 7-DOF arms, which were designed to mimic human arms so that the robot can perform human-like upper-body movements.Over the past few years, the reliability of the robot has become a critical issue.When only one of the actuators was malfunctioning, the robot was not able to perform the tasks at hand, which caused serious problems, especially in time-critical situations.This issue raised a research question on how to allow high-degree-of-freedom humanoid robots to continue their operation even when some actuators malfunction or are damaged, especially with anthropomorphic 7-DOF arms that have redundancy in their design.The process that allows the robot system to generate a new motion trajectory for completing the tasks with the remaining actuators is called joint failure recovery.
One way to handle the problem of some actuators malfunctioning is designing and developing robots with redundancy mechanisms [4][5][6][7][8]; redundant robots are designed to have more degrees of freedom than required to perform their tasks.These redundant joints can be used to replace the failed joints and allow the robot to complete its tasks under the joint failure condition in many different ways.For example, Lewis and Maciejewski [9] used a 6 DOFs joint space PUMA560 robot manipulator for the end-effector, which requires 3 DOFs task space, and defined inverse kinematic joint velocities redistribution for the redundant joints after a joint failed and locked at a position.Elsayed et al. [10] proposed a differential-kinematics-based method to control the movement of neighbor joints of the failed joint in a snake robot in order to recover from the failure to regain the snake-like shape and resume locomotion.Chen et al. [11] proposed an optimization method to optimally control the velocity and torque of remaining joints after a locked-joint failure occurred in order to reduce the joint parameter jump of a space manipulator when carrying a load.
The concept of using redundancy in robots became commonly applied.Instead of using complex computation to reconfigure the remaining joints, another approach to handling the failure recovery problem is using soft computing techniques [12,13].Nature provides several good examples of using adaptation to solve a problem.Animals can adapt to their injuries; although adapted gait may not be the same as normal gait, the animals can still move from one place to another.Cully et al. [14] were inspired by those animals and proposed a trial-and-error learning algorithm from the robot's previous performance before damage occurred.Cully et al. demonstrated the algorithm performance on a hexapod robot adapting to recover from damaged legs in fire damage scenarios; the robot could adapt within two minutes.Dereli and Köker [15] used a quantum-behaved particle swarm optimization algorithm (Quantum PSO) to solve the inverse kinematic of a 7 DOFs serial robot manipulator.Compared with other swarm intelligence techniques, including particle swarm optimization (PSO), firefly algorithm (FA), and artificial bee colony (ABC), Dereli and Köker found that the estimated position of the end effector by Quantum PSO was more accurate and required less computation time.There are research studies that use different bio-inspired artificial intelligence methods for solving inverse kinematics of the 7-DOF robotics manipulator, including the slime mold algorithm [16], boundary-restricted PSO [17], firefly algorithm [18], and artificial bee colony [19].
From the literature, the strategy for robots to recover from joint failure is using redundant design and then performing computational reconfiguration of the remaining functional joints to cope with the failed joint.Inverse kinematics [20,21], quadratic programming algorithms [22], and optimization methods [11,23] are often used for joint reconfiguration; however, these methods require complex computations, which could lead to high computation resources and be time-consuming when used in real situations.The approach using metaheuristic methods or soft computing algorithms could help reduce the computational complexity and determine a good joint-reconfiguration solution for redundant robots within an acceptable computation time, as shown in [15].Unlike in industrial robot manipulators, joint failure recovery of a humanoid receptionist robot must preserve the robot's communication ability by its basic gestures.Gestures are a part of non-verbal communication that are important for human-robot interactive communication [24].In this study, we proposed the use of bio-inspired artificial intelligence algorithms for the failure recovery of a service semi-humanoid robot with redundant manipulators to maintain its emblematic gestures within a given reconfiguration time constraint.

Namo Robot
Namo is a semi-humanoid receptionist robot, as shown in Figure 1.Namo's head can move with three degrees of freedom.Namo has two anthropomorphic arms designed to mimic human arms so that most emblematic gestures can be naturally performed by the robot.With these two arms, Namo can perform 13 pre-defined gestures.In this study, we used the 4 main gestures for her normal operations as a receptionist robot.Figure 1 illustrates Namo's 4 main emblematic gestures, including Wai (or Thai greeting), Bye, Salute, and Side Invite.We paid particular attention to the Wai gesture as it is the traditional greeting of Thailand and an important part of Thai social behavior.
of non-verbal communication that are important for human-robot interactive communication [24].In this study, we proposed the use of bio-inspired artificial intelligence algorithms for the failure recovery of a service semi-humanoid robot with redundant manipulators to maintain its emblematic gestures within a given reconfiguration time constraint.

Namo Robot
Namo is a semi-humanoid receptionist robot, as shown in Figure 1.Namo's head can move with three degrees of freedom.Namo has two anthropomorphic arms designed to mimic human arms so that most emblematic gestures can be naturally performed by the robot.With these two arms, Namo can perform 13 pre-defined gestures.In this study, we used the 4 main gestures for her normal operations as a receptionist robot.Figure 1 illustrates Namo's 4 main emblematic gestures, including Wai (or Thai greeting), Bye, Salute, and Side Invite.We paid particular attention to the Wai gesture as it is the traditional greeting of Thailand and an important part of Thai social behavior.Each arm has seven degrees of freedom.The actuators are located on Namo's arm, as illustrated in Figure 2a.The forward kinematics of Namo's arms were analyzed with the Denavit-Hartenberg Convention [25].Figure 2b illustrates the kinematic chains with Denavit-Hartenberg frames of Namo's right arm as an example.The parameter setting for the Denavit-Hartenberg convention of Namo's arm is shown in Table 1.Each arm has seven degrees of freedom.The actuators are located on Namo's arm, as illustrated in Figure 2a.The forward kinematics of Namo's arms were analyzed with the Denavit-Hartenberg Convention [25].Figure 2b illustrates the kinematic chains with Denavit-Hartenberg frames of Namo's right arm as an example.The parameter setting for the Denavit-Hartenberg convention of Namo's arm is shown in Table 1.

Gesture Similarity Measurement
Unlike joint failure recovery for robot manipulators, joint failure recovery for humanoid receptionist robots requires not only the correct end-effector position but the overall robot arms' positions must also represent the correct emblematic gestures; the output combination of elbow, wrist, and palm positions should be perceived by humans as emblematic gestures.We proposed a gesture similarity measurement to be the measure for evaluating the similarity between the new joint configuration and the original configuration for each gesture.The gesture similarity score consists of four weighted terms from four components of the arm, including the elbow, wrist, fingertip, and palm orientation, as in score = (welbow•delbow) + (wwrist•dwrist) + (wtip•dtip) + (ws•ds), (1) where d is the Euclidean distance between the 3D space position of the reconfigured joint angle and that of the reference joint angle of the gesture.Greater distance expresses less similarity to the reference gesture.The Euclidean distance is used for the elbow, wrist, and fingertip components.Figure 3b illustrates the frame-based model of Namo's right arm in simulation, as shown in Figure 3a; the reconfigured gesture of Namo's right arm is shown in orange, while the reference gesture is shown in blue.The orange dotted line denotes the distance vector between the joint position of the reconfigured and reference gesture at the same component.
For the hand orientation, we used the distance between rotation quaternions instead.Assuming that the reconfigured wrist is at the same position as the reference wrist, as illustrated in Figure 3c, we calculated the different angles from the distance between the rotation quaternion of the reconfigured hand, p, and that of the reference hand, q.Then, ds is the angular distance between the reconfigured hand orientation and the reference hand orientation.
For Wai, both of Namo's arms need to be in the correct position in order to represent the Thai greeting properly.Therefore, we added two more components in (1) to include the distance between the wrist of the left and right hands, dwrist_LR, and the distance between the fingertip of the left and right hands, dtip_LR, in the gesture similarity measurement, as

Gesture Similarity Measurement
Unlike joint failure recovery for robot manipulators, joint failure recovery for humanoid receptionist robots requires not only the correct end-effector position but the overall robot arms' positions must also represent the correct emblematic gestures; the output combination of elbow, wrist, and palm positions should be perceived by humans as emblematic gestures.We proposed a gesture similarity measurement to be the measure for evaluating the similarity between the new joint configuration and the original configuration for each gesture.The gesture similarity score consists of four weighted terms from four components of the arm, including the elbow, wrist, fingertip, and palm orientation, as in score = (w elbow where d is the Euclidean distance between the 3D space position of the reconfigured joint angle and that of the reference joint angle of the gesture.Greater distance expresses less similarity to the reference gesture.The Euclidean distance is used for the elbow, wrist, and fingertip components.Figure 3b illustrates the frame-based model of Namo's right arm in simulation, as shown in Figure 3a; the reconfigured gesture of Namo's right arm is shown in orange, while the reference gesture is shown in blue.The orange dotted line denotes the distance vector between the joint position of the reconfigured and reference gesture at the same component.
For the hand orientation, we used the distance between rotation quaternions instead.Assuming that the reconfigured wrist is at the same position as the reference wrist, as illustrated in Figure 3c, we calculated the different angles from the distance between the rotation quaternion of the reconfigured hand, p, and that of the reference hand, q.Then, d s is the angular distance between the reconfigured hand orientation and the reference hand orientation.
For Wai, both of Namo's arms need to be in the correct position in order to represent the Thai greeting properly.Therefore, we added two more components in (1) to include the distance between the wrist of the left and right hands, d wrist_LR , and the distance between the fingertip of the left and right hands, d tip_LR , in the gesture similarity measurement, as where d offset is the offset value for the thickness of the hand.In this study, the weight values for ( 1) and ( 2 where doffset is the offset value for the thickness of the hand.In this study, the weight values for ( 1) and ( 2) were set as equal weight values for all components.

Bio-Inspired Joint Reconfiguration Method for Failure Recovery
To solve the problem of joint failure recovery for humanoid receptionist robots, the method should find the reconfigured joint angles of the remaining joints that allow the robots to perform emblematic gestures; the output gestures may not be completely the same as the reference gestures but still communicate the meanings of those gestures.Similar to the way that animals adapt themselves to walk after a leg injury, bio-inspired artificial intelligence methods simulate self-organized mechanisms occurring in nature and use them to solve complex optimization problems [26].In this study, we proposed the use of bio-inspired artificial intelligence methods for the joint reconfiguration of humanoid receptionist robots when one reference joint angle set for each emblematic gesture is available to the robots.
Once that joint failure occurs, the robot cannot control the failed joint to a desired angle.In this study, we proposed to lock the joint at a specific angle.We determined the locked angle by averaging the joint angles of that failed joint from all reference emblematic gesture sets; the average joint angle could potentially be a less restrictive constraint to the robot system to regenerate a good gesture reconfiguration of the remaining joints for all emblematic gestures.Under the constraint that the joint angle of the failed joint is locked at a specific angle, the bio-inspired artificial intelligence methods, e.g., genetic algorithm (GA), bacteria foraging optimization algorithm (BFOA), and artificial bee colony (ABC), are used as optimizers to generate joint angle sets of the remaining joints and determine a good joint reconfiguration solution that minimizes the difference between the reconfigured gestures and the reference gestures.The similarity to the reference gesture set is the objective function for the joint failure recovery problem; the greater similarity indicates the better joint reconfigured solution.We used the proposed gesture similarity measurement in (1) as the fitness evaluation of the generated joint reconfiguration.After running iterations of bio-inspired artificial intelligence methods, the reconfigured gesture sets were obtained.Figure 4 illustrates the process of the gesture reconfiguration methods

Bio-Inspired Joint Reconfiguration Method for Failure Recovery
To solve the problem of joint failure recovery for humanoid receptionist robots, the method should find the reconfigured joint angles of the remaining joints that allow the robots to perform emblematic gestures; the output gestures may not be completely the same as the reference gestures but still communicate the meanings of those gestures.Similar to the way that animals adapt themselves to walk after a leg injury, bio-inspired artificial intelligence methods simulate self-organized mechanisms occurring in nature and use them to solve complex optimization problems [26].In this study, we proposed the use of bio-inspired artificial intelligence methods for the joint reconfiguration of humanoid receptionist robots when one reference joint angle set for each emblematic gesture is available to the robots.
Once that joint failure occurs, the robot cannot control the failed joint to a desired angle.In this study, we proposed to lock the joint at a specific angle.We determined the locked angle by averaging the joint angles of that failed joint from all reference emblematic gesture sets; the average joint angle could potentially be a less restrictive constraint to the robot system to regenerate a good gesture reconfiguration of the remaining joints for all emblematic gestures.Under the constraint that the joint angle of the failed joint is locked at a specific angle, the bio-inspired artificial intelligence methods, e.g., genetic algorithm (GA), bacteria foraging optimization algorithm (BFOA), and artificial bee colony (ABC), are used as optimizers to generate joint angle sets of the remaining joints and determine a good joint reconfiguration solution that minimizes the difference between the reconfigured gestures and the reference gestures.The similarity to the reference gesture set is the objective function for the joint failure recovery problem; the greater similarity indicates the better joint reconfigured solution.We used the proposed gesture similarity measurement in (1) as the fitness evaluation of the generated joint reconfiguration.After running iterations of bio-inspired artificial intelligence methods, the reconfigured gesture sets were obtained.Figure 4 illustrates the process of the gesture reconfiguration methods with single reference sets.Joint reconfiguration can be conducted separately for each emblematic gesture.

Performance Analysis
We demonstrated the performance of the bio-inspired gesture reconfiguration method with three techniques: GA, BFOA, and ABC.We explored the differences in using different techniques for the same failure recovery problem on the Namo robot.The locked joint angle for each of the joints that are averaged from the reference gesture can be expressed in Table 2.Because the performance of each algorithm depends on how the solution representation and parameter setting are set appropriately to the problem, the parameter tuning is performed using a grid search to determine the most appropriate model parameter set for each algorithm before making a comparison of the performance between methods.The grid search was carried out for the Wai gesture, which is the most complex gesture among other emblematic gestures.The tuned parameter sets given the maximum similarity gesture were then used in joint reconfiguration when each joint failed to function.The experiment setup for these algorithms is explained as follows.

Performance Analysis
We demonstrated the performance of the bio-inspired gesture reconfiguration method with three techniques: GA, BFOA, and ABC.We explored the differences in using different techniques for the same failure recovery problem on the Namo robot.The locked joint angle for each of the joints that are averaged from the reference gesture can be expressed in Table 2.Because the performance of each algorithm depends on how the solution representation and parameter setting are set appropriately to the problem, the parameter tuning is performed using a grid search to determine the most appropriate model parameter set for each algorithm before making a comparison of the performance between methods.The grid search was carried out for the Wai gesture, which is the most complex gesture among other emblematic gestures.The tuned parameter sets given the maximum similarity gesture were then used in joint reconfiguration when each joint failed to function.The experiment setup for these algorithms is explained as follows.

Genetic Algorithm
The GA was inspired by the evolution of life in nature.Each chromosome has genotypes that can be expressed to the phenotypes of the lifeform.Through natural selection among the population of lifeforms, better individuals can exist, mate, and reproduce new offspring.The chromosomes of the new offspring can inherit some of their parents' genotypes through the crossover process and may have some deviated genotypes from the parents through the mutation process.With a number of generation cycles, it is possible that the offspring from the later generation can evolve to survive in the environment better than their ancestors.GA simulates such an evolution process and uses the process to solve many complex problems [27][28][29][30].Further details about GA can be found in [27].
In this study, the genetic representation of the chromosome is set as a sequence of 6 real-valued numbers, which correspond to the 6 remaining joint angles to be reconfigured.A population of chromosomes was randomly generated with uniform distribution.The parents for crossover operations were chosen at random.We chose a one-point crossover for this problem.The population in the next generation was reproduced by generation rollover, where the new, better offspring replaced the worst individuals.The key parameters that regulate the evolution process for GA include the population size, crossover rate, mutation rate, mutation step size, and the maximum number of generation cycles.

Bacteria Foraging Optimization Algorithm
BFOA was inspired by the foraging behavior of E. coli and M. xanthus bacteria [31].With chemotaxis, which is the movement based on their senses of the changing concentration of nutrients in the environment, bacteria can move toward the food location; additionally, they can release chemical substances to attract and recruit other bacteria in the area to the location as well.These bacteria move with the flagella that rotate to create force and push them in one direction [32].They can perform two movement patterns: swimming and tumbling.When they sense the target, they move with the swimming pattern toward the target.The tumbling pattern allows the bacteria to move in a random direction.Bacteria use these two movement patterns in their foraging to increase the foraging area and to collaboratively find their food; many studies [33,34] have successfully used BFOA to solve complex problems.Further details about BFOA can be found in [27].
Similar to the GA algorithm, each bacterium is a sequence of 6 real-valued numbers.The control parameters for BFOA include the population size, the number of chemotactic steps, the swimming length, the number of swim steps, the number of elimination-dispersal events, the number of reproduction steps, and the probability of elimination.Additionally, the control parameters for the swarming dynamics of the bacteria population that should be set appropriately to the application include the depth of attractant released by the cell, the width of the attractant signal, the height of the repellant effect, and the width of the repellant effect.

Artificial Bee Colony Algorithm
ABC was inspired by the foraging behavior of honey bees [35].During foraging, bees can be in three statuses.The bees that find food sources are called employed bees.Onlooker bees are those staying at the hive and waiting for information.The employed bees fly back to the hive and share information about the quality and location of the food sources they found through waggle dances.Scout bees are bees that explore a new food source.ABC simulates the bee foraging behavior with these three phases to find a good solution for complex problems in many research areas [35,36].Further details about ABC can be found in [37].
The dimension of the search space in this problem is 6 according to the 6 remaining joint angles to be reconfigured.Each employed bee represents the solution or food source location in the search space.The best solution has the minimum difference from the reference gesture.The control parameters for ABC include the number of employed bees, the limit number that employed bees will search until abandoning the area to explore other areas in the search space, and the maximum number of cycles.Note that ABC uses the same number of employed bees for onlooker bees at the hive.While employed bees exploit the food source area that they found, onlooker bees can be recruited to explore the adjacent area from the food source the employed found.

Results and Discussion
In this study, we used bio-inspired artificial intelligence algorithms, including GA, BFOA, and ABC, for joint reconfiguration after a joint failure of a service semi-humanoid robot with redundant manipulators to maintain its emblematic gestures.To be used instead of using complex, time-consuming computation of inverse kinematics, the bio-inspired joint reconfiguration results must be within the acceptable difference from the reference gestures; in addition, the computation time must be low enough to allow quick maintenance so that the service robot could continue her duty.All the experiments were run on a computer with a 12th Gen Intel (R) Core (TM) i7-12700K (5 GHz) with 64 GB RAM, running on Ubuntu 22.04 with Visual Studio Code IDE, implemented with Python 3.9 and NumPy 1.20.3.

Parameter Set Tuning for Optimal Solution
We performed a grid search on varying control parameter values of each algorithm on the Wai gesture in order to determine a good combination of parameters that yields the minimum gesture difference from the reference gesture within 1 s of computation time for joint reconfiguration.We conducted an experiment in the case where Joint 2 failed as an example in this study.The tuned parameter sets given the maximum similarity gesture for GA, BFOA, and ABC are reported in Table 3.ABC returned the best joint reconfiguration solution that had minimum difference from the reference gesture, while the best solution from GA had the largest difference from the reference gesture.The best joint reconfiguration solution from ABC had a similar difference value to BFOA but took a little less computation time than BFOA.Although the population size for ABC seems smaller than BFOA, as shown in Table 3, which may be the reason for the lower computation time, ABC uses the same number for both employed bees and onlooker bees, so the actual population size is the same as BFOA.In terms of ease of use, ABC has a smaller number of control parameters than BFOA; hence, it seems more convenient to apply ABC in real joint reconfiguration applications.We used the tuned parameter sets of each algorithm for emblematic gesture reconfiguration on the Namo robot when Joint 2 failed to function.The experiment was repeated 100 times in order to test if the tuned parameter sets could be used for other emblematic gestures.The resulting gesture similarity measurement to the reference gestures and resulting computation time are reported in Tables 4 and 5, respectively.The tuned parameter sets from performing a grid search on joint reconfiguration with the Wai gesture can be used for other gestures as well.Using the Kruskal-Wallis test, it can be concluded that using different algorithms demonstrated significantly different effects on gesture similarity and computation time for joint reconfiguration.Compared to other algorithms, after 100 repetitions, ABC resulted in the best values in many aspects.ABC yielded lower difference values from the reference gestures, except for the lowest difference in the Salute gesture; however, the values are not significantly different.ABC produced more reliable results, as shown in Table 4, with the lowest SD and interquartile range for all gestures.When comparing two unmatched algorithms' solutions with the Mann-Whitney test, it was found that most of the algorithms' solutions are distinctive from each other in terms of gesture similarity (p = 0.000), except for the Wai gesture for which BFOA and ABC did not show the difference (p = 0.072).On the other hand, the Mann-Whitney test on computation time suggested that all algorithms' solutions returned with distinct computation time from other algorithms' solutions (p = 0.000), except for the Salute and Side Invite gestures for which BFOA and ABC did not show the difference (p = 0.274 and p = 0.642, respectively).BFOA used the lower computation time to yield the solutions in most cases, as shown in Table 4. Nevertheless, the primary concern in choosing the algorithm for real application is the quality of the solution; the computation time is the secondary concern.Hence, the results suggested that it is possible to use ABC with the tuned parameter set for joint reconfiguration after failure.
Figure 5 illustrates the best output gestures of the four main emblematic gestures of the Namo robot in simulation from GA, BFOA, and ABC in comparison with the reference gestures.Although the output gestures from ABC are most similar to the reference gestures, the differences are not much and cannot be visually perceivable.In Table 4, the minimum difference values for the Wai and Salute gestures are higher than other gestures.From Figure 5, the output gestures from all algorithms are the same.This may be because the locked joint angle of the failed joint limits the possibility of obtaining better-reconfigured gestures.
As gradient-based optimization algorithms are also found to be efficient for solving inverse kinematics problems, we verified that the chosen optimization methods could perform better than gradient-based algorithms.We compared the performance of emblematic gesture reconfiguration when Joint 2 failed to function for the Wai gesture between using the gradient descent optimization algorithm with momentum [38] and the ABC algorithm.The tuned parameter set of the ABC algorithm used in the comparison is shown in Table 3.For the gradient descent algorithm, the decay rate was set as 0.9.The learning rate was set as 0.1 and 0.05 for comparison.Both algorithms were run with random initial seeds for 1000 iterations.Figure 6 illustrates the gesture similarity scores from gradient descent (GD) and ABC algorithms over all iterations in comparison.The best gesture similarity scores from the gradient descent algorithm with 0.1 and 0.05 learning rates are 113.95 and 112.83 mm, respectively.The best gesture similarity score from ABC is 57.16 mm.The time spent to complete 1000 iterations are 1.3550, 1.3483, and 1.0740 s for GD_0.1, GD_0.05, and ABC, respectively.Figure 6 shows that ABC can quickly converge to the solution with a better gesture similarity score than the gradient descent algorithms.The performance of gradient-based algorithms depends on the initial seed; it could lead to a local optimum.Many of the bio-inspired algorithms, including ABC, utilized a population of agents to search for the solutions collectively, so they have a greater chance of finding global optimal solutions.

Joint Reconfiguration for All Possible Joint Failures
It is interesting to investigate whether the tuned parameter sets from one scenario can be extended to use in other joint failure scenarios.In this experiment, we used the tuned parameter sets from Joint 2 failure to reconfigure joint angles after the failure of all possible joints.Table 6 shows the comparison of algorithm performance in terms of the minimum difference values of the output gestures from the reference gestures, the computation time used for obtaining the best joint reconfiguration, the median difference values, and the interquartile range from 100 repetitions.The best values from the comparison are shown in bold.As gradient-based optimization algorithms are also found to be efficient for solving inverse kinematics problems, we verified that the chosen optimization methods could perform better than gradient-based algorithms.We compared the performance of emblematic gesture reconfiguration when Joint 2 failed to function for the Wai gesture between using the gradient descent optimization algorithm with momentum [38] and the ABC algorithm.The tuned parameter set of the ABC algorithm used in the comparison is shown in Table 3.For the gradient descent algorithm, the decay rate was set as 0.9.The learning rate was set as 0.1 and 0.05 for comparison.Both algorithms were run with random initial seeds for 1000 iterations.Figure 6 illustrates the gesture similarity scores from gradient descent (GD) and ABC algorithms over all iterations in comparison.The best gesture similarity scores from the gradient descent algorithm with 0.1 and 0.05 learning rates are 113.95 and 112.83 mm, respectively.The best gesture similarity score from ABC is 57.16 mm.The time spent to complete 1000 iterations are 1.3550, 1.3483, and 1.0740 s for GD_0.1, GD_0.05, and ABC, respectively.Figure 6 shows that ABC can quickly converge to the solution with a better gesture similarity score than the gradient descent algorithms.The performance of gradient-based algorithms depends on the initial seed; it could lead to a local optimum.Many of the bio-inspired algorithms, including ABC, utilized a population of agents to search for the solutions collectively, so they have a greater chance of finding global optimal solutions.

Joint Reconfiguration for All Possible Joint Failures
It is interesting to investigate whether the tuned parameter sets from one scenario can be extended to use in other joint failure scenarios.In this experiment, we used the tuned parameter sets from Joint 2 failure to reconfigure joint angles after the failure of all possible joints.Table 6 shows the comparison of algorithm performance in terms of the minimum difference values of the output gestures from the reference gestures, the computation time used for obtaining the best joint reconfiguration, the median difference values, and the interquartile range from 100 repetitions.The best values from the comparison are shown in bold.The results showed that the best reconfigurations, which are the reconfiguration with the minimum difference value, from BFOA and ABC, were significantly better than those from GA (Wilcoxon matched-pairs signed-rank test, p = 0.000 and p = 0.000, respectively).Between the BFOA and ABC pair groups, the Wilcoxon matched-pairs signed-rank test did not show significant differences in the minimum difference values (p = 0.399).When considering the algorithm performance on the reliability of the results, the Wilcoxon matched-pairs signed-rank test showed significant differences in median difference values and the interquartile range of resulting difference values between BFOA and ABC (p = 0.000).Hence, it may be concluded that BFOA can provide a chance to yield better joint reconfiguration, but ABC can be relied upon for determining good joint reconfiguration.
From the gesture similarity measure, the failure at one joint position could contribute to the difficulty of joint reconfiguration on each emblematic gesture differently.The best gesture difference values for the Wai gesture in Table 6 ranged the widest among the values from all gestures, from 2.2689 to 93.7706 units.Figure 7 illustrates the best reconfiguration outputs of the Wai gesture as an example.To perform the Wai gesture beautifully, the robot's palms of both hands should be put together at the chest level.From Figure 7, the output reconfigured gestures when Joints 2 and 3 failed to have more errors than in other cases.If we use the three-dimensional Cartesian coordinate system, where the x-axis is in the direction of the robot's face, the y-axis is pointed to the side of the robot, and the z-axis is pointed upward, Joint 2 is responsible for the control of the upper arm up and down to the side of the robot body (or rotating around the x-axis).Joint 3 is responsible for the control of the lower arm from the elbow to open or close (or rotate around the z-axis).When joint 2 or 3 fails and is locked at the averaged angle from all the reference gestures, the space for the robot hands to reach and join together at the chest area is limited.The joint method has attempted to determine a good solution with low gesture difference values, but the locked joint angle at the averaged angle from all reference gestures seemed to limit the search space.In our future study, we plan to include the locked joint angle in the problem so that the joint reconfiguration method can determine a good joint angle to lock the failed joint and may allow better-reconfigured gestures than the output in this study.

The Analysis of Output Gestures through the Gesture Similarity Measurement
Considering the resulting gesture similarity measurement from all possible joint failures, we found that the strategy to lock the failed joint at some specific joint angle had an immense impact on the success of performing the emblematic gestures.The average joint angle from all gestures may not always be suitable.We analyzed the distance of each

The Analysis of Output Gestures through the Gesture Similarity Measurement
Considering the resulting gesture similarity measurement from all possible joint failures, we found that the strategy to lock the failed joint at some specific joint angle had an immense impact on the success of performing the emblematic gestures.The average joint angle from all gestures may not always be suitable.We analyzed the distance of each component of the robot arm in all best-reconfigured gestures when each failed joint occurred.Figure 8 illustrates the boxplot of all distance components in the gesture similarity measurement in (1) and ( 2) for all possible joint failures.Note that the distance components of the elbow, wrist, fingertip, and palm orientation, as shown in Figure 7a-d, are calculated for all emblematic gestures in this study, but the distance components between the left and right hands, as shown in Figure 7e-f, are used for the Wai gesture only.
Joints 1-3 are located in the shoulder area and are designed to control the movement of the robot's upper arm.Joint 1 is responsible for controlling the upper arm to swing forward and backwards.Joint 2 controls the upper arm to swing up and down to the side of the robot.Joint 3 controls the upper arm to rotate inward and outward from the robot's body.When one of these joints fails, it affects all the components along the arm.The results in Table 6 show that the minimum difference values of the reconfigured outputs when the joints in this area failed were greater than those when other joints failed for all gestures.The boxplot in Figure 8 also confirms that when either of the joints in this group failed, the distance values between the reconfigured outputs and reference gestures of all components became high.Among the joints in this group, Joint 3 seemed to cause problems in performing the most emblematic gestures when it failed.Joints 1-3 are located in the shoulder area and are designed to control the movement of the robot's upper arm.Joint 1 is responsible for controlling the upper arm to swing forward and backwards.Joint 2 controls the upper arm to swing up and down to the side of the robot.Joint 3 controls the upper arm to rotate inward and outward from the robot's body.When one of these joints fails, it affects all the components along the arm.The results in Table 6 show that the minimum difference values of the reconfigured outputs when the joints in this area failed were greater than those when other joints failed for all gestures.

Figure 3 .
Figure 3.The gesture similarity measurement: (a) Namo's model in simulation; (b) the frame-based comparison between the reconfigured gesture and the reference gesture of Namo's right arm; (c) the concept for hand orientation similarity measurement.

Figure 3 .
Figure 3.The gesture similarity measurement: (a) Namo's model in simulation; (b) the frame-based comparison between the reconfigured gesture and the reference gesture of Namo's right arm; (c) the concept for hand orientation similarity measurement.
Sensors 2023, 23, x FOR PEER REVIEW 6 of 18with single reference sets.Joint reconfiguration can be conducted separately for each emblematic gesture.

Figure 4 .
Figure 4.The block diagram for the proposed gesture reconfiguration method.

Figure 4 .
Figure 4.The block diagram for the proposed gesture reconfiguration method.

Figure 5 .
Figure 5.The emblematic gestures expressed by the Namo robot in simulation: (a) the reference gestures, (b) the output gestures from GA, (c) the output gestures from BFOA, and (d) the output gestures from ABC.

Figure 5 .
Figure 5.The emblematic gestures expressed by the Namo robot in simulation: (a) the reference gestures, (b) the output gestures from GA, (c) the output gestures from BFOA, and (d) the output gestures from ABC.

Figure 6 .
Figure 6.The comparison between gesture similarity scores from gradient descent (GD) and ABC algorithms over the iterations.

Figure 6 .
Figure 6.The comparison between gesture similarity scores from gradient descent (GD) and ABC algorithms over the iterations.
(a) Distance of elbow (b) Distance of wrist (c) Distance of fingertip (d) Distance of palm orientation (e) Distance between left and right wrists (f) Distance between left and right fingertips

Figure 8 .
Figure 8.The boxplot of the distance components between the best reconfiguration outputs and the reference gestures in all possible joint failures on the Namo robot: (a) distance of the elbow component; (b) distance of the wrist component; (c) distance of the fingertip component; (d) distance of the palm orientation component; (e) distance between the left and right wrists for the Wai gesture; (f) distance between the left and right fingertips for the Wai gesture.

Figure 8 .
Figure 8.The boxplot of the distance components between the best reconfiguration outputs and the reference gestures in all possible joint failures on the Namo robot: (a) distance of the elbow component; (b) distance of the wrist component; (c) distance of the fingertip component; (d) distance of the palm orientation component; (e) distance between the left and right wrists for the Wai gesture; (f) distance between the left and right fingertips for the Wai gesture.

Table 1 .
Parameter setting for the Denavit-Hartenberg convention of Namo's right arm.

Table 1 .
Parameter setting for the Denavit-Hartenberg convention of Namo's right arm.

Table 2 .
The joint angles for all emblematic gestures in this study and the averaged joint angles.

Table 2 .
The joint angles for all emblematic gestures in this study and the averaged joint angles.

Table 3 .
The tuned parameter sets and best joint reconfiguration solutions when joint 2 failed.

Table 4 .
The descriptive statistics of difference from reference gestures of GA, BFOA, and ABC reconfigured gestures from 100 repetitions using the tuned parameter sets.
The best values are shown in bold.

Table 5 .
The descriptive statistics of computation time of GA, BFOA, and ABC output gestures reconfigured gestures from 100 repetitions using the tuned parameter sets.

Gestures Descriptive Statistics of Computation Time (Seconds) Kruskal-Wallis Test MIN MAX MEAN SD Median Interquartile Range
The best values are shown in bold.

Table 6 .
The comparison of algorithm performance on joint reconfiguration for all joint failures using the tuned parameter sets.

Table 6 .
The comparison of algorithm performance on joint reconfiguration for all joint failures using the tuned parameter sets.