Functional Electrical Stimulation System for Drop Foot Correction Using a Dynamic NARX Neural Network †

: Neurological diseases may reduce Tibialis Anterior (TA) muscle recruitment capacity causing gait disorders, such as drop foot (DF). The majority of DF patients still retain excitable nerves and muscles which makes Functional Electrical Stimulation (FES) an adequate technique to restore lost mobility. Recent studies suggest the need for developing personalized and assist-as-needed control strategies for wearable FES in order to promote natural and functional movements while reducing the early onset of fatigue. This study contributes to a real-time implementation of a trajectory tracking FES control strategy for personalized DF correction. This strategy combines a feedforward Non-Linear Autoregressive Neural Network with Exogenous inputs (NARXNN) with a feedback PD controller. This control strategy advances with a user-speciﬁc TA muscle model achieved by the NARXNN’s ability to model dynamic systems relying on the foot angle and angular velocity as inputs. A closed-loop, fully wearable stimulation system was achieved using an ISTim stimulator and wearable inertial sensor for electrical stimulation and user’s kinematic gait sensing, respectively. Results showed that the NARXNN architecture with 2 hidden layers and 10 neurons provided the highest performance for modelling the kinematic behaviour of the TA muscle. The proposed trajectory tracking control revealed a low discrepancy between real and reference foot trajectories (goodness of ﬁt = 77.87%) and time-effectiveness for correctly stimulating the TA muscle towards a natural gait and DF correction. methodology, S.C., A.C. and J.M.M.; A.C.; J.F. and S.C. A.C.; writing—orig-inal draft S.C. A.C.; writing—review and J.F., J.M.M. C.P.S.


Introduction
Stroke is the leading cause of death and disability globally, often resulting in paralysis for stroke survivors. Nevertheless, paralyzed subjects still retain excitable peripherical nerves and muscle tissues that may be re-established with Functional Electrical Stimulation (FES) by bypassing the biological lesion and conducting the necessary stimulus to induce a muscular contraction [1]. FES generates movement by stimulating the nervous tissue that innervates muscles. The stimulation pulses can be adapted by tuning parameters such as amplitude, frequency, or width. The pulse parameter modelling is relevant when considering different muscle types, and their non-linear and time-variant dynamics behaviour [2].
Drop Foot (DF) is a gait disorder that results from a reduced ability or total inability to contract the TA muscle. FES-based DF correction systems currently available in the market fail to take into consideration the time-variant dynamics of the electrically stimulated antagonistic muscle pairs for knee flexion and extension and for ankle joint dorsi-and plantarflexion during all gait phases. The ILC presented promising results in improving the gait of incomplete spinal cord injury subjects; however, the automatic ILC gain tuning could not find adequate parameters for all subjects, demanding manual configuration. Bouri et al. [26] proposed a feedforward and feedback control strategy for tracking the knee angle trajectory in paraplegic patients, demonstrating the importance of using a single feedforward component (predefined muscle activation patterns) to effectively reduce time delay. The works developed by Ferrarin et al. [24] and Ferrante et al. [25] relied on the use of goniometers to accurately track the knee extension trajectory by determining the stimulation pulse width. These studies compared the use of an open-loop (single inverse muscle models), closed-loop (single PID controllers), and a combination of feedback (PID controllers) and feedforward (neural network muscle models) components, concluding that the feedback and feedforward control strategy yielded better tracking performance, achieving a time delay of 0.21 ± 0.02 s and a root mean square error (RMSE) of 3.4 ± 0.3 in [24] and an RMSE of 3.2 in [25]. The trajectory tracking control strategies based on ILC, or repetitive control demonstrate great potential to adapt stimulation under periodic conditions such as gait. However, tuning the controller parameters to the subject characteristics still a considerable challenge. The combination of PID and neural network controllers have shown promising achievements; nonetheless, further exploration of such strategies considering fully wearable FES systems for real-time gait assistance is currently needed.
This study presents a real-time trajectory tracking FES control strategy by combining a feedforward Non-Linear Autoregressive Neural Network with Exogenous inputs (NARXNN) with a feedback PD controller for personalized DF correction. Based on previous findings [5,6,19] on modelling of muscle parameters, we hypothesized that the use of a dynamic inverse model (NARXNN) would capture the nonlinear dynamic behaviour of the electrically stimulated muscle, and thus predict the pulse width based on the desired joint kinematic trajectory. The novel aspect of this study relies on the real-time implementation into a compact embedded system and performance evaluation of a novel user specific NARXNN model of the TA muscle as a feedforward control. The microcontroller processes the control strategy based on acquired data from a wearable inertial sensor placed on the foot and communicates with the ISTim stimulator [28] to impose a user-specific and timely triggered FES. Additionally, the present work presents a control benchmarking analysis, comparing the performance of the feedback and feedforward control strategy with the use of a single feedforward neural network and a single feedback PD. This study presented promising results indicating that the use of a feedforward and feedback controller provides considerable improvements in performance potentiating further research with pathologic end-users.
The present work extends literature findings through (i) the use of a fully wearable FES system integrating the control and sensor processing algorithms into a microcontroller system, addressing the high demand for closed-loop control FES systems for daily motion assistance; (ii) the creation of a real-time NARXNN model that accurately represents the TA muscle non-linear dynamics as the feedforward component of a trajectory tracking feedback and feedforward control strategy; and, (iii) the inclusion of a user-specific TA muscle model into the control strategy, as well as the continuous and real-time feedback from the user's foot kinematics promoting a personalized FES assistance.

Materials and Methods
FES systems are meant to be small-sized and practical to actively restore motor function in daily living and allow the execution of everyday-tasks that otherwise would not be completed. In this sense, we proposed a wearable system for DF correction, extending our previous work [29]. As demonstrated in Figure 1, it is composed by the STM NUCLEO-32F303K8 ® processing unit to execute the control strategy and gait event detection algorithm; a modular stimulation unit from the ISTim Modular Stimulation System [28] to deliver the stimulation pulses to the TA muscle according to the desired trajectory; and, Machines 2021, 9,253 4 of 18 the MPU6050 wearable Inertial Measurement Unit (IMU) that acquires the foot kinematics in real-time. The FES system is powered by battery at 3.7 V with a capacity of 1000 mAh and dimensions of 5.84 × 29.5 × 51 mm 3 . It ensures autonomy for at least eight hours considering that the FES system consumption reaches up to 100 mAh. The algorithms present in the processing unit use the acquired foot kinematics in the sagittal plane (foot angle and foot angular velocity). The foot angular velocity is the gyroscope pitch value and the foot angle is calculated by fusing the data from the accelerometer and the gyroscope, using a complementary filter [30]. The complementary filter is represented by Equation (1), as follows: where Θ is the estimated foot angle in the sagittal plane, θ g is the foot angle integrated from the gyroscope pitch axis, θ a is the foot angle estimated from the accelerometer pitch axis and α g and α a are the filter constants, that were tuned to 0.98 and 0.02, respectively.
where Θ is the estimated foot angle in the sagittal plane, is the foot angle integrated from the gyroscope pitch axis, is the foot angle estimated from the accelerometer pitch axis and and α are the filter constants, that were tuned to 0.98 and 0.02, respectively. The proposed FES system was developed to re-establish motor control of the foot in stroke survivors that present severe walking disability. Consequently, the FES system will be important to assist gait at reduced self-selected walking speeds, ranging from 1-2.5 km/h [31]. The main goal of the proposed system is to assist drop foot patients efficiently with a compact, few intrusive, and wearable device. Nevertheless, the system is modular, allowing to scale up all functionalities namely, the number of sensors, number of channels, and control strategies according to other application scenarios.
The present work is divided into three main steps, the (i) NARXNN and Muscle Data Acquisition, the (ii) NARXNN Building, and the (iii) Experimental Procedure, as demonstrated in Figure 1. The first step addresses the acquisition of foot angle and angular velocity data retrieved from the TA muscle stimulation modulating the stimulation pulse width using specific filtered random noise signals. The obtained data was subsequently used to train, validate, and test a NARX neural network model to create an inverse model of the TA muscle. In the last step, the NARX neural network performance was tested when incorporated into different control strategies under experimental procedures divided in two phases. The chosen microcontroller was the STM NUCLEO-32F303K8 ® (STMicroelectronics, Geneva, Switzerland), since it is small but has a high range of peripheric features, with a length of 50.29 mm and a width of 18.54 mm. It features I2C and USART communication interfaces and provides 2 USART peripherals. It has an Arm ® Cortex ® -M4 32-bit CPU with 72 MHz maximum CPU frequency and features 64 Kbytes of Flash memory and 12 Kbytes The proposed FES system was developed to re-establish motor control of the foot in stroke survivors that present severe walking disability. Consequently, the FES system will be important to assist gait at reduced self-selected walking speeds, ranging from 1-2.5 km/h [31]. The main goal of the proposed system is to assist drop foot patients efficiently with a compact, few intrusive, and wearable device. Nevertheless, the system is modular, allowing to scale up all functionalities namely, the number of sensors, number of channels, and control strategies according to other application scenarios.
The present work is divided into three main steps, the (i) NARXNN and Muscle Data Acquisition, the (ii) NARXNN Building, and the (iii) Experimental Procedure, as demonstrated in Figure 1. The first step addresses the acquisition of foot angle and angular velocity data retrieved from the TA muscle stimulation modulating the stimulation pulse width using specific filtered random noise signals. The obtained data was subsequently used to train, validate, and test a NARX neural network model to create an inverse model of the TA muscle. In the last step, the NARX neural network performance was tested when incorporated into different control strategies under experimental procedures divided in two phases.
The chosen microcontroller was the STM NUCLEO-32F303K8 ® (STMicroelectronics, Geneva, Switzerland), since it is small but has a high range of peripheric features, with a length of 50.29 mm and a width of 18.54 mm. It features I2C and USART communication interfaces and provides 2 USART peripherals. It has an Arm ® Cortex ® -M4 32-bit CPU with 72 MHz maximum CPU frequency and features 64 Kbytes of Flash memory and 12 Kbytes of SRAM [32]. The processing unit communicates with the IMU and the ISTim via I2C, and serial USART, respectively. The sampling frequency of the system for data acquisition from MPU6050 was set to 500 Hz to capture the varying dynamics of the foot movement during gait [27]. This sampling frequency was used for all procedures conducted in the present study. The proposed wearable system was designed to provide personalized and time-effective assistance, compensate external disturbances, and provide natural movement detection algorithms.

Functional Electrical Stimulation
The electrical stimulation was conducted using the ISTim Modular Stimulation System [26,27], which delivers biphasic squared electrical pulses. The electrical pulse parameters range between 0-50 V, 0-503 µs, and 0-200 Hz, for pulse amplitude, width, and frequency, respectively [33]. The electrical stimulation was delivered to the TA muscle using two skin-surface, self-adhesive electrodes (5 cm; Medel GmbH, Hamburg, Germany) placed over the common peroneal nerve, allowing dorsiflexion of the foot. Considering the anode-and cathode-first biphasic stimulation [34], the cathode electrode was placed near to the head of the fibula and the anode electrode placed on the middle of the fibula.

Participants
Regarding the creation of the reference trajectories, the participants were approached through a mailing list across the University of Minho community describing the study goal, protocol, and duration. The participants were recruited according to a set of inclusion criteria, as follows: (i) healthy locomotion; (ii) more than 18 years old; (iii) body mass ranging from 45 to 90 kg; and (iv) body height ranging from 1.50 to 1.90 m. We selected 10 able-bodied subjects (6 males, 4 females) for the data acquisition procedure. The participants' mean age was 23.9 ± 1.64 years, with a mean height of 174.9 ± 7.07 cm, and mean weight of 69.7 ± 7.14 kg.
The trajectory tracking control validation procedures were conducted with an able 23-year-old female subject with 162 cm and 60 kg. All participants provided written and informed consent, according to the ethical conduct defined by the University of Minho Ethics Committee (CEICVS 006/2020) that follows the standards set by the declaration of Helsinki and the Oviedo Convention.

Modeling the Tibialis Anterior Muscle
The proposed empirical model represents the inverse dynamics of the electrically stimulated muscle. It predicts the pulse width values needed to deliver a personalized stimulation signal to control muscle contraction given the desired trajectory, towards correcting DF in real-time.

NARX Neural Network
NARX is a class of discrete-time non-linear systems that establish non-linear relationships between past observations and future outputs [18]. These models are useful for FES research purposes due to the small number of required parameters and their ability to represent the nonlinear dynamic behaviour of the electrically stimulated muscle [35]. The NARXNN model is represented by Equation (2), as follows: where y(t) is the output predicted by the model, u(t) is the input at time t. f is the nonlinear function that describes the system's behaviour and n and m are the regression orders of the input and output, respectively. Since the function f is a Multilayer Perceptron (MLP), the model represents a NARXNN. The proposed NARXNN was trained in Series-Parallel mode for more efficient training, considering that the true output is available (pulse width). The NARXNN is a type of recursive neural network that only has feedback between the output and input layers [5]. The dynamic inverse NARXNN model represented by Equation (2) was trained using the foot angle (θ(t)) and its angular velocity ( . θ(t)) as exogenous reference inputs, represented by u(t) in Equation (2). During training, the NARXNN learned the pulse width (P W) as the desired output that would generate the considered foot angle (θ(t)) and respective angular velocity ( . θ(t)). For real-time operating conditions and validation purposes, NARXNN considers the estimated pulse width (P W) as input (feedback loop) in addition to the foot angle (θ(t)) and its angular velocity ( . θ(t)), as represented in Figure 2. By trial and error, we verified that the NARXNN with only one delayed input(PW(t)) fitted the trade-off between maximum accuracy and real-time efficiency.
Machines 2021, 9, x FOR PEER REVIEW 6 of 18 recursive neural network that only has feedback between the output and input layers [5].
The dynamic inverse NARXNN model represented by Equation (2) was trained using the foot angle ( (t)) and its angular velocity ( ( )) as exogenous reference inputs, represented by ( ) in Equation (2). During training, the NARXNN learned the pulse width (PW ) as the desired output that would generate the considered foot angle ( (t)) and respective angular velocity ( ( ) ). For real-time operating conditions and validation purposes, NARXNN considers the estimated pulse width (PW ) as input (feedback loop) in addition to the foot angle ( (t)) and its angular velocity ( ( )), as represented in Figure 2. By trial and error, we verified that the NARXNN with only one delayed input (PW ( )) fitted the trade-off between maximum accuracy and real-time efficiency. The NARXNN considers as input the estimated pulse width (PW ( − 1)), the foot angle ( ( )), and the foot angle velocity ( ( )) to generate new predictions (PW ( )).

NARX Model Data Acquisition
A data acquisition procedure was performed to personalize the NARXNN model to each user's TA muscle characteristics. It consists of applying different stimulation signals to the TA muscle and recording the resulting foot kinematics (foot angle and angular velocity) using the IMU placed on the subject's foot. This procedure is represented in Figure  1-NARXNN and Muscle Data Acquisition.
The pulse width was chosen as the controlled variable and the stimulation frequency was adjusted to 30 Hz, the pulse amplitude to 28 V, and the minimum and maximum pulse width to 30 µs and 120 µs, respectively. The NARXNN was trained with real foot movement data from the IMU obtained after stimulating the TA muscle. For this, Filtered Random Noise (FRN) signals were used to shape the pulse width parameter. A frequency analysis of foot kinematics when walking with speeds between 1 Km/h and 2 Km/h determined that the most representative frequencies range between 0.23 and 0.34 Hz (The power spectrum of kinematic signals for each speed is represented in Figure A1 of appendix A). For assessing the model robustness, TA muscle stimulations were performed by adjusting the pulse width values using FRN signals with three incremental frequencies of 0.1 Hz, 0.2 Hz, and 0.3 Hz to generate slow, intermediate, and fast motion responses, respectively [28]. Three experimental trials were conducted for each FRN pulse frequency to collect diversified data for robust model training. The experimental trials were conducted with a healthy 23-year-old female subject. The system must consider no contact forces during the swing phase; thus, the data acquisition was performed with the subject in a seated position with both feet elevated from the ground.

NARX Model Building
Data (foot angle, foot angular velocity, and the pulse width signals) obtained from the NARXNN and Muscle Data Acquisition procedure were normalized between 0 and 1 using the min-max normalization method. The normalized data were used to train the personalized dynamic inverse model. The goal was to train the model with foot angle, foot angular velocity as inputs, and the expected pulse width as output in order to build a stimulation pulse adaptive neural network. The model was created and preliminarily tested using ® running on an ® computer with an ® i73537U processor at 2 GHz. This procedure is represented in Figure 1-NARXNN Building. The NARXNN considers as input the estimated pulse width (P W(t − 1)), the foot angle (θ(t)), and the foot angle velocity ( . θ(t)) to generate new predictions (P W(t)).

NARX Model Data Acquisition
A data acquisition procedure was performed to personalize the NARXNN model to each user's TA muscle characteristics. It consists of applying different stimulation signals to the TA muscle and recording the resulting foot kinematics (foot angle and angular velocity) using the IMU placed on the subject's foot. This procedure is represented in Figure 1-NARXNN and Muscle Data Acquisition.
The pulse width was chosen as the controlled variable and the stimulation frequency was adjusted to 30 Hz, the pulse amplitude to 28 V, and the minimum and maximum pulse width to 30 µs and 120 µs, respectively. The NARXNN was trained with real foot movement data from the IMU obtained after stimulating the TA muscle. For this, Filtered Random Noise (FRN) signals were used to shape the pulse width parameter. A frequency analysis of foot kinematics when walking with speeds between 1 Km/h and 2 Km/h determined that the most representative frequencies range between 0.23 and 0.34 Hz (The power spectrum of kinematic signals for each speed is represented in Figure A1 of Appendix A). For assessing the model robustness, TA muscle stimulations were performed by adjusting the pulse width values using FRN signals with three incremental frequencies of 0.1 Hz, 0.2 Hz, and 0.3 Hz to generate slow, intermediate, and fast motion responses, respectively [28]. Three experimental trials were conducted for each FRN pulse frequency to collect diversified data for robust model training. The experimental trials were conducted with a healthy 23-year-old female subject. The system must consider no contact forces during the swing phase; thus, the data acquisition was performed with the subject in a seated position with both feet elevated from the ground.

NARX Model Building
Data (foot angle, foot angular velocity, and the pulse width signals) obtained from the NARXNN and Muscle Data Acquisition procedure were normalized between 0 and 1 using the min-max normalization method. The normalized data were used to train the personalized dynamic inverse model. The goal was to train the model with foot angle, foot angular velocity as inputs, and the expected pulse width as output in order to build a stimulation pulse adaptive neural network. The model was created and preliminarily tested using MATLAB ® running on an ASUS ® computer with an INTEL ® CORE TM i73537U processor at 2 GHz. This procedure is represented in Figure 1-NARXNN Building.
The trials' data were randomly (using "rand" MATLAB function) divided as 70% for training, 15% for validation, and 15% for testing. The NARXNN was built using hyperbolic tangent as activation function and the Levenberg-Marquardt as the training algorithm. The weights were initialized randomly, between −1 and 1, and the NARXNN was trained with two different stopping criteria: upon reaching 500 training epochs or if the validation error increases six times in a row, to avoid overfitting. The validation error consisted of determining the Mean Squared Error (MSE) by comparing the expected values with those output by the NARXNN. We studied NARXNN performance by varying the number of neurons (5, 10, and 20) and hidden layers (1 and 2) towards finding the best performing NN with the smaller number of parameters to be used in the real-time control strategy. The performance of the NARXNN was assessed with the testing dataset, in terms of the goodness of fit (GOF) and the Pearson correlation coefficient (ρ). The GOF provides a measure of how similar a signal is when compared to a given reference, in terms of amplitude. The Pearson correlation coefficient measures linear similarity between the signals.

Trajectory Tracking Control Strategy
A trajectory tracking control strategy was implemented in the proposed system using a PD controller paired with a NARXNN model. Figure 3 depicts the closed-loop control strategy, where the feedback loop is fed by the foot angle and foot angle velocity, both measured by the foot IMU sensor in real-time.
The trials' data were randomly (using "rand" MATLAB function) divided as 70% for training, 15% for validation, and 15% for testing. The NARXNN was built using hyperbolic tangent as activation function and the Levenberg-Marquardt as the training algorithm. The weights were initialized randomly, between −1 and 1, and the NARXNN was trained with two different stopping criteria: upon reaching 500 training epochs or if the validation error increases six times in a row, to avoid overfitting. The validation error consisted of determining the Mean Squared Error (MSE) by comparing the expected values with those output by the NARXNN. We studied NARXNN performance by varying the number of neurons (5, 10, and 20) and hidden layers (1 and 2) towards finding the best performing NN with the smaller number of parameters to be used in the real-time control strategy. The performance of the NARXNN was assessed with the testing dataset, in terms of the goodness of fit (GOF) and the Pearson correlation coefficient ( ). The GOF provides a measure of how similar a signal is when compared to a given reference, in terms of amplitude. The Pearson correlation coefficient measures linear similarity between the signals.

Trajectory Tracking Control Strategy
A trajectory tracking control strategy was implemented in the proposed system using a PD controller paired with a NARXNN model. Figure 3 depicts the closed-loop control strategy, where the feedback loop is fed by the foot angle and foot angle velocity, both measured by the foot IMU sensor in real-time. is generated by combining the feedforward control command ( ( )) and the feedback control command ( ( )) and is sent to the stimulator, which delivers the stimulation to tibialis anterior muscle accordingly. The subsequent foot movement is continuously measured by the IMU placed on the subject's foot in terms of real foot angular velocity ( ) and foot angle (t). The PD controller only considered the foot angle (t) as the reference trajectory, while the Dynamic Inverse Model (NARXNN), considered as references both the foot angle (t) and the foot angle velocity ( ).
The error value, ( ), represents the PD controller input and it is computed as indicated in Equation (3).
where ( ) is the desired foot trajectory angle and ( ) is the real foot angle measured by the IMU. The PD control command is given by Equation (4).
where ( ) is the control variable output of the controller, ( ) is the error value given by Equation (3), ( ) is the real foot angular velocity, and and are the proportional and derivative gains, respectively. The pulse width value that is sent to the stimulator, , is a sum of the value predicted by the model, , and the control variable from the PD controller, , given by Equation (5). The control command U s (t) (pulse width) is generated by combining the feedforward control command (U m (t)) and the feedback control command (U c (t)) and is sent to the stimulator, which delivers the stimulation to tibialis anterior muscle accordingly. The subsequent foot movement is continuously measured by the IMU placed on the subject's foot in terms of real foot angular velocity . θ(t) and foot angle θ(t). The PD controller only considered the foot angle θ R (t) as the reference trajectory, while the Dynamic Inverse Model (NARXNN), considered as references both the foot angle θ R (t) and the foot angle velocity . θ R (t).
The error value, e(t), represents the PD controller input and it is computed as indicated in Equation (3).
where θ R (t) is the desired foot trajectory angle and θ(t) is the real foot angle measured by the IMU. The PD control command is given by Equation (4).
where U c (t) is the control variable output of the controller, e(t) is the error value given by Equation (3), . θ(t) is the real foot angular velocity, and K p and K d are the proportional and derivative gains, respectively. The pulse width value that is sent to the stimulator, U s , is a sum of the value predicted by the model, U m , and the control variable from the PD controller, U c , given by Equation (5).
The trajectory tracking control strategy relies on the gait event detection in order to evoke muscle contraction according to each stage of the gait cycle. The used gait event algorithm was proposed in [36], and relies on the foot's angular velocity to detect in real- Toe-Off (TO) and Middle Mid-Swing (MMSW) using an adaptive finite state machine. The detected gait events are used to trigger the trajectory tracking control during the swing phase given that this is the most affected phase in DF patients [37].

Response Time Delay Removal
It is important to emphasize that the FES actuation has a subject-specific delay linked to the subject's muscular response time. Therefore, it is important to anticipate this delay in order to track the correct foot angle trajectory in the correspondent gait phase. The response time delay is continually measured during gait, and the response time delay removed in each gait cycle is retrieved from the previous gait cycle. The time delay removal strategy starts by determining the stance phase duration (T st ), by measuring the time between heel strike and toe-off events, both detected by the gait event algorithm in real-time [36]. Subsequently, the subject specific time delay (T d ) is computed as the time difference between the toe-off events of the reference foot angle trajectory and the measured foot angle trajectory. Figure 4 depicts a case example of a subject-specific time delay removal. The delay removal is achieved by triggering stimulation at a specific time for each gait cycle, denominated as stimulation time (T stim ), determined as the difference between T st and T d , as indicated in Equation (6).

Reference Trajectory Acquisition
The foot reference trajectory was obtained from 10 healthy subjects walking on a treadmill at three different speeds (1 km/h, 1.5 km/h, and 2 km/h) as mentioned in Section 2.2. An IMU was placed on the subject's foot to measure his/her kinematic behaviour. The foot angle and foot angular velocity were acquired in real-time during 10 gait cycles for each speed. Note that each gait cycle starts and begins when foot angle and foot angular velocity are 0 • and 0 rad/s, respectively. For each walking speed, the foot angle and foot angular velocity reference trajectories were created by performing the average of the 10 retrieved signals, resulting in three-foot angle reference trajectories (Figure 5a-c) and three foot angular velocity reference trajectories (Figure 5d-f).
Machines 2021, 9, x FOR PEER REVIEW 9 of 18 retrieved signals, resulting in three-foot angle reference trajectories (Figure 5a-c) and three foot angular velocity reference trajectories (Figure 5d-f).

Trajectory Tracking Control Validation
The trajectory tracking control was validated considering an experimental protocol divided into two phases. For both phases, the control strategies were applied at a frequency of 125 Hz. PD controller was tuned manually until the system was stable, achieving values of 1.5 and 0.7 for and , respectively. The inertial sensor was placed on the surface of the foot. The system was validated with an able 23-year-old female subject with 162 cm and 60 kg [38,39]. The performance of the trajectory tracking control was assessed using GOF and the root mean square error (RMSE). The GOF and RMSE provide objective measures to compare the reference signals with the measured signals obtained by applying the FES control strategy. The advances in this study may be further explored in disabled subjects, since the response of the relaxed muscles of able subjects is similar to that of paralyzed muscle limbs under electrical stimulation [40][41][42][43][44].

Phase I Protocol
The phase I protocol aims the control benchmarking analysis by comparing the performance of feedforward and feedback trajectory tracking control strategy (NARXNN & PD) with the (i) single PD controller (for compensation purposes) and (ii) Open-loop dynamic inverse NARXNN control strategy (OP), as represented in Figure 6. This comparative analysis focused on selecting the most effective controller towards the DF correction application. The validation was performed with the subject seated, keeping the foot elevated from the ground, allowing the ankle to move without restriction. These experimental conditions are easily reproducible in consequent trials and aim at allowing the testing to be conducted without the existence of ground contact forces to the foot. Figure  7a represents the experimental setup. The control strategies were validated with the reference trajectory for three different speeds: 1 km/h, 1.5 km/h, and 2 km/h, conducting 4 trials for each speed.

Trajectory Tracking Control Validation
The trajectory tracking control was validated considering an experimental protocol divided into two phases. For both phases, the control strategies were applied at a frequency of 125 Hz. PD controller was tuned manually until the system was stable, achieving values of 1.5 and 0.7 for K p and K d , respectively. The inertial sensor was placed on the surface of the foot. The system was validated with an able 23-year-old female subject with 162 cm and 60 kg [38,39]. The performance of the trajectory tracking control was assessed using GOF and the root mean square error (RMSE). The GOF and RMSE provide objective measures to compare the reference signals with the measured signals obtained by applying the FES control strategy. The advances in this study may be further explored in disabled subjects, since the response of the relaxed muscles of able subjects is similar to that of paralyzed muscle limbs under electrical stimulation [40][41][42][43][44].

Phase I Protocol
The phase I protocol aims the control benchmarking analysis by comparing the performance of feedforward and feedback trajectory tracking control strategy (NARXNN & PD) with the (i) single PD controller (for compensation purposes) and (ii) Open-loop dynamic inverse NARXNN control strategy (OP), as represented in Figure 6. This comparative analysis focused on selecting the most effective controller towards the DF correction application. The validation was performed with the subject seated, keeping the foot elevated from the ground, allowing the ankle to move without restriction. These experimental conditions are easily reproducible in consequent trials and aim at allowing the testing to be conducted without the existence of ground contact forces to the foot. Figure 7a represents the experimental setup. The control strategies were validated with the reference trajectory for three different speeds: 1 km/h, 1.5 km/h, and 2 km/h, conducting 4 trials for each speed.

Phase II Protocol
The phase II protocol was performed to validate the feedforward and feedback control strategy (PD & NARXNN) for DF correction during gait with the subject walking on  . During experimental trials, the subject was instructed not to lift the foot during the swing phase, behaving similar to a DF patient (pseudo-DF-gait). The stimulation was applied in random gait cycles during the simulated DF gait of the subject. The random application of the electrical stimulus aimed to minimize the occurrence of voluntary muscle contractions. Preliminary experimental validations have been performed in literature with healthy subjects using FES to correct drop foot [38,39]. One trial was conducted for each walking speed until 10 gait cycles were completed.

NARXNN Testing
The dynamic inverse NARX was tested alone after training and validation, to infer its ability to predict target FRN signals, under a different number of layers and neurons. This procedure is represented in Figure 1 as the last step of the NARXNN building block. Figure 8 shows the predicted signals (pulse width) overlapped to the reference FRN signals, and respective performance (GOF = 89.05%, 83.9%, and 69.49%) obtained for one trial using a NARXNN with two hidden layers and 20 neurons.

Phase II Protocol
The phase II protocol was performed to validate the feedforward and feedback control strategy (PD & NARXNN) for DF correction during gait with the subject walking on the treadmill at three speeds: 1 km/h, 1.5 km/h, and 2 km/h. The range of walking speeds between 1 and 2 km/h aims to focus the system validation on the most severe cases of walking disability as a consequence of a stroke [31]. The validation setup can be seen in Figure 7b. During experimental trials, the subject was instructed not to lift the foot during the swing phase, behaving similar to a DF patient (pseudo-DF-gait). The stimulation was applied in random gait cycles during the simulated DF gait of the subject. The random application of the electrical stimulus aimed to minimize the occurrence of voluntary muscle contractions. Preliminary experimental validations have been performed in literature with healthy subjects using FES to correct drop foot [38,39]. One trial was conducted for each walking speed until 10 gait cycles were completed.

NARXNN Testing
The dynamic inverse NARX was tested alone after training and validation, to infer its ability to predict target FRN signals, under a different number of layers and neurons. This procedure is represented in Figure 1 as the last step of the NARXNN building block. Figure 8 shows the predicted signals (pulse width) overlapped to the reference FRN signals, and respective performance (GOF = 89.05%, 83.9%, and 69.49%) obtained for one trial using a NARXNN with two hidden layers and 20 neurons. The NARNN performance for tracking low, medium, and high-frequency FRN signals was accessed considering a different number of neurons (5, 10, and 20) in a single and double-layer configuration. The performance results in terms of average GOF and ρ coefficient were estimated based on the experimental data and are depicted in Table 1.  The NARNN performance for tracking low, medium, and high-frequency FRN signals was accessed considering a different number of neurons (5, 10, and 20) in a single and double-layer configuration. The performance results in terms of average GOF and ρ coefficient were estimated based on the experimental data and are depicted in Table 1. The system computation times were analysed to determine its real-time performance when running in the microcontroller architecture. The obtained computation times regarding the NARXNN prediction, the IMU data acquisition, the communication with the stimulator and the gait event detection algorithm are listed in Table 2. The combination of these values represents the system operating frequency which stands higher than the stimulation frequency (30 Hz). Therefore, the implemented dynamic inverse models present time effective responses, which stand adequate for real-time control. Considering the Tables 1 and 2 results, the NARXNN with two hidden layers and 10 neurons in each hidden layer was the selected chosen architecture to use in the control strategies of the following experimental procedures (two-phase protocol), to privilege the trade-off between accuracy and time-efficiency. The results (GOF) obtained regarding the three different control strategies validated at three different speeds are presented in Table 3 and Figure 9.  The results (GOF) obtained regarding the three different control strategies validated at three different speeds are presented in Table 3 and Figure 9.  The system validation under phase II protocol was conducted with a healthy subject on a treadmill at 1 km/h, 1.5 km/h, and 2 km/h. Results under phase II protocol for the speed of 1 km/h can be seen in Figure 10. By analysing Figure 10a, at the beginning of the

Phase II Protocol
The system validation under phase II protocol was conducted with a healthy subject on a treadmill at 1 km/h, 1.5 km/h, and 2 km/h. Results under phase II protocol for the speed of 1 km/h can be seen in Figure 10. By analysing Figure 10a, at the beginning of the swing phase, the foot angle is 20 • lower than the reference trajectory (the reference trajectory retrieved as described in Section 2.4.2). In order to compensate for this disturbance, the controller increases the applied pulse width, as shown in Figure 10c. As the real foot angle approaches the desired trajectory the applied pulse width decreases, as expected, in order to avoid unneeded muscle stimulation, and consequently to prevent the early onset of fatigue ( Figure 4). The reference trajectory was set to zero from heel-strike to push-off events to assist only when needed for drop foot correction, aiming at delaying the muscle fatigue. The pulse width is not constant throughout the swing phase, since it changes according to the difference between the desired trajectory and the real foot angle. The foot angle is able to reach the desired reference angles before the heel strike and, due to the pulse width decrease at the heel strike, it is also able to slowly descend towards the ground, generating a natural movement. of fatigue (Figure 4). The reference trajectory was set to zero from heel-strike to push-off events to assist only when needed for drop foot correction, aiming at delaying the muscle fatigue. The pulse width is not constant throughout the swing phase, since it changes according to the difference between the desired trajectory and the real foot angle. The foot angle is able to reach the desired reference angles before the heel strike and, due to the pulse width decrease at the heel strike, it is also able to slowly descend towards the ground, generating a natural movement. The gait cycles related to corrected DF gait by FES were averaged and compared with the pseudo-DF gait and with the healthy gait, to determine if the applied control strategy improved the pseudo-DF gait pattern. The obtained RMSE results comparing the foot angle from healthy gait with pseudo-DF and corrected DF gait are depicted in Table 4. These results suggest that the proposed trajectory tracking control was able to improve the foot motion in 21.41%, 45.82%, and 37.76% for the walking speeds of 1 km/h, 1.5 km/h, and 2 km/h, respectively.  Figure 11 shows that in the pseudo-DF gait the foot does not rise above 0° at heel strike. However, when the trajectory tracking control strategy is applied, the foot follows the desired trajectory, raising above 0°, and demonstrating a healthy behaviour. The gait cycles related to corrected DF gait by FES were averaged and compared with the pseudo-DF gait and with the healthy gait, to determine if the applied control strategy improved the pseudo-DF gait pattern. The obtained RMSE results comparing the foot angle from healthy gait with pseudo-DF and corrected DF gait are depicted in Table 4. These results suggest that the proposed trajectory tracking control was able to improve the foot motion in 21.41%, 45.82%, and 37.76% for the walking speeds of 1 km/h, 1.5 km/h, and 2 km/h, respectively.  Figure 11 shows that in the pseudo-DF gait the foot does not rise above 0 • at heel strike. However, when the trajectory tracking control strategy is applied, the foot follows the desired trajectory, raising above 0 • , and demonstrating a healthy behaviour.

Discussion
A real-time wearable FES system for DF correction was presented and validated in a two-phase protocol to compare different control approaches and to infer about the system's ability to correct DF during gait. Table 1 analysis suggests that the difference in GOF between the results for the NARXNN with 20 neurons trained with 1 and 2 hidden layers is reduced, being 1.28% and 1.3% for low and medium-frequency FRN signals, respectively. For the high-frequency FRN signals, the difference in GOF increases to 8.01%. Despite this difference for high-frequency FRN signals, the ρ values stood similar with average difference values of 0.095, 0.007, and 0.017 regarding low, medium, and high-frequency FRN signals respectively. The NARXNNs showed higher performance for low-frequency signal predictions, with a GOF of 81.41%, when compared to the performance for high-frequency signal predictions, with a GOF of 68.07%. The prediction time responses of the models were also studied, showing that the NARXNNs with 10 neurons had the fastest response (7.0 ± 0.327 ms).
The obtained results for phase I protocol show that the feedforward and feedbackbased trajectory tracking control strategy had the best performance when compared to the other control strategies. However, for the highest speed reference (2 km/h) the accuracy has decreased 69.54% (GOF = 23.72%), which might be related with the fact that the NARXNN was trained with FRN signals with frequencies ranging from 0.1-0.3 Hz. The inclusion of a NARXNN into a closed-loop PD controller increased prediction performance (Table 3), and a visual inspection of Figure 9 shows that the corrected DF gait trajectory presents higher similarity comparing with the healthy gait trajectory. The lower performance for the 2 km/h speed might be related to the considerable differences in initial conditions, i.e., the difference between the initial real foot angle and the initial reference trajectory values, creating a delayed pulse width, which results in a delayed tracking response. This issue was not observed in the phase II protocol since the initial conditions were fixed, i.e., the user started the walking on a treadmill in foot flat event, where the foot angle is approximate 0°. The results obtained for phase I protocol point show considerable improvements (62%) in using a feedback and feedforward control strategy (combining a PID and a neural network), as previously advances in similar studies [17].
Additionally, the proposed feedback and feedforward control strategy presents considerable improvements when comparing pseudo-DF gait and corrected DF gait with

Discussion
A real-time wearable FES system for DF correction was presented and validated in a two-phase protocol to compare different control approaches and to infer about the system's ability to correct DF during gait. Table 1 analysis suggests that the difference in GOF between the results for the NARXNN with 20 neurons trained with 1 and 2 hidden layers is reduced, being 1.28% and 1.3% for low and medium-frequency FRN signals, respectively. For the high-frequency FRN signals, the difference in GOF increases to 8.01%. Despite this difference for highfrequency FRN signals, the ρ values stood similar with average difference values of 0.095, 0.007, and 0.017 regarding low, medium, and high-frequency FRN signals respectively. The NARXNNs showed higher performance for low-frequency signal predictions, with a GOF of 81.41%, when compared to the performance for high-frequency signal predictions, with a GOF of 68.07%. The prediction time responses of the models were also studied, showing that the NARXNNs with 10 neurons had the fastest response (7.0 ± 0.327 ms).
The obtained results for phase I protocol show that the feedforward and feedbackbased trajectory tracking control strategy had the best performance when compared to the other control strategies. However, for the highest speed reference (2 km/h) the accuracy has decreased 69.54% (GOF = 23.72%), which might be related with the fact that the NARXNN was trained with FRN signals with frequencies ranging from 0.1-0.3 Hz. The inclusion of a NARXNN into a closed-loop PD controller increased prediction performance (Table 3), and a visual inspection of Figure 9 shows that the corrected DF gait trajectory presents higher similarity comparing with the healthy gait trajectory. The lower performance for the 2 km/h speed might be related to the considerable differences in initial conditions, i.e., the difference between the initial real foot angle and the initial reference trajectory values, creating a delayed pulse width, which results in a delayed tracking response. This issue was not observed in the phase II protocol since the initial conditions were fixed, i.e., the user started the walking on a treadmill in foot flat event, where the foot angle is approximate 0 • . The results obtained for phase I protocol point show considerable improvements (62%) in using a feedback and feedforward control strategy (combining a PID and a neural network), as previously advances in similar studies [17].
Additionally, the proposed feedback and feedforward control strategy presents considerable improvements when comparing pseudo-DF gait and corrected DF gait with healthy gait, namely, 21.41%, 45.82%, and 37.76% for the walking speeds of 1 km/h, 1.5 km/h, and 2 km/h, respectively. These results suggest that the use of FES under the proposed feedback and feedforward control strategy allowed a healthier motor control of the foot during gait. Despite promising results, the existing angle error between reference trajectory and corrected DF trajectory may be related to the used reference. The creation of the angle reference included subject with considerable difference in height which may not be the most adequate for the subjected that conducted the system validation. Nevertheless, the results from our study regarding the feedback and feedforward control strategy during gait stand in accordance with a study with a similar experimental setup [9]. The trajectory tracking control strategy proposed in our study achieved normalized RMSE of 0.305, 0.203, and 0.239 degrees for walking speeds of 1, 1.5, and 2 km/h, respectively. The iterative learning control proposed in [9,45] presented similar performance in the initial iterations (normalized RMSE of 0.25 degrees); however it converged to a minimal error performance in the subsequent control iterations (normalized RMSE of 0.05 degrees). Moreover, the study [11] advances in the use of FES to control the foot pitch and roll motion in paretic gait, while achieving promising results, having normalized RMSE values for the foot pitch of 0.08 degrees. Despite requiring further experimental trials, our study contribution regarding similar studies [9] relates to the use of a fully wearable FES system that runs in real-time a feedback and feedforward control strategy to correct drop foot gait.
Similar trajectory tracking strategies have been proposed in the literature [17,46]. However, the study conducted in this paper advances [17] by embedding a feedforward and feedback control strategy in a completely wearable and real-time FES system. Moreover, the work in [21] proposes a wearable FES prototype using a fuzzy logic control, however, the obtained results suggested the need to include a parameter adjustment algorithm to run during the stimulation. The present study tackles the need to develop wearable FES systems with embedded control, and the use of a NARXNN allows the automatic parameter adjustment to provide an optimized stimulation pattern. The FES system was created to be personalized for each subject, requiring minimal-to-null inputs from the end-user for real-time use. In this sense, the pulse intensity and frequency parameters are initially fixed to the best-suited values, not requiring any type of adjustment during stimulation. Moreover, the NARXNN automatically adjusts the pulse width in real-time to the subject's needs.

Limitations and Future Directions
In this study, we presented an initial design of feedback and feedforward control strategy by combining a PD and a NARX neural network-integrated in an FES wearable system, for real-time DF correction during gait. Our main goal was to conduct initial tests aiming for the system validation and to evaluate the NARX as a feedforward component from the feedback and feedforward control strategy. The main limitation of this study relies on the reduced number of subjects used for validation purposes. In fact, despite promising results and the real-time validations conducted, the proposed system will undergo extensive validation with an increased population size by including patients with drop foot as the target end-users. The NARX neural network was able to modulate TA muscle for slow walking speeds. Despite this, the model will be improved to allow assistance in faster walking speeds, by training the NARXNN with FRN signals with higher frequency. Additionally, future developments on this research will include the execution of the stimulation strategies in different days for the same subject to assess the stimulation response variability day by day. This research will also progress to improve the control time-effectiveness and time-delay quantification during gait at different speeds.

Conclusions
This paper proposes a real-time trajectory tracking control strategy, based on a NARXNN combined with a PD, to control the foot angle trajectory using a newly designed wearable FES system for assist-as-needed DF correction. Under DF gait conditions, the assistance was focused on the swing phase, which was timely detected through an adaptive gait event detection algorithm. The findings suggest that combining feedforward NARX and a feedback PD controller may yield the best performance when compared to the use of a single feedforward NARX or a single feedback PD controller. Furthermore, the optimized NARXNN architecture showed to be adequate in correcting DF of the participating subject in real-time settings and its inclusion in the feedback and feedforward control strategy enabled personalized and time-effective assistance by considering the user-specific non-linear TA muscle characteristics.
The study conducted in this paper allowed to conclude that the FES system stood as an effective tool for real-time electrical stimulation of the TA muscle. Moreover, the proposed control strategy combining a NARXNN with a PD controller revealed promising results, despite the reduced number of participants in the experimental trials. These results potentiate further investigation with the proposed system with an heterogenous group of participants. Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.

Conflicts of Interest:
The authors declare no conflict of interest. Figure A1 represents the spectral frequency analysis conducted for the kinematic signals depicted in Figure 5. Results of Figure A1 state that the frequencies with the highest power for the speeds of 1 km/h ( Figure A1a), 1.5 km/h ( Figure A1b), and 2 km/h ( Figure A1c) are 0.23 Hz, 0.29 Hz, and 0.34 Hz, respectively. Furthermore, Table A1 presents the frequencies with highest power and energy share below 0.3 Hz for the speeds of 1 km/h, 1.5 km/h and 2 km/h.

Conclusions
This paper proposes a real-time trajectory tracking control strategy, based on a NARXNN combined with a PD, to control the foot angle trajectory using a newly designed wearable FES system for assist-as-needed DF correction. Under DF gait conditions, the assistance was focused on the swing phase, which was timely detected through an adaptive gait event detection algorithm. The findings suggest that combining feedforward NARX and a feedback PD controller may yield the best performance when compared to the use of a single feedforward NARX or a single feedback PD controller. Furthermore, the optimized NARXNN architecture showed to be adequate in correcting DF of the participating subject in real-time settings and its inclusion in the feedback and feedforward control strategy enabled personalized and time-effective assistance by considering the user-specific non-linear TA muscle characteristics.
The study conducted in this paper allowed to conclude that the FES system stood as an effective tool for real-time electrical stimulation of the TA muscle. Moreover, the proposed control strategy combining a NARXNN with a PD controller revealed promising results, despite the reduced number of participants in the experimental trials. These results potentiate further investigation with the proposed system with an heterogenous group of participants. Informed Consent Statement: Informed consent was obtained from all subjects involved in the study.

Conflicts of Interest:
The authors declare no conflict of interest. Figure A1 represents the spectral frequency analysis conducted for the kinematic signals depicted in Figure 5. Results of Figure A1 state that the frequencies with the highest power for the speeds of 1 km/h ( Figure A1a), 1.5 km/h ( Figure A1b), and 2 km/h ( Figure  A1c) are 0.23 Hz, 0.29 Hz, and 0.34 Hz, respectively. Furthermore, Table A1 presents the frequencies with highest power and energy share below 0.3 Hz for the speeds of 1 km/h, 1.5 km/h and 2 km/h.