Real-Time Identification of Knee Joint Walking Gait as Preliminary Signal for Developing Lower Limb Exoskeleton

Abstract

An exoskeleton is a device used for walking rehabilitation. In order to develop a proper rehabilitation exoskeleton, a user’s walking intention needs to be captured as the initial step of work. Moreover, every human has a unique walking gait style. This work introduced a wearable sensor, which aimed to recognize the walking gait phase, as the fundamental step before applying it into the rehabilitation exoskeleton. The sensor used in this work was the IMU sensor, used to recognize the pitch angle generated from the knee joint while the user walks, as information about the walking gait cycle, before doing the investigation on how to identify the walking gait cycle. In order to identify the walking gait cycle, Neural Network has been proposed as a method. The gait cycle identification was generated to recognize the gait cycle on the knee joint. To verify the performance of the proposed method, experiments have been done in real-time application. The experiments were carried out with different processes such as walking on a flat floor, climbing up, and walking down stairs. Five subjects were trained and tested using the system. The experiments showed that the proposed method was able to recognize each gait cycle for all users as they wore the sensor on their knee joints. This study has the potential to be applied on an exoskeleton rehabilitation robot as a further research experiment.

1. Introduction

Walking is a very important activity for humans in carrying out their daily activities. Without the ability to walk, humans will find it difficult to carry out their activities; as occurring to those who have had a stroke, recovering from a stroke, the elderly who have lost the muscle power to walk, or those suffering from spinal cord injuries. Thanks to the rapid development of medical device technologies, the problem of walking difficulties is increasingly capable of being tackled with the help of technology. One of the solutions is an exoskeleton robot that is intended to help humans walk. Such a robot can be classified into two types, namely lower limb and upper limb, which have been reviewed in [1,2,3,4]. The lower limb exoskeleton is not only used for medical purposes, such as assisting human walking [5], training or rehabilitation due to stroke, post stroke, and spinal cord injury [6,7], but also for military purposes or lifting heavy objects [8], and for gaining the ability for walking long distances [9]. However, in developing the lower limb exoskeleton, the most challenging part is to understand the human walking intention: whether it can predict a movement or only react to an existing one.

In order to understand the human walking intention, some methods have been proposed by researchers. As presented by Castagneri, et al. [10], the Statistical Gait Analysis (SGA) is introduced to process muscle cycle activation patterns, extracted from a functional walk by using an EMG sensor. Meng, et.al in [11], they used EMG sensors and a hidden Markov model (HMM) in order to recognize the gait phases. In another study, the gait cycle could also be recognized by using sensors placed on the plantar of the feet, as presented by Pawin, et.al [12], calculating the locus of Zero Moment Point (ZMP) from the walking gait cycle estimated by using Force Sensitive Resistors (FSRs). In order to classify the normal gait cycle, they implemented a neural network Kim et al. [13] proposed a simple gait segmentation from two sensorized insoles and developed eight flexible capacitive costume-made sensors to calculate the gait parameters and segment the gait cycle phased and subphases. Using the foot plantar for gait recognition was also proposed in [14,15], using a different approach. Moreover, Luu, et al. [16] presented their work using the EEG sensor in order to recognize the walking condition by adding the independent component analysis and k-means clustering. In addition, Villarreal et al. [17] proposed the gait phase pattern from the parameters of the mechanical variable.

The gait cycle recognition can also be analyzed from the gait flow image thanks to current technological developments. As proposed in [18], Kinect depth cameras were used to recognize the human gait recognition from the modeled body parts of a human to identify a walking person. The gait image also can be analyzed with different approaches such as Lacus-Kanade’s approach [19], extracting features from the human body [20,21], generating the 3D dynamic by using the active triangulation principle [22], using the Kohonen Self-Organizing Mapping (KSOM) neural network [23], and deriving the full model on a 7-link system and then performing the Denavit-Hartenberg method to analysis the kinematic and the Euler-Lagrane equation for the dynamic analysis [24].

Nevertheless, using the camera or gait image to recognize the human walking gait cycle may be complicated due to the need of setting up camera to collect data. When using the gait cycle as the preliminary signal for activating the exoskeleton robot, the signal needs to be identified while the suit is worn on the human body. Therefore, the use of a wearable sensor is necessary to this work. Tiny wearable sensors, namely an accelerometer or a gyroscope, or the combination of these two sensors, can be used, commonly called the IMU sensor. As shown in [25], they used a single accelerometer for the Parkinson’s disease (PD) patients by fitting a model equation to the relationships of the subject’s walking behavior. Whereas Hu et al. [26] used a tri-axial accelerometer on the waist to collect the body acceleration and derive the kinematic equation for human-walking model. Mota et al. [27] placed the sensor on the shank and used the complementary filter to obtain the body motion. As for the IMU sensor, several researchers have used this sensor in order to recognize the human walking phases with different sensor placement and approaches. The sensor can be placed in the user’s trouser pocket [28], pelvis [29], on the hip and knee joint [30,31], and chest [32]. Besides sensors, the actuators also play an important role [33].

The aim of this work is to identify the walking gait cycle where the IMU sensor can be attached to the human body; so that it can be integrated to the exoskeleton robot. In contrast with our previous work [32], in this work, we identified the human walking gait cycle by only looking at the information provided by the IMU sensors (only pitch angle) placed on both thighs for the knee joint. A number of methods to identify the gait cycle of the exoskeleton user has been presented. Recent literatures are summarized in

Table 1. A summary of recent methods on gait cycle identification.

No	Sensors	Reference	Description
1	EMG	[10]	Statistical Gait Analysis Algorithm
2	EMG	[11]	Hidden Markov Model Algorithm
3	Force Sensitive Resistor	[12]	Placed on plantar
4	Kinect depth cameras	[19]	Captured human walking then perform the body part modeling
5	IMU	[24]	The sensor was placed on waists
6	IMU	[26]	The sensor was placed on trouser pocket
7	IMU	[27]	The sensor placed on pelvis
8	IMU	[28]	The sensor placed on hip joints
9	IMU	[32]	The sensor placed on chest

. This work will focus on identifying the human walking gait cycle by using the IMU sensor, which will be placed on the thigh for determine the gait phase produced by the knee joint. In order to recognize the gait phase from the IMU pitch angle, which is produced by knee joint, the neural network method is implemented in this proposed system by real-time application. Experiments carried out in this work include walking on a flat surface, climbing up and walking down stairs with multiple samples.

To deliver a complete discussion, this paper is organized as follows: Section 2 provides the information about the sensors and their placement. Section 3 describes the proposed method and experiments. Section 4 presents the results of the experiments and Section 5 provides the concluding remarks and the future work of this investigation.

2. Sensor Placement

The sensor used in this work was the IMU 6050 sensor to measure the pitch angle produced by the knee while the user was walking. The reason of using the IMU sensor is that this sensor has an immediate response to any movement occurring towards it, and it is convenient to put close to the knee joint so that the signal will be more accurate. This is different from previous work [13], where a foot plantar sensor was used, needing much sensor configuration and the signal having to be wait until the user stepped their foot on the right position. As for sensor placement in this work, presented on Figure 1, Figure 1a shows the mechanical equipment of the lower limb exoskeleton for the knee joints, and Figure 1b shows the position of the sensor on the center of the thigh, about 5 cm above from the center point of the knee for each leg. The IMU sensor position on the knee joint was the X+ headed downward of the user and the X− to the opposite direction. The Y+ pointed to the left side of the user and the Y- to the opposite direction. The Z+ was directed to the rear of the user and the Z− to the opposite direction. The complete mechanical design of the lower limb exoskeleton can be seen on Figure 2, where Figure 2a–c show the prototype with different points of view. In this work, the IMU sensor was used to give the gait cycle recognition of the leg in order to generate assistance from the actuator mounted to the robot. The mechanical was designed to assist the knee joint for rehabilitation purposes. As presented on Figure 2c, the robot was equipped with two servo motors as the actuator, and a mini-PC Intel NUC as the main controller to control the motion and the gait cycle identification.

3. Methods

The block diagram of the system is shown in Figure 3. The IMU sensors were connected to the Arduino microcontroller. Prior to the gait cycle recognition, initially the IMU sensor needed to capture the pitch signal generated from the knee joint when the user was walking. Afterward, the signal was forwarded to the main controller i.e., mini PC via serial communication. In this main controller, the signal was processed, producing the whole gait signal. These signals would be generated when the user was walking. Furthermore, the signal given by the signal processing was used as the reference signal to the gait cycle identification process. In this work, Neural Network (NN) was used as the proposed method in order to recognize the gait phase produced by the users when they were walking. The architecture of the proposed method is presented on Figure 4, where the left and right pitch IMU sensor data are used as the reference signal. The output of the NN can be modeled by Equation (1). Back propagation method is used to determine the weights of the model.

$$\mathrm{O}=H^{\left(3\right)}\left[W^{\left(3\right)}{H}^{\left(2\right)}\left[W^2{H}^{\left(1\right)}\left[W^{\left(1\right)}I\right]\right]\right]$$

(1)

where:

• O = Output
• I = Input
• H⁽¹⁾ = Activation function in layer 1
• H⁽²⁾ = Activation function in layer 2
• H⁽³⁾ = Activation function in layer 3
• W⁽¹⁾ = Weight matric for layer 1
• W⁽²⁾ = Weight matric for layer 2
• W⁽³⁾ = Weight matric for layer 3

The NN architecture consisted of two nodes at the input layers and two hidden layers. These hidden layers had ten nodes for the first hidden layer and six nodes for the second one. The output layer generated three nodes. These nodes represented each gait phase as shown on Figure 5. The walking phase on Figure 5, consists of heel strike (HS), contralateral toe off (CTO), mid stance, contralateral heel strike (CHS), toe off (TO), mid swing, and finally returning to heel strike (HS). In reality, the initial gait phase of every user will be different according to the first foot initial contact. If the signal recognized the HS after initial contact for the first foot, then the other foot would present the CHS signal, and so on. Moreover, the binary output digits from the NN processing can be seen on

Table 2. The binary output digits from the neural network result.

No	Digits Binary Output	Gait Cycle
1	000	Initial
2	001	Heel strike
3	010	Contralateral toe off
4	011	Mid stance
5	100	Contralateral heel strike
6	101	Toe off
7	110	Mid swing

, where each phase possesses its own binary code in this table.

The signal monitoring was developed in this work using C# programming as can be seen on Figure 6a. When the proposed method was implemented in the prototype, the proposed method needed to be normalized into a suitable number. In order to normalize the input angle, it had to be no more than 130°. This is because of the maximum stride angle produced by a normal human walking is 130°, which can be seen on Figure 6b. The maximum stride angles were collected from the sample user posture and maximum pitch angle of the IMU when the user was walking, and the normalization value was generated from the IMU data read while the user was walking.

4. Results

This section will discuss some experiments which have been completed in real-time application. In order to verify the performance of the proposed method in identifying the gait cycle phase, the experiment was carried out in three different procedures such as walking on a flat floor, ascending stairs and descending stairs, which can be seen on Figure 7a–c, respectively. This experiment involved some users as seen in

Table 3. Experimental users’ data.

No	Walking on Flat Floor					Climbing Up and Down the Stairs
	User	Age (Years)	Height (cm)	*F/M	Weight (Kg)	User	Age (Years)	Height (cm)	*F/M	Weight (Kg)
1	User A	50	160	M	60	User A	17	155	F	50
2	User B	17	155	F	50	User B	21	168	M	60
3	User C	21	168	M	60	User C	45	150	F	65
4	User D	21	165	M	65
5	User E	45	150	F	65

*F/M: Female/Male.

. There were five users for walking on the flat floor procedure and three users for ascending and descending stairs procedures. This experiment also involved male and female users with different ages, weight, and heights as described in Table 3. In the training phase every user had to walk two gait cycles. The purpose of the training phase purpose was to obtain the weight of the NN algorithm using back propagation method. After this process, the test phase was performed. The tests comprised of experiments of walking on a flat surface, followed by the experiments of walking up and down stairs.

The first experiment was walking on a flat surface (floor), taken for two cycles of walking, where the results of each user are presented on Figure 8. On Figure 8, all the users used their right leg for the initial step, denoted by the higher signal produced by the right leg compared to the left. As for the results, at the heel strike phase, User E generated the lowest angle while striding the left leg and User B for the right leg. On the other hand, User D generated the highest angle while striding the right leg and User C and User A for the left leg. Furthermore, when the contralateral toe off (CTO) phase occurred while the users were walking, the right leg was in a supporting position in the front while the other began to lift off the floor. In this phase, User E produced the lowest signal for the right and left leg, while the highest signal was produced by User A. Meanwhile, in the mid stance phase User A generated the highest angle for the left and right legs. The lowest angle was produced by User B for the left leg and User D for the right leg. In the contralateral heel strike (CHS) phase, the feet were placed on the floor, the left foot in the front of right foot. In this phase, the lowest angle was generated by User C for the left leg and User B for the right leg, while the highest signal was produced by User D for the left leg and User A for the right leg. Moreover, in the toe off position, the highest angle was produced by User B for the left leg and User D for the right leg. The lowest angle was produced by User B for the right leg and User D for the left leg. The last phase was the mid swing phase where the left leg would be stepping on behind the right leg, while the right leg was in swinging position in front of left leg. In this phase, User E produced the lowest angle at the left leg and User D for the right leg. As for the highest signal generated by the User A for the left leg and User C for the right leg. The summary of the angles produced by each user can be seen on

Table 4. The angle of each gait phase generated by users.

User	Heel Strike (HS)		Contralateral Toe Off (CTO)		Mid Stance		Contralateral Heel Strike (CHS)		Toe off (TO)		Mid Swing
	Pitch Left (°)	Pitch Right (°)	Pitch Left (°)	Pitch Right (°)	Pitch Left (°)	Pitch Right (°)	Pitch Left (°)	Pitch Right (°)	Pitch Left (°)	Pitch Right (°)	Pitch Left (°)	Pitch Right (°)
User A	58	121	63	108	108	80	116	73	106	75	63	116
User B	51	117	56	101	92	77	109	66	110	63	56	113
User C	58	120	59	95	103	77	107	70	89	76	61	117
User D	49	128	62	106	98	75	122	68	77	77	60	87
User E	43	121	49	89	106	76	112	68	79	64	54	111

. From this experiment it can be concluded that each user has their own habits for their daily walking activities.

Table 5. Confusion matrix flat surface (in percentage).

		Prediction
Real		Initial	Heel Strike	Contralateral Toe off	Mid Stance	Contralateral Heel Strike	Toe off	Mid Swing
	Initial	100	0	0	0	0	0	0
	Heel strike	0.2	98.1	1.6	0	0	0.1	0
	Contralateral toe off	0	0.8	97.6	0.3	0	1.2	0
	Mid stance	0	0	0.4	98.6	0.2	0	0.7
	Contralateral heel strike	0	0	0	0.7	96.7	1.7	0.8
	Toe off	0	0	0.7	0	0.2	97.2	1.8
	Mid swing	0	0	0	1.3	0.1	0.1	98.4

shows the confusion matrix between the prediction and the real gait circle in the flat surface. The results presented in the Table 5 are the average value of five users.

The second experiment was tested out to verify the system when the users were climbing up stairs. In order to record the signal, the users ascend four steps, each having a height of about 20 cm. Information of the users can be seen in Table 3, the subjects consisting of two women and a man with different ages and heights. As seen on Figure 9, each user generated the same trend when climbing upstairs. The gait phase trends generated by the users were mid stand, mid swing, toe off (TO), and contralateral toe off (CTO). From Figure 9, the User B generated the highest signal of each phase. From the experiment, it can be concluded that the most common phase generated by the users were TO for the right leg and CTO for the left leg. The confusion matrix on the climbing up the stairs test for three subjects is presented in

Table 6. Confusion Matrix climbing up in percentage.

		Prediction
Real		Initial	Heel strike	Contralateral toe off	Mid stance	Contralateral heel strike	Toe off	Mid swing
	Initial	100	0	0	0	0	0	0
	Heel strike	0.4	96.3	1.9	0.2	0.8	0.3	0
	Contralateral toe off	0	0.8	97.6	0.3	0	1.2
	Mid stance	0	0	0.6	98.1	0.6	0.2	0.5
	Contralateral heel strike	0	0.2	0.3	0.7	96.2	1.6	0.9
	Toe off	0	0	0.6	0.4	0.4	97.1	1.5
	Mid swing	0	0	0.1	0.8	0.6	0.2	98.2

.

Further, when the users were going down a staircase, the signal generated can be seen on Figure 10. On Figure 10, the gait phase occurred when the user walked down the stairs, also mid stand, mid swing, toe off (TO), and contralateral toe off (CTO) for all users. In addition, the confusion matrix on the test when all subjects are going down is presented in

Table 7. Confusion Matrix climbing down in percentage.

		Prediction
Real		Initial	Heel strike	Contralateral toe off	Mid stance	Contralateral heel strike	Toe off	Mid swing
	Initial	100	0	0	0	0	0	0
	Heel strike	0.2	97.2	0.9	0	0.7	0.5	0.3
	Contralateral toe off	0	0.5	97.5	0.2	0.1	1.5	0
	Mid stance	0	0	0.4	98.6	0.3	0.4	0.2
	Contralateral heel strike	0	0	0.2	0.4	97.2	1.3	0.8
	Toe off	0	0	0.5	0.6	0.5	96.8	1.4
	Mid swing	0	0	0.3	0.6	0.8	0.3	97.9

. Compared with the gait movement of the users, the system is able to predetermine phases of gait of the users in every surface with successful rate of 98%.

5. Conclusions

The Neural Network has been proposed as the gait cycle recognition in this work, which was aimed to recognize the gait phase on the knee joint of the users. Experiments have been completed in real-time application. From the experiment results, the proposed method was found to be able to recognize the gait phases of human walking, whether they walked on a flat floor, ascending stairs, or descending stairs. The results of each user showed different signals, depending on their walking habits in daily activities. In the future, we will integrate this sensor to the prototype of an exoskeleton robot for use as a rehabilitation robot on the knee joint. Nevertheless, the prototype of the exoskeleton that will be developed should be appropriate to the sensor’s characteristic, the actuator chosen, and mechanical design should be emphasized in the next project. Selecting the proper actuator for the exoskeleton has been achieved. Therefore, it can be used for further study. Moreover, due to development of artificial intelligence, it is possible to predict the human walking intention by establishing the human–machine coupling relationship for the future.