Recognition of Gait Activities Using Acceleration Data from A Smartphone and A Wearable Device ^†

Abstract

Activity recognition is an important task in many fields, such as ambient intelligence, pervasive healthcare, and surveillance. In particular, the recognition of human gait can be useful to identify the characteristics of the places or physical spaces, such as whether the person is walking on level ground or walking down stairs in which people move. For example, ascending or descending stairs can be a risky activity for older adults because of a possible fall, which can have more severe consequences than if it occurred on a flat surface. While portable and wearable devices have been widely used to detect Activities of Daily Living (ADLs), few research works in the literature have focused on characterizing only actions of human gait. In the present study, a method for recognizing gait activities using acceleration data obtained from a smartphone and a wearable inertial sensor placed on the ankle of people is introduced. The acceleration signals were segmented based on the automatic detection of strides, also called gait cycles. Subsequently, a feature vector of the segmented signals was extracted, which was used to train four classifiers using the Naive Bayes, C4.5, Support Vector Machines, and K-Nearest Neighbors algorithms. Data was collected from seven young subjects who performed five gait activities: (i) going down an incline, (ii) going up an incline, (iii) walking on level ground, (iv) going down stairs, and (v) going up stairs. The results demonstrate the viability of using the proposed method and technologies in ambient assisted living contexts.

1. Introduction

Activity or action recognition is the process of inferring and assigning a label of a class to a dataset element of at least one source, such as video, images, or portable devices. One of the more active areas of research in activity recognition corresponds to the recognition of Activities of Daily Living (ADLs) [1] and instrumental ADLs [2] using acceleration data of portable and wearable devices [3,4]. These ADLs are usually very different from one another, for example, brushing teeth, preparing meals, and ambulating.

Gait analysis is the discipline that studies the locomotion and ambulation of people using techniques that branch from a brief observation to sophisticated computerized measurements [5]. Gait analysis is important because human gait may reflect early warning signs of senile dementia and Alzheimer’s disease [6] or the assessment of frailty [7]. In addition to this, the recognition of the human gait can be useful to identify the characteristics of the places where people move, which is important for monitoring their activities [8], abnormal behaviors [9], and risk factors [10]. Human gait is defined as the cyclic movement of the feet in which each foot alternates contact with the support surface [11]. To begin walking it is necessary to start from a situation of mechanical stability in standing. The walking activity involves the entire musculoskeletal system and various postural reflexes.

Periodic patterns occur in typical human gait captured by accelerometers, for instance, there are positive peaks in swing phases when a foot moves forward in a specific axis [12]. People change their gait due to age-related changes and pathological changes, which makes it difficult to distinguish patterns in the acceleration signals. Elderly people, for instance, drag their feet and have shorter step length. Gait events, such as heel strike and toe-off, are detected based on characteristic accelerations and decelerations detected as peaks [13].

In the present study, we present a method for recognizing gait activities using acceleration data obtained from a smartphone and from a wearable device based on the identification of human strides and using supervised classification. The remaining of the paper is organized as follows. In Section 2 related work is summarized. Section 3 and Section 4 detail the method proposed for recognizing gait activities and its evaluation, respectively. Finally, the conclusions and future directions are presented in Section 5.

2. Related Work

Technological advances have allowed collecting data of the activities of people mainly from the use of video cameras [14,15], environmental sensors [16], and portable devices [4,17,18]. The latter have the advantages of working properly outdoors and of not being sensitive to occlusion or lighting.

The most studied ambulating or gait activities are walking, jogging and running. For example, in [19] the authors propose a one-dimensional (1D) Convolutional Neural Network (CNN) -based method for recognizing those three activities using accelerometer data collected from smartphones of five subjects. The raw acceleration signals are combined in a vector magnitude and segmented in windows of 10 and 20 s. In another study, walking was classified into three speeds: slow, normal and fast [20]. Acceleration data were obtained from 25 subjects who carried a smartphone. Eight features were extracted from the acceleration magnitude vector, and seven classification techniques were applied for classifying the walking data by the speed.

In [21], the activities of running, being still, jumping and walking were classified using Support Vector Machines (SVM). Data was collected from a smartphone placed inside the pocket of 11 subjects. The features were extracted from raw acceleration segments of 5.12 s based on the Discrete Cosine Transform (DCT) coefficients and were selected using Principal Component Analysis (PCA). In another work, accelerometer and gyroscope signals were used to classify gait actions [22]. A smartphone was placed on the thigh and an Inertial Measurement Unit (IMU) on the chest of 24 subjects with the aim of classifying six activities: sitting, standing, lying, walking, running, and cycling. Time and frequency-domain features were extracted from the segmented signals (15 s) and classified using five different algorithms.

In contrast to previous work, two important aspects of our study can be highlighted: (i) the acceleration signals were dynamically segmented based on the automatic detection of the strides of the participants, and (ii) a double-comparison was made in two dimensions, comparing two devices (smartphone vs. IMU) and comparing two types of acceleration signals (forward-direction vs. magnitude vector). The objective of using two types of devices, one specialized (IMU) with a specific configuration, and another that most people are carrying nowadays (smartphone), is to assess whether the computational techniques applied are independent of the type of sensing device and its configuration.

3. Methods

The proposed method for recognizing gait activities is presented in Figure 1 and detailed below. The method is divided into two stages: (i) strides detection and (ii) gait classification. Data acquisition is detailed in Section 3.1. Once the data has been collected, the acceleration signals are segmented into strides (see Section 3.2). Finally, descriptive features are extracted from the signals in order to classify the gait activities, as it is detailed in Section 3.3.

3.1. Setup

Two devices that incorporate accelerometer sensors were used in data capturing. The first was a smartphone with the Android operating system using Physics Toolbox Suite (https://www.vieyrasoftware.net/) for acquiring acceleration data. The second was a LPMS-B2 miniature Inertial Measurement Unit (IMU) using LPMS-Control software (https://lp-research.com/software/) for recording acceleration data. The sampling rate of both devices was 200 Hz.

Seven test subjects (24–35 years) were asked the place where they typically carry their smartphone, and the majority answered that in the hand. For this reason, all subjects were asked to carry the smartphone in the right hand during capture sessions, while the IMU was firmly attached on the right ankle using an elastic band. All subjects declared not having any mobility impairment in their limbs. In Figure 2a, the IMU placed on the right ankle of a test subject is shown.

The gait activities to be performed were: (i) going down an incline, (ii) going up an incline, (iii) walking on level ground, (iv) going down stairs, and (v) going up stairs. The test subjects were asked to perform each activity three times at a speed comfortable to them. The environments to carry out the activities were the same for all subjects, two of them indoors (Figure 2c,d), and the other outdoors (Figure 2b). To start and finish a new trial it was required that the subjects remained in an anatomical position and performed a controlled movement to synchronize both devices.

3.2. Strides Detection

As mentioned earlier, strides are characterized by periodic movements of the limbs, so in this study they are used as a unit of analysis. At this stage, the acceleration signals captured during the test sessions are preprocessed and segmented according to the steps 1–3 in Figure 1. The complete description of the strides detection stage is detailed in Algorithm 1. This process for detecting strides is generic and can be applied to acceleration signals from the smartphone or IMU. Figure 3 shows the process for segmenting the acceleration signals $acc$ in strides using the data of a “walking on level ground” trial of a test subject.    

Algorithm 1: Summary of the strides segmentation stage.

Inputs: $acc$: time-series comprising three axes: $\{x,y,z\}$. a: filter coefficient. w: window size for searching strides. Output: $ac{c}_{FD}$: Array of segmented forward-direction signal. $ac{c}_{XYZ}$: Array of segmented magnitude-vector of acceleration. Notation: $FD$ signal representing forward-direction axis-signal. $XYZ$ magnitude vector of acceleration. S signal can be $ac{c}_x$, $ac{c}_y$, o $ac{c}_z$. Start Preprocessing acceleration signals: for time-series S ∈ acc do $ac{c}_{FD}\leftarrow segmenting(FD,P_b,P_e)$; $ac{c}_{XYZ}\leftarrow segmenting(XYZ,P_b,P_e)$; End

In the preprocessing step, a low-pass filter is applied to the acceleration signals $ac{c}_X, ac{c}_Y, ac{c}_Z$ of both devices, using the coefficient a which filters the highest frequencies. According to this filter, if the value of the coefficient is 1 the signal remains the same. The value of a directly depends on the magnitude of the signal to be filtered, which was manually selected at 0.05. The purpose of the filter is to avoid artifacts due to movement and to filter the noise in acceleration signals. A pseudo-Gaussian smoothing function is then applied to the filtered signals based on w window sliding-average. As an example, in Figure 3a,b the raw acceleration signals $acc$ and the filtered and smoothed signals $S^{*}$ are shown, respectively. The first step is applied to the three axes of acceleration.

Because the signal aligned with the sagittal plane of people is the one that records a greater magnitude of acceleration, a search is conducted to identify which of the three acceleration signals $S_x^{*}, S_y^{*}, S_z^{*}$ has greater power (second step) and it will be labeled as the forward-direction signal $ac{c}_{FD}$. In addition, the acceleration magnitude vector $ac{c}_{XYZ}$, also called magnitude of total acceleration, is calculated in order to use information of the three signals combined. This vector has been widely used as it is invariant to device rotations [23].

In the third step, the patterns in the forward-direction signal related to strides are searched. These patterns are consistent to the heel strike events included in $ac{c}_{FD}$ [13]. The high local maxima or peaks P are extracted using w as sample-rate. The value of w is related to the cadence of people, and a value of 200 frames was manually selected because the optimal cadence of young people, between 21–40 years, is 48 and 60 steps/min [24]. In Figure 3c the peaks related to strides in the $ac{c}_{FD}$ signal are shown. Finally, the $ac{c}_{FD}$ and $ac{c}_{XYZ}$ signals are segmented based on the beginning $P_b$ and the end $P_e$ of each peak P. A simple way to find the limits of P is to search for the local minimums in the signals. The output of this stage is illustrated in Figure 3. The first and last strides were discarded in all trials because their anatomical characteristics with respect to the gait are atypical.

3.3. Gait Classification

In the second stage, a supervised classification approach was implemented. Five descriptive features were extracted from each segment in the signals $ac{c}_{FD}$ and $ac{c}_{XYZ}$ of both smartphone and IMU: (i) width of the peak ($time\_step$), (ii) height of the peak ($height$), (iii) mean of the acceleration ($mean$), (iv) standard deviation of the acceleration ($std$), and (v) power of the acceleration ($power$). The 2D feature space for smartphone and IMU data are illustrated in Figure A1, Figure A2 of the Appendix A, respectively. As it can be noticed, data or instances of the “walking on level ground” activity can be seen more separable than the other activities from the different combinations of the features.

In the last step, the feature vectors of both smartphone and IMU acceleration data are used for building the gait activity classifiers. Four inference algorithms were selected because they are appropriate to deal with problems involving unbalanced data [25,26]. Naive Bayes (NB) classifier is a classification method founded on the Bayes theorem based on the estimated conditional probabilities [27]. C4.5 is a decision tree classifier that builds a binary classification tree; determined by a splitting criterion, features are selected as branching points that separate the two classes [28]. Support Vector Machines (SVM) classify unseen information by deriving selected features and constructing a high dimensional hyperplane to separate the data points into two classes [29]. K-Nearest Neighbors (KNN) is a non-parametric method that does not need any modeling or explicit training phase before the classification process [30].

4. Results and Discussion

In order to evaluate the gait classifiers, feature vectors were divided into three treatments: (i) including only features extracted from the forward-direction signal $ac{c}_{FD}$, (ii) including only features extracted from the acceleration magnitude vector $ac{c}_{XYZ}$, and (iii) including both the $ac{c}_{FD}$ and $ac{c}_{XYZ}$ signals. The treatments were the same for smartphone and IMU acceleration data as it is shown in

Table 1. Correctly classified instances (%) using acceleration data from smartphone and IMU. NB: Naive Bayes; SVM: Support Vector Machines; and KNN: K-Nearest Neighbors.

Device	Treatment	Features	NB	C4.5	SVM	KNN	Avg ($\mathit{\sigma }$)
Smartphone	$ac{c}_{FD}$	5	53.4	67.6	50.3	68.6	60.0 (9.5)
	$ac{c}_{XYZ}$	5	52.8	61.4	50.0	58.6	55.7 (5.2)
	$ac{c}_{FD}+ac{c}_{XYZ}$	9	61.0	63.8	61.4	79.3	66.4 (8.7)
IMU	$ac{c}_{FD}$	5	60.0	63.8	58.6	81.0	67.1 (10.3)
	$ac{c}_{XYZ}$	5	50.7	68.6	49.7	75.9	61.2 (13.1)
	$ac{c}_{FD}+ac{c}_{XYZ}$	9	59.7	68.6	63.8	85.5	69.4 (11.4)

. While the $FD$ signal was used for segmenting the strides in $FD$ and $XYZ$, the feature $timestep$ is the same in treatments (i) and (ii). Data partition was made using leave-one-out cross-validation, which indicates that the classifiers were trained with the instances of six subjects (85.7% of instances) and tested with the instances of the remaining subject (14.3% of instances), repeating the process for the instances of each subject.

According to the classification results, the features extracted from the forward-direction signal improve the classification of the gait activities that when using the features extracted from the acceleration magnitude vector, both for smartphone data as for IMU data. In addition, in both cases, combining the characteristics of $ac{c}_{FD}$ and $ac{c}_{XYZ}$ improves the correctly classified instances. In general, a better inter-class classification average is obtained using the IMU acceleration data, although the standard deviation is smaller for the smartphone acceleration data. These values can be confirmed in the box-plot shown in Figure 4. Regarding inference algorithms, in general, Naive Bayes and Support Vector Machines score the lowest results, while C4.5 and KNN score the best. In particular, features of the third treatment in combination with the KNN algorithm score the best results with 79.3% for the smartphone and 85.5% for the IMU.

For an in-depth analysis of the classification results for each gait activity,

Table 2. Classification results for each gait activity using the data treatment $ac{c}_{FD}+ac{c}_{XYZ}$ and the KNN algorithm. TPR = true positive rate; TNR =true negative rate.

	Smartphone		IMU
Gait Activity	TPR	TNR	TPR	TNR
Going down an incline	0.871	0.964	0.871	0.959
Going up an incline	0.900	0.964	0.900	0.918
Walking on level ground	0.914	0.900	0.800	0.982
Going down stairs	0.600	0.972	0.825	0.980
Going up stairs	0.450	0.940	0.875	0.976
Weighted average	0.793	0.946	0.855	0.960

shows the results of sensitivity (or true positive rate TPR) and specificity (or true negative rate (TNR)) for the treatment that includes $ac{c}_{FD}$ and $ac{c}_{XYZ}$ features using the KNN algorithm.

According to these two metrics and for the smartphone data, the activity “walking on level ground” has the best rate of true positives, while the activities of “going up stairs” and “going down stairs” score the worst record; in turn, “going down stairs” has the best specificity, which indicates that the instances of the other activities were not confused mainly with this class. Regarding IMU data, the two actions of “going up an incline” and “going down an incline” score the best sensitivity, while the activity “walking on level ground” scores the best true negative rate of all the records. These results can be observed in the confusion matrices presented in Figure 5.

As it can be seen in Figure 5a, the classification model was overfitted to the “walking on level ground” activity since it scores the highest sensitivity (0.914) but the lowest specificity (0.900) among the five classes. Therefore, despite the fact that the majority of their instances were correctly classified, there were also many instances of the “going down stairs” and “going up stairs” activities that were confused with this class; almost the same number of predicted instances of “going up stairs” (45%) were misclassified as “walking on level ground” (38%). On the other hand, misclassified instances of “walking on level ground” in Figure 5b were predicted as “going down an incline” (7%) or “going up an incline” (13%). Although acceleration data of smartphone scored the highest classification value individually (0.914), when using IMU data, classification values greater than 0.800 were obtained for all the actions.

One of the advantages of using KNN as an inference algorithm is that it is not necessary to parameterize the classification models, so it depends entirely on the training set. Another advantage is that it is easy to implement this algorithm for a multi-class problem. However, the efficiency of this algorithm declines as the size of the training set grows, thus an extra task could be needed for choosing the most representative instances before using the models. Another disadvantage of the KNN algorithm is that it is very sensitive to outliers, so the selection of the number of neighbors is important. The number of neighbors k selected was 5, which was chosen to be consistent with the number of classes, and Euclidean distance was selected as distance criteria.

Regarding the relevance of the studied features, they were ranked using Information Gain [31], which evaluates the worth of each feature by measuring the entropy with respect to the class, i.e., gait activities. According to this analysis, the most relevant features that coincided in both data (smartphone and IMU) were: (i) FD-$std$ (0.641 and 0.798), (ii) FD-$height$ (0.662 and 0.516), and (iii) $time\_step$ (0.538 and 0.471). New treatments, or data partitions, were generated and classifiers were constructed using KNN considering the five most relevant features for both sets, obtaining a correct classification rate of 64.1% and 77.9%, for smartphone and IMU data, respectively. In both cases, the results of the second treatment (58.6% and 75.9%) of Table 1 are improved, which corresponds to using only the features extracted from $XYZ$, but they are less than the results of the first and third treatments, which corresponds to using the features extracted from $FD$ (68.6% and 81.0%), and the combination of $FD$ and $XYZ$ (79.3% and 85.5%). Therefore, the selection of relevant features did not help to improve the classification using the current data.

Finally, the results rely on the conditions of the study, i.e., the gait speed, the test subjects group, the environmental setting, among others. On the other hand, the proposed method is robust to the anatomical reference in which the sensing devices are carried or placed and to their rotation, as long as their position do not vary during the test session. Additionally, only the values of a and w have to be adapted in the preprocessing step; the first variable depends on the gait speed, and the second of the sampling rate.

5. Conclusions

In the present study, a method for recognition of gait activities was proposed based on the use of acceleration data of a commercial device widely adopted by the population (including seniors [32]), and a specialized low-cost device whose placement in the lower limb has been studied in the field [33]. These devices are a smartphone and an IMU, respectively.

This method of detecting gait that uses strides, or gait cycles, as a segmentation unit is independent of the sensing device of 3D acceleration. Currently, most portable devices incorporate a 3D acceleration sensor, from smartphones to lenses, such as Google Glass. Thus, we are considering to capture and evaluate acceleration data obtained by other commercial devices, such as smartwatches or fitness trackers.

The results obtained in this study are promising, and despite the conditions and limitations discussed previously, the gait activities studied are very similar to each other. This work is in development and the lessons learned will be used to improve the computational techniques applied. One of them to implement the method for online execution using the models learned. In addition, data from other populations will be collected, such as older adults, since the characteristics of their gait are different [34].