# Detecting Steps Walking at very Low Speeds Combining Outlier Detection, Transition Matrices and Autoencoders from Acceleration Patterns

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Related Work

## 3. Sensor and Data Series

## 4. Outlier Pre-Detection

## 5. Transition Matrices

_{i}, the transition matrices are computed form ${a}_{g}\left({t}_{i-N},\dots ,{t}_{i},\dots ,{t}_{i+N}\right)$ and ${a}_{Hx}\left({t}_{i-N},\dots ,{t}_{i},\dots ,{t}_{i+N}\right)$. Each sample is assigned to a state according to the distance to the mean value of the series in terms of the standard deviation of the series. We have used eight states as follows (for the particular case of ${a}_{g}\left(t\right)$):

- m = mean value of (${a}_{g}\left({t}_{i-N},\dots ,{t}_{i},\dots ,{t}_{i+N}\right)$)
- std = standard deviation of $({a}_{g}\left({t}_{i-N},\dots ,{t}_{i},\dots ,{t}_{i+N}\right))$
- S
_{1}if ${a}_{g}\left(t\right)$ − m ≤ −1.5*std - S
_{2}if ${a}_{g}\left(t\right)$ − m ≤ −std and ${a}_{g}\left(t\right)$ − m > −1.5*std - S
_{3}if ${a}_{g}\left(t\right)$ − m ≤ −0.5*std and ${a}_{g}\left(t\right)$ −m > −std - S
_{4}if ${a}_{g}\left(t\right)$ − m ≤ 0 and ${a}_{g}\left(t\right)$ − m > −0.5*std - S
_{5}if ${a}_{g}\left(t\right)$ − m ≤ 0.5*std and ${a}_{g}\left(t\right)$ − m > 0 - S
_{6}if ${a}_{g}\left(t\right)$ − m ≤ std and a_{g}(t) − m > 0.5*std - S
_{7}if ${a}_{g}\left(t\right)$ − m ≤ 1.5*std and ${a}_{g}\left(t\right)$ − m > std - S
_{8}if ${a}_{g}\left(t\right)$ − m > 1.5*std

_{i}. The transition matrix counts the number of times that being at state j goes to state k in the next instant of time. A final regularization is performed to convert counts to probabilities by dividing the cumulative count of each row, as captured in Equation (4), where the component in row j and column k of the transition matrix is calculated.

## 6. Autoencoders

_{1}and f

_{2}are activation functions such as the sigmoid function). In our case, x corresponds to the vector calculated from serializing the transition matrix T, as described in the previous section.

## 7. Results

#### 7.1. Experiment Set-up and Database

- to stand still for 5 s (this information will be used to validate the calibration of the gravity sensor and to mark the start of the data);
- walk at a speed of 60 steps per minute during 60 s;
- to stand still for 5 s (this information will be used to validate the calibration of the gravity sensor and facilitate the automatic split of the recorded data into segments of single activities);
- walk at a speed of 30 steps per minute during 60 s;
- to stand still for 5 s;
- walk at a speed of 40 steps per minute during 60 s;
- to stand still for 5 s;
- slide (walk without separating the feet from the ground) at a speed of 30 steps per minute during 60 s;
- to stand still for 5 s;
- sit down and up 10 times;
- to stand still for 5 s;
- walk around a chair (in circles) at a speed of 30 steps per minute during 60 s; and,
- to stand still for 5 s.

#### 7.2. Implemented Algorithm Details

- Define the maximum and minimum cadences of steps to be detected (c
_{max}and c_{min}) in steps per minute → in our case c_{max}= 40 and c_{min}= 30 - Set the outlier detection window to ${T}_{out}=\frac{60}{{c}_{max}}s$
- For each T
_{out}= 1.5 s of ${a}_{g}\left(t\right),{a}_{Hx}\left(t\right)$ centered at ${t}_{c}$, calculate the Mahalanobis distance $\mathrm{md}\left({t}_{c}\right)$ from $\left({a}_{g}\left({t}_{c}\right),{a}_{Hx}\left({t}_{c}\right)\right)$ and $\left[\left({a}_{g}\left({t}_{c}-0.75\right),{a}_{Hx}\left({t}_{c}-0.75\right)\right),\dots ,\left({a}_{g}\left({t}_{c}+0.75\right),{a}_{Hx}\left({t}_{c}+0.75\right)\right)\right]$ - For all ${t}_{c}$ in (0:${t}_{max}$), if $\mathrm{md}\left(t={t}_{c}\right)>\mathrm{th}$ then consider ${t}_{c}$ an outlier (th = 3 has been empirically selected).
- For each ${t}_{c}$. corresponding to an outlier use $\left[\left({a}_{g}\left({t}_{c}-0.12\right),{a}_{Hx}\left({t}_{c}-0.12\right)\right),\dots ,\left({a}_{g}\left({t}_{c}+0.12\right),{a}_{Hx}\left({t}_{c}+0.12\right)\right)\right]$ to feed an autoencoder with a single hidden layer with 5 hidden units.
- Calculate the Pearson correlation index between the input and output of the autoencoder as a similarity index to decide if the outlier corresponds to a step.

- Define the maximum and minimum cadences of steps to be detected (c
_{max}and c_{min}) in steps per minute → in our case c_{max}= 40 and c_{min}= 30 - Set the outlier detection window to ${T}_{out}=\frac{60}{{c}_{max}}s$
- For each T
_{out}= 1.5 s of ${a}_{g}\left(t\right),{a}_{Hx}\left(t\right)$ centered at ${t}_{c}$, calculate the Mahalanobis distance $\mathrm{md}\left({t}_{c}\right)$ from $\left({a}_{g}\left({t}_{c}\right),{a}_{Hx}\left({t}_{c}\right)\right)$ and $\left[\left({a}_{g}\left({t}_{c}-0.75\right),{a}_{Hx}\left({t}_{c}-0.75\right)\right),\dots ,\left({a}_{g}\left({t}_{c}+0.75\right),{a}_{Hx}\left({t}_{c}+0.75\right)\right)\right]$ - For all ${t}_{c}$ in (0:${t}_{max}$), if $\mathrm{md}\left(t={t}_{c}\right)>\mathrm{th}$ then consider ${t}_{c}$ an outlier (th = 3 has been empirically selected).
- For each ${t}_{c}$ corresponding to an outlier use $\left[\left({a}_{g}\left({t}_{c}-0.12\right),{a}_{Hx}\left({t}_{c}-0.12\right)\right),\dots ,\left({a}_{g}\left({t}_{c}+0.12\right),{a}_{Hx}\left({t}_{c}+0.12\right)\right)\right]$ in order to estimate the transition matrix as described in Section 5 (being N = 6). The values of ${a}_{g}\left(t\right)\mathrm{and}{a}_{Hx}\left(t\right)$ are mapped into 8 different states (this number has been empirically selected) as described in Section 5. The states are assigned depending on the distance of each pair ${a}_{g}\left(t\right)$, ${a}_{Hx}\left(t\right)$ to the mean values of ${a}_{g}\left[T\right]$ and ${a}_{Hx}\left[T\right]$ in the previously selected 240 ms time window centered at each outlier in terms of their standard deviation. This normalization is required in order to compensate different user weights.
- Use the transition matrices to feed an autoencoder with a single hidden layer with five hidden units.
- Calculate the Pearson correlation index between the input and output of the autoencoder as a similarity index to decide if the outlier corresponds to a step.

#### 7.3. Autoencoders Based on Acceleration Data around Outlier Pre-Detected Points

#### 7.4. Autoencoders Based on Transition Matrices around Outlier Pre-Detected Points.

## 8. Conclusions

## Acknowledgments

## Author Contributions

## Conflicts of Interest

## References

- Wang, A.; Chen, G.; Yang, J.; Zhao, S.; Chang, C. A Comparative Study on Human Activity Recognition Using Inertial Sensors in a Smartphone. IEEE Sens. J.
**2016**, 16, 4566–4578. [Google Scholar] [CrossRef] - Hassanalieragh, M.; Page, A.; Soyata, T.; Sharma, G.; Aktas, M.; Mateos, G.; Kantarci, B.; Andreescu, S. Health Monitoring and Management Using Internet-of-Things (IoT) Sensing with Cloud-based Processing: Opportunities and Challenges. In Proceedings of the 2015 IEEE International Conference on Services Computing (SCC), New York, NY, USA, 27 June–2 July 2015. [Google Scholar]
- Avci, A.; Bosch, S.; Marin-Perianu, M.; Marin-Perianu, R.; Havinga, P. Activity recognition using inertial sensing for healthcare, wellbeing and sports applications: A survey. In Proceedings of the 2010 23rd International Conference on Architecture of Computing Systems (ARCS); VDE: Hannover, Germany, 2010. [Google Scholar]
- Lara, O.D.; Labrador, M.A. A survey on human activity recognition using wearable sensors. IEEE Commun. Surv. Tutor.
**2013**, 15, 1192–1209. [Google Scholar] [CrossRef] - Bassett, D.R., Jr; John, D. Use of pedometers and accelerometers in clinical populations: Validity and reliability issues. Phys. Ther. Rev.
**2013**, 15, 135–142. [Google Scholar] [CrossRef] - Feng, Y.; Wong, C.K.; Janeja, V.; Kuber, R.; Mentis, H.M. Comparison of tri-axial accelerometers step-count accuracy in slow walking conditions. Gait Posture
**2017**, 53, 11–16. [Google Scholar] [CrossRef] [PubMed] - Sandroff, B.M.; Motl, R.W.; Pilutti, L.A.; Learmonth, Y.C.; Ensari, I.; Dlugonski, D.; Klaren, R.E.; Balantrapu, S.; Riskin, B.J. Accuracy of StepWatch™ and ActiGraph Accelerometers for Measuring Steps Taken among Persons with Multiple Sclerosis. PLoS ONE
**2014**, 9, e93511. [Google Scholar] [CrossRef] [PubMed] - O’Connell, S.; ÓLaighin, G.; Quinlan, L.R. When a Step Is Not a Step! Specificity Analysis of Five Physical Activity Monitors. PLoS ONE
**2017**, 12, e0169616. [Google Scholar] [CrossRef] [PubMed] - Motl, R.W.; Snook, E.M.; Agiovlasitis, S. Does an accelerometer accurately measure steps taken under controlled conditions in adults with mild multiple sclerosis? Disabil. Health J.
**2011**, 4, 52–57. [Google Scholar] [CrossRef] [PubMed] - Korpan, S.M.; Schafer, J.L.; Wilson, K.C.; Webber, S.C. Effect of ActiGraph GT3X+ Position and Algorithm Choice on Step Count Accuracy in Older Adults. J. Aging Phys. Act.
**2015**, 23, 377–382. [Google Scholar] [CrossRef] [PubMed] - Moon, Y.; McGinnis, R.S.; Seagers, K.; Motl, R.W.; Sheth, N.; Wright, J.A., Jr; Ghaffari, R.; Sosnoff, J.J. Monitoring gait in multiple sclerosis with novel wearable motion sensors. PLoS ONE
**2017**, 12, e0171346. [Google Scholar] [CrossRef] [PubMed] - Arcidiacono, C.; Porto, S.M.; Mancino, M.; Cascone, G. A threshold-based algorithm for the development of inertial sensor-based systems to perform real-time cow step counting in free-stall barns. Biosyst. Eng.
**2017**, 153, 99–109. [Google Scholar] [CrossRef] - DIrican, A.C.; Aksoy, S. Step Counting Using Smartphone Accelerometer and Fast Fourier Transform. Sigma J. Eng. Nat. Sci.
**2017**, 8, 175–182. [Google Scholar] - Fasel, B.; Duc, C.; Dadashi, F.; Bardyn, F.; Savary, M.; Farine, P.A.; Aminian, K. A wrist sensor and algorithm to determine instantaneous walking cadence and speed in daily life walking. Med. Biol. Eng. Comput.
**2017**, 55, 1773–1785. [Google Scholar] [CrossRef] [PubMed] - Munoz-Organero, M.; Lotfi, A. Human movement recognition based on the stochastic characterisation of acceleration data. Sensors
**2016**, 16, 1464. [Google Scholar] [CrossRef] [PubMed] - Godfrey, A.; Morris, R.; Hickey, A.; Del Din, S. Beyond the front end: Investigating a thigh worn accelerometer device for step count and bout detection in Parkinson’s disease. Med. Eng. Phys.
**2016**, 38, 1524–1529. [Google Scholar] [CrossRef] [PubMed] - Yuen, K.V.; Ortiz, G.A. Outlier detection and robust regression for correlated data. Comput. Methods Appl. Mech. Eng.
**2017**, 313, 632–646. [Google Scholar] [CrossRef] - Selmanaj, D.; Corno, M.; Savaresi, S.M. Hazard Detection for Motorcycles via Accelerometers: A Self-Organizing Map Approach. IEEE Trans. Cybern.
**2016**, PP, 1–12. [Google Scholar] [CrossRef] [PubMed] - Liang, Z.; Martell, M.A.C.; Nishimura, T. A Personalized Approach for Detecting Unusual Sleep from Time Series Sleep-Tracking Data. In Proceedings of the 2016 IEEE International Conference on Healthcare Informatics (ICHI), Chicago, IL, USA, 4–7 October 2016; pp. 18–23. [Google Scholar]
- Ma, M.X.; Ngan, H.Y.; Liu, W. Density-based Outlier Detection by Local Outlier Factor on Largescale Traffic Data. In IS&T International Symposium on Electronic Imaging Science and Technology 2016: Image Processing: Machine Vision Applications IX; Society for Imaging Science and Technology (IS&T): Springfield, VA, USA, 2016; pp. 1–4. [Google Scholar]
- Tang, J.; Ngan, H.Y. Traffic Outlier Detection by Density-Based Bounded Local Outlier Factors. Inf. Technol. Ind.
**2016**, 4, 6–18. [Google Scholar] - Palma, C.; Salazar, A.; Vargas, F. Automatic Detection of Deviations in Human Movements Using HMM: Discrete vs. Continuous. In International Symposium on Visual Computing; Springer International Publishing: Cham, Switzerland, 2016; pp. 534–543. [Google Scholar]
- Medrano, C.; Igual, R.; García-Magariño, I.; Plaza, I.; Azuara, G. Combining novelty detectors to improve accelerometer-based fall detection. Med. Biol. Eng. Comput.
**2017**, 55. [Google Scholar] [CrossRef] [PubMed] - Khan, S.S.; Karg, M.E.; Kulić, D.; Hoey, J. Detecting falls with X-Factor Hidden Markov Models. Appl. Soft Comput.
**2017**, 55, 168–177. [Google Scholar] [CrossRef] - Le, Q.V. A Tutorial on Deep Learning Part 2: Autoencoders, Convolutional Neural Networks and Recurrent Neural Networks. Google Brain. 2015. Available online: http://www.cs.mcgill.ca/~dprecup/courses/ML/Materials/dl-tutorial2.pdf (accessed on 28 September 2017).
- MathWorks Documentation. The Autoencoder class in Matlab. Available online: https://es.mathworks.com/help/nnet/ref/autoencoder-class.html (accessed on 28 September 2017).

**Figure 2.**Acceleration samples around outliers corresponding to slow walking steps. Each color represents a different sample.

Participant ID | Age | Gender | Normal Walk |
---|---|---|---|

1 | 24 | M | Y |

2 | 41 | F | Y |

3 | 45 | M | Y |

Sim Thr | Recall | Precision | F Score |
---|---|---|---|

0.40 | 0.77 | 0.50 | 0.61 |

0.50 | 0.75 | 0.53 | 0.62 |

0.60 | 0.67 | 0.59 | 0.63 |

0.70 | 0.50 | 0.65 | 0.56 |

0.80 | 0.33 | 0.73 | 0.46 |

0.90 | 0.02 | 0.50 | 0.04 |

Sit down | Walk in Circles | Slide | Walk 60 spm | |
---|---|---|---|---|

% detected as → | 0.00 | 6.83 | 55.82 | 37.36 |

Sim Thr | Recall | Precision | F Score |
---|---|---|---|

0.40 | 0.88 | 0.50 | 0.64 |

0.50 | 0.79 | 0.54 | 0.64 |

0.60 | 0.73 | 0.57 | 0.64 |

0.70 | 0.67 | 0.64 | 0.65 |

0.80 | 0.60 | 0.74 | 0.67 |

0.90 | 0.44 | 0.78 | 0.56 |

Sit down | Walk in Circles | Slide | Walk 60 spm | |
---|---|---|---|---|

% detected as → | 0.00 | 5.07 | 46.20 | 48.73 |

© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Muñoz-Organero, M.; Ruiz-Blázquez, R.
Detecting Steps Walking at very Low Speeds Combining Outlier Detection, Transition Matrices and Autoencoders from Acceleration Patterns. *Sensors* **2017**, *17*, 2274.
https://doi.org/10.3390/s17102274

**AMA Style**

Muñoz-Organero M, Ruiz-Blázquez R.
Detecting Steps Walking at very Low Speeds Combining Outlier Detection, Transition Matrices and Autoencoders from Acceleration Patterns. *Sensors*. 2017; 17(10):2274.
https://doi.org/10.3390/s17102274

**Chicago/Turabian Style**

Muñoz-Organero, Mario, and Ramona Ruiz-Blázquez.
2017. "Detecting Steps Walking at very Low Speeds Combining Outlier Detection, Transition Matrices and Autoencoders from Acceleration Patterns" *Sensors* 17, no. 10: 2274.
https://doi.org/10.3390/s17102274