# IMU Consensus Exception Detection with Dynamic Time Warping—A Comparative Approach

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Technical Primitives

#### 2.1. Dynamic Time Warping

**x**and

**z**can first be checked using different sequence sizes for N and M, respectively, i.e.,

_{n}, n = 1, 2, …, N, and z

_{m}, m = 1, 2, …, M, and denoted as a cost function δ(n, m). δ(n, m) is thus defined as

**x**and

**z**. The contiguous path Ω, recorded and warped from the first cell (1, 1) to the final cell (N, M), provides the total warping cost Σ as:

**x**and

**z**, as shown in Figure 1. Lower scores of Σ imply higher similarity between sequences of unequal length. According to this alignment, the optimal warping path Ω becomes the best fit of the two sequences, and paves the way to achieving the greatest similarity of the aligned sequences, even if they are unequal in length. Here, three points are attributed to DTW to enhance the dynamic matching as follows:

- Controlled endpoint matching
- Floating adjustable local matching
- The most correlated global matching

#### 2.2. Neural Network Matching

_{j}, j = 1, 2, …, for conventional activation:

_{j}, j = 1, 2, …, are the inputs of the neuron, w

_{j}is the corresponding weight of i

_{j}, and θ is a real bias constant. While the linear combination of the inputs simply manipulate the signals for weighted synthesis, the activation function g(∙), which is inspired by biological brains, mimics a biological synaptic excitation and inhibition, transforming the activation level of the neuron into two basic state outputs: either on or off.

_{i}= w

_{i}

_{, new}− w

_{i}

_{, old}, E

_{d}is the error function, and η is the learning ratio, 0 < η < 1. The approximated error would converge iteratively under a cyclic feed-forward back-propagation evolution procedure to reach an accurate prediction. The layered BPNN will well adapt the system response based on learning from actual instances. Through iteratively propagating the errors backward to update the structural interconnection weights internally, a satisfactory precision can still be obtained, even for a complex problem.

#### 2.3. Equalization Preprocessor Ahead of ANN

#### 2.3.1. The DFT Interpolation Preprocessor

**x**and

**x**= [x

_{1}, x

_{2}, …, x

_{n}, …, x

_{N}]

^{T}, the DFT of

**x**is

**x**to generate a new sequence

**z**= [z

_{1}, z

_{2}, …, z

_{m},…, z

_{M}], where M > N. A procedure is first issued to separate the DFT sequence into two sequences of equal length, and then to pad M–N zeros in between the two sequences as shown below:

#### 2.3.2. The Convolutional Preprocessor

**x**and

**h**, are convoluted directly to result in

**z**in the time domain, their corresponding frequency-domain pairs, $F\left(x\right)$ and $\text{}F\left(h\right)$, are multiplied to gain $F\left(z\right)$, i.e.,

**x**and

**h**are two input sequences with lengths N and Q respectively. To comprehend the temporal operation, the indices of the label of the two sequences are discretized as n, n = 1, 2, …, N, and q, q = 1, 2, …, Q for

**x**and

**h**, respectively. Sequence

**z**corresponds to the output, indexed by m, m = 1, 2, …, M, of the operation. The discrete convolution of the two finite sequences is then rewritten as

_{m}is obtained by summing the products of the two sequences, x and h, all over the indices of n, where h has been flipped and shifted temporally ahead by m + 1.

**x**,

**h**, and

**z**, an equation can easily be set up to explain the relationship among the sequences according to (12), i.e.,

**h**is made to be adjustable in length through the convolutional operation. While it is independent of the contents of

**h**, the length of the output sequence

**z**is alternatively dependent on the sequence length of

**h**. For certain given N and M for sequences

**x**and

**z**, a nonzero sequence

**h**of automatically adjustable length M–N+1 suffices to make up the difference between M and N, and makes it possible to output the stretched

**z**sequences from their corresponding

**x**inputs even with different lengths N. To preserve as much as possible the original discriminative information carried by

**x**, a series of impulses, based on the operator’s intuition for the system’s impulse response, was selected for

**h**. This selection could not only mitigate the loss of the original discriminative information of the input sequence but could also simplify the conversion. The impulse train of the constant sequence is, in fact, a moving average operator in the time domain or a low-pass filter in the frequency domain. Due to the variable length of

**x**, the length of

**h**might vary case by case. According to (13), the outputs of the preprocessor can be compensated such that they have the same expected size, a fixed length of M, and this can be applied accordingly as the input for the subsequent ANN postprocessor.

## 3. Materials and Methods

- Confirming first the communication via Bluetooth between the IMU and the mobile phone;
- Labeling the pattern which the volunteer wants to generate by entering a numerical code; and
- Recording a pattern representing the motion of the volunteer by pressing the start/stop button on the app.

## 4. Simulated Results and Discussions

#### 4.1. Methodological Verification by Operational Elaboration

**x**

_{a}and

**x**

_{c}exhibit a similar tendency, and

**x**

_{a}and

**x**

_{b}tend towards being contrary to one another.

**x**

_{a}and

**x**

_{b}, and between

**x**

_{a}and

**x**

_{c}, respectively. There are fewer distorted alignments between

**x**

_{a}and

**x**

_{c}than between

**x**

_{a}and

**x**

_{b}, as can be seen by comparing Figure 9a,b. As shown in Figure 9, the fewer the distorted alignments, the higher the similarity between the compared patterns. This means that the motion behind

**x**

_{c}is closer to that behind

**x**

_{a}than to that behind

**x**

_{b}. With reference to (4) in order to calculate warping costs, Figure 10a,b show the optimal warping paths for the alignment of Figure 9a,b, respectively. As Figure 10 illustrates, a route that directly connects the upper left corner in the diagram with the lower right corner without any form of detour is the optimal match, and retains the lowest matching cost. The greater the detour the exhibited by the route, the greater the distortion implied of the point-to-point alignments (Figure 9), and the greater the expenditure of warping cost. In brief, a shorter warping path implicitly represents a lower warping cost. In the case shown in Figure 10b, it requires less warping cost than the case shown in Figure 10a. To confirm this explanation, the corresponding total warping costs for both cases were calculated as 0.86 and 0.38, respectively, i.e., the cost degrees were consistent with the eye checking of the original designated patterns.

**z**s for the three inputs

**x**

_{a},

**x**

_{b}, and

**x**

_{c}in Figure 8 using both of the preprocessors. Both the preprocessors ensure that the length of their output sequences will be 15 points and normalize into the range of [0, 1] for better recognition in next ANN stage. Figure 11 and Figure 12 show that all of the transformed

**z**s, which have been resampled as 15-point sequences, still approximately maintain the tendency of their original inputs. This fact potentially supports confidence in the success of the corresponding pattern recognition when implementing a consecutive ANN.

#### 4.2. Recognition Performance Validations

**z**

_{t}

_{1}and

**z**

_{t}

_{2}, were then used to match the unlabeled testing sample

**z**

_{s}. The decision-making mechanism for assigning sample

**z**to a specific category can thus be defined in a simple way as:

**z**) renders DTW comparable with outputs of the ANN inference. The assessment set k = 5 for the cross-validation, i.e., each fold contained 26 sample sequences for the circulation of the folded replications. Average accuracy was eventually produced after the five replications had been completed. Table 2 shows the performance of the folded replications and the average accuracy.

#### 4.3. Consideration of Model Preparation

## 5. Real-Time Application

#### 5.1. Matching Pattern Paradigm (MPP) Selection

#### 5.2. Performance of Real-Time Abnormality Detection

## 6. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## References

- De Sá, J.P.M. Pattern Recognition: Concept, Methods and Applications; Springer-Verlag: Berlin/Heidelberg, Germany, 2001. [Google Scholar]
- Jain, A.K.; Duin, R.P.W.; Mao, J. Statistical pattern recognition: A review. IEEE Trans. Pattern Anal. Mach. Intell.
**2000**, 22, 4–37. [Google Scholar] [CrossRef] - Hirschberg, D.; Meyers, G. Combinatorial Pattern Matching; Springer-Verlag: Berlin/Heidelberg, Germany, 1996. [Google Scholar]
- Brunelli, R. Template Matching Techniques in Computer Vision: Theory and Practice, 1st ed.; John Wiley & Sons: Hoboken, NJ, USA, 2009. [Google Scholar]
- Hu, W.; Gharuib, A.; Hafez, A. Template match object detection for inertial navigation systems. Positioning
**2011**, 2, 78–83. [Google Scholar] [CrossRef] - Brown, L.G. A survey of image registration techniques. ACM Comput. Surv.
**1992**, 24, 325–376. [Google Scholar] [CrossRef][Green Version] - Bay, H.; Ess, A.; Tuytelaars, T.; Gool, L.V. Speeded-up robust features. Comput. Vis. Image Underst.
**2008**, 110, 346–359. [Google Scholar] [CrossRef] - Klevans, R.L.; Rodman, R.D. Voice Recognition; Artech House Inc.: Norwood, MA, USA, 1997. [Google Scholar]
- Prabhakar, O.P.; Sakoe, N.K. A survey on voice command recognition technique. Int. J. Adv. Res. Comput. Sci. Softw. Eng.
**2013**, 3, 576–585. [Google Scholar] - Sakoe, H.; Chiba, S. Dynamic programming algorithm optimization for spoken word recognition. IEEE Trans. Acoust. Speech, Signal Process.
**1978**, 26, 43–49. [Google Scholar] [CrossRef] - Fang, C. From Dynamic Time Warping (DTW) to Hidden Markov Model (HMM); Final project report, ECE742 Stochastic Decision; University of Cincinnati: Cincinnati, OH, USA, 19 March 2009. [Google Scholar]
- Bartolucci, F.; Farcomeni, A.; Pennoni, F. Latent Markov Models for Longitudinal Data; Chapman and Hall/CRC: Boca Raton, FL, USA, 2012. [Google Scholar]
- Derawi, M.O.; Bours, P.; Holien, K. Improved cycle detection for accelerometer based gait authentication. In Proceedings of the 2010 Sixth International Conference on Intelligent Information Hiding and Multimedia Signal Processing, Darmstadt, Germany, 15–17 October 2010. [Google Scholar]
- Wu, J.; Konrad, J.; Ishwar, P. Dynamic time warping for gesture-based user identification and authentication with Kinect. In Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Process, Vancouver, BC, Canada, 26–31 May 2013. [Google Scholar]
- Raheja, J.L.; Minhas, M.; Prashanth, D.; Shah, T.; Chaudhary, A. Robust gesture recognition using Kinect: A comparison between DTW and HMM. Optik-Int. J. Light Electron Opt.
**2015**, 126, 1098–1104. [Google Scholar] [CrossRef] - Gabayan, K.; Lansel, S. Programming-by-Example Gesture Recognition; Report CS229; Department of Statistics, Stanford University: Stanford, CA, USA, 2006. [Google Scholar]
- Barczewska, K.; Drozd, A. Comparison of methods for hand gesture recognition based on dynamic time warping algorithm. In Proceedings of the 2013 Federated Conference on Computer Science and Information Systems, Kraków, Poland, 8–11 September 2013. [Google Scholar]
- Chou, Y.-H.; Cheng, H.-C.; Cheng, C.-H.; Su, K.-H.; Yang, C.-Y. Dynamic time warping for IMU based activity detection. In Proceedings of the 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Budapest, Hungary, 9–12 October 2016. [Google Scholar]
- Cheng, Y.-W.; Chen, P.-Y.; Yang, C.-Y.; Samani, H. IMU based activity detection for post mini-stroke healthcare. In Proceedings of the 2016 International Conference on System Science and Engineering (ICSSE), Puli, Taiwan, 7–9 July 2016. [Google Scholar]
- Michalski, R.S.; Carbonell, J.G.; Mitchell, T.M. Machine Learning: An Artificial Intelligence Approach; Morgan Kaufmann Publishers Inc.: Burlington, MA, USA, 1986. [Google Scholar]
- Haykin, S. Neural Networks—A Comprehensive Foundation; Prentice Hall: Upper Saddle River, NJ, USA, 1998. [Google Scholar]
- Bishop, M.C. Pattern Recognition and Machine Learning; Springer-Verlag: New York, NY, USA, 2006. [Google Scholar]
- Hayes, M.H. Schaums Outline of Digital Signal Processing; McGraw Hill: New York, NY, USA, 1999. [Google Scholar]
- Ciaurri, Ó.; Varona, J. A Whittaker-Shannon-Kotel’nikov sampling theorem related to the Dunkl transform. Proc. Am. Math. Soc.
**2007**, 135, 2939–2947. [Google Scholar] [CrossRef] - Pacola, E.R.; Quandt, V.I.; Schneider, F.K.; Sovierzoski, M.A. The wavelet transform border effect in EEG spike signals. In Proceedings of the World Congress on Medical Physics and Biomedical Engineering, Beijing, China, 26–31 May 2012. [Google Scholar]
- Gray, R.M.; Goodman, J.W. Fourier Transforms; Kluwer Academic Publishers: Boston, MA, USA, 1995. [Google Scholar]
- Chou, Y.-H.; Chen, P.-Y.; Yang, C.-Y.; Wang, J.-J. Make various-length vectors equal in length: A convolutional approach. In Proceedings of the 2015 International Conference on Fuzzy Theory and Its Applications, Yilan, Taiwan, 18–20 November 2015. [Google Scholar]
- Tjacket. Available online: https://www.mytjacket.com/ (accessed on 15 April 2019).
- TDK Invensense MPU-6500. Available online: https://www.invensense.com/products/motion-tracking/6-axis/mpu-6500/ (accessed on 15 April 2019).

**Figure 10.**Optimal warping paths along with accumulated warping cost calculations corresponding to Figure 9. (

**a**) Between

**x**

_{a}and

**x**

_{b}; (

**b**) Between

**x**

_{a}and

**x**

_{c}.

**Figure 11.**DFT interpolation preprocessed sequences

**z**

_{a},

**z**

_{b}, and

**z**

_{c}from inputs

**x**

_{a},

**x**

_{b}, and

**x**

_{c}, respectively.

**Figure 12.**Convolutionally equalized preprocessed sequences

**z**

_{a},

**z**

_{b}, and

**z**

_{c}from inputs

**x**

_{a},

**x**

_{b}, and

**x**

_{c}, respectively.

Specifications of Mobile Phone | Specifications of Desktop Computer | ||
---|---|---|---|

Model | ASUS ZenFone ZC500TG | Processor | Intel i7-3770, 3.40 GHz |

Processor | MediaTek 6580 QuadCore, 1.3 GHz | Memory | DDR3-1600 8 GB |

Memory | 2GB RAM | Storage | Toshiba SATAII 1TB, 7200 rpm |

Storage | 8GB eMMC Flash | Operation System | Microsoft Windows 7 |

Operation System | Android 5.1 | Display | AMD Radeon HD 6670, 2 GB |

Display | 1280 × 720 pixels, 5 inches | USB | USB 3.0 |

Training Folder | Validation Folder | Correct Count | Error Count | Execution Time (sec) | Accuracy (%) |
---|---|---|---|---|---|

#1, #2, #3, #4 | #5 | 24 | 2 | 1.84 | 92.31 |

#1, #2, #3, #5 | #4 | 25 | 1 | 1.95 | 96.15 |

#1, #2, #4, #5 | #3 | 24 | 2 | 1.76 | 92.31 |

#1, #3, #4, #5 | #2 | 25 | 1 | 2.21 | 96.15 |

#2, #3, #4, #5 | #1 | 24 | 2 | 2.13 | 92.31 |

Average | 24.4 | 1.6 | 1.98 | 93.85 |

Training Folder | Validation Folder | Correct Count | Error Count | Execution Time (sec) | Accuracy (%) |
---|---|---|---|---|---|

#1, #2, #3, #4 | #5 | 22 | 4 | 3.58 | 84.62 |

#1, #2, #3, #5 | #4 | 24 | 2 | 2.84 | 92.31 |

#1, #2, #4, #5 | #3 | 22 | 4 | 2.93 | 84.62 |

#1, #3, #4, #5 | #2 | 23 | 3 | 2.74 | 88.46 |

#2, #3, #4, #5 | #1 | 24 | 2 | 3.37 | 92.31 |

Average | 22.75 | 3.25 | 3.02 | 87.50 |

Training Folder | Validation Folder | Correct Count | Error Count | Execution Time (sec) | Accuracy (%) |
---|---|---|---|---|---|

#1, #2, #3, #4 | #5 | 22 | 4 | 2.74 | 84.62 |

#1, #2, #3, #5 | #4 | 24 | 2 | 3.53 | 92.31 |

#1, #2, #4, #5 | #3 | 21 | 5 | 3.48 | 80.77 |

#1, #3, #4, #5 | #2 | 23 | 3 | 2.92 | 88.46 |

#2, #3, #4, #5 | #1 | 23 | 3 | 3.56 | 88.46 |

Average | 22.6 | 3.4 | 3.25 | 86.92 |

Model | Prediction Accuracy (%) | Prediction Execution Time (sec) | Model Preparing Time (sec) | ||
---|---|---|---|---|---|

Feature Preprocessing Time (sec) | ANN Training Time (sec) | Total Model Preparing Time (sec) | |||

DTW | 93.85 | 1.98 | − | − | 4.72 |

ANN with Preprocessor 1 * | 87.5 | 3.02 | 3.6 | 7.06 | 10.66 |

ANN with Preprocessor 2 ** | 86.92 | 3.25 | 1 | 9.72 | 10.72 |

Method to Generate Matching Pattern Paradigm | Accuracy (%) |
---|---|

Averaging with Convolutional Equalizer | 91.54 |

Averaging with DFT Interpolation Equalizer | 97.69 |

Randomly Selection without Averaging | 95.38 |

Motion | Test Count of Every Volunteer | Averaged Missed Count | Averaged Error Count | Averaged Accuracy (%) | Averaged Time to Response (s) |
---|---|---|---|---|---|

Sit-down | 30 | 2.79 | 0.86 | 87.86 | 1.17 |

Stand-up | 30 | 2.07 | 1.11 | 89.40 | 1.52 |

Average | 30 | 2.43 | 0.98 | 88.63 | 1.34 |

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Yang, C.-Y.; Chen, P.-Y.; Wen, T.-J.; Jan, G.E. IMU Consensus Exception Detection with Dynamic Time Warping—A Comparative Approach. *Sensors* **2019**, *19*, 2237.
https://doi.org/10.3390/s19102237

**AMA Style**

Yang C-Y, Chen P-Y, Wen T-J, Jan GE. IMU Consensus Exception Detection with Dynamic Time Warping—A Comparative Approach. *Sensors*. 2019; 19(10):2237.
https://doi.org/10.3390/s19102237

**Chicago/Turabian Style**

Yang, Chan-Yun, Pei-Yu Chen, Te-Jen Wen, and Gene Eu Jan. 2019. "IMU Consensus Exception Detection with Dynamic Time Warping—A Comparative Approach" *Sensors* 19, no. 10: 2237.
https://doi.org/10.3390/s19102237