# Navigating Virtual Environments Using Leg Poses and Smartphone Sensors

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Related Work

## 3. Theoretical Background

#### 3.1. Physical Equipment

#### 3.2. Dataset Description

#### 3.3. Machine Learning Techniques

#### 3.3.1. Regression

^{2}), which provides the variance in the outcome that is explained by the regression model, is given as:

#### 3.3.2. Artificial Neural Networks

#### 3.3.3. K-Nearest Neighbor

_{q}. This causes the closer neighbors to have greater weights. But, the processing time to fit for many neighbors can be time consuming. The distance metric applied to two vectors u and v in Weighted KNN is given as [23]:

#### 3.3.4. Decision Trees

#### 3.3.5. Ensemble Decision Trees

#### 3.4. Validation Process

## 4. Methodology

#### 4.1. Data Collection and Preprocessing stage

#### 4.2. Training Stage

- Normality Check: With regards to linear regression, where normality of the data is one of the requirement, is verified using normal Q-Q plot. Further confirmation of normality is handled using Shapiro-Wilk normality test.
- Salient variables identification: Stepwise regression is used to identify the salient independent variables and see if the adjusted R
^{2}value could be improved by just using these, instead of the whole list of independent variables. - Handing multiple outcomes: As the number of dependent outcomes are five, binomial based Logistic regression could be not applied. Thus, non-linear regression was applied on the same dataset with a Poisson regression. ANOVA was applied on the results of the Poisson regression to understand their applicability.

- Training and Testing Dataset: During the training process, both the training and the validation datasets were used. During the training phase, the weights are adjusted using the training data set and the validation data set is used to control overfitting.
- Variants of Machine Learning Techniques used: KNN is applied using Weighted KNN, Cubic KNN and Cosine KNN. By varying the number of neighbors, the accuracy of classification is tested. With regards to decision trees, three variants such as Simple (with number of splits equal to 4), Medium (with number of splits equal to 20) and Complex (with number of splits equal to 100) Trees are tested.

#### 4.3. Testing Stage

## 5. Results and Discussion

#### 5.1. Regression

#### 5.1.1. Linear Regression

^{2}value obtained for the linear regression is 0.2588, which is low and thus the applicability of linear regression for this dataset is questionable. Figure 3 shows the normal Q-Q plot.

^{10}, which is lower than the $\alpha $ value, indicates that the null hypothesis (that the sample comes from a population that is normally distributed) is rejected. The same was confirmed using the Kolmogorov-Smirnov test, where the obtained p-value was 2.995 × 10

^{5}. Thus, the non-applicability of linear regression to this dataset is confirmed.

^{2}value of 0.2661, confirming the non-applicability of linear regression to this dataset:

#### 5.1.2. Non-Linear Regression

^{2}value for ANOVA was 0.2626259. The obtained p-value using the Pearson method was 1.142181 × 10

^{10}. This indicates a lack of fit for this regression. Thus, we move on to apply other machine learning techniques.

#### 5.2. Artificial Neural Networks

#### 5.3. K-Nearest Neighbors

#### 5.4. Decision Trees

#### 5.5. Ensemble Decision Trees

#### 5.6. Comparison of Different Approaches

#### 5.7. Limitations and Future Work

## 6. Conclusions

## Supplementary Materials

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Olshannikova, E.; Ometov, A.; Koucheryavy, Y.; Olsson, T. Visualizing Big Data with augmented and virtual reality: Challenges and research agenda. J. Big Data
**2015**. [Google Scholar] [CrossRef] - Boletsis, C. The New Era of Virtual Reality Locomotion: A Systematic Literature Review of Techniques and a Proposed Typology. Multimodal Technol. Interact.
**2017**, 4, 24. [Google Scholar] [CrossRef] - Bruder, G.; Steinicke, F. Threefolded Motion Perception During Immersive Walkthroughs. In Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology, New York, NY, USA, 13–15 November 2014; pp. 177–185. [Google Scholar]
- Borrego, A.; Latorre, J.; Llorens, R.; Alcañiz, M.; Noé, E. Feasibility of a walking virtual reality system for rehabilitation: Objective and subjective parameters. J. Neuroeng. Rehabil.
**2016**, 13. [Google Scholar] [CrossRef] [PubMed] - Zank, M.; Kunz, A. Using Locomotion Models for Estimating Walking Targets in Immersive Virtual Environments. In Proceedings of the 2015 International Conference on Cyberworlds (CW), Visby, Sweden, 7–9 October 2015; pp. 229–236. [Google Scholar]
- Cakmak, T.; Hager, H. Cyberith virtualizer: A locomotion device for virtual reality. In Proceedings of the SIGGRAPH, Vancouver, BC, Canada, 10–14 August 2014. [Google Scholar]
- Bozgeyikli, E.; Raij, A.; Katkoori, S.; Dubey, R. Locomotion in Virtual Reality for Individuals with Autism Spectrum Disorder. In Proceedings of the 2016 Symposium on Spatial User Interaction, New York, NY, USA, 8–9 August 2016; pp. 33–42. [Google Scholar]
- Skopp, N.A.; Smolenski, D.J.; Metzger-Abamukong, M.J.; Rizzo, A.A.; Reger, G.M. A Pilot Study of the VirtuSphere as a Virtual Reality Enhancement. Int. J. Hum. Comput. Interact.
**2014**, 30, 24–31. [Google Scholar] [CrossRef] - Wilson, P.T.; Kalescky, W.; MacLaughlin, A.; Sanders, B.W. VR locomotion: Walking > walking in place > arm swinging. In Proceedings of the 15th ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry, Zhuhai, China, 3–4 December 2016. [Google Scholar]
- Ferracani, A.; Pezzatini, D.; Bianchini, J.; Biscini, G.; del Bimbo, A. Locomotion by Natural Gestures for Immersive Virtual Environments. In Proceedings of the 1st International Workshop on Multimedia Alternate Realities, New York, NY, USA, 10–16 October 2016; pp. 21–24. [Google Scholar]
- Comparing Leaning-Based Motion Cueing Interfaces for Virtual Reality Locomotion—IEEE Conference Publication. Available online: https://ieeexplore.ieee.org/document/7893320 (accessed on 16 December 2018).
- Cardoso, J.C.S. Comparison of Gesture, Gamepad, Gaze-based Locomotion for VR Worlds. In Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology, New York, NY, USA, 2–4 November 2016; pp. 319–320. [Google Scholar]
- Bozgeyikli, E.; Raij, A.; Katkoori, S.; Dubey, R. Point & Teleport Locomotion Technique for Virtual Reality. In Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play, New York, NY, USA, 19–21 October 2016; pp. 205–216. [Google Scholar]
- Riehle, T.H.; Anderson, S.M.; Lichter, P.A.; Whalen, W.E.; Giudice, N.A. Indoor inertial waypoint navigation for the blind. In Proceedings of the 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Osaka, Japan, 3–7 July 2013; pp. 5187–5190. [Google Scholar]
- Real-Time Human Movement Mapping to a Virtual Environment—IEEE Conference Publication. Available online: https://ieeexplore.ieee.org/document/7519395 (accessed on 16 December 2018).
- Williamson, B.; Wingrave, C.; LaViola, J.J. Full Body Locomotion with Video Game Motion Controllers. In Human Walking in Virtual Environments: Perception, Technology, Applications; Springer: Berlin, Germany, 2013; pp. 351–376. [Google Scholar]
- Ponto, K.; Kimmel, R.; Kohlmann, J.; Bartholomew, A.; Radwir, R.G. Virtual exertions: A user interface combining visual information, kinesthetics and biofeedback for virtual object manipulation. In Proceedings of the 2012 IEEE Symposium on 3D User Interfaces (3DUI), Costa Mesa, CA, USA, 4–5 March 2012; pp. 85–88. [Google Scholar]
- Bypassing the Natural Visual-Motor Pathway to Execute Complex Movement Related Tasks Using Interval Type-2 Fuzzy Sets—IEEE Journals & Magazine. Available online: https://ieeexplore.ieee.org/document/7491211 (accessed on 16 December 2018).
- Xing, H.; Li, J.; Hou, B.; Zhang, Y.; Guo, M. Pedestrian Stride Length Estimation from IMU Measurements and ANN Based Algorithm. 2017. Available online: https://www.hindawi.com/journals/js/2017/6091261/ (accessed on 16 December 2018).
- Renaudin, V.; Susi, M.; Lachapelle, G. Step Length Estimation Using Handheld Inertial Sensors. Sensors
**2012**, 12, 8507–8525. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Wang, J.; Shao, W.; Song, Z. Student’s-t Mixture Regression-Based Robust Soft Sensor Development for Multimode Industrial Processes. Sensors
**2018**, 18, 3968. [Google Scholar] [CrossRef] [PubMed] - Šíma, J. Neural networks between integer and rational weights. In Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA, 14–19 May 2017; pp. 154–161. [Google Scholar]
- Vitola, J.; Pozo, F.; Tibaduiza, D.A.; Anaya, M. A Sensor Data Fusion System Based on k-Nearest Neighbor Pattern Classification for Structural Health Monitoring Applications. Sensors
**2017**, 17, 417. [Google Scholar] [CrossRef] [PubMed] - Freund, Y.; Schapire, R.E. Experiments with a New Boosting Algorithm. In Proceedings of the Thirteenth International Conference on International Conference on Machine Learning, San Francisco, CA, USA, 31 May–2 June 1996; pp. 148–156. [Google Scholar]
- Yang, S.; Zhang, C.; Wu, W. Binary output layer of feedforward neural networks for solving multi-class classification problems. arXiv, 2018; arXiv:1801.07599. [Google Scholar]
- Kruijff, E.; Marquardt, A.; Trepkowski, C.; Lindeman, R.W.; Hinkenjann, A.; Maiero, J.; Riecke, B.E. On Your Feet!: Enhancing Vection in Leaning-Based Interfaces through Multisensory Stimuli. In Proceedings of the 2016 Symposium on Spatial User Interaction, Tokyo, Japan, 15–16 October 2016. [Google Scholar]

Question | Score |
---|---|

(1) Are you interested in first person computer games and virtual reality applications? Please grade from 1 to 5. 5 is very, while 1 is not at all. | Average = 5.0 |

(2) What is your preferred game controller for first person games)? (a) Keyboard and mouse (b) Gamepads (c) The proposed approach (d) Other. | (a) 44/44 (b) 0/44 (c) 0/44 (d) 0/44 |

(3) What is your preferred controller for virtual reality applications while sitting on a chair? (a) Keyboard mouse (b) Hand held controllers (c) The proposed approach (d) Other. | (a) 3/44 (b) 19/44 (c) 22/44 (d) 0/44 |

(4) Which of the above gives you a more realistic experience in VR applications while sitting on a chair? (a) Keyboard and mouse (b) Hand held controllers (c) The proposed approach (d) Other. | (a) 1/44 (b) 17/44 (c) 26/44 (d) 0/44 |

(5) How easy was it to move in virtual words using the proposed approach? Please grade from 1 to 5. 5 is very, while 1 is not at all. | Average = 4.4 |

Regression | ANN | KNN | Decision Trees | Ensemble | |
---|---|---|---|---|---|

Accuracy | Lack of fit | 67.7% | 70.3% | 69.4% | 72.2% |

14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | |
---|---|---|---|---|---|---|---|---|

Weighted KNN | 77.9% | 77.6% | 78.3% | 77.9% | 76.9% | 78.6% | 76.2% | 77.6% |

Cubic KNN | 73.4% | 73.4% | 71.0% | 71.0% | 72.1% | 69.7% | 69.0% | 70.0% |

Cosine KNN | 75.5% | 74.8% | 75.2% | 76.2% | 74.8% | 74.8% | 75.2% | 75.9% |

Artificial Neural Networks | KNN | Decision Trees | Ensemble Decision Trees (Bag) | |
---|---|---|---|---|

Accuracy | 84.2% | 78.6% | 71.7% | 80.3% |

Training Time (Sec) | 0.0313 | 0.036150 | 0.024026 | 2.159735 |

Artificial Neural Networks | KNN | Decision Trees | Ensemble Decision Trees (Bag) | |
---|---|---|---|---|

Accuracy | 84.1% | 76.9% | 72.76% | 74.83% |

Runtime execution (1 classification) (Sec) | 0.0192 | 0.002264 | 0.000908 | 0.246718 |

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Tsaramirsis, G.; Buhari, S.M.; Basheri, M.; Stojmenovic, M.
Navigating Virtual Environments Using Leg Poses and Smartphone Sensors. *Sensors* **2019**, *19*, 299.
https://doi.org/10.3390/s19020299

**AMA Style**

Tsaramirsis G, Buhari SM, Basheri M, Stojmenovic M.
Navigating Virtual Environments Using Leg Poses and Smartphone Sensors. *Sensors*. 2019; 19(2):299.
https://doi.org/10.3390/s19020299

**Chicago/Turabian Style**

Tsaramirsis, Georgios, Seyed M. Buhari, Mohammed Basheri, and Milos Stojmenovic.
2019. "Navigating Virtual Environments Using Leg Poses and Smartphone Sensors" *Sensors* 19, no. 2: 299.
https://doi.org/10.3390/s19020299