Static and Dynamic Activity Detection with Ambient Sensors in Smart Spaces
Abstract
:1. Introduction
2. Related Work
2.1. Multimedia-Based Activity Detection
2.2. Wearable-Based Activity Detection
2.3. Discussion
3. Proposed System for Static and Dynamic Activity Detection
- (1)
- The system should satisfy four in-the-wild conditions given in [16] so that natural behavior of human is not restricted.
- (2)
- The system should not violate user privacy. Privacy is always a big concern in sensor based systems [34] where data is stored for future use. Sensors such as camera and microphone raises questions on privacy. We instead are using low-resolution (4 × 16) thermal sensors. With such a small resolution, it is impossible to identify a user, while maintaining non-intrusive, accurate, and real-time activity detection.
3.1. Hardware Setup
3.2. Static Activity Detection Method
3.3. Dynamic Activity Detection Method
4. Results
4.1. Static Activity Detection
4.1.1. Data Collection
4.1.2. Analysis
- Accuracy =
- Precision =
- Recall/Sensitivity =
- Specificity =
4.1.3. Data Collection
4.2. Dynamic Activity Detection
Analysis
5. Conclusions
Author Contributions
Conflicts of Interest
References
- Teixeira, T.; Dublon, G.; Savvides, A. A survey of human-sensing: Methods for detecting presence, count, location, track, and identity. ACM Computing Surveys 2010, 5, 59–69. [Google Scholar]
- Agarwal, Y.; Balaji, B.; Gupta, R.; Lyles, J.; Wei, M.; Weng, T. Occupancy-driven energy management for smart building automation. In Proceedings of the 2nd ACM Workshop on Embedded Sensing Systems for Energy-Efficiency in Building, Zurich, Switzerland, 2 November 2010; pp. 1–6. [Google Scholar]
- Liang, Y.; Zhang, R.; Wang, W.; Xiao, C. Design of energy saving lighting system in university classroom based on wireless sensor network. Commun. Netw. 2013, 5, 55–60. [Google Scholar] [CrossRef]
- Offermans, S.; Gopalakrishna, A.K.; van Essen, H.; Özçelebi, T. Breakout 404: A smart space implementation for lighting services in the office domain. In Proceedings of the 2012 Ninth International Conference on Networked Sensing (INSS), Antwerp, Belgium, 11–14 June 2012; pp. 1–4. [Google Scholar]
- Hsu, Y.L.; Chou, P.H.; Chang, H.C.; Lin, S.L.; Yang, S.C.; Su, H.Y.; Chang, C.C.; Cheng, Y.S.; Kuo, Y.C. Design and implementation of a smart home system using multisensor data fusion technology. Sensors 2017, 17, 1631. [Google Scholar] [CrossRef] [PubMed]
- Piyathilaka, L.; Kodagoda, S. Human activity recognition for domestic robots. In Field and Service Robotics; Springer: Berlin, Germany, 2015; pp. 395–408. [Google Scholar]
- Bourke, A.; O’brien, J.; Lyons, G. Evaluation of a threshold-based tri-axial accelerometer fall detection algorithm. Gait Posture 2007, 26, 194–199. [Google Scholar] [CrossRef] [PubMed]
- Sung, J.; Ponce, C.; Selman, B.; Saxena, A. Unstructured human activity detection from rgbd images. In Proceedings of the 2012 IEEE International Conference on Robotics and Automation (ICRA), Saint Paul, MN, USA, 14–18 May 2012; pp. 842–849. [Google Scholar]
- Mukhopadhyay, S.C. Wearable sensors for human activity monitoring: A review. IEEE Sens. J. 2015, 15, 1321–1330. [Google Scholar] [CrossRef]
- Niu, W.; Long, J.; Han, D.; Wang, Y.F. Human activity detection and recognition for video surveillance. In Proceedings of the 2004 IEEE International Conference on Multimedia and Expo (ICME), Taipei, Taiwan, 27–30 June 2004; Volume 1, pp. 719–722. [Google Scholar]
- Ryoo, M.S. Human activity prediction: Early recognition of ongoing activities from streaming videos. In Proceedings of the 2011 IEEE International Conference on Computer Vision (ICCV), Barcelona, Spain, 6–13 November 2011; pp. 1036–1043. [Google Scholar]
- Albukhary, N.; Mustafah, Y. Real-time Human Activity Recognition. IOP Conf. Ser. Mater. Sci. Eng. 2017, 260, 012017. [Google Scholar] [CrossRef] [Green Version]
- Song, K.T.; Chen, W.J. Human activity recognition using a mobile camera. In Proceedings of the 2011 8th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), Incheon, Korea, 23–26 November 2011; pp. 3–8. [Google Scholar]
- Zhao, H.; Liu, Z. Shape-based human activity recognition using edit distance. In Proceedings of the 2nd International Congress on Image and Signal Processing, Tianjin, China, 17–19 October 2009; pp. 1–4. [Google Scholar]
- Wang, Y.; Huang, K.; Tan, T. Human activity recognition based on R transform. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA, 17–22 June 2007; pp. 1–8. [Google Scholar]
- Vaizman, Y.; Ellie, K.; Lanckriet, G. Recognizing Detailed Human Context In-the-Wild from Smartphones and Smartwatches. arXiv, 2016; arXiv:1609.06354. [Google Scholar] [CrossRef]
- Basu, C.; Rowe, A. Tracking Motion and Proxemics Using Thermal-Sensor Array. arXiv, 2015; arXiv:1511.08166. [Google Scholar]
- Shah, A.K.; Ghosh, R.; Akula, A. A spatio-temporal deep learning approach for human action recognition in infrared videos. In Optics and Photonics for Information Processing XII; International Society for Optics and Photonics: Bellingham, WA, USA, 2018; Volume 10751, p. 1075111. [Google Scholar]
- Akula, A.; Shah, A.K.; Ghosh, R. Deep learning approach for human action recognition in infrared images. Cogn. Syst. Res. 2018, 50, 146–154. [Google Scholar] [CrossRef]
- Ke, S.R.; Thuc, H.; Lee, Y.J.; Hwang, J.N.; Yoo, J.H.; Choi, K.H. A review on video-based human activity recognition. Computers 2013, 2, 88–131. [Google Scholar] [CrossRef]
- Caba Heilbron, F.; Escorcia, V.; Ghanem, B.; Carlos Niebles, J. Activitynet: A large-scale video benchmark for human activity understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 961–970. [Google Scholar]
- Lara, O.D.; Labrador, M.A. A survey on human activity recognition using wearable sensors. IEEE Commun. Surv. Tutor. 2013, 15, 1192–1209. [Google Scholar] [CrossRef]
- Cornacchia, M.; Ozcan, K.; Zheng, Y.; Velipasalar, S. A survey on activity detection and classification using wearable sensors. IEEE Sensors J. 2017, 17, 386–403. [Google Scholar] [CrossRef]
- Yao, S.; Swetha, P.; Zhu, Y. Nanomaterial-Enabled Wearable Sensors for Healthcare. Adv. Healthc. Mater. 2018, 7, 1700889. [Google Scholar] [CrossRef] [PubMed]
- Varatharajan, R.; Manogaran, G.; Priyan, M.; Sundarasekar, R. Wearable sensor devices for early detection of Alzheimer disease using dynamic time warping algorithm. Clust. Comput. 2018, 21, 681–690. [Google Scholar] [CrossRef]
- Mathie, M.; Celler, B.G.; Lovell, N.H.; Coster, A. Classification of basic daily movements using a triaxial accelerometer. Med. Biol. Eng. Comput. 2004, 42, 679–687. [Google Scholar] [CrossRef] [PubMed]
- Gao, L.; Bourke, A.; Nelson, J. Evaluation of accelerometer based multi-sensor versus single-sensor activity recognition systems. Med. Eng. Phys. 2014, 36, 779–785. [Google Scholar] [CrossRef] [PubMed]
- Olguın, D.O.; Pentland, A.S. Human activity recognition: Accuracy across common locations for wearable sensors. In Proceedings of the 2006 10th IEEE International Symposium on Wearable Computers, Montreux, Switzerland, 11–14 October 2006; pp. 11–14. [Google Scholar]
- Bayat, A.; Pomplun, M.; Tran, D.A. A study on human activity recognition using accelerometer data from smartphones. Procedia Comput. Sci. 2014, 34, 450–457. [Google Scholar] [CrossRef]
- Kwapisz, J.R.; Weiss, G.M.; Moore, S.A. Activity recognition using cell phone accelerometers. ACM SIGKDD Explor. Newsl. 2011, 12, 74–82. [Google Scholar] [CrossRef]
- Zhuang, J.; Liu, Y.; Jia, Y.; Huang, Y. User Discomfort Evaluation Research on the Weight and Wearing Mode of Head-Wearable Device. In Proceedings of the International Conference on Applied Human Factors and Ergonomics, Orlando, FL, USA, 21–25 July 2018; Springer: Berlin, Germany, 2018; pp. 98–110. [Google Scholar]
- Belapurkar, N.; Shelke, S.; Aksanli, B. The case for ambient sensing for human activity detection. In Proceedings of the 8th International Conference on the Internet of Things, Santa Barbara, CA, USA, 15–18 October 2018; p. 40. [Google Scholar]
- Belapurkar, N.; Harbour, J.; Shelke, S.; Aksanli, B. Building Data-Aware and Energy-Efficient Smart Spaces. IEEE Internet Things J. 2018, 5, 4526–4537. [Google Scholar] [CrossRef]
- Sundmaeker, H.; Guillemin, P.; Friess, P.; Woelfflé, S. Vision and challenges for realising the Internet of Things. In Cluster of European Research Projects on the Internet of Things; European Commision: Brussels, Belgium, 2010; Volume 3, pp. 34–36. [Google Scholar]
- Melexix. Thermal Sensor Description. Available online: https://www.melexis.com/en/product/MLX90621/Far-Infrared-Sensor-Array-High-Speed-Low-Noise (accessed on 10 January 2019).
- Venkatesh, J.; Aksanli, B.; Chan, C.S.; Akyürek, A.S.; Rosing, T.S. Scalable-application design for the iot. IEEE Softw. 2017, 34, 62–70. [Google Scholar] [CrossRef]
- Schmidhuber, J. Deep learning in neural networks: An overview. Neural Netw. 2015, 61, 85–117. [Google Scholar] [CrossRef] [Green Version]
- Abadi, M.; Barham, P.; Chen, J.; Chen, Z.; Davis, A.; Dean, J.; Devin, M.; Ghemawat, S.; Irving, G.; Isard, M.; et al. Tensorflow: A system for large-scale machine learning. In Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation, Savannah, GA, USA, 2–4 November 2016; Volume 16, pp. 265–283. [Google Scholar]
- How Do Object Distance and Focal Length Affect Depth of Field? Available online: https://physicssoup.wordpress.com/2015/04/26/how-do-object-distance-and-focal-length-affect-depth-of-field/ (accessed on 10 January 2019).
- Robert, W.; Alistair, A.; David, B. Comparative study on connected component labeling algorithms for embedded video processing systems. In Proceedings of the 2010 International Conference on Image Processing, Computer Vision, & Pattern Recognition, Las Vegas, NV, USA, 12–15 July 2010. [Google Scholar]
- Bracewell, R.N.; Bracewell, R.N. The Fourier Transform and Its Applications; McGraw-Hill: New York, NY, USA, 1986; Volume 31999. [Google Scholar]
- Szeliski, R. Computer Vision: Algorithms and Applications; Springer Science & Business Media: Berlin, Germany, 2010. [Google Scholar]
- Martin-Diaz, I.; Morinigo-Sotelo, D.; Duque-Perez, O.; Romero-Troncoso, R.D.J. Advances in Classifier Evaluation: Novel Insights for an Electric Data-Driven Motor Diagnosis. IEEE Access 2016, 4, 7028–7038. [Google Scholar] [CrossRef]
- Jones, E.; Oliphant, T.; Peterson, P. SciPy: Open source scientific tools for Python. Available online: http://www.citeulike.org/group/19049/article/13344001 (accessed on 15 February 2019).
- ASU. The Average Walking Stride Length. Available online: https://livehealthy.chron.com/average-walking-stride-length-7494.html (accessed on 10 January 2019).
Classifier | STAND | SOC | SOG | LOG | Overall |
---|---|---|---|---|---|
Logistic | 99.94 | 99.97 | 99.94 | 99.95 | 99.90 |
SVM | 99.91 | 99.93 | 99.89 | 99.90 | 99.82 |
DecTree | 99.80 | 99.71 | 99.42 | 99.63 | 99.28 |
RandFor | 99.97 | 99.94 | 99.92 | 99.96 | 99.90 |
NaiveB | 65.62 | 74.75 | 70.17 | 76.03 | 43.29 |
NN | 99.98 | 99.96 | 99.99 | 99.97 | 99.96 |
Classifier | STAND | SOC | SOG | LOG |
---|---|---|---|---|
Logistic | 99.88 | 99.95 | 99.89 | 99.89 |
SVM | 99.82 | 99.87 | 99.78 | 99.80 |
DecTree | 99.60 | 99.43 | 98.87 | 99.23 |
RandFor | 99.96 | 99.89 | 99.85 | 99.92 |
NaiveB | 53.36 | 0.09 | 54.79 | 0.83 |
NN | 99.97 | 99.93 | 99.98 | 99.95 |
© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Shelke, S.; Aksanli, B. Static and Dynamic Activity Detection with Ambient Sensors in Smart Spaces. Sensors 2019, 19, 804. https://doi.org/10.3390/s19040804
Shelke S, Aksanli B. Static and Dynamic Activity Detection with Ambient Sensors in Smart Spaces. Sensors. 2019; 19(4):804. https://doi.org/10.3390/s19040804
Chicago/Turabian StyleShelke, Sagar, and Baris Aksanli. 2019. "Static and Dynamic Activity Detection with Ambient Sensors in Smart Spaces" Sensors 19, no. 4: 804. https://doi.org/10.3390/s19040804
APA StyleShelke, S., & Aksanli, B. (2019). Static and Dynamic Activity Detection with Ambient Sensors in Smart Spaces. Sensors, 19(4), 804. https://doi.org/10.3390/s19040804