Design, Implementation, and Field Testing of a Privacy-Aware Compliance Tracking System for Bedside Care in Nursing Homes
- A privacy-aware mechanism that ensures only the consented users will be tracked even when she or he works in several different rooms during her/his shift, which protects the privacy of patients and other unrelated persons who might come to the view of the Kinect sensor.
- An occlusion-resilient human activity tracking mechanism that works well in crowded rooms in nursing homes.
- A mechanism to deliver personalized and discreet alerts to the user on detection of a non-compliant activity in realtime.
- A lease-based mechanism that strikes a good balance between meeting the privacy requirement and the usability of the system.
2. Background and Related Work
3. System Design
3.1. System Requirement
- Privacy protection. The system must track only the consented nursing assistant, not any other person that might appear in front of the Kinect sensor, including the patient in the room. This is essential to protect the privacy of the patients. The US federal regulation requires that an explicit consent is required before a patient can be tracked. Because it is unlikely for a patient to agree to be tracked all the time for any purpose, the system must provide maximum provision that he or she is not tracked.
- Reliable non-compliance activity recognition. The rooms in a nursing home typically are crowded with limited space. For example, a room may consist of several pieces of furniture such as wheelchairs, tables, walkers, etc. The activity recognition algorithm must be robust despite the occlusion caused by these pieces.
- Realtime alert. To have any positive impact to a nursing assistant, the alert should be delivered in realtime in a way that he or she could accept when he or she performed a non-compliant activity.
- Multi-room support. A nursing assistant is typically assigned to several rooms during a shift. To maximize the effectiveness of our system, we must be able to continuously track the nursing assistant regardless which room he or she goes in to do his or her job duties.
- Balance the privacy protection and usability of the system. While protecting the privacy of the patients, the system must strive to be usable by nursing assistants in a way that is too burdensome for them to use it.
3.2. System Design Rationale
3.2.1. Privacy Protection Design
3.2.2. Robust Activity Recognition Algorithm Design
3.2.3. Realtime Alert Design
3.2.4. Multi-Room Support Design
3.2.5. Design for Usability while Protecting Patient Privacy
4. System Implementation
4.1. System Overview
- Kinect input: The streams of the color and depth frames from connected Kinect sensors.
- Network input: HTTP requests sent by the PhoneApps over the WiFi network.
- Registration request: It is sent when a user has requested to register with the system. The response for the request indicates whether or not the registration has succeeded. Without the lease mechanism, this request is only initiated by the user via the WatchApp, the registration request will be broadcast to all known KinectServers. When the lease mechanism is enabled, the PhoneApp would periodically sends the registration request to the KinectServer that the user had manually registered with earlier when the user is temporarily unregistered (due to occlusion or due to her/him stepping out of the view of the Kinect sensor).
- Activity status request: It is sent periodically once a user has registered with the system. The response indicates whether or not a back-bending activity has just been detected.
- Stop-tracking request: It is sent when the user wishes the KinectServer to stop tracking temporarily due to privacy concerns, for example, when she is about to give the patient a sponge-bath to make sure that the system absolutely will not track anyone in the room. Once the stop-tracking request is received, the KinectServer also terminates the current lease, if any, for the user.
- Server and ServerTask to handle HTTP communications. The Server component serves as the HTTP listener for the KinectServer. Upon receiving an HTTP request, the Server creates a new ServerTask to handle the message. Both the HTTP request and the response contain a JSON object.
- KinectDriver to handle motion sensing data from the Kinect sensor. In addition, It also handles the user registration request in conjunction with the Activity Recognizer. If the user is properly registered, the user is added to the User object. For each registered user, the KinectDriver tracks her/him for possible non-compliance activities.
- Kinect Coordinator to dispatch the registration and activity requests to a particular KinectDriver and the corresponding User.
4.2. User Registration Mechanism
4.3. Gesture Recognition and Activity Detection
4.4. Alerts and Feedback
- Normal state, denoted as . It is represented by a by a posture configuration that is within the boundary of the requirement. In our study, it refers to the range when the user is considered to have her/his back straight up.
- Abnormal state, denoted as . It is represented by a by a posture configuration that is outside the boundary of the requirement. In our study, it refers to the range when the user is considered to have her/his back bent too much.
5. Field Study
5.1. Analysis of the Logged Data
5.2. Usability Survey Result
- “It helps me to realize how I can hurt myself by moving residents wrong to transfer a resident”.
- “I think it would be a good idea”.
- “I think it should be repeated once it’s working properly. It would be a great source for proper body mechanics”.
- “It was a good experience”.
- “It was nice to participate”.
- “I enjoyed the opportunity but it needs a little fixing”.
- “Thanks for thinking of ways to keep us safe”.
- “It was different and I enjoyed it”.
6. Conclusions and Future Work
Conflicts of Interest
|PACTS||Privacy-Aware Compliance Tracking System|
|XML||eXtensible Markup Language|
|HTTP||HyperText Transfer Protocol|
|ARP||Address Resolution Protocol|
|DHCP||Dynamic Host Configuration Protocol|
|NFC||Near Field Communication|
- Katz, J.N. Lumbar disc disorders and low-back pain: Socioeconomic factors and consequences. J. Bone Jt. Surg. 2006, 88, 21–24. [Google Scholar] [CrossRef]
- Marras, W.S.; Lavender, S.A.; Leurgans, S.E.; Rajulu, S.L.; Allread, W.G.; Fathallah, F.A.; Ferguson, S.A. The Role of Dynamic Three-Dimensional Trunk Motion in Occupationally-Related Low Back Disorders: The Effects of Workplace Factors, Trunk Position, and Trunk Motion Characteristics on Risk of Injury. Spine 1993, 18, 617–628. [Google Scholar] [CrossRef] [PubMed]
- Videman, T.; Battié, M.C. Spine update: The influence of occupation on lumbar degeneration. Spine 1999, 24, 1164–1168. [Google Scholar] [CrossRef] [PubMed]
- Mohammed, S.; Singh, D.; Johnson, G.T.; Xu, P.; McCluskey, J.D.; Harbison, R.D. Evaluation of occupational risk factors for healthcare workers through analysis of the Florida Workers’ Compensation Claims Database. Occup. Dis. Environ. Med. 2014, 2, 77. [Google Scholar] [CrossRef]
- Lun, R.; Zhao, W. A Survey of Applications and Human Motion Recognition with Microsoft Kinect. Int. J. Pattern Recognit. Artif. Intell. 2015, 29, 1555008. [Google Scholar] [CrossRef]
- Zhao, W.; Espy, D.D.; Reinthal, M.; Feng, H. A feasibility study of using a single Kinect sensor for rehabilitation exercises monitoring: A rule based approach. In Proceedings of the IEEE Symposium on Computational Intelligence in Healthcare and E-health, Orlando, FL, USA, 9–12 December 2014; pp. 1–8. [Google Scholar]
- Zhao, W. A concise tutorial on human motion tracking and recognition with Microsoft Kinect. Sci. China Inf. Sci. 2016, 59, 93101. [Google Scholar] [CrossRef]
- Lin, J.S.; Kulic, D. Online Segmentation of Human Motion for Automated Rehabilitation Exercise Analysis. IEEE Trans. Neural Syst. Rehabil. Eng. 2014, 22, 168–180. [Google Scholar] [CrossRef] [PubMed]
- Li, Y.; Luo, X.; Wang, W.; Zhao, W. Human Action Recognition with Skeleton Data Using Extreme Learning Machine. In Proceedings of the Chinese Intelligent Automation Conference; Springer: Berlin, Germany, 2017; pp. 449–456. [Google Scholar]
- Zhao, W.; Lun, R.; Espy, D.D.; Reinthal, M. Rule based realtime motion assessment for rehabilitation exercises. In Proceedings of the IEEE Symposium on Computational Intelligence in Healthcare and E-health, Orlando, FL, USA, 9–12 December 2014; pp. 133–140. [Google Scholar]
- Zhao, W.; Lun, R.; Espy, D.D.; Ann Reinthal, M. Realtime Motion Assessment For Rehabilitation Exercises: Integration Of Kinematic Modeling With Fuzzy Inference. J. Artif. Intell. Soft Comput. Res. 2014, 4, 267–285. [Google Scholar] [CrossRef]
- Zhao, W.; Reinthal, M.A.; Espy, D.D.; Luo, X. Rule-Based Human Motion Tracking for Rehabilitation Exercises: Realtime Assessment, Feedback, and Guidance. IEEE Access 2017, 5, 21382–21394. [Google Scholar] [CrossRef]
- Hachaj, T.; Ogiela, M.R. Rule-based approach to recognizing human body poses and gestures in real time. Multimedia Syst. 2014, 20, 81–99. [Google Scholar] [CrossRef]
- Debar, H.; Wespi, A. Aggregation and correlation of intrusion-detection alerts. In Recent Advances in Intrusion Detection; Springer: Berlin, Germany, 2001; pp. 85–103. [Google Scholar]
- Morin, B.; Mé, L.; Debar, H.; Ducassé, M. A logic-based model to support alert correlation in intrusion detection. Inf. Fusion 2009, 10, 285–299. [Google Scholar] [CrossRef]
- Valeur, F.; Vigna, G.; Kruegel, C.; Kemmerer, R. Comprehensive approach to intrusion detection alert correlation. IEEE Trans. Dependable Secure Comput. 2004, 1, 146–169. [Google Scholar] [CrossRef]
- Hravnak, M.; Edwards, L.; Clontz, A.; Valenta, C.; DeVita, M.A.; Pinsky, M.R. Defining the incidence of cardiorespiratory instability in patients in step-down units using an electronic integrated monitoring system. Arch. Intern. Med. 2008, 168, 1300–1308. [Google Scholar] [CrossRef] [PubMed]
- Chipara, O.; Lu, C.; Bailey, T.C.; Roman, G.C. Reliable clinical monitoring using wireless sensor networks: Experiences in a step-down hospital unit. In Proceedings of the 8th ACM Conference on Embedded Networked Sensor Systems, Zurich, Switzerland, 2–5 November 2010; pp. 155–168. [Google Scholar]
- Imhoff, M.; Kuhls, S. Alarm algorithms in critical care monitoring. Anesth. Analg. 2006, 102, 1525–1537. [Google Scholar] [CrossRef] [PubMed]
- Roesch, M. Snort: Lightweight Intrusion Detection for Networks. In Proceedings of the 13th Systems Administration Conference, Seattle, WD, USA, 7–12 November 1999; Volume 99, pp. 229–238. [Google Scholar]
- Hayes, D.L.; Levine, P.A. Pacemaker timing cycles. In Cardiac Pacing and ICDs; Blackwell Publishing: Malden, MA, USA, 2002; pp. 265–321. [Google Scholar]
- Wang, P.; Kramer, A.; Estes, N.; Hayes, D.L. Timing cycles for biventricular pacing. Pac. Clin. Electrophysiol. 2002, 25, 62–75. [Google Scholar] [CrossRef]
- Cobelli, C.; Carson, E. Introduction to Modeling in Physiology and Medicine; Academic Press: Cambridge, MA, USA, 2008. [Google Scholar]
- Sommer, R.; Paxson, V. Outside the closed world: On using machine learning for network intrusion detection. In Proceedings of the IEEE Symposium on Security and Privacy, Berkeley, CA, USA, 22–25 May 2010; pp. 305–316. [Google Scholar]
- Becker, K.; Thull, B.; Käsmacher-Leidinger, H.; Stemmer, J.; Rau, G.; Kalff, G.; Zimmermann, H.J. Design and validation of an intelligent patient monitoring and alarm system based on a fuzzy logic process model. Artif. Intell. Med. 1997, 11, 33–53. [Google Scholar] [CrossRef]
- Convertino, V.A.; Moulton, S.L.; Grudic, G.Z.; Rickards, C.A.; Hinojosa-Laborde, C.; Gerhardt, R.T.; Blackbourne, L.H.; Ryan, K.L. Use of advanced machine-learning techniques for noninvasive monitoring of hemorrhage. J. Trauma Acute Care Surg. 2011, 71, S25–S32. [Google Scholar] [CrossRef] [PubMed]
- King, A.; Fortino, K.; Stevens, N.; Shah, S.; Fortino-Mullen, M.; Lee, I. Evaluation of a smart alarm for intensive care using clinical data. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Diego, CA, USA, 28 August–1 Septermber 2012; pp. 166–169. [Google Scholar]
- Murukannaiah, P.K.; Singh, M.P. Platys: An Active Learning Framework for Place-Aware Application Development and Its Evaluation. ACM Trans. Softw. Eng. Methodol. 2015, 24, 1–33. [Google Scholar] [CrossRef]
- Zhao, W.; Espy, D.D.; Reinthal, M.; Ekelman, B.; Goodman, G.; Niederriter, J. Privacy-Aware Human Motion Tracking with Realtime Haptic Feedback. In Proceedings of the IEEE International Conference on Mobile Services, New York, NY, USA, 27 June–2 July 2015; pp. 446–453. [Google Scholar]
- Zhao, W.; Lun, R.; Gordon, C.; Fofana, A.B.M.; Espy, D.D.; Reinthal, M.A.; Ekelman, B.; Goodman, G.D.; Niederriter, J.E.; Luo, X. A Human-Centered Activity Tracking System: Toward a Healthier Workplace. IEEE Trans. Hum.-Mach. Syst. 2017, 47, 343–355. [Google Scholar] [CrossRef]
- Zhao, W.; Wu, Q.; Padaraju, V.; Bbela, B.; Reinthal, A.; Espy, D.; Luo, X.; Qiu, T. A Privacy-Aware Compliance Tracking System for Skilled Nursing Facilities. In Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, Banff, AB, Canada, 5–9 October 2017; pp. 3568–3573. [Google Scholar]
- Zhao, W.; Lun, R.; Gordon, C.; Fofana, A.B.M.; Espy, D.D.; Reinthal, A.; Ekelman, B.; Goodman, G.D.; Niederriter, J.E.; Luo, C.; et al. LiftingDoneRight: A Privacy-Aware Human Motion Tracking System for Healthcare Professionals. Int. J. Handheld Comput. Res. 2016, 7, 1–15. [Google Scholar] [CrossRef]
- Zhao, W. Building Dependable Distributed Systems; John Wiley & Sons: Hoboken, NJ, USA, 2014. [Google Scholar]
- Chai, H.; Zhao, W. Towards trustworthy complex event processing. In Proceedings of the 5th IEEE International Conference on Software Engineering and Service Science, Beijing, China, 27–29 June 2014; pp. 758–761. [Google Scholar]
- Chai, H.; Zhao, W. Byzantine Fault Tolerant Event Stream Processing for Autonomic Computing. In Proceedings of the IEEE 12th International Conference on Dependable, Autonomic and Secure Computing, Dalian, China, 24–27 August 2014; pp. 109–114. [Google Scholar]
- Zhao, W. Optimistic Byzantine fault tolerance. Int. J. Parallel Emerg. Distrib. Syst. 2016, 31, 254–267. [Google Scholar] [CrossRef]
- Zhao, W. Performance optimization for state machine replication based on application semantics: A review. J. Syst. Softw. 2016, 112, 96–109. [Google Scholar] [CrossRef]
- Zhao, W. Fast Paxos Made Easy: Theory and Implementation. Int. J. Distrib. Syst. Technol. 2015, 6, 15–33. [Google Scholar] [CrossRef]
- Lun, R.; Gordon, C.; Zhao, W. Tracking the activities of daily lives: An integrated approach. In Proceedings of the Future Technologies Conference (FTC), San Francisco, CA, USA, 6–7 December 2016; pp. 466–475. [Google Scholar]
- Lun, R.; Gordon, C.; Zhao, W. The design and implementation of a Kinect-based framework for selective human activity tracking. In Proceedings of the 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Budapest, Hungary, 9–12 October 2016; pp. 002890–002895. [Google Scholar]
- Luo, X.; Lv, Y.; Zhou, M.; Wang, W.; Zhao, W. A laguerre neural network-based ADP learning scheme with its application to tracking control in the Internet of Things. Pers. Ubiquitous Comput. 2016, 20, 361–372. [Google Scholar] [CrossRef]
- Luo, X.; Zhang, D.; Yang, L.T.; Liu, J.; Chang, X.; Ning, H. A kernel machine-based secure data sensing and fusion scheme in wireless sensor networks for the cyber-physical systems. Future Gener. Comput. Syst. 2016, 61, 85–96. [Google Scholar] [CrossRef]
- Luo, X.; Deng, J.; Wang, W.; Wang, J.H.; Zhao, W. A Quantized Kernel Learning Algorithm Using a Minimum Kernel Risk-Sensitive Loss Criterion and Bilateral Gradient Technique. Entropy 2017, 19, 365. [Google Scholar] [CrossRef]
- Luo, X.; Xu, Y.; Wang, W.; Yuan, M.; Ban, X.; Zhu, Y.; Zhao, W. Towards Enhancing Stacked Extreme Learning Machine With Sparse Autoencoder by Correntropy. J. Frankl. Inst. 2017. [Google Scholar] [CrossRef]
- Zhao, W. Systems and Methods for Privacy-Aware Motion Tracking with Notification Feedback. U.S. Patent App. 15/190,669, 29 December 2016. [Google Scholar]
|Room||Total of Registered||Daily Average Before/After||Total NCA||Total Number|
|Number||Time (Minutes)||Lease Mechanism||Time (Minutes)||of NCAs|
|1||316.08 (daily mean 4.95)||3.02/18.78||16.74||427 (daily mean 5.3)|
|2||311.56 (daily mean 2.64)||1.56/7.11||2.04||575/7.2|
|3||89.16 (daily mean 1.11)||0.88/5.23||19.26||345/4.3|
|4||171.24 (daily mean 2.14)||1.14/6.80||30.54||620/7.8|
© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Zhao, W.; Wu, Q.; Reinthal, A.; Zhang, N. Design, Implementation, and Field Testing of a Privacy-Aware Compliance Tracking System for Bedside Care in Nursing Homes. Appl. Syst. Innov. 2018, 1, 3. https://doi.org/10.3390/asi1010003
Zhao W, Wu Q, Reinthal A, Zhang N. Design, Implementation, and Field Testing of a Privacy-Aware Compliance Tracking System for Bedside Care in Nursing Homes. Applied System Innovation. 2018; 1(1):3. https://doi.org/10.3390/asi1010003Chicago/Turabian Style
Zhao, Wenbing, Qing Wu, Ann Reinthal, and Nian Zhang. 2018. "Design, Implementation, and Field Testing of a Privacy-Aware Compliance Tracking System for Bedside Care in Nursing Homes" Applied System Innovation 1, no. 1: 3. https://doi.org/10.3390/asi1010003