AI-Driven Privacy in Elderly Care: Developing a Comprehensive Solution for Camera-Based Monitoring of Older Adults

Featured Application

An artificial intelligence-driven elderly monitoring system that preserves the visual privacy of the subjects under surveillance while providing essential visual context and essential wellness status to caregivers.

Abstract

The need for privacy in elderly care is crucial, especially where constant monitoring can intrude on personal dignity. This research introduces the development of a unique camera-based monitoring system designed to address the dual objectives of elderly care: privacy and safety. At its core, the system employs an AI-driven technique for real-time subject anonymization. Unlike traditional methods such as pixelization or blurring, our proposed approach effectively removes the subject under monitoring from the scene, replacing them with a two-dimensional avatar. This is achieved through the use of YOLOv8, which facilitates accurate real-time person detection and pose estimation. Furthermore, the proposed system incorporates a fall detection algorithm that utilizes a residual causal convolutional network together with motion features of persons to identify emergency situations and promptly notify caregivers in the event of a fall. The effectiveness of the system is evaluated to emphasize its advanced privacy protection technique and fall detection capabilities using several metrics. This evaluation demonstrates the system’s proficiency in real-world applications and its potential to enhance both safety and privacy in elderly care environments.

1. Introduction

The global demographic landscape is experiencing a notable shift, with a rapidly growing elderly population [1]. This demographic change also brings an increase in health-related risks among elderly people, most notably a heightened incidence of fall accidents. The World Health Organization reports that around 28–35% of elderly persons suffer a fall annually [2], posing significant risks, especially for those living alone. In such cases, a fall can lead to severe, potentially life-threatening consequences if assistance is not promptly provided. Therefore, in recent years, many systems have been introduced for effective monitoring and rapid response systems to ensure the safety and well-being of the elderly population.

The development of various monitoring technologies, predominantly focused on fall detection and prevention [3,4,5], has been a key response to this challenge. These systems are categorized into wearable device-based, radar-based, and camera-based solutions [6]. While camera-based solutions are popular and easy to install, their implementation raises substantial privacy concerns. With the elderly population valuing their independence and dignity, the intrusive nature of continuous monitoring can be a source of discomfort and stress. Therefore, smart home technologies like camera monitoring systems, increasingly adopted to support aging in place, must balance effective monitoring with privacy protection [7].

In this context, the preferences of older adults for privacy-protective surveillance methods, particularly those utilizing avatar-based systems, become highly relevant. According to one study [7], older adults naturally have concerns regarding both visual and behavioral privacy, with a stronger inclination towards maintaining visual privacy. A significant majority of healthy older adults show a preference for technologies that transform their actual images into avatars, rather than more invasive methods. These avatar-based systems effectively maintain the functionality of surveillance—essential for safety and security—while addressing the crucial concerns of visual privacy. By replacing real images with avatars, such systems can reduce the feelings of intrusion and discomfort associated with being constantly monitored, thereby aligning with elderly people’s expectations for dignity and independence. Such innovative approaches in smart home technologies not only support the practical aspects of aging in place but also prioritize the emotional and psychological well-being of elderly individuals, thus promoting greater acceptance among older adults.

Building on these challenges, our research presents a novel contribution to the field of elderly care, particularly in addressing the delicate balance between effective monitoring and preserving privacy. First, we introduce an advanced artificial intelligence (AI)-driven camera-based monitoring system based on the YOLOv8 model, employing innovative techniques for real-time subject anonymization. This approach significantly mitigates privacy concerns by replacing traditional invasive video feeds with non-intrusive two-dimensional (2D) avatars. Second, our system incorporates a fall detection algorithm developed using temporal variation in various motion features and a causal convolutional network classifier, ensuring timely responses to emergencies. Our work contributes to the technological and ethical advancements in elderly care and thoughtfully addresses the critical ethical considerations of privacy and dignity. This research represents a significant step towards blending AI technology with a respect for the privacy and dignity of elderly people, emphasizing the importance of safeguarding personal space and autonomy in the development of elderly care systems.

2. Related Works

Elderly monitoring systems have significantly benefited from advancements in sensors, surveillance systems, and the Internet of Things (IoT), which facilitate real-time monitoring and enhanced connectivity. These technologies provide a detailed overview of an elderly individual’s environment and health status [5]. Following this, developments in AI, specifically in person detection [6], pose estimation [7], and action recognition [8], have greatly improved the accuracy of these systems. These AI-driven techniques allow for precise identification and automated analysis of elderly movements, aiding in early detection of potential health risks.

2.1. Camera Surveillance Systems

Camera surveillance systems have emerged as a vital component in elderly monitoring, leveraging advanced image processing and AI algorithms. The integration of high-resolution cameras with real-time video analytics has enabled more detailed observation of elderly activities, ensuring safety by providing prompt alerts to caregivers [8,9,10]. Innovations in this domain include the use of infrared or thermal imaging to enhance monitoring capabilities during low-light or no-light conditions [11,12,13,14,15]. Additionally, the implementation of machine learning models, such as convolutional neural networks (CNNs), has significantly improved the ability of these systems to recognize and interpret complex human behaviors [16,17]. This progress has been vital in not just detecting emergencies like falls or unusual inactivity, but also in understanding daily patterns and deviations that might indicate health or well-being concerns.

Furthermore, the adoption of edge computing in camera surveillance systems marks a crucial development in the area of elderly surveillance systems. By processing data locally, smart camera systems are more cost-effective, reduce latency, and decrease response time, which is essential for prompt interventions in emergency situations [18,19,20,21]. Moreover, edge computing aids in addressing privacy concerns, as sensitive data can be processed and stored locally, minimizing the need for transmission to remote servers. This feature is particularly important in elderly care, where privacy and data security are paramount. However, recent advances in edge computing rely on minimal data transmission, meaning that only essential information, such as alerts or behavioral data, is transmitted to servers and caregivers. While this method effectively addresses visual privacy concerns [7], it does not provide caregivers with a complete picture of a scene. This limitation is of the utmost concern, since fall detection and behavioral analysis models, although useful, are not infallible and are prone to false alarms and missed incidents.

2.2. Artificial Intelligence in Elderly Care

In elderly care, the application of AI has seen a transformative breakthrough, particularly through the deployment of sophisticated algorithms and neural network architectures for pose detection, fall detection, and activity monitoring. Human pose estimation [22,23], serving as the core component of such automated analysis systems, lays the foundation for subsequent analyses like fall detection and activity monitoring. State-of-the-art algorithms such as PoseNet [24] and BlazePose [25] have revolutionized pose estimation with 3D and 2D image data using CNN architectures, significantly impacting research and practical applications in this field. These algorithms excel in accurately identifying and tracking human body positions in real time, which is critical for monitoring elderly behavior.

Fall detection models rely heavily on object detection or pose estimation models. These models typically utilize a combination of spatial and temporal data processed through advanced machine learning algorithms like CNNs and Long Short-Term Memory networks (LSTMs) [26,27]. The spatial aspect involves analyzing the posture and position of an individual, while the temporal component examines movement patterns over time. However, some models also use direct classification approaches using pose/skeleton data, as presented in [28,29,30], where the pose of a person is used to classify whether or not a fall has occurred. Moreover, other research incorporates object detection models and bounding box techniques for fall classification, as shown in [31,32]. Recent research has employed diverse and innovative strategies [33,34] to enhance the reliability and accuracy of fall detection, a critical component in ensuring the safety of elderly people.

Pose estimation models have also led research in the direction of activity recognition. Recent advancements in machine learning (ML) allow these models to utilize skeletal information derived from pose estimation [35], coupled with spatial–temporal data and/or other robust features, to effectively classify human activities [17,36]. As shown in the analysis presented in [6], vision-based activity recognition models have shown greater robustness compared to motion sensor-based monitoring systems. This is primarily due to the distinct visual features that can be separated into a variety of actions. In contrast, motion sensor features may display similar speeds and patterns across different actions, leading to potential ambiguities in classification. Therefore, several elderly monitoring systems utilize these models to automate the behavioral monitoring of elderly subjects [12]. Some research also combines pose data with handcrafted features to achieve fall detection classification, as presented in the method proposed in [37], where distances and angle features between pose keypoints between consecutive frames were used to detect falls. Similarly, in [38], other biometric features from pose keypoints were extracted and used to train a random forest model to classify falls.

Despite the advancements in skeleton-based fall detection methods [39], there remain significant gaps in the technology, particularly in terms of privacy and user-centric approaches for elderly surveillance. Enhancing the privacy aspects of monitoring systems, for instance, does not just require adhering to ethical standards but also increasing the acceptance and comfort level of elderly individuals who are being monitored. Privacy-preserving methods, such as anonymization of video data, can foster greater trust and willingness among elderly users [40,41,42,43,44].

2.3. Subject Anonymization

In elderly care, privacy-preserving monitoring systems often utilize ambient sensors or wearables to provide passive monitoring, thereby addressing the privacy concerns posed by camera surveillance [42]. These technologies capture essential data regarding the physical well-being and activity levels of elderly people without the invasiveness [45,46,47,48,49] associated with video recordings. However, while sensors and wearables offer a less intrusive means of observation, they may not always provide the comprehensive visual context necessary for certain caregiving tasks. Furthermore, several vision-based monitoring systems [10,26,33,50,51] have proven to be more capable of detecting emergency situations, such as falls, making them more popular among caregivers and less inconvenient for elderly people than wearables.

Subject anonymization is a technique widely used in computer vision for enhancing visual privacy and making it harder to identify subjects under surveillance. Early works in this field focused on methods such as face identification and warping [52] to render the faces of subjects unidentifiable. Some common anonymization methods [53] in video surveillance, including blanking, obfuscation, abstraction, and encryption, safeguard personal identities by first detecting individuals in video feeds and then applying privacy-enhancing techniques to these areas. Blanking replaces sensitive regions with background imagery, obfuscation (via pixelation, blurring, or warping) and abstraction (employing edge detection, avatars, or silhouettes) modify visual details to prevent recognition, and encryption ensures data privacy by restricting access to selected areas with decryption keys, offering strong privacy safeguards. Recent innovations in visual privacy techniques [54,55] focus on whole-body or face detection/segmentation, altering visual representations through methods like blurring, pixelation [56], and embossing; replacing them with silhouettes, skeleton representations, and three-dimensional avatars; or complete invisibility, further advancing privacy protection. Yet the literature indicates that many of these methods might necessitate manual image manipulation or entail substantial processing time, limiting their applicability in real-time systems.

Other privacy techniques specifically designed for elderly monitoring systems include the use of depth images, rather than solely RGB images, to develop fall detection systems [40,57]. Depth images, by only revealing spatial information, enable accurate fall detection without compromising visual privacy. Infrared/thermal camera-based monitoring systems [11,12,58,59], on the other hand, capture heat signatures, further reducing visible details and enhancing privacy for elderly people. While these methods offer effective monitoring while prioritizing individual privacy, they also restrict the visual information often needed for contextual understanding in caregiving.

Given the outlined challenges and advancements in privacy-preserving technologies for elderly monitoring, there is a pressing need for automated systems capable of providing efficient subject anonymization while ensuring effective surveillance. The system proposed in this paper uses the latest advancements in computer vision and AI to automatically protect individuals’ identities while still keeping a close eye on their well-being. It is designed to dynamically adapt to the complexities of real-world scenarios, ensuring that privacy does not compromise the capacity to identify and react to emergencies. With a core focus on the fundamental objectives of surveillance—safety and well-being—the proposed system introduces a privacy-aware approach that retains the integrity of elderly monitoring. This innovative solution effectively meets the dual demands of preserving privacy and delivering comprehensive care, creating an ethical surveillance system through the application of state-of-the-art technology.

3. Materials and Methods

This section details the systematic approach undertaken to design, implement, and evaluate the proposed surveillance system. It describes the technological frameworks and procedural steps that collectively facilitate the achievement of a privacy-preserving elderly monitoring system.

3.1. System Design

The system design, as shown in Figure 1, integrates a Wi-Fi-enabled Internet protocol (IP) camera (TP-Link, Hong Kong), a high-speed Wi-Fi router (TP-Link, Hong Kong), and a desktop server computer (ASUS, Taipei City, Taiwan) equipped with an NVIDIA RTX 3050 graphics processing unit (GPU) (Gigabyte Technology, New Taipei City, Taiwan), an Intel i5 processor (INTEL, Hong Kong) with 12 cores, and 32 GB of random-access memory (RAM) (Micron, Singapore). This setup captures and wirelessly transmits a video stream for real-time processing, including a machine learning model and a privacy-preserving anonymization technique. The processed video stream and elderly status are then securely uploaded to the cloud and made accessible to caregivers through a smartphone application.

The system begins with an IP camera capturing live video in the living space of an elderly individual, as shown in Figure 2. This video stream is processed using YOLOv8 [60], a machine learning model that identifies the presence of a person in the scene. Once detection occurs, the process splits into two main functions: ensuring the subject’s privacy and analyzing their posture for safety concerns.

To protect privacy, the system employs subject anonymization by using the bounding box information of the detected individual and replacing their visible image with background. This method maintains the individual’s privacy by hiding their appearance in the video feed. Concurrently, pose keypoints are extracted using YOLOv8 to form a skeleton representation of the person. This skeleton aids in adjusting the 2D avatar’s posture to accurately reflect the person’s movements, ensuring a realistic representation.

The skeleton representation is crucial for the fall detection algorithm, which analyzes the person’s movements and posture. If the algorithm detects a pattern indicative of a fall, it immediately triggers an alert for assistance. Finally, the system securely transmits the anonymized video stream, now featuring the 2D avatar, along with any alerts from the fall detection algorithm to the caregiver through the cloud. In the event of a fall, the system sends an alert to the caregiver that includes both the fall alarm and a real-time image of the scene without modifications. This ensures that the caregiver receives comprehensive information about the fall and the actual condition of the monitored person, enabling a prompt response.

3.2. Person Detection and Subject Anonymization

For a system focused on crucial safety tasks like monitoring elderly people, picking the right machine learning model is important. It needs to be not only real-time and reliable but also computationally efficient. After looking into several options, this study chose YOLOv8 for its strong ability to detect people and estimate their poses in real time. Developed by Ultralytics and released in January 2023, YOLOv8 [60] has quickly become a favorite in both practical applications and academic research, due to its reliable and accurate detection and classification capabilities. Several researchers have used YOLOv8 for multiple tasks, including multi-object pedestrian tracking [61], student behavior detection in classroom [62], road defect detection [63], safety helmet detection [64], and human pose estimation [65].

The YOLOv8 architecture, shown in Figure 3, is designed to be efficient and not demanding on system resources, meaning that it can run smoothly on various types of hardware, from personal computers to more powerful GPU-based systems. This flexibility is vital for ensuring that the proposed elderly monitoring system can operate effectively across different settings without needing specialized equipment. The YOLOv8’s architecture consists of IP camera-to-fine (c2f) modules, which allow the model to first identify larger, more general areas where a person might be (the “coarse” part) and then refine its focus to detect finer details within those areas (the “fine” part). This approach is particularly effective for identifying not just the bounding boxes that indicate where a person is in a video but also the specific pose keypoints or body joints.

In our proposed system for elderly monitoring, the subject anonymization process was developed using the YOLOv8 model’s capabilities. The algorithm, as shown in Figure 4, initiates by setting a background image filled entirely with black pixels, creating an empty buffer for the background of subsequent frames. As the system processes each frame of the video stream, it employs YOLOv8 (yolov8s-pose.pt) to detect bounding boxes for persons in the frame. The original model is converted to support the open neural network exchange (ONNX) runtime to increase the inference speed across CPU and GPU architectures. This step identifies the regions within the frame that require anonymization or privacy protection. Following detection, the algorithm assesses whether any individuals have been identified. If they have, it proceeds to selectively update the regions within these bounding boxes using content from a predefined or updated background buffer image, effectively anonymizing the subject by replacing their visual representation with backgrounds of previously acquired frames. Conversely, if no individuals are detected within a frame, the algorithm updates the entire background buffer with the current frame, refreshing the scene in the absence of significant foreground activity. This dynamic updating mechanism allows the system to maintain an up-to-date background representation of the monitored environment.

For frames where individuals are detected, the system further uses human pose keypoint data to extract the joint angles of the skeleton representation. Leveraging this information, the algorithm adjusts a 2D avatar to match the detected pose, overlaying this avatar within the bounding box in place of the person’s image. Moreover, the area outside the detected bounding boxes is utilized to update the existing background buffer image, ensuring that the scene information remains current in each frame. This step not only preserves the anonymity of the monitored individual but also retains a visual cue of their activities and movements, which is crucial for monitoring well-being and detecting potential emergencies. This process works in an iterative way, with the system continuously checking whether the video stream has concluded. If the stream is ongoing, the algorithm advances to the next frame, repeating this procedure until the end of the stream, ensuring comprehensive and continuous monitoring throughout.

For animating the 2D avatar, this study utilizes Panda3D, an open-source game engine that is supported in the Python programming language. Pose data, extracted in the form of pixel coordinates from the YOLOv8 model, identify 17 keypoints that represent a human figure’s posture within a frame, as shown in Figure 5a. To align these keypoints with the rig joints of the 2D avatar, the raw keypoint data are transformed into joint angle data, ensuring that the avatar’s movements accurately mirror those of the detected human posture, as shown in Figure 5b. Additionally, two extra keypoints are generated at the midpoint between keypoints {5, 6} and {11, 12} to determine the upper torso center and lower torso center coordinates, respectively. These coordinates are used for calculating the neck and spine angles, aligning with the 2D avatar’s rig. To ensure smooth and natural transitions in the avatar’s movements across consecutive video frames, a 3rd-order polynomial fitting is applied to the extracted joint angle data to prevent any sudden jumps or jerk motions.

Given three keypoints, A, B, and C, where B is the joint for which the angle is calculated and A and C are adjacent joints, the angle at B can be calculated using the dot product formula. Using vectors $\overrightarrow{BA}$ and $\overrightarrow{BC}$ from the pixel coordinates of these points, the angle $\theta$ for joint B can be calculated as shown in Equation (1):

$$\theta =arccos\left(\frac{\overrightarrow{BA}·\overrightarrow{BC}}{‖\overrightarrow{BA}‖‖\overrightarrow{BC}‖}\right)$$

(1)

However, considering the relative orientation of joints and given the planar nature of 2D avatar animation, this study uses a more direct approach to calculate the angles in 2D space, which can be represented as Equation (2), below:

$$\theta =arctan2\left(u_y,u_x\right)-arctan2(v_y,v_x)$$

(2)

where $u_y$ and $u_x$ are the x and y axis components of $\overrightarrow{BA}$ and $v_y$ and $v_x$ are the x and y axis components of $\overrightarrow{BC}$. This method significantly enhances 2D avatar animation by accurately accounting for vector orientation, which is crucial for replicating human movements. Utilizing the arctan2 operator, which assesses both vector magnitude and direction, ensures that angles are measured across the full 360° range, from 180° to −180°. This capability allows for capturing motion in all four quadrants, an important feature for animating complex movements with lifelike accuracy.

The study applies a 3rd-order polynomial fitting to smooth the transitions of these angles over time, ensuring natural movement of the 2D avatar. Given a set of joint angles $\theta$ over time$t$, the polynomial fitting can be represented using Equation (3):

$$\widehat{\theta }\left(t\right)=a_0+a_{1}t+a_2{t}^2+a_3{t}^3$$

(3)

where $a_0$, $a_1$, $a_2$, and $a_3$are the coefficients determined by fitting the polynomial to the historical data of joint angles using the least squares method.

3.3. Fall Detection Method

This study implements a hierarchical architecture for fall detection, as shown in Figure 6, using bounding boxes and pose keypoints from the YOLOv8 model, originally used for subject anonymization in Section 3.2. These outputs are used for the extraction of motion features, including the average velocity of keypoints, the body’s center, the bounding box’s center, and the aspect ratio of the bounding box. To accommodate frame rate variations across different video acquisition systems, motion features are normalized by calculating respective pixel/second velocities.

Thereafter, a 1D causal CNN is employed for binary classification, determining “fall” or “not fall” scenarios in each frame. This network uses the extracted features and their temporal variations. The causal nature of the network guarantees that predictions are made based on present and past data only. The size of the training and inference window is set to correspond to 1 s or the number of frames per second, allowing for a consistent temporal scope in analyzing the video data.

The average pose keypoint velocity is calculated by measuring the change in the position of each keypoint across consecutive frames and dividing it by the time elapsed between these frames. If $P_{i,t}$ and $P_{i,t+1}$ represent the positions of keypoint $i$ at times $t$ and$t+1$, respectively, the velocity $V_i$ for keypoint $i$ is calculated as shown in Equation (4):

$$V_i=\frac{‖P_{i,t+1}-P_{i,t}‖}{\mathsf{\Delta }t}$$

(4)

The average velocity ($V_{avg}$) across all N keypoints can be calculated as shown in Equation (5):

$$V_{avg}=\frac{1}{N}\sum _{i=1}^N{V}_i$$

(5)

The body center ($C_{body}$) is determined using the positions of the shoulders and hips. The midpoints $M_{shoulder}$ and $M_{hip}$ are found by averaging the positions of the left and right shoulders ($P_{\mathrm{LS}}$ and $P_{\mathrm{RS}}$) and the left and right hips ($P_{\mathrm{LH}}$ and $P_{\mathrm{R}\mathrm{H}}$), respectively, as shown in Equations (6)–(8):

$$M_{shoulder}=\frac{P_{\mathrm{L}\mathrm{S}}+P_{\mathrm{R}\mathrm{S}}}{2}$$

(6)

$$M_{hip}=\frac{P_{\mathrm{L}\mathrm{H}}+P_{\mathrm{R}\mathrm{H}}}{2}$$

(7)

$$C_{body}=\frac{M_{shoulder}+M_{hip}}{2}$$

(8)

The center of the bounding box ($C_{bbox}$) is calculated by taking the average of the x and y coordinates of the bounding box’s corners. If $B_{tl}$ and $B_{br}$ represent the top-left and bottom-right corners of the bbox, respectively, then $C_{bbox}$ can be calculated using Equation (9):

$$C_{bbox}=\frac{B_{tl}+B_{br}}{2}$$

(9)

The aspect ratio ($AR$) of the bounding box is determined by the ratio of its width to its height. The width can be calculated as the difference between the x components of the left ($B_{tl}$) and right ($B_{tr}$) corners, while height can be calculated from the y components of the top ($B_{tl}$) and bottom ($B_{bl}$) corners. Therefore, the equation for calculating $AR$ can be formulated as in Equation (10):

$$AR=\frac{B_{tr}\left(x\right)-B_{tl}\left(x\right)}{B_{bl}\left(y\right)-B_{tl}\left(y\right)}$$

(10)

The temporal variation ($\sigma$) of the features is calculated over a sliding window containing the sequence of feature values $[F_t, F_{t+1}, F_{t+2}, \dots , F_{t+n}]$ over time $t$. Within this window, the temporal variation is calculated using Equation (11):

$$\sigma =\sqrt{\frac{1}{k}\sum _{i=t-k+1}^t{\left(F_i-\mu \right)}^2}$$

(11)

where $F_i$ represents the feature value at each frame within the window, $\mu$ is the mean of these values, and $k$ is the total number of frames in the window. This equation quantifies the variability of feature values around the mean, providing a measure of temporal variation across the video frames.

Building on previous research on fall detection systems, this research aims to develop a classification model that can be seamlessly implemented across various scenes. While numerous studies have demonstrated that analyzing variations in features allows for the creation of rule-based methods to detect falls and non-falls within a monitored environment, it is crucial to acknowledge that these rules necessitate a detailed examination of body and motion features, along with expert knowledge of biodynamics. Consequently, this study opted for a 1D causal convolutional neural network (CNN) with residual connections as the model of choice for learning and classification.

One-dimensional CNNs were first used for sequence-to-sequence generation of audio data, as shown in [66]. Over time, various adaptations, including causal and dilated causal CNNs, were developed to cater to a broad spectrum of time-series and sequence datasets necessitating the maintenance of strict causal relationships during training and inference, as shown in [67,68,69,70]. Notably, 1D convolutional networks have demonstrated significant efficacy in applications related to human motion modeling [71,72,73] and classification of complex patterns within 1D time-series datasets [74,75,76,77]. This versatility and robust performance highlight the potential of 1D CNNs to capture intricate temporal dynamics, making them a compelling choice for studies focusing on sequence data regression and classification, eliminating the need for complex feature engineering.

This study employs 1D dilated causal CNN layers with residual connections, as illustrated in Figure 7, to construct the classification model. Given that body motion features and bounding box attributes have been previously extracted and their temporal variations are used for training, the model architecture is intentionally simplified. It consists of three residual blocks followed by a fully connected layer with 32 neurons with sigmoid activation at the network’s tail. The output node consists of a single neuron with sigmoid activation for the binary classification of fall and non-fall scenarios. Each residual block in the model starts with a dilated causal CNN layer, which uses a specific number of filters and a dilation parameter, followed by a Scaled Exponential Linear Unit (SELU) [78] activation layer. This setup allows for the initial expansion of the network’s receptive field, enabling it to capture wider nonlinear temporal patterns and without a significant increase in computational complexity. After this, another causal CNN layer is applied, using the same number of filters but without dilation, and this is also followed by an SELU activation layer. To integrate learned features while preserving information from earlier layers, each block includes a skip connection that undergoes a 1 × 1 convolution. This convolution is designed to match the number of filters to the block’s final layer before adding it to the block’s output. The model is structured with three residual blocks, each set to progressively increase the number of filters (16, 32, and 64) and dilation parameters (1, 2, and 3, respectively). This progressive design allows the network to hierarchically extract and refine features from simpler to more complex patterns. Incrementally expanding the number of filters and dilation rates ensures that the model can learn from different motion characteristics to distinguish between normal activities and fall incidents based on temporal patterns within the data.

The proposed network is implemented using PyTorch version 2.1.0, a popular deep learning framework among machine learning researchers. During the training phase, the Adam [79] optimization algorithm is employed, chosen for its effective handling of sparse gradients and its adaptability in different scenarios. The initial learning rate is set to 0.001, a standard starting point that balances the speed and stability of the learning process. As training progresses, the Adam algorithm automatically adjusts this learning rate based on the training data’s behavior, ensuring optimal convergence rates.

4. Results

This section presents the outcomes achieved through the implementation of the subject anonymization technique and the fall detection algorithm.

4.1. Dataset

This study used two publicly available datasets for an evaluation of the proposed system. The first dataset used in this study was the University of Rzeszow Fall Detection (URFD) dataset [80], created at the University of Rzeszow, and comprises 70 sequences at a resolution of 640 × 480, including 30 falls and 40 activities of daily living, such as sitting, crouching, picking up objects, lying on the floor, and lying on a bed or couch. The dataset features a total of 13,000 images—3000 dedicated to fall sequences and 10,000 to daily activities. They are categorized into three classes: non-falling activities, post-fall scenarios (not used for classification in this study), and active falls (used for classification).

The other dataset was curated by the “Laboratoire Electronique, Informatique et Image” (LE2I) [81] and contains 191 annotated videos from various elderly living environments, including home (60 videos), coffee room (70 videos), office (33 videos), and lecture room environments (28 videos), 130 out of which show falls and the rest of which show daily activities. This dataset includes a range of actions, such as walking, sitting on a chair, sitting on a sofa, and bending over, providing a comprehensive view of typical elderly movements. These activities are particularly valuable as they enable the evaluation of fall detection models to determine whether they can accurately distinguish actions like sitting and bending from actual falls. This dataset was captured using a single camera with a resolution of 320 × 240 and a frame rate of 25 frames per second. However, in this study, these videos were resized to 640 × 480 resolution to match the input resolution of YOLOv8.

4.2. Subject Anonymization

The proposed anonymization system was tested using the videos from the LE2I dataset to evaluate the algorithm’s effectiveness. With regard to performance, in terms of detection accuracy, utilizing the YOLOv8 model (yolov8s-pose.pt), the system was able to reliably detect the person’s location in over 99% of the video frames. However, challenges arose in scenarios with significant occlusion, where more than half of the subject’s body was obscured by environmental objects such as desks or tables. In these instances, the model occasionally encountered difficulties in detecting keypoints with high confidence. Nonetheless, it consistently achieved success in identifying the upper body’s bounding box, even under such challenging conditions. Limitations were notably observed in rare cases when parts of the subject exited the camera’s field of view, resulting in detection failures. To mitigate these limitations, future implementations could consider the strategic placement of cameras within the environment or the integration of multiple cameras to avoid blind spots, thereby enhancing the system’s overall robustness and reliability in diverse settings.

To visually demonstrate the anonymization process utilized in this study, four images are presented to illustrate the stages: an original video frame is shown in Figure 8a, showing a scene prior to any processing. Figure 8b displays the privacy frame, where the person has been removed from the scene using their bounding box. The skeleton representation, as predicted by the YOLOv8 model’s pose detection, is shown in Figure 8c. Finally, Figure 8d presents the 2D avatar superimposed onto the scene, representing the person’s movements in an anonymized manner. Additionally, another action, a person sitting, is shown in Figure 9. Both Figure 8 and Figure 9 originate from the “lecture room” video of the LE2I dataset. This scene is well-lit, offering a baseline for fair comparison. Furthermore, two frames from the “home” scene of the LE2I dataset, which is moderately lit, are presented in Figure 10 and Figure 11. Despite varying lighting conditions, scene setups, and the actions of the person, the proposed method effectively detected the person in the scenes and superimposed the 2D avatar, demonstrating its ability to perform consistently across different environments and lighting conditions.

4.3. Fall Detection

To evaluate the proposed fall detection system, bounding boxes and motion features were extracted from video frames, as detailed in Section 3.3. The training of the proposed 1D residual causal CNN model utilized videos exclusively from the “office” scene of the LE2I dataset. The average pose velocity and its temporal variation for a video containing a fall and another without a fall are shown in Figure 12a and Figure 12b, respectively.

Given that there were only 33 videos for the “office” scene, which is insufficient for training a deep learning model effectively, this study incorporated a data augmentation technique. This approach rolled the feature data along the time-series axis to increase the dataset. Specifically, the time-series data underwent a shift of 25 frames along the time axis for each video frame. Iterating this process and adjusting the frame shift by a factor of 25 frames, the size of the training dataset was substantially increased by 100 times. To illustrate this method, Figure 13 shows (a) the original average pose velocity, (b) the data after a 25-frame shift to the right, and (c) the data following a 25-frame shift to the left, demonstrating the augmentation’s effect on the dataset’s variation in fall location for the time series. Using this augmentation method, the model was trained for 100 epochs, and the model’s weights were saved for inference from the rest of the dataset.

This study utilized the two benchmark datasets mentioned in Section 4.1 to evaluate the proposed model. The model trained on the “coffee” scene of the LE2I dataset was tested on the rest of the LE2I dataset scenes and also on the URFD dataset. Given the binary classification nature of the fall detection, the evaluation metrics employed in this research are detailed below [33]:

• Accuracy: The percentage of items correctly classified, calculated as (TP + TN)/(TP + TN + FP + FN).
• Precision: The proportion of true-positive identifications among all positive predictions, calculated as TP/(TP + FP).
• Recall/sensitivity: The true-positive rate or the model’s ability to identify positive instances accurately, calculated as TP/(TP + FN).
• Specificity: Often used together with recall, specificity measures the true-negative rate or the model’s ability to correctly identify negative instances, calculated as TN/(TN + FP).
• F1-Score: This denotes the harmonic mean between precision and recall, calculated as 2 × (Precision × Recall)/(Precision + Recall)

Table 1. Frame-wise evaluation results for the LE2I and URFD datasets using the proposed fall detection method.

Dataset	Accuracy [%]	Precision [%]	Sensitivity [%]	Specificity [%]	F1-Score
LE2I	98.86	90.02	91.11	99.35	0.905
URFD	96.23	97.18	92.56	98.40	0.948

presents the frame-wise evaluation results for the LE2I and URFD datasets using the proposed fall detection method. For the LE2I dataset, the method achieved an accuracy of 98.86%, a precision of 90.02%, a sensitivity of 91.11%, a specificity of 99.35%, and an F1-Score of 0.905. In comparison, on the URFD dataset, the method attained an accuracy of 96.23%, a precision of 97.18%, a sensitivity of 92.56%, a specificity of 98.40%, and an F1-Score of 0.948. These results indicate that the proposed method performs accurately across different datasets, excelling particularly in specificity, reducing false alarms while maintaining high accuracy and precision. The results of the proposed method are comparable to those of the methods of the original datasets [80,81] and other state-of-the-art methods [33,34] evaluated with these datasets.

Furthermore, when evaluating the system on a video-wise basis, both accuracy and precision reached 100% for the tested datasets. This suggests that, while frame-wise metrics provide insight into the method’s performance at a granular level, the video-wise evaluation showcases the system’s ability to accurately classify fall or non-fall events in entire video sequences with perfect precision. This highlights the method’s effectiveness in real-world applications, where accurate event classification over live video streams is critical.

4.4. System Latency

In the proposed system equipped with an NVIDIA RTX 3050 Ti GPU with an Intel i5 processor (12 cores) and 32GB of RAM, the system’s latency demonstrates notable variation depending on the hardware setup. When utilizing the GPU for inference, it achieves up to 72 frames per second (fps), and when solely relying on the CPU, it maintains 43 fps. This performance improvement is due to the ONNX optimization of the YOLOv8 model and the proposed fall detection model. When the system’s resources are limited to using only four CPU cores, it still attains a reasonable speed of 14 fps. This performance demonstrates the system’s ability to sustain real-time processing capabilities across a variety of hardware setups, emphasizing its adaptability and potential applicability in end-user scenarios that involve low-cost hardware. Additionally, by employing a smaller YOLOv8 model like YOLOv8-nano, the system may achieve even faster speeds, further enhancing its accessibility and performance in budget-friendly hardware setups. The proposed system architecture may also scale effectively to manage feeds from multiple cameras simultaneously to address different room and blind spot issues without sacrificing processing efficiency. This scalability ensures comprehensive monitoring in various scenarios, ensuring that no blind spots are left unwatched.

5. Conclusions

The results achieved through the proposed fall detection and subject anonymization method, as tested on the LE2I and URFD datasets, demonstrate the system’s efficacy in ensuring privacy while maintaining high accuracy in fall detection across various real-world scenarios. The method’s performance, with a frame-wise accuracy reaching 98.86% and a precision of up to 97.18%, alongside the video-wise accuracy and precision both reaching 100%, demonstrates its robustness and reliability. Particularly noteworthy is the system’s specificity, which significantly reduces false alarms, a crucial factor in elderly monitoring systems to prevent unnecessary panic or stress.

The subject anonymization aspect of this study plays a pivotal role in addressing privacy concerns, a significant barrier to the acceptance of monitoring technologies among elderly people. By effectively anonymizing individuals in the video through the removal of identifiable features and replacing them with 2D avatars, this method respects the privacy of the monitored individuals. This approach not only ensures the utility of the monitoring system for safety and wellness purposes but also promotes its acceptance among elderly users who might be concerned about their privacy and dignity.

In conclusion, while current state-of-the-art technologies in fall detection offer high accuracy, they often require substantial computational resources and typically overlook the ethical and user-centric aspects of monitoring. This lack of attention to user preferences, particularly around visual privacy, can significantly hinder the acceptance and applicability of these technologies among elderly users, who may prioritize dignity and privacy in their care solutions. This study offers a robust solution that effectively balances the dual objectives of accurate fall detection and thorough privacy protection. The high accuracy and low false-positive rate indicate significant progress in elderly care technology. Importantly, this research highlights the innovative application of artificial intelligence not just for utility in fall detection but also in addressing design needs related to privacy concerns. Its emphasis on privacy protection through subject anonymization is likely to encourage greater acceptance among the elderly population, making it a significant step forward in the development of assistive technologies that cater to the sensitive needs of this demographic.

Some of the limitations observed in this study are inherent to using 2D avatar representations to model complex 3D human movements. While these 2D avatars are effective for basic monitoring, they struggle to accurately convey complex actions from certain angles, which can lead to ambiguities in interpreting the subject’s actual posture. Future research could focus on integrating optimized 3D modeling techniques, which offer a more detailed and dynamic representation of human poses while preserving real-time performance. Future work will also focus on refining the fall detection and anonymization algorithms for enhanced accuracy and real-time processing, exploring multi-camera setups to reduce occlusions, and integrating advanced privacy-preserving techniques. Additionally, efforts will aim at improving user acceptance through ethical considerations to ensure that the technology not only meets the highest standards of efficiency and privacy but also considers the major aspects of social ethics.