1. Introduction
Demographic ageing in humans means that to date, 12% of the global population are aged over 60 years, and this number is likely to double within a few decades [
1]. Ageing leads to a higher prevalence of complications that may benefit from exercise therapy. Such an increase in ageing will mean that the rapid development of science and medicine, as well as the introduction of new technologies and methodologies utilized by health systems, will be needed. Increased knowledge has been gained regarding new treatment regimes for a growing number of chronic diseases and traumas, but with consequential increases in social and economic costs [
2]. It is well-known that rehabilitation forms an important part of a typical overall treatment plan, which can be delivered, for instance, by utilizing therapeutic exercise (physiotherapy). The performance of physical activity has many advantages in older people with dementia, and can positively affect the preservation of cognitive abilities [
3]. Stroke patents may also benefit from physical activities, which can result in improved recovery rates.
However, the success of rehabilitation largely depends on keeping the patient interested and motivated in the continuation of treatment. Factors influencing adherence to the continuation of physical education depend on whether people continue to receive professional assistance and counselling after the completion of the initial training [
4]. Among the main reasons for the termination of continued professional assistance and counselling are forgetfulness, a lack of further supervision and motivation, and time restraints (for example: attending the rehabilitation center).
The use of exercise therapy delivered remotely using posture recognition and interactive content may have a positive impact on enabling patients to perform exercise, as well as their willingness to continue training and rehabilitation programs [
5].
Events such as the recent Covid-19 pandemic reinforce the need for remote exercise therapy with feedback from a doctor, which would be very beneficial for many patients with different disabilities.
Traditionally, exercise therapy consists of demonstrating exercises, observation and evaluation by a health professional, which in turn requires special training and significant face-to-face contact with a patient. However, modern computer and sensor technologies could be utilized to augment (or where appropriate, replace) direct intervention by health professionals. Such technologies that can capture specific postures will be able to determine whether or not the exercise regimes provided to the patient are proving the beneficial postural changes over time, with reference to those obtained from healthy adults. With the capabilities of motion capture systems advancing significantly in recent years, and with motion capture systems being more accessible and effective, they allow the kinematics of the human body to be measured and recorded with sufficient accuracy in real time, even using web cameras.
Two main types of motion capture systems are widely used: those which use markers, and those which estimate joint and limb segment parameters based on neural network training from marker systems. The first requires use of a special suit, or a removable system of sensors (active or passive markers) attached to the human body. The second type, such as those provided by Microsoft Kinect, Intel RealSense, Structure Core and others, use color and depth data, as well as image recognition algorithms, to retrieve the data. These systems can record kinematic data and perform analysis of the human body’s movements in real time.
In addition, the development and availability of these sensors opens more opportunities, as it makes it possible to create bespoke courses of rehabilitation, and to monitor their implementation [
6,
7,
8,
9,
10,
11]. Similar applications have been developed for different patient groups, but the most widely represented software has been designed for post-stroke patients [
12,
13,
14,
15,
16]. Software has also been designed for people with neurological diseases [
17], including cerebral palsy [
18], multiple sclerosis [
19] and traumatic brain injuries [
20].
However, the algorithms used by these systems to estimate the accuracy of execution of movements by such patients are not fully described in the literature. Two of those algorithms can, however, be distinguished by their differing mode of operation. The first is based on the use of dynamic time warping (DTW), along with fuzzy logic [
7], and the other is based on the recognition of different body segment postures and trajectories [
21]. However, the use of a home-based system, using virtual rehabilitation and offering the possibility of communication with a doctor, is more convenient for the patient, and also allows the course of rehabilitation to be altered by adding new exercises, if necessary. DTW is, however, difficult to apply when compared to posture estimation algorithms. Anton et al. utilized the recognition of postures together with trajectories, which resulted in an accuracy of posture estimation of 91.9%, and detection of movements of 95.16% [
21].
Recent advances in machine learning have led to the use of machine learning algorithms in many studies, including posture classification [
22,
23]. The objective of these studies is to classify the sitting postures via conventional algorithms and deep learning-based algorithms using the body pressure distribution data from pressure sensors [
22]. After classifying the sitting postures using several classifiers, average and maximum classification rates of 97.20% and 97.94%, respectively, were obtained from nine subjects with a support vector machine using the radial basis function kernel. Through a comparison of the application of the convolutional neural network (CNN) and conventional machine learning algorithms, the effectiveness of an approach [
23] wherein the CNN algorithm is applied was shown (average value of accuracy = 0.953). However, machine learning-based algorithms have problems with a computational complexity that lead to an inability of real-time implementation (in reference [
22], the authors stressed this point) and the need for resources for training.
These examples of previous research in the use of posture recognition algorithms provide strong arguments for the continued research and development of such algorithms.
The aim of this research was to develop simpler and more efficient identification algorithms for posture and exercise classification within healthy participants, as well as to evaluate these using Kinect V2. The main contributions of our work can be summarized as follows. Three algorithms for the classification of different postures were developed and evaluated. The effectiveness of these algorithms was based on a total error of vector lengths and a total error of angles, and the multiplication of these two parameters was proved. To compare the effectiveness of classification algorithms, a database was created from the descriptions of the 573 known postures, as well as 903 postures which were not related to them. It was shown that the algorithms presented in this study were demonstrated to be reasonably accurate, and could potentially form the basis for developing a simple system for the remote monitoring of rehabilitation involving exercise therapy.
The remainder of this paper is organized as follows. In
Section 2, we describe the Microsoft Kinect V2-based approach to the automatic classification of human exercise movement and present three algorithms for posture classifications. In
Section 3, we compare the effectiveness of the three developed classification algorithms by means of a database that was created from the descriptions of the 573 known postures and 903 postures which were not correctly performed. In
Section 4, we discuss the results and how they can be interpreted from the perspective of previous studies, and of the working hypotheses. Future research directions also are highlighted. Finally, we present the conclusions in
Section 5.
4. Discussion
The aim of this study was to determine accurate posture and exercise classification algorithms with low-cost sensors such as Microsoft Kinect, which has also led to the development of different virtual rehabilitation programs [
13,
26]. The use of such sensors can have many advantages. Firstly, they highlight interactivity and motivation, and they can also be used at home. This is important for people who live in remote areas, where there may not be experts who are locally available. In addition, the technique can be adapted to the needs of any patient group [
27], or animals [
28,
29,
30,
31].
The comparison of this sensor with a professional optical motion capture system has demonstrated that it has the accuracy sufficient for both the tasks and data generation capability needed by specialists in the field of rehabilitation [
8].
However, the question of how to evaluate the correctness of the exercise is still not certain, as the literature is only represented by a limited number of articles [
7,
21]. The previous research has demonstrated a most accurate posture classification of 91.9%, and for movement, a most accurate posture classification of 95.16% [
21]. This study demonstrated a slight increase in the accuracy by using three different algorithms and by setting up a threshold level for: total error of vector lengths; total error of angles; and multiplication of vector errors by angle errors (as in [
21]). Calculating sensitivity and specificity, the classification accuracy of the algorithms was obtained, with the best result shown by the algorithm using the total error of angles (94.9%). This algorithm showed better results when compared with previous research based on a multiplication of the total errors algorithm. This new algorithm also requires considerably fewer parameters for the classification of postures and exercise movements. The previous study, which showed the best accuracy for the posture classification, used 30 variables of the posture descriptor, such as angles and vector lengths [
21]. However, the second algorithm in this research used only 17 variables of posture descriptor, which significantly improved the efficiency of the method.
In our study, when evaluating the classification accuracy of the exercises, we used results for the average accuracy of each participant and the average accuracy of the exercises, which were 94.3% (SD 1.7%) and 94.2% (SD 1.8%), respectively. Those results are practically the same as those of the previous research [
21], but our algorithm, as mentioned above, requires considerably fewer parameters for the classification of postures and exercise movements. More advanced marker-based motion capture systems can also be used to improve the classification accuracy of algorithms. Previous research [
32] has demonstrated that the static error of tracking passive markers with Oqus (Qualisys) cameras was 0.15 mm and a dynamic 0.26 mm, with much higher tracking frequencies than those used by the Kinect V2 sensor.
The definition of human posture can be applied not only to the creation of applications for rehabilitation, but also for monitoring the lives of older people, such as in the recording of a sudden fall. According to statistics, 28–35% of people over 65 years of age experience a fall [
33], after which they often need a period of rehabilitation. Such a monitoring system could detect a person’s posture, and alert relatives, neighbors or close friends in cases where the person’s positional data indicates the possibility of a heart attack, stroke or other complication; such a posture, for example, could be lying down on the floor. The time factor in attending to such situations is very crucial, being directly correlated to the person’s recovery.
More studies are required to develop classification algorithms for the various medical applications mentioned, as this study had a number of limitations, outlined below.
Limited tested sample size and reference database for healthy subjects.
Healthy and young subjects were recruited without any disabilities.
Different races, nationalities and type of disability may influence the results, as well as affect anthropometric data.
Kinect sensors are not consistent in data collection for different environments, and different types of clothing can significantly change the accuracy of the detection of joints, as was noticed in our study.
Future planned research is to use the Qualisys system to improve the algorithm by reducing the number of limitations.
Video analysis is widely applied in the context of human movement detection, and real-time implementation using reliable algorithms based on the postural recognition of healthy persons should provide postural data that can be used to assess the effectiveness of clinically prescribed exercise regimes for patients, as well as allow for variations in exercise regime, dependent on the data collected. Such data would be useful in optimized treatment by exercise therapy.
The advantages of such an approach could also be extended to veterinary applications. Very few studies address automatic video-based analysis of animals—for example, canine behavior as a means of monitoring animal health and wellbeing [
28,
29,
30]—with some of these studies using a 3D Kinect camera to detect joint position. In [
28], the authors present a system capable of identifying static postures for canines that does not rely on hand-labeled data at any point, although the system can only identify the “standing,” “sitting” and “lying” postures with approximately 70%, 69% and 94% accuracy, respectively. Paper [
29] presents a depth-based tracking system for the automatic detection of animals’ postures and body segments, as well as an exhaustive evaluation on the performance of several classification algorithms, based on both a supervised and a knowledge-based approach. Furthermore, Barnard et al. addressed a problem of automatic behavioral analysis of kenneled dogs using 3D video monitoring [
30]. Dog body segment detection was done using standard Structural Support Vector Machine classifiers, and the automatic tracking of the dog was also implemented. However, this tool has a high margin for improvement.
A number of studies were also found in the literature using wide-ranging applications in the biomechanics of animals, as well as in prosthetics to prevent injuries, monitoring rehabilitation after surgical operations, choosing the appropriate orthopedic devices and prostheses, training and others [
34,
35,
36]. Therefore, the classification algorithm of posture can also be useful in not only human medicine, but also veterinary applications, influencing veterinary intervention using exercise regimes, as well as monitoring animals’ health and behavior. Further studies using the Qualisys system and neural network, which would be trained to recognize a dog’s skeleton using cost-effective video cameras, are planned; so far, such work has only been carried out for humans.