Application of Fuzzy and Rough Logic to Posture Recognition in Fall Detection System

Considering that the population is aging rapidly, the demand for technology for aging-at-home, which can provide reliable, unobtrusive monitoring of human activity, is expected to expand. This research focuses on improving the solution of the posture detection problem, which is a part of fall detection system. Fall detection, using depth maps obtained by the Microsoft Kinect sensor, is a two-stage method. We concentrate on the first stage of the system, that is, pose recognition from a depth map. For lying pose detection, a new hybrid FRSystem is proposed. In the system, two rule sets are investigated, the first one created based on a domain knowledge and the second induced based on the rough set theory. Additionally, two inference aggregation approaches are considered with and without the knowledge measure. The results indicate that the new axiomatic definition of knowledge measures, which we propose has a positive impact on the effectiveness of inference and the rule induction method reducing the number of rules in a set maintains it.


Introduction and Description Problem
Fuzzy [1] and rough [2] sets provide tools for the analysis of significant imperfections of data and knowledge. The former allows classification of objects as belonging to a given degree to a set or relation. The latter provides approximations in cases where the information is incomplete. In this paper, we demonstrate how the mentioned theories can be merged into a hybrid system to improve the solution of the posture detection problem, which is a part of a fall detection system.
Considering that the population is aging rapidly, the demand for assistive technology for aging at home which can provide reliable, unobtrusive monitoring of human activity is expected to expand. One important aim of assistive technology is to provide prolonged independent living in a safe, home like environment without changing everyday lifestyle. Falls are a severe problem within the growing aging population. Many efforts have been undertaken to develop reliable methods of fall detection. The increasing number of studies in this area have allowed us to identify the major challenges and issues for fall detection technology, especially: performance, usability, and acceptance by the elderly. Fall detection systems need to be as accurate and reliable as possible both in terms of high sensitivity and specificity. In practice, this means that fall detectors must reliably distinguish between falls and activities of daily living (ADL) robustly, sustaining at low false alarm ratio. The method should not limit the placement of the sensors, or be sensitive to volatile environmental conditions. Such detection systems fall into two major categories, that is, wearable sensors and context-aware systems [3]. The main advantages of wearable sensors are size, usability, power consumption, and costs of use. The availability of cheap, embedded inertial sensors used in smartphones and smartwatches has contributed to the (i) New measures: • A new subsethood measure for fuzzy values is proposed and its validity is proved with the help of the example of use; • A new knowledge measure for FSs is introduced and its significance is proved with the help of the example of use; (ii) A new hybrid system is proposed and used in a real decision making problem, i.e., a fall detection system for the elderly, in particular in a posture detection system: • The proposed knowledge measure is applied to fuzzy inference problems; • A rule induction method is applied to reduce the number of rules in a set while maintaining the effectiveness of the inference process and significantly improve the performance of a approximate reasoning.
The paper is organized as follows. In Section 2 related works are presented. In Section 3 methodology and data descriptions are proposed. In addition, elements of the fuzzy and rough sets theory as well as new measures of precedence and knowledge based on precedence indicators with their applications to fuzzy inference are presented. Finally, the experimental results of simulations of a hybrid approach to the fall detection problems are described in Section 4.

Related Work
Recently, depth cameras have been used in fall detection [11,12]. Ref. [13] applied the skeletal model obtained from Kinect SDK to fall detection. Ref. [14] proposed employing 3D joint tracking information to estimate the walking speed and to extract features describing the movements of a person going down the stairs. However, a person can be in one of many poses before a fall, so the skeleton extraction model may fail, or be unreliable during fall motion [15,16]. In [16] a two-stage fall detection method is proposed. Temporal segmentation of the vertical state time series of a person tracked in 3D is used in the first stage to identify on-ground events. In the second stage the confidence that the event was preceded by a fall is calculated, using a set of decision trees and features extracted from ground-based events. The improvement of fall detection reliability by combining depth and inertial sensors was proposed in [17]. Recent work demonstrates that merging the depth with accelerometer signal improves human activity recognition [18]. A more detailed overview of recent fall detection methodology using depth sensors is provided in [19]. Other approaches are based, for example, on convolutional neural networks (CNNs). However, due to the limited amount of data, their performance is limited. In [20] the authors used transfer learning where pre-traning on the ImageNet dataset AlexNet architecture was applied to accelerometric data, achieving an accuracy of 96.4%. Additionally, the authors of [21] also used depth data, however extracted from videos and thus applied to 3D-CNN. The detection of falls base on videos relies on multiple frames and uses more complex models, thus it can be considerably slower. By using data augmentation, they increased the model accuracy from 69.6% to 92.4% [22]. In this work we perform detection and classification of body contour on depth images. This approach ensures the privacy of the monitored person and is very effective in terms of processing speed. Our method involves merging the techniques mentioned above, fuzzy sets theory and rough sets theory. Despite the popularity of machine learning approaches, issues may arise with the use of simulated human fall event data. Firstly, the small number of actors, may not be sufficient to represent the entire population in terms of variability in human properties (i.e., height) or human biomechanics [23]. Scarcity of data may be problematic (especially for deep learning) so approaches other than traditional supervised classification are being investigated [24]. Another solution to address the lack of data is a customization of the parameters of the decision system to a person's physical characteristics [25]. Our approach leverages the ease of customization and explainability of a fuzzy inference system by reducing the number of rules, allowing to build a linguistically understandable classifier maintaining high detection accuracy.

Methodology, Data, Theory and Tools Descriptions
For the purpose of to this article, we propose a new hybrid diagnostic system based on fuzzy and rough sets theory. To be specific, two rule sets are investigated, the first one created based on a domain knowledge and the second constructed by the rough set theory along with the main area of research which is concentrated on the concept of fuzzy information measure, and therefore the knowledge measure. These measures together with various aggregation methods are used to evaluate the accuracy of the classification of rule sets in the decision-making process.

Methodology and Data
The main goal of this research was to compare two approaches to posture recognition in fall detection: I. Knowledge Approach and II. Rough Set Approach. In the first approach a method based on a domain knowledge was used to generate a set of rules, the cardinality of which results from the combinatorial characteristic of this method. In turn, in the second approach induction method based on rough sets (described in Section 3.3) was used to reduce a set of rules. Next, both sets of rules were used in the fuzzy inference and evaluation process separately. Additionally, expert knowledge was used for modeling the selection of the parameters for the fuzzification function (described in Section 4). This combination of fuzzy and rough solutions is a novelty to the systems studied in the literature on fall detection problems. The concept of a hybrid approach (that we call a FuzzyRoughSystem, or FRSystem), presented in Figure 1, was based on three processes: Data Acquisition Process, Fuzzy Inference Process and Evaluation Process. In the Data Acquisition Process, Kinect v1 cameras and an inertial motion sensor were used. The inertial sensors: PS Move and x-IMU collected data at 60 Hz and 256 Hz rates, respectively. The cameras were placed in different locations (one the front of the room parallel to the floor and the second one on the ceiling, facing down), in each case, the camera could be static or mounted on an active head. To preserve the user's privacy, only the depth maps were analyzed. Depth maps were acquired using USB protocol, while accelerometric data were streamed wirelessly from the accelerometer using the Bluetooth protocol. For data acquisition, the OpenNI library was used, while the IMU sensor's software was prepared based on the source codes provided by the manufacturer.
As a result, 5990 depth maps were collected in the UR Fall Detection Dataset. These depth maps were acquired using two Microsoft Kinect cameras from two different viewpoints. Each of the 30 distinct falls had about 150 labelled frames. The depth maps were stored as PNG16 images with 640 × 480 resolution. The fall detection system, based on the images, was carried out in two stages: detection of a lying pose based on a single depth map and character movement analysis using dynamic transitions, however, in this work, we focused on the first stage of the system. Features describing the silhouette of a person at a given moment were determined as a result of the clustering of 600 images depicting characters in various poses, including during a fall and while performing ADL actions were analyzed. Ultimately, the following descriptors were selected from the set of features: • H/W-the ratio of the height of the person's bounding box to its width in the segmented point cloud.

•
H/H max -the ratio of the height of the person's surrounding box in the current frame to the physical height of the person. • max(σ x , σ z )-the maximum standard deviation of the values of points belonging to the character from its center of gravity along the axes of the Kinect camera coordinate system. • P 40 -the ratio of the number of points, lying no more than 40 cm above the floor, to the number of all points (belonging to the character point cloud).
Before we present and discuss the implementation of the new system (Section 4), we will recall some facts and introduce new elements in the fuzzy sets theory or rough sets theory.

Fuzzy Set Theory
Firstly, we recall the concept of a fuzzy set (relation) (cf. [26]). We consider fuzzy sets in a set P = ∅.
All fuzzy sets on P will be denoted per FS(P) and the membership function describing the degree of belonging of p ∈ P to R is µ R (P).

Basic Operations
In this chapter, we will focus on the elementary operations (fuzzy negations and implication functions built on [0, 1]) used in fuzzy reasoning, which is the basis of our novel system and which will also be recalled in Section 3.2.3.
Example 1 (cf. [28]). Examples of fuzzy negations N are: , the Sugeno family of fuzzy (strong) negations, where λ ∈ (−1, ∞) and for λ = 0 we get the classical fuzzy negation. Examples of fuzzy implications I are: . Now, we recall the basic and the most important operation on fuzzy sets, i.e., an aggregation function.

Knowledge Measure
We will focus on an important measure, that is, the measure of fuzzification, that is, the knowledge measure. We propose to use this measure in the process of fuzzy inference when drawing conclusions from premises (in aggregating premises). Before we move on to a new idea of measuring knowledge in the fuzzy set environment/theory, we need to present a certain tool useful for the operation of fuzzy values, that is, a measure of inclusion of fuzzy values called a precedence indicator.

Precedence Indicator
Research on fuzzy sets began with the concept of Zadeh (1965) Bandler and Kohout (1980) proposed a new measure subsethood grade/precedence indicator of a fuzzy set in another fuzzy set which is based on a considering the infimum of an appropriate aggregation of implication operators. This idea of Bandler and Kohout inspired many authors to study fuzzy subsethood measures as the type of function σ : FS(P) × FS(P) → [0, 1] with the different axiomatizations that have been proposed are not equal and they hinge on the examined applications. Based on this fact, and drawing inspiration from the works [32][33][34][35] in this paper we propose a new list of axiomatization for fuzzy precedence measure Prec : 1] as the class of implication operators which allows us to: 1.
Construct a new precedence indicator inspired by the axiomatic definition of the fuzzy subsethood measures; 2.
Construct new knowledge measures using a new precedence indicator; 3.
Apply new knowledge measures in fuzzy inference, as an illustrative example of the effectiveness of the proposed new measures.
Now we propose the constructive method of the precedence indicator based on an aggregation and negation functions. Proposition 1. Let N denote a fuzzy negation (i.e., an antytonic operation that fulfils N(0) = 1, N(1) = 0) and A is the aggregation A ≤ max. Then is the precedence indicator.
Here are some examples of the precedence indicators that satisfy Proposition 1.

Example 4.
For A = A mean and standard negation N we have 1.
or for Sugeno negation with λ = 1 we have 2.
We pay attention to the fact that precedence indicators create a subclass of fuzzy implication functions as we observe in the following example. Example 5. The following operations are implication function but not precedence indicators: for p, q ∈ [0, 1].

Knowledge Measure
In this part of the work, we consider the crucial concept of information in the setting of uncertainty, that is, the idea of the knowledge measure of a fuzzy set, and suggest a new construction process for it by use of a precedence indicator. Cognitively, the knowledge measure is dual to the entropy measure of the arbitrary fuzzy set which gives the average values/height of fuzziness/ambiguity existing in the fuzzy set. Similarly, we can wonder about the average amount of knowledge present in the fuzzy set. Thus, a knowledge measure of a fuzzy set needs to satisfy the following axiomatic postulates. We propose some generalisation (in the fourth axiom) of the axiomatic definition of knowledge measure presented in [7,8].
Definition 7. For R ∈ FS(P) a knowledge measure would satisfy the following properties: K1 K(R) has maximum value iff R is a crisp set, i.e., R(p i ) = 0 or 1 for all p i ∈ P, K2 K(R) has minimum value iff R is the most fuzzy set, i.e., R(p i ) = 0.5 for all p i ∈ P, K3 K(R * ) ≥ K(R), where R * is a crisped version (sharpened) of R, K4 K(R) = K(R N ), where R N is the duality (complement) of set R for strong fuzzy negation N, i.e., R N (p) = N(R(p)), p ∈ P (for classic negation N we obtain a complement relation of R).
We suggest the following construction method of the knowledge measure.

Proposition 2. Let
Prec be a precedence indicator that satisfies Proposition 1, where aggregation A is symmetric and N is the strong negation with an equilibrium point 0.5 (i.e., N(0.5) = 0.5) for R ∈ FS(P), card(P) = n, n ∈ N, then is a knowledge measure.
(K1) is obvious with the assumption about R, Prec, and their properties. Because for a crisp relation of R we have: 1.
(K3) If R * is crisper than R, that is, Based on Proposition 1 and for In a similar way we consider the case R(p i ) < 0.5. (K4) Based on Proposition 1 we observe for the symmetric aggregation A: as a consequence we have K(R N ) = K(R), which completes the proof.

Example 6.
If in Proposition 2 we used precedence indicators satisfying Proposition 1 with A ∈ {A mean , A gmean , A 2mean , min, max} and N is standard (classical) negation, then we obtain knowledge measure K(R) for R ∈ FS(P).

Knowledge Measure and Fuzzy Inference (Mamdani)
The known and popular area of fuzzy logic and its extensions application is approximate reasoning, where from uncertainty/imprecise inputs/fuzzy premises or rules we often obtain uncertainty/imprecise inferences. Approximate reasoning has been used in many fields, for example, medical diagnosis, expert systems and control systems.
The main goal of this part of the paper is to explore the more general algorithm of approximate reasoning by using the general modus ponens property with the arbitrary aggregation functions next to the new knowledge measure. In the beginning, an algorithm for multi conditional approximate reasoning based on the new aggregation-based composition rules is proposed. The use of knowledge measure in fuzzy reasoning is a new accent in the classical model of inference. Thus we obtain a modification of the standard fuzzy reasoning method.
Approximate reasoning is the procedure where a possible uncertainty/imprecise conclusion is implied from a collection of uncertainty/imprecise premises. The classical modus ponens schema, was extended by Zadeh [36] to fuzzy reasoning in the following way and we obtained the GMP, that is, Generalized Modus Ponens:

Proposition: If p is D then q is E Fact: p is D' -----------------------------------------q is E',
where E is the fuzzy set in the universe Q. The main plus of the GMP is that we can obtain new information even if D and D are different. Usually, in the GMP the fuzzy rule is represented using a fuzzy relation R on the referential set P × Q. Existing different methods to build R can be used [37]. The most promising: R(p, q) = I(D(p), E(q)), where I is an implication function. We may build the implication function from the aggregation function: I(p, q) = A(1 − p, q) with A(1, 0) = A(0, 1) = 1. Thus we can also create the relation R using the aggregation function by specific assumptions.
The fuzzy inference process is as follows where A, B are aggregation functions on [0, 1]. The basic inference process has the form presented in Figure 1.
Our novelty in the fuzzy inference in the process of aggregating premises is the proposal to use the combination of aggregation and knowledge measure as the following new operator: where R is a fuzzy set on P, where cardP = n. Thus premises data in the given rule and K knowledge measure created by Proposition 2 and A, B are aggregation functions.

Rough Set Theory
The rough set theory use the indiscernibility relation to discover information about objects in an information system.

Definition 8 ([38]
). An information system (IS) is an ordered quadruple (U,AT,V,f) where U is a finite nonempty set of objects, AT is a finite nonempty set of attributes, V = a∈AT V a ; is a nonempty finite set of values of attributes, where V a is the domain of attribute a, and f : U × AT → V is an information function such that f (x, a) ∈ V a for all x ∈ U and a ∈ AT.
A decision table is a type of information system. In the decision table the set AT = A ∪ D; A is a set of attributes, and D is set of decisions, D ∩ A = ∅. Whereas, a concept is the set of all cases with the same decision value [39].

Definition 9 ([2]). For each subset of attributes A ⊆ AT a binary indiscernibility relation IND(A)
on U can be determined as follows: Let a ∈ A, v ∈ V, and p = (a, v) be an attribute-value pair. The set [p] of all cases from U for which attribute a has value v is called a block of attribute-value pairs [40]. The rule induction Algorithm 1 LEM2 [39], in order to find a local covering of an input set, explores the space of attribute-value pairs. Let X be a subset of U and P be a nonempty collection of nonempty sets of attributevalue pairs. The set P is a minimal complex of X if and only if X depends on P and no proper subset P of P exists such that X depends on P [39]. ρ is a local covering of X if and only if the following conditions are satisfied: each member P of ρ is a minimal complex of X, 2.

Algorithm 1 LEM2
Input: a set X Output: a single local covering ρ of set X X := G;

5.
Pose: Based on the collected data, two rule sets were generated independently. The first one, a result of the Rough Set Approach, contained 44 rules: 10 rules for the pose notLy, 34 rules for the pose mayLy and 10 rules for the pose isLy. The second one, a result of the Knowledge Approach (FRSystem, Figure 1), contained 81 rules ((3 cases( f unctions)) 4 f eatures , [46]): 13 for the pose notLy, 52 rules for the pose mayLy and 16 rules for the pose isLy.
Next, in the Fuzzy Inference Process, a modified version of the basic Mamdani model was applied to obtain a posture decision (lying or not). Namely, in fuzzy inference, in the process of aggregating premises, a combination of aggregation and knowledge measure was used (new aspect by applying the operator O R , see Section 3.2.3) constructed using a new precedence indicator. The effectiveness of the new measure was compared with the classic model without using the knowledge measure (the Sections 3 and 4 in the FRSystem (Figure 2)) and also the effectiveness of applying different aggregations in the fuzzy inference process was analyzed. To demonstrate the effectiveness of the proposed hybrid approach the following characteristics were used: where TP is the number of correct isLy classifications, TN is the number of correct notLy classifications, FP is the number of notLy classifications as isLy and FN the number of isLy classifications as notLy • specificity • precision • sensitivity REC = TP TP + FN (16) in the Evaluation Process. Note that accuracy means how close a measurement is to the actual or expected value. The precision says how close the sets of measurements are to each other. The recall is characterized as the percentage of relevant results that are correctly classified by the used model, and specificity means the percentage of true negative results. Finally, the rules used in inference (I. Rough Set Approach and II. Knowledge Approach) were assessed based on: the number of correct classifications of the rule, the effectiveness of the rule in the set and the effectiveness of the rule within the decision class.
We assumed that the effectiveness of the rule in the set can be expressed as follows: the In turn the effectiveness of the rule within the decision class can be determined as follows: Based on the above-mentioned measures, a rule ranking was created. First, the strongest rules from the set classification point of view were identified. Then, among the strongest rules, the ones which turned out to be the most effective within the decision class were selected. In this way, the rules that were critical to pose detection were indicated. The rules that were critical to pose detection were indicated. Finally, we use the center of gravity method for the defuzzification process.
To measure the effectiveness of our approach, the above-mentioned characteristics: accuracy (ACC), specificity (SPE), precision (PRE), and recall (REC) (sensitivity) were used. We studies the following cases: • Determination of the effectiveness of classic fuzzy inference (without the knowledge measure and without the rule reduction) in fall detection problems, Table 1; • Assessment of the impact of different aggregation functions and different knowledge measures, i.e., precedence indicators, on the effectiveness of classification of the reduced and nonreduced rules, using the FRSystem, Table 2; • Verification of the effectiveness of the different knowledge measure construction methods in the FRSystem, proposed by us and others known from the literature, Table 3. • Estimation of the effectiveness of each rule in the whole set and within the decision class. Tables 1-3 show the experimental results obtained during the given dataset analysis. Presented outcomes in Table 1 maintain a high level of classification comparable to [6]. However, the next studies show that we observe progress in our classification results if we use the FRSystem (as can be seen in the result in Tables 1 and 2) where the results are grouped for the original set of rules and after their selection by the rough method. Moreover, we compare the effectiveness of different aggregation functions used in the fuzzy inference, in the process of aggregating premises. We present the best results obtained for knowledge measures that satisfy Proposition 2 and are used in FRSystem. In particular, in K 1 we use in operator O R aggregation functions A 2mean and B = F (A 2mean used in the precedence indicator used in the Knowledge measure K) (which we denote as K 1 (A 2mean , B F , A 2mean ). Similarly, we created K 2 (A 2mean , B F , max), K 3 (min, B F , A 2mean ), K 4 (A mean , B F , A mean ), K 5 (A mean , B min , max). In the presented results, we assume the results of each class we aggregate by the maximum. The best results we obtained are marked in bold. As can be seen, the best performance is obtained for K 2 used in the FRSystem, with the following measures: ACC (96.9%), PRE (96.2%), SPE (87.8%) and REC (99.9%). What is more, we may say that the application of a reduced set of rules retained the classification level, that is, we obtained results with an acceptable difference of error, in a limit of the error at the level of about 0.01 (see Table 2). Thus, paths I and II in the FRSystem are comparable in the effectiveness aspect, but reducing the number of rules also has another important and positive effect on our model because we do not have to take into account all the attribute-value relationships. Only the most important relationships are selected in the induction process. A smaller and at the same time, optimal set of rules is easier for experts to evaluate.
Moreover, in Table 3 we compare our best results (we denote by K the knowledge measure built-in to the proposed method and used in the FRSystem) with other methods to build knowledge measures known in the literature (unlike our approach, the dependence (precedence indicator) of a given fuzzy value on the extreme (certain value) is not taken into account), such as: [7]. There, the fuzzy and dual values are taken into account while in our approach the given fuzzy value is compared by subsethood measure with the extremes (the largest and the smallest certain value), which gives a more complete picture of the uncertainty contained in the measurements. We observe the higher effectiveness of the proposed new knowledge measure (see Table 3). For comparison we take K from case K 2 from the result presented in Table 2: where for p, q ∈ [0, 1] we have In order to identify the most relevant attribute values (from a classification view point) for each decision class the rules were assessed first on the whole set, and then on the concepts. As a result, the values of the attributes clearly defining the detection of a lying or non-lying position are indicated and presented in Table 4. It should be noted that, the H/W attribute did not occur in the reduced set of rules, among the conditions of the most efficient rules for the notLy decision class. The absence of this attribute did not affect the quality of classification within this class in relation to the non-reduced set of rules. The remaining conjunctions of conditions for the most effective reduced and non-reduced rules were identical.

Conclusions
In this paper, we have provided the initial results of a very interesting new approach to the selection of appropriate aggregation functions and a set of rules for fuzzy inference in the problem of fall detection, especially posture detection. Moreover, the main research was concentrated on investigating the concept of a fuzzy information measure, presenting a new axiomatic definition for the knowledge measure, and using theirs in the proposed hybrid system. The results obtained for the mentioned aspects indicate the positive results of the new approach. Out of 81 rules (see [46]), by applying the LEM2 algorithm we indicate 44 rules (see [47]) which allow us to significantly reduce the dimensionality of the studied problem and facilitate its analysis while maintaining a high level of classification comparable to [6].
Our goal for future work is to develop this research on both theoretical and practical grounds. For example, we would like, in cooperation with an Elderly care home in Rzeszow, to expand the data set and develop some new methods to represent data, for example, a hybrid method that uses fuzzy and rough sets concerning uncertainty, so we will use interval-valued fuzzy set theory. In addition, the developed hybrid inference method seems to be very promising for use with different input data sets in the future. In particular, new measures of information may prove useful for the issues or methodologies observed in the works [7,8], where the proposed knowledge measure is utilized to calculate the weights vector, when weights are partially known and other when weights are completely unknown in economic terms, in multiple attribute decision-making methods or in image thresholding based on a fuzzy accuracy measure.

Data Availability Statement:
The dataset used in this research work is on the website http://fenix. univ.rzeszow.pl/~mkepski/.

Conflicts of Interest:
The authors declare no conflict of interest.