Adapted Binary Particle Swarm Optimization for Efficient Features Selection in the Case of Imbalanced Sensor Data

Abstract

Daily living activities (DLAs) classification using data collected from wearable monitoring sensors is very challenging due to the imbalance characteristics of the monitored data. A major research challenge is to determine the best combination of features that returns the best accuracy results using minimal computational resources, when the data is heterogeneous and not fitted for classical algorithms that are designed for balanced low-dimensional datasets. This research article: (1) presents a modification of the classical version of the binary particle swarm optimization (BPSO) algorithm that introduces a particular type of particles called sensor particles, (2) describes the adaptation of this algorithm for data generated by sensors that monitor DLAs to determine the best positions and features of the monitoring sensors that lead to the best classification results, and (3) evaluates and validates the proposed approach using a machine learning methodology that integrates the modified version of the algorithm. The methodology is tested and validated on the Daily Life Activities (DaLiAc) dataset.

1. Introduction

The development and large scale deployment of IoT monitoring devices has made significant amounts of data available, which may be used for improving the care of elders. In this context, the identification of daily living activities out of monitoring data is very important for understanding the elders behaviour and assessing deviations from daily routines, which may signal illness progression and timely planning of intervention processes. For example, in the project ReMIND [1] the enhancement of the quality of life of the elders with mild neurocognitive impairments is considered, and one major challenge addressed by the ReMIND project is the stimulating of the physical activity through music using James robots.

Therefore, the challenge of the daily living activities (DLAs) classification in real-time using low computational resources is critical as those elders might suffer unexpected falls caused by their critical health condition or sudden switches from one DLA to another DLA caused by memory problems. One way to increase the classification speed is to reduce the number of features of the data samples; however, that challenge is even more complex when the data is imbalanced and heterogeneous for each monitored person. In [2] the reduction of the number of features was addressed by proposing a novel bio-inspired algorithm. However, in that approach, the experimental DLAs datasets were balanced and normalized prior to the application of the feature selection.

The challenges of the aging population [3] have been addressed by various types of information and communication technology (ICT) systems in the last years, but few of them evaluated the challenges related to the acceptance and to the usability of the ambient assisted living (AAL) systems [4]. Those challenges are even more complex in particular cases, such as the ones where the elders have symptoms of dementia [5] or symptoms of Parkinson’s disease [6].

The recent developments for the internet of things (IoT) [7] and wearable computing led to the development of various types of health monitoring systems, which use data collected from various sources [8]. Moreover, these systems enable the continuous monitoring of elders; however, there are still challenges that must be considered, such as the need for high processing power and the low latency. This challenge is even more complex in the case of imbalanced data with many features and samples.

The classification of the DLAs using data that is generated by wearable devices has various applications in healthcare, such as falls detection [9], falls prediction [10], the classification of the activities of the elders that live alone [11], the classification of the DLAs of the elders that live in free-living conditions [12], the detection of anomalies in DLAs [13] and behaviour recognition for robust healthcare [14].

In the literature there are many datasets generated by sensors [15], which are used in the prediction of DLAs; however, the identification of the most appropriate techniques that return the best results, in various contexts, is still challenging due to the many dimensions that must be considered, such as the selection of the features, the balancing of the samples and the selection of the classifiers applied in the classification of the DLAs.

Real time data can be collected using various types of monitoring sensors, such as gyroscope sensors, orientation sensors, accelerometer sensors, and pressure sensors. The data generated by monitoring sensors are very useful in research, especially in the health research domain, as a thorough and complex analysis of this data can reveal a great deal of useful information, such as patterns in the daily living behaviour of the persons that are monitored, which might indicate particular health conditions, adherence to a particular type of routine, and anomalies in the normal daily behaviour.

The representative challenges related to the analysis of the data generated by the sensors, which monitor the DLAs are: (1) the determination of the best positions of the monitoring sensors that return the best classification results, (2) the determination of the features of the data generated by each monitoring sensor, which have the most significant influence on the final classification results, and (3) the determination of the combinations of features that correspond to each monitoring sensor, which return the best classification results for each monitored activity. Compared to the classical algorithms from the literature for feature selection that do not consider the particularities of the DLAs data, the approach presented in this article considers the fact that groups of features belong to different monitoring sensors, which are placed on different parts of the bodies of the monitored subjects.

The main contributions of this research article are:

(1)

The development of a modified version of the standard binary particle swarm optimization (BPSO) algorithm, which introduces sensors particles characterized by weights proportional to the importance of the monitoring sensors and used further in the equations for the updating of the velocities of the standard particles;

(2)

The adaptation of the proposed algorithm for DLAs data reflecting these adaptations in the way in which the objective function is defined and in the ranking of the features returned by the proposed algorithm;

(3)

The evaluation and the validation of the adapted version of the BPSO algorithm using a machine learning methodology developed in-house, which compares the proposed algorithm with other feature selection approaches, and which uses as experimental support Daily Life Activities (DaLiAc) dataset [16].

The primary reasons to adapt BPSO for DLAs data are: (1) the flexibility to modify the classical BPSO, either by altering the mathematical equations or by introducing novel concepts in the original version, (2) the fact that the standard classical algorithms, in particular the standard versions of BPSO, do not exploit the nature of the DLAs datasets at maximum, and (3) the fact that in the case of the DLAs the data is generated by several monitoring sensors described by groups of features—information that might lead to better results if it is integrated in the feature selection algorithms used in the processing of this type of data.

The remaining sections of the article are organized as follows: Section 2 illustrates the research background, Section 3 presents the materials and methods used in this article, Section 4 describes the most representative results obtained in the experiments, Section 5 presents a discussion of those results, and Section 6 concludes the article.

2. Background

The first subsection addresses feature selection challenges related to imbalanced data with many features generated by monitoring sensors. The second subsection presents literature approaches that consider feature selection based on the particle swarm optimization (PSO) algorithm [17]. The third subsection considers various approaches from the literature that use the DaLiAc dataset for experimental support.

2.1. Feature Selection Challenges for Imbalanced Data Generated by Monitoring Sensors

In [18] the high dimensionality and the class imbalance properties of the datasets are tackled. Compared to the approach presented in this article, the authors of that article are motivated by medical applications from real-world systems. Their feature selection method aims to achieve a better separability between the minority class and the majority class. In the approach presented in this article the majority or the minority properties of the classes of the samples are not considered when the features are selected. However, the approach presented in this article considers the variability of the monitored data for each class for each feature, and it also considers the relations between features.

The authors of [19] address one of the major challenges of the feature selection in the case of datasets with many features, namely the ignorance of the class imbalance problem and thus in the majority of the cases, the selected features are biased towards majority classes. In the approach presented in this article the imbalance is addressed by the way in which the objective function of the adapted version of the BPSO algorithm is defined, as all the classes are treated equally.

In [20], the authors deal with the following two major challenges in machine learning, namely the high dimensionality and the class-imbalance. The cardinality of the set of features is penalized by a scaling factors technique, while the class-imbalance problem is addressed using a method based on a support vector machine (SVM) [21]. In the approach presented in this article, each class is treated equally when the objective function is defined, and moreover, the number of features is penalized in the objective function, increasing its value when the number of features is small.

The selection of features for imbalanced datasets is challenging as an effective learning model should be constructed, reducing the memory consumption and the time consumption—two objectives, which are addressed in the approach presented in this article. In [22] that challenge is addressed by rough-set-based feature selection algorithm for imbalanced data (RSFSAID), a novel algorithm for feature selection. Similar to the approach from this article, those authors use PSO. However, in their approach, PSO is applied in the determination of the optimized parameters for the proposed algorithm. They also study the lower and the upper boundary regions, when they define the significance of the features. In the approach from this article the minimum and the maximum values of the ranges of variability are considered in the definition of the matrix of variability.

The authors of [23] address the challenge of feature selection in the case of high dimensional imbalanced datasets from the perspective of the neglecting of the features interaction. Many traditional approaches applied in the case of datasets with imbalanced classes are biased towards the majority classes and the authors of that article propose a method for feature selection based on the interaction information (II). In this article the interactions between the features are considered, defining a matrix that approaches the features from the perspective of the range of variability. Moreover, this article considers particular characteristics of the DLAs datasets that are not approached in that article, and proposes a more generic algorithm for feature selection.

2.2. Feature Selection Approaches Based on Particle Swarm Optimization

The authors of [24] apply PSO in feature selection, considering three goals, which are also targeted by the approach presented in this research article: (1) the maximization of the classification performance, (2) the minimization of the number of the selected features and (3) the minimization of the computational time. However, they propose, as future research work, the development of an approach based on PSO, which is multi-objective and which simultaneously minimizes the number of the selected features and maximizes the performance of the classification. In the approach presented in this article, both of those two objectives are considered in the definition of the objective function.

In [25], a method is presented in which the features are selected using a hybrid version of PSO, which also integrates a local search strategy. That method is called hybrid particle swarm optimization with local search (HPSO-LS) and determines efficiently the discriminative features with reduced correlations. The objective function of that approach uses the k-nearest neighbor (k-NN) [26] method, and, between two solutions that return the same accuracy, it considers that solution with the smaller number of features. However, that approach is expensive in terms of computational resources, as it requires the training of a k-NN classifier for each particle in each iteration of the algorithm.

The authors of [27] considered an algorithm for feature selection based on PSO with learning memory (PSO-LM). Compared to the approach from this article, that approach introduces a memory learning strategy with the objective to balance the global exploration and the local exploitation in the algorithm. Moreover, their objective function is similar to the objective function of the adapted version of the BPSO algorithm described in this article, as it uses two parts as follows: The first part corresponds to the accuracy of the prediction model trained using the selected features and the second part corresponds to the ratio between the number of the selected features and the total number of features. However, the approach presented in this article is different, as, instead of using a classifier for the first part of the objective function as in the case of that approach, which considers the k-NN classifier, this article introduces an algorithm, which considers the variability of the monitored data, and which also considers the fact that the data is imbalanced.

In [28], an approach is considered for the joint moment prediction, in which the features are selected using a method based on BPSO and the objective function is based on the variance accounted for (VAF). Similar to the approach from this article, their approach is tested and validated on DLAs data. However, that approach does not consider the minimization of the features as a distinctive part of the objective function and the data considered in their experiments for the validation of the approach contains only gait patterns of running, therefore it can not be generalized very easily.

Another approach for feature selection based on PSO is presented in [29]. The method presented in that article is called hybrid particle swarm optimization with a spiral-shaped mechanism (HPSO-SSM) and is based on the following three improvements: (1) the enhancement of the diversity in the searching process using a logistic map sequence, (2) the introduction of two new parameters in the original formulas, used in the updating of the positions, and (3) the adopting of a spiral-shaped mechanism as a local search operator. Their method presents broad applications, as it is validated using twenty classic benchmark classification datasets from the University of California, Irvine (UCI) machine learning repository. However, compared to the method presented in this article, it is not very focused, as it does not exploit the nature of those datasets at maximum.

2.3. Machine Learning Approaches from the Literature That Consider the Daily Life Activities (DaLiAc) Dataset

The Daily Life Activities (DaLiAc) dataset was introduced in [16] as a benchmark dataset that is analyzed in the context of a hierarchical classification system for DLAs. Compared to the method presented in this article, which evaluates the performance of the DLAs classification using 10-fold cross validation, the method presented in that article considers a leave-one-subject-out procedure. The authors also address the challenge of a high number of features, which leads to a very high computational complexity in the case of embedded systems or real time applications and they even suggest the reduction of the features, giving, as an illustrative example, the sequential feature selection.

The authors of [30] use the DaLiAc dataset as experimental support for a method based on a classifier ensemble of randomized trees, and their objective is to maximize the recognition performance, both in the replacement scenario and in the relocation scenario. Compared to the approach presented in this article, they used more K-folds cross-validation schemes with K from $\{2,3,5,10,20\}$ and they also used the accuracy to measure the performance. They also included junk data in their experiments and therefore their results are not directly comparable with the results obtained in this article.

The human activity recognition is approached in [31] by combining a number of classifiers, and the method presented in that article, which is based on a combination of models outperforms other types of classifiers combination models. However, the challenge that still remains is how to increase the robustness of that method against interperson variability. The method based on BPSO described in this article addresses that challenge partially by assigning a weight to each monitoring sensor and consequently the weights are different and adapted for each monitored subject.

The algorithms applied in [32] for improving the accuracy of activity recognition are k-NN, logistic regression (LR), naive bayes (NB), random forest (RF), extremely randomized trees (ERT) and SVM. The method described in that article consists of four phases, namely segmentation, feature extraction, feature selection and evaluation. In the case of the DaLiAc dataset, the overall mean classification is $89.6\%$. The work described in this article is inspired from the work from that article as follows: (1) our work considers the selection of the features according to the positions of the sensors that generate them, (2) it considers the RF algorithm, and (3) it applies the proposed research method on several datasets. However, one weakness of that work is that it does not address the importance of each monitoring sensor, even though it addresses the optimal number of sensors, the positions of the sensors, and the optimal number of features for each sensor.

The method presented in [33] considers the DaLiAc dataset from a different perspective than the other articles, which consider it as experimental support, converting the inertial sensor signals to images and then applying a convolutional neural network (CNN) [34] for images classification. Even if that method leads to better classification results than the traditional classification methods, it is also subject to the limitations of CNNs, such as the class imbalance, the overfitting and poor performance when the input data is small. Moreover, CNNs are also computationally expensive and, in this article, that challenge is addressed by trying to reduce the number of features so that the classifiers can be trained faster and the classification results are optimized.

3. Materials and Methods

This section presents the main concepts and the pseudo-code of the adapted version of BPSO.

3.1. Mathematical Formulation of the Optimization Problem Approached Using an Adapted Variant of the BPSO Algorithm

The objective of the optimization problem is to select an optimal number of features in the case of data generated by sensors that monitor various types of DLAs in order to: (1) minimize the number of selected features and to (2) maximize the value of a mathematical formula based on a heuristic introduced by us, which describes the range of variability of the DLAs using the selected features. Those two sub-objectives are reflected in the objective function of the adapted version of BPSO algorithm.

The optimization variables are represented by arrays of zeros and ones of the form $X=(x_1,\dots ,x_{24})$, such that $x_i\in \{0,1\}$ for any $i\in \{1,\dots ,24\}$. A value equal to 0 means that a feature is not selected and a value equal to 1 means that a feature is selected. The search space is thus represented by $2^{24}=\mathrm{16,777,216}$ possible variables.

The optimization constraint is the maximum value of the selected features, which is equal to six as six is also the number of features generated by each monitoring sensor in the data used as experimental support. The search space is thus limited to fewer variables, more specifically ${\sum }_{i=0}^6\left(\genfrac{}{}{0pt}{}{24}{i}\right)=\mathrm{190,051}$. That constraint is considered in this article using the ranking of the features according to an heuristic developed in-house and is applied to the best results obtained using the adapted version of BPSO, when they are compared to the results obtained using other methods in Section 4.

3.2. Matrix of Variability of DLAs Sensors Data for Each Monitored Subject

The matrix of variability M has the form:

$$M=\left[\begin{array}{ccc}{a}_{1,1}& \dots & a_{1,N_{DLAs}}\\ \dots & \dots & \dots \\ a_{D,1}& \dots & a_{D,N_{DLAs}}\end{array}\right]$$

(1)

where D is the number of features of the data generated by the monitoring sensors, which is also the dimension of the search space in the adapted version of BPSO, and $N_{DLAs}$ is the number of daily living activities. Each line i of the matrix M, where $i=1,\dots ,D$, is a permutation of the set $\{1,2,\dots ,N_{DLAs}\}$, such that the order of the elements of that set is given by the increasing order of the ranges of variability of each monitored DLA. The range of variability for a DLA a and the corresponding feature f is given by the formula:

$$r\_va{r}_{a,f}=v\_ma{x}_{a,f}-v\_mi{n}_{a,f}$$

(2)

where $r\_va{r}_{a,f}$ is the range of variability for the DLA a and the feature f, $v\_ma{x}_{a,f}$ is the maximum value monitored by the feature f for the DLA a and $v\_mi{n}_{a,f}$ is the minimum value monitored by the feature f for the DLA a.

3.3. Metric for the Evaluation of the Matrix of Variability of DLAs Sensors Data for Each Monitored Subject

The value of the metric $M_{\Delta }$ used in the evaluation of the matrix of variability M of DLAs sensors data is given by:

$$\begin{array}{c}{M}_{\Delta }=\sum _{i=1}^{N_{DLAs}}(|max\left\{j\right|j\in \mathbb{N},1\le j\le N_{DLAs}\wedge \exists k\in \mathbb{N},1\le k\le D,M_{k,j}=i\}\hfill \\ \hfill -min\left\{j\right|j\in \mathbb{N},1\le j\le N_{DLAs}\wedge \exists k\in \mathbb{N},1\le k\le D,M_{k,j}=i\}+1|).\end{array}$$

(3)

3.4. Heuristic for Features Ranking in the Optimal Solution Returned by the Adapted BPSO Algorithm for Each Monitored Subject

In order to restrict the number of selected features to a maximum threshold, a matrix R of variability is proposed. The matrix R is defined by the set of all $r\_va{r}_{a,f}$ such that a takes values from the set $\{1,\dots ,N_{DLAs}\}$ and f takes values from the set $\{1,\dots ,D\}$:

$$R=\left[\begin{array}{ccc}r\_va{r}_{1,1}& \dots & r\_va{r}_{1,D}\\ \dots & \dots & \dots \\ r\_va{r}_{N_{DLAs},1}& \dots & r\_va{r}_{N_{DLAs},D}.\end{array}\right]$$

(4)

The feature $f_1$ is considered better than the feature $f_2$ if the value returned by the $Comparator(f_1,f_2)$ described next is greater than or equal to 0:

$$Comparator(f_1,f_2)=\sum _{i=1}^{N_{DLAs}}s(r\_va{r}_{i,f_1}-r\_va{r}_{i,f_2}),$$

(5)

such that for any real number x the value of $s\left(x\right)$ is 1 if $x\ge 0$ or $-1$ otherwise.

3.5. Mathematical Description of the Objective Function of the Adapted Version of the BPSO Algorithm

The inputs of Algorithm 1 are: p—The particle for which the fitness value is computed, D—The number of dimensions of the search space, M—A precomputed matrix, which is presented in more details next, $M_{\Delta }$—A metric used for the evaluation of that matrix, and $DLAs$—The set of daily living activities monitored by the monitoring sensors. The output of the algorithm is r—The fitness value of the particle p.

Algorithm 1: The objective function of the adapted BPSO algorithm.

Initially, the value of r is 0 (line 1), the value of x is equal to the position of the particle p (line 2) and $F_{selected}$ is an empty set (line 3). For each d from the set $\{1,\dots ,D\}$, if the value of $x_d$ is equal to 1 then the feature $f_d$ is included in the set of selected features $F_{selected}$ (lines 4–8). The initial value of $\Delta$ is 0 (line 9). For each activity a from the set of daily living activities $DLAs$ the steps from (lines 11–23) are repeated. The initial values of $mi{n}_a$ and $ma{x}_a$ are equal to $+\infty$ (line 11) and $-\infty$ (line 12). For each feature f from the set of features $F_{selected}$, the values of $mi{n}_a$ and $ma{x}_a$ are updated in (lines 14–19) as follows: If the value of $M_{f,a}$ is less than the value of $mi{n}_a$ (line 14) then $mi{n}_a$ is initialized to the value of $M_{f,a}$ (line 15) and if the value of $M_{f,a}$ is greater than the value of $ma{x}_a$ (line 17) then $ma{x}_a$ is initialized to the value of $M_{f,a}$ (line 18). If $F_{selected}\ne \varnothing$ (line 21) then the value of $\Delta$ is incremented with $\left|ma{x}_a-mi{n}_a+1\right|$ (line 22). In (line 25) the value of r is updated using the formula:

$$r=\frac{1}{2}\times \left(\frac{\Delta }{M_{\Delta }}+\frac{\left|D\right|-\left|F_{selected}\right|}{\left|D\right|}\right).$$

(6)

The ideal value of r is 1 and corresponds to the case when $\Delta =M_{\Delta }$ and the number of selected features is 0, and the worst value of r is 0 and corresponds to the case when $\Delta =0$ and all features are selected. Consequently, r takes values from the interval $[0,1]$ and the objective of the algorithm is to maximize that value. The algorithm returns the final value of r in (line 26).

3.6. Adapted BPSO Algorithm for Feature Selection in the Case of DLAs Sensors Data

The BPSO algorithm is adapted after the binary version of the PSO algorithm [35]. A method similar to the one presented in [36] is considered in order to the restrict the values of the positions of the particles to take values from the set $\{0,1\}$.

The inputs of the adapted BPSO Algorithm 2 are: $N_{particles}$—The number of particles, $N_{iterations}$—The maximum number of iterations, $c_1$—The acceleration coefficient for the cognitive component, $c_2$—The acceleration coefficient for the social component, $w_{min}$— The minimum value for the inertia, $w_{max}$—The maximum value for the inertia, $V_{min}$—The minimum possible value for the velocity, $V_{max}$—The maximum possible value for the velocity, D—The number of dimensions of the search space, and $N_{sensors}$— The number of monitoring sensors. The output $g_{best}$ describes the position of the best particle after the maximum number of iterations $N_{iterations}$.

The adapted BPSO algorithm starts with the initialization of the particles that correspond to each monitoring sensor (line 1) and then for each particle that corresponds to a monitoring sensor, the corresponding fitness value is calculated (lines 2–4). Next, the weights of each sensor particle are computed (lines 5–7) in such a way that the weights are proportional to the fitness values of the sensor particles.

The particles swarm is initialized randomly (line 8) and the value of the initial iteration i is set to 0 in (line 9). The algorithm is then executed for a maximum number of iterations $N_{iterations}$ given as input.

In each iteration for each particle from the swarm (lines 11–15), the fitness value is computed in (line 12) using the $ObjectiveFunction$ and the functions $UpdatedLocalBest$ and $UpdateGlobalBest$ update the values of $p_{best}$ and $g_{best}$ as follows:

$$p_{best}=\left\{\begin{array}{cc}p\hfill & \mathrm{if} Fitness\left(p\right)>Fitness\left(p_{best}\right)\hfill \\ p_{best}\hfill & \mathrm{otherwise}\hfill \end{array}\right.$$

(7)

$$g_{best}=\left\{\begin{array}{cc}{p}_{best}\hfill & \mathrm{if} Fitness\left(p_{best}\right)>Fitness\left(g_{best}\right)\hfill \\ g_{best}\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(8)

The inertia is updated (line 16) considering the values of i, $w_{min}$ and $w_{max}$ as follows:

$$w=\left\{\begin{array}{cc}{w}_{max}\hfill & \mathrm{if} i=0\hfill \\ max(w-i,w_{min})\hfill & \mathrm{if} i>0.\hfill \end{array}\right.$$

(9)

Algorithm 2: Adapted BPSO Algorithm.

Next, for each particle from the swarm (lines 17–30) and for each dimension of the particle (lines 18–29) the new value of the velocity of the particle for dimension d is computed, taking into consideration the following components:

• the inertia component: $w\times v_d^{old}$,
• the cognitive component: $c_1\times r_1\times (p_{best,d}-x_d^{old})$,
• the social component: $c_2\times r_2\times (g_{best,d}-x_d^{old})$,
• the sensors component: ${\sum }_{k=1}^{N_{sensors}}{t}_k\times r_{k+2}\times (s_{k,d}-x_d^{old})$.

If the computed value is not in the interval $(V_{min},V_{max})$, then that value is updated either to $V_{min}$ or to $V_{max}$ (lines 20–22) as follows:

$$v_d^{new}=\left\{\begin{array}{cc}{V}_{min}\hfill & \mathrm{if} v_d^{new}<V_{min}\hfill \\ V_{max}\hfill & \mathrm{if} v_d^{new}>V_{max}.\hfill \end{array}\right.$$

(10)

The velocity is converted into a probability value (line 23) using the exponential:

$$S\left(v_d^{new}\right)=\frac{1}{1+e^{-v_d^{new}}}.$$

(11)

In (lines 24–28) the value of a random value $r_0$ from $[0,1]$ is applied in order to update the position of the particle as follows:

$$x_d^{new}=\left\{\begin{array}{cc}1\hfill & \mathrm{if} r_0<S\left(v_d^{new}\right)\hfill \\ 0\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(12)

In (line 31) the current iteration i is incremented with 1 and finally in (line 33) the algorithm returns the value of $g_{best}$.

4. Results

The adapted version of the BPSO algorithm was written and evaluated in Java in IntelliJ IDEA. The feature selection experiments for the adapted BPSO algorithm were performed in IntelliJ IDEA. The experiments for feature selection using FFS and BFE were performed in Konstanz Information Miner (KNIME) [37]. The experiments for feature selection using RF were performed in JetBrains PyCharm, using the sklearn library from Python. The experiments for feature selection based on the adapted versions of GA and DE were performed in IntelliJ IDEA. The properties of the system in which the experiments were performed are:

(1)

processor properties: Intel(R) Core(TM) i5-7600K CPU @ 3.80GHz 3.80 GHz;

(2)

installed memory (RAM) properties: 16.0 GB;

(3)

system type properties: 64-bit Operating System, x64-based processor.

4.1. Machine Learning Methodology for the Classification of DLAs Based on Adapted BPSO

The steps of the machine learning methodology that is applied in the classification of the DLAs using data generated by monitoring sensors are described in more details in the next subsections. In the Feature Selection step, the methodology integrates the algorithm that is presented in this article, namely the adapted BPSO algorithm. The methodology is also used as support in order to validate and to test this algorithm.

4.1.1. DLAs Sensors Data

In Figure 1, the placement of the monitoring sensors for the DaLiAc dataset is illustrated.

The main characteristics of the DaLiAc dataset are presented in

Table 1. DaLiAc dataset characteristics.

Characteristic	Value
number of daily living activities (DLAs)	13
number of subjects	19
number of monitoring sensors	4
total number of features	24
sampling frequency	200 Hz

.

In the case of the DaLiAc dataset the following activities are monitored:

(1)

$DL{A}_1$—Sitting;

(2)

$DL{A}_2$—Lying;

(3)

$DL{A}_3$—Standing;

(4)

$DL{A}_4$—Washing dishes;

(5)

$DL{A}_5$—Vacuuming;

(6)

$DL{A}_6$—Sweeping;

(7)

$DL{A}_7$—Walking outside;

(8)

$DL{A}_8$—Ascending stairs;

(9)

$DL{A}_9$—Descending stairs;

(10)

$DL{A}_{10}$—Treadmill running;

(11)

$DL{A}_{11}$—Bicycling (50 watt);

(12)

$DL{A}_{12}$—Bicycling (100 watt);

(13)

$DL{A}_{13}$—Rope jumping;

The DaLiAc dataset contains information about 19 subjects (eight females and eleven males) aged $26\pm 8$ years. The data was acquired using four SHIMMER (Shimmer Research, Dublin, Ireland) sensors. Each of those sensors is equipped with a triaxial accelerometer (three features) and a triaxial gyroscope (three features). The accelerometer measures the proper acceleration while the gyroscope measures the velocity and the angular velocity. The sensors were placed on the left ankle, on the right hip, on the chest and on the right ankle of the monitored subjects. The frequency of sampling of those sensors is 200 Hz.

Figure 2 summarizes the number of samples of each daily living activity for each monitored subject. As can be seen in the figure, the walking outside activity ($DL{A}_7$) contains the biggest number of samples for each monitored subject and moreover the samples for each activity are not balanced.

4.1.2. Feature Selection

In this step, various combinations of feature selection techniques are investigated. Four different types of feature selection techniques are applied:

(1)

The first technique considers the features that correspond to the monitoring sensors that are used for collecting the data from the monitoring sensors: six features (chest sensor), six features (right wrist sensor), six features (left ankle sensor), six features (right hip sensor);

(2)

The second technique considers the following three feature selection algorithms from literature: the forward feature selection (FFS) algorithm [38], the backward features elimination (BFE) algorithm [39] and the random forest (RF) [40];

(3)

The third technique considers the BPSO algorithm adapted for data generated by monitoring sensors placed on the bodies of the monitored subjects;

(4)

The fourth technique considers adapted versions of genetic algorithm (GA) [41] and differential evolution (DE) [42] for feature selection using the same objective function as in the case of the adapted BPSO algorithm;

4.1.3. Cross Validation

The approach presented in this article is based on 10-fold cross validation. Thus, the data is split randomly in 10 folds of approximately equal size and the prediction model is run 10 times, such that, in each run, the testing data is represented by a different fold from those 10 folds and the training data is represented by the remaining nine folds.

4.1.4. Machine Learning Classification Model

The machine learning classification model is used in the classification of the daily living activities performed by the monitored subjects based on the data from the monitoring sensors. The machine learning classification model that is applied in this article is RF. RF is an ensemble classification approach that is based on the development of a collection of decision trees and has applications in many domains, such as medicine, ecology, bioinformatics, astronomy, and agriculture, and, moreover, it is a very good option for imbalanced datasets.

4.1.5. DLAs Classification

The final output of the machine learning methodology is represented by the classified DLAs, considering the raw sensors data generated during the monitoring of the DLAs. The metrics that are applied in the evaluation of the models used for the classification of the DLAs are the recall, the precision, and the F-measure.

4.2. Feature Selection Results for DLAs Data Generated by Monitoring Sensors Using the Adapted BPSO Algorithm

The configuration of the parameters of the adapted BPSO algorithm applied in the experiments is summarized in

Table 2. Configuration of the adapted binary particle swarm optimization (BPSO) algorithm parameters.

Parameter	Significance	Value
$N_{iterations}$	number of iterations	30
$N_{particles}$	number of particles	20
$c_1$	cognitive component value	2
$c_2$	social component value	2
$V_{min}$	minimum value of velocity	$-0.5$
$V_{max}$	maximum value of velocity	$0.5$
$w_{min}$	minimum value of inertia	$0.500$
$w_{max}$	maximum value of inertia	$0.901$

.

The mappings between the sensor particles weights and the sensor positions for the DaLiAc dataset are: $t_1$—C, $t_2$—RW, $t_3$—LA and $t_4$—RH.

Table 3. Sensor particle weights for the DaLiAc dataset for each monitored subject.

Monitored Subject/Sensor Particle Weight	${\mathit{t}}_1$	${\mathit{t}}_2$	${\mathit{t}}_3$	${\mathit{t}}_4$
$S_1$	0.478	0.567	0.448	0.507
$S_2$	0.544	0.490	0.466	0.500
$S_3$	0.570	0.535	0.418	0.477
$S_4$	0.498	0.559	0.470	0.473
$S_5$	0.565	0.492	0.465	0.478
$S_6$	0.482	0.529	0.465	0.524
$S_7$	0.472	0.502	0.528	0.498
$S_8$	0.564	0.520	0.435	0.481
$S_9$	0.537	0.554	0.429	0.480
$S_{10}$	0.509	0.505	0.472	0.514
$S_{11}$	0.508	0.494	0.461	0.537
$S_{12}$	0.536	0.520	0.474	0.470
$S_{13}$	0.594	0.510	0.441	0.455
$S_{14}$	0.555	0.480	0.475	0.490
$S_{15}$	0.497	0.525	0.489	0.489
$S_{16}$	0.573	0.490	0.450	0.487
$S_{17}$	0.555	0.527	0.451	0.467
$S_{18}$	0.571	0.484	0.453	0.492
$S_{19}$	0.532	0.524	0.472	0.472

summarizes the weights of the sensor particles for the DaLiAc Dataset, for each monitored subject.

In

Table 4. Summary of the adapted BPSO algorithm mean results for each monitored subject.

Subject	${\mathit{t}}_{\mathit{m}}$	${\mathit{f}}_{\mathit{m}}$	${\mathit{NRF}}_{\mathit{m}}$	${\mathit{a}}_{\mathit{m}}$
$S_1$	684,476	0.818	8	$91.0\%$
$S_2$	910,064	0.806	6	$93.7\%$
$S_3$	739,985	0.795	7	$92.4\%$
$S_4$	714,869	0.799	7	$90.0\%$
$S_5$	726,932	0.811	7	$90.4\%$
$S_6$	614,417	0.814	7	$94.3\%$
$S_7$	733,800	0.796	5	$86.0\%$
$S_8$	625,166	0.808	7	$91.1\%$
$S_9$	665,925	0.800	7	$93.3\%$
$S_{10}$	658,837	0.804	6	$89.7\%$
$S_{11}$	761,430	0.802	7	$88.2\%$
$S_{12}$	793,377	0.803	7	$92.8\%$
$S_{13}$	802,259	0.798	7	$92.1\%$
$S_{14}$	760,730	0.803	6	$90.9\%$
$S_{15}$	813,828	0.811	7	$90.7\%$
$S_{16}$	885,900	0.805	7	$91.0\%$
$S_{17}$	875,175	0.798	6	$92.7\%$
$S_{18}$	774,223	0.793	7	$91.1\%$
$S_{19}$	753,161	0.798	6	$91.7\%$

is presented as a summary of the experiments for each monitored subject S, as follows: the mean running time in milliseconds $t_m$, the mean fitness value $f_m$, the mean number of returned features $NR{F}_m$, and the mean accuracy $a_m$. For each monitored subject, five experiments were performed. The value of $NR{F}_m$ is from $\{5,6,7,8\}$.

In Figure 3, the evolution of the fitness value in each iteration of the adapted BPSO algorithm for each subject is presented. The fitness value is improved after 30 iterations for each monitored subject.

4.3. Comparison of the Results Obtained Using the Adapted Version of BPSO for Feature Selection with the Results Obtained Using Other Methods

Both for the adapted versions of GA and DE for feature selection, the same objective function was applied as in the case of the adapted version of BPSO. Moreover, the number of returned features was restricted to six in the case of GA, DE, and BPSO, using the heuristic for features ranking introduced in this article, which returns the results presented in Appendix A. In the case of the adapted version of BPSO the configuration parameters described in the previous section were applied.

The following configurations are applied in the case of the other methods:

(1)

FFS and BFE—The standard configurations from KNIME, a threshold for the number of features equal to six and a random drawing strategy;

(2)

RF—The standard configurations from sklearn, a number of estimators equal to 1000, a maximum number of features equal to six;

(3)

GA—20 chromosomes, 30 iterations, $CR$ (crossover rate) = $0.5$, $MR$ (mutation rate) = $0.5$;

(4)

DE—20 agents, 30 iterations, $CR$ (crossover probability) = $0.5$, F (differential weight) = $1.0$.

Both for the adapted versions of GA and DE for feature selection, the same objective function was applied, as in the case of the adapted version of BPSO. Moreover, the number of returned features was restricted to six in the case of GA, DE, and BPSO, using the heuristic for features ranking introduced in this article that returns the results presented in Appendix A.

In

Table 5. The running time for forward feature selection (FFS), backward features elimination (BFE), random forest (RF), binary particle swarm optimization (BPSO), genetic algorithm (GA) and differential evolution (DE) for each monitored subject, in milliseconds.

Subject/Features Selection	FFS	BFE	RF	BPSO	GA	DE
$S_1$	333,842	2,517,009	914,742	636,930	1,930,886	1,282,519
$S_2$	409,865	5,253,360	1,117,351	953,376	2,213,781	1,519,955
$S_3$	333,872	3,661,512	800,075	799,151	1,766,370	1,271,330
$S_4$	337,070	4,481,319	751,205	716,961	1,787,177	1,269,005
$S_5$	348,089	1,632,155	886,658	703,276	1,836,603	1,316,106
$S_6$	309,679	1,569,169	680,149	658,894	1,613,077	1,146,485
$S_7$	368,906	1,704,399	827,109	705,606	1,882,546	1,373,507
$S_8$	326,191	1,540,248	863,979	588,280	1,663,440	1,207,692
$S_9$	349,503	4,865,604	723,893	652,314	1,791,708	1,282,526
$S_{10}$	345,890	3,366,558	695,436	575,586	1,750,748	1,285,746
$S_{11}$	329,980	2,669,384	814,888	822,193	1,733,939	1,255,230
$S_{12}$	342,686	1,580,970	830,055	868,976	1,832,432	1,319,197
$S_{13}$	336,975	4,311,050	905,903	849,842	1,855,374	1,294,919
$S_{14}$	333,746	3,019,609	902,391	892,880	1,761,730	1,253,549
$S_{15}$	326,056	3,081,598	873,733	915,939	1,648,931	1,233,069
$S_{16}$	349,341	3,455,577	1,109,170	931,524	1,838,868	1,303,958
$S_{17}$	322,297	1,534,359	730,817	664,468	1,715,616	1,287,910
$S_{18}$	318,742	5,478,390	715,003	875,047	1,644,804	1,157,746
$S_{19}$	319,628	4,515,086	782,075	675,779	1,662,081	1,141,452

, the running times for each feature selection approach are presented. The running time of BPSO is much better than the running time of BFE, GA and DE, but slightly worse than the running time of FFS and comparable to the running time of RF.

In Figure 4, the selected features for each feature selection approach are presented, namely FFS, BFE, RF, BPSO, GA, and DE, for each monitored subject.

4.4. Comparison of the DLAs Classification Results Obtained Using the Adapted Version of BPSO with the Results Obtained Using Other Methods

In

Table 6. The random forest accuracy classification results for each feature selection approach for the DaLiAc dataset for each monitored subject ${}^1$.

Subject/Features Selection	C	RW	LA	RH	FFS	BFE	RF	BPSO	GA	DE
$S_1$	$90.9\%$	$93.7\%$	87.1%	$89.2\%$	$91.7\%$	92.8%	$91.5\%$	$\underline{\underline{\underline{92.1\%}}}$	$91.5\%$	$90.1\%$
$S_2$	$94.1\%$	97.1%	88.8%	$90.6\%$	$97.5\%$	$97.5\%$	$\underline{\underline{\underline{97.0\%}}}$	$95.8\%$	$96.8\%$	$94.2\%$
$S_3$	$91.9\%$	$94.0\%$	86.7%	$90.3\%$	$\underline{\underline{\underline{93.0\%}}}$	93.4%	$94.0\%$	93.4%	93.4%	$92.7\%$
$S_4$	$90.5\%$	$96.2\%$	87.5%	$89.5\%$	95.9%	95.9%	$95.4\%$	$\underline{\underline{\underline{95.7\%}}}$	$93.5\%$	$91.2\%$
$S_5$	$92.1\%$	$95.4\%$	82.3%	$90.7\%$	95.3%	95.3%	$\underline{\underline{\underline{94.9\%}}}$	$94.7\%$	$88.8\%$	$93.6\%$
$S_6$	$\underline{\underline{\underline{95.5\%}}}$	$94.1\%$	88.3%	$93.3\%$	$95.1\%$	$96.8\%$	$95.4\%$	96.6%	$93.2\%$	$94.4\%$
$S_7$	$94.6\%$	$\underline{\underline{\underline{95.0\%}}}$	87.1%	$93.0\%$	$96.1\%$	95.7%	$94.5\%$	$92.5\%$	$91.9\%$	$91.2\%$
$S_8$	$92.1\%$	$93.9\%$	85.5%	$88.3\%$	$96.9\%$	$96.9\%$	95.8%	$\underline{\underline{\underline{95.1\%}}}$	$93.8\%$	$88.7\%$
$S_9$	$92.3\%$	$93.4\%$	83.6%	$88.2\%$	94.4%	$\underline{\underline{\underline{94.3\%}}}$	$94.0\%$	$94.7\%$	$89.6\%$	$94.0\%$
$S_{10}$	94.5%	$\underline{\underline{\underline{94.1\%}}}$	87.0%	$88.4\%$	$95.0\%$	$95.0\%$	94.5%	$92.4\%$	$93.7\%$	$90.0\%$
$S_{11}$	$91.7\%$	$91.4\%$	84.7%	$88.6\%$	92.1%	$\underline{\underline{\underline{91.9\%}}}$	$92.7\%$	$90.4\%$	$88.9\%$	$89.3\%$
$S_{12}$	$94.3\%$	$94.1\%$	85.1%	$90.5\%$	94.6%	$95.6\%$	$\underline{\underline{\underline{94.5\%}}}$	$94.0\%$	$93.3\%$	$\underline{\underline{\underline{94.5\%}}}$
$S_{13}$	$92.6\%$	$94.1\%$	86.0%	$89.8\%$	$\underline{\underline{\underline{93.2\%}}}$	$92.8\%$	$91.9\%$	94.0%	$91.4\%$	$91.4\%$
$S_{14}$	$91.4\%$	95.6%	86.3%	$91.3\%$	$\underline{\underline{\underline{95.4\%}}}$	$95.9\%$	$95.1\%$	$95.0\%$	$90.9\%$	$94.1\%$
$S_{15}$	$91.6\%$	93.2%	88.8%	$91.7\%$	$\underline{\underline{\underline{93.0\%}}}$	$93.3\%$	$91.2\%$	$92.5\%$	$93.3\%$	$90.0\%$
$S_{16}$	$91.5\%$	$\underline{\underline{\underline{91.6\%}}}$	81.3%	$90.1\%$	$94.2\%$	$94.2\%$	$91.5\%$	92.8%	$91.2\%$	$89.3\%$
$S_{17}$	$95.1\%$	$\underline{\underline{\underline{95.3\%}}}$	89.2%	$94.5\%$	95.8%	$95.9\%$	$94.4\%$	$94.5\%$	$92.8\%$	$95.0\%$
$S_{18}$	$89.0\%$	$\underline{\underline{\underline{93.3\%}}}$	83.6%	$88.1\%$	93.5%	93.5%	$91.1\%$	$92.6\%$	$88.7\%$	$93.9\%$
$S_{19}$	$92.1\%$	$\underline{\underline{\underline{96.0\%}}}$	88.4%	$92.2\%$	96.2%	$97.0\%$	96.2%	$93.4\%$	$92.9\%$	$91.2\%$
Average	$92.5\%$	$\underline{\underline{\underline{94.2\%}}}$	86.1%	$90.4\%$	94.6%	$94.9\%$	$93.9\%$	$93.8\%$	$92.0\%$	$92.0\%$

${}^1$ (1) The best results are underlined with one line, (2) the second best results are underlined with two lines, (3) the third best results are underlined with three lines, (4) the worst results are underlined with one dash line.

the DLAs classification results for each feature selection approach are presented, when the applied classification algorithm is RF. For eight out of the 19 monitored subjects, BPSO returns classification results that are from the top three results. Moreover, BPSO returns the best classification result in the case of the ninth monitored subject and this result is very promising, showing that further improvements of the algorithm might lead to even better classification results. Another promising observation is that GA returns the best classification result in the case of the fifteenth monitored subject and DE returns the best classification result in the case of the eighteenth monitored subject. Therefore, further improvements of the objective function and the application of other bio-inspired algorithms might lead to better classification results.

5. Discussion

In this section, a critical discussion and the main focus is on the following aspects is presented: (1) the results obtained when the adapted BPSO algorithm is applied for each type of DLA and (2) the comparison of the performance of the machine learning approach based on he BPSO algorithm with the performance of other approaches from the literature that consider the same benchmark dataset.

5.1. Application of Adapted BPSO Algorithm for DLAs Classification

The experiments were performed in KNIME as follows: the input data for each feature selection approach is read from an Excel Reader (XLS) node, the X-Partitioner node represents the beginning of the cross validation and it applies a number of validations equal to 10 and random sampling, two Number To String nodes are used for converting the data types of the labels of the samples from the training data and from the testing data from number to string, in order to prepare the data for classification, the Random Forest Learner node is used for creating a classification model and the split criterion is Information Gain Ration, the Random Forest Predictor node is used for predicting the results, considering the data resulted from the Random Forest Learner node and the testing data as input, one String to Number node is used for converting the data type of the predicted labels from string to number, the X-Aggregator node represents the end of the cross-validation, and finally the Scorer node is used for computing the accuracy.

In Figure 5, the classification results for the DaLiAc Dataset for each monitored subject are presented. The best results were obtained in the case of the activities sitting ($DL{A}_1$) and lying ($DL{A}_2$) and the worst results were obtained in the case of the activities bicycling (50 watt) ($DL{A}_{11}$) and bicycling (100 watt) ($DL{A}_{12}$).

5.2. Comparison of the Performance of the BPSO Based Approach with the Performance of Literature Approaches

In this subsection, the results obtained using the machine learning methodology based on BPSO are compared with some of the best literature results.

Table 7. Comparison of the results obtained using the adapted version of the BPSO algorithm with the results obtained in the literature using the DaLiAc dataset as the experimental support.

Approach	Result	Method
Our approach	$93.8\%$ accuracy	The features are selected using an adapted version of BPSO and the data is classified using RF
(Zdravevski et al. [32])	$93.4\%$ accuracy	A method that applies the score-drift feature selection, based on an algorithm for feature extraction, selection, and classification
(Leutheuser et al. [16])	$89.6\%$ accuracy	A hierarchical multi-sensor based classification system
(Hur et al. [33])	$96.4\%$ accuracy	A method that is based on Iss2Image-UCNet6

presents a comparison of the results obtained, using the adapted version of the BPSO algorithm with the results obtained in the literature.

As can be seen in the table, the approach proposed in this article returns results comparable to the ones from the literature. The method presented in [33] returns much better results; however, in our approach we consider a maximum of six features and, in that approach, all features are used, therefore the two approaches are not directly comparable.

6. Conclusions

In this article, we presented a novel approach for feature selection in the case of data that is generated by sensors that monitor various types of DLAs. The method was tested and validated on the DaLiAc dataset. The experimental results show that the adapted version of BPSO presented in this article is comparable to other classical algorithms such as FFS, BFE, and RF, in terms of running time and classification performance.

The running time of the adapted version of BPSO is better than the running time of BFE and the developed objective function also returns promising results in combination with other bio-inspired algorithms, such as GA and DE. Moreover, for some monitored subjects the approach based on BPSO returns results that are much better than the ones obtained using classical algorithms for feature selection. Even if the proposed method based on BPSO returns results, which are not as good as the ones obtained using classical methods for some monitored subjects, further improvements of the objective function might lead to better results.

For example, the method presented in [33] performs significantly better,; however, it requires complex computational resources and image processing transformations.

As future research work the following research directions are proposed: (1) The testing and the validation of the proposed methodology using other validation types, such as the leave-one-subject validation type or the leave-two-subjects validation type, (2) the proposal of another heuristic for the ranking of the features returned by the adapted version of BPSO and (3) the comparison of the performance of the adapted version of BPSO with the performance of similar approaches based on other types of bio-inspired algorithms.