Improved Gender Recognition during Stepping Activity for Rehab Application Using the Combinatorial Fusion Approach of EMG and HRV

Gender recognition is trivial for physiotherapist, but it is considered a challenge for computers. The electromyography (EMG) and heart rate variability (HRV) were utilized in this work for gender recognition during the stepping exercise using a stepper. The relevant features were extracted and selected. The selected features were then fused to automatically predict gender recognition. However, the feature selection for gender classification became a challenge to ensure better accuracy. Thus, in this paper, a feature selection approach based on both the performance and the diversity between the two features from the rank-score characteristic (RSC) function in a combinatorial fusion approach (CFA)(Hsu et al) was employed. Then, the features from the selected feature sets were fused using a CFA. The results were then compared with other fusion techniques such as naive bayes (NB), decision tree (J48), k-nearest neighbor (KNN) and support vector machine (SVM). Besides, the results were also compared with previous researches in gender recognition. The experimental results showed that the CFA was efficient and effective for feature selection. The fusion method was also able to improve the accuracy of the gender recognition rate. The CFA provides much better gender classification results which is 94.51% compared to Barani’s work (90.34%), Nazarloo’s work (92.50%), and other classifiers.


Introduction
Rehabilitation is a physiotherapy activity to recover partially or totally the motor abilities of a patient.Usually, physiological devices (PDs) such as steppers, upright bicycles and treadmills are used to assist in the rehabilitation process.One of the PDs used in this study is a modified stepper rather than a conventional stepper.It was built using a smart material known as Magnetorheological (MR) fluid; installed as an MR valve on the stepper.The conventional stepper machine was slightly modified and can be controlled using the concept of MR valve (see Figure 1) for rehab applications.The mechanical properties such as stiffness, viscosity, and hardness of the MR-based damper will change subjected to the induced magnetic fluxes.The modified stepper is a current-induced device that varies the stiffness level (i.e., low, medium, high) of the damper.The characteristics of the MR fluid require the users to make an effort to do lower limb exercise resistance from a low to high stiffness level on the stepper.In addition, before the Ampere current is induced to the MR-based stepper, the system interface should have the ability to identify the gender of a person using it.The main reason is that the strength level of men and women is different [1][2][3].Thus, if the same load is applied to both men and women during exercise, it will cause unhealthy fitness, for example, it will cause them to be tired, unfit and other similar states.

The Need for Gender Recognition Using Biometrics
In conjunction to that, there is a need for a system development without any human intervention, in this case from a physiotherapist, such as automatic gender recognition.Gender recognition is trivial for a physiotherapist or the population in general.However, it is considered very challenging for computers.Gender recognition can be very helpful in association in terms of recognizing the individual user's personality.Currently, computer vision systems play imperative roles in our life with the development of Visual Surveillance and Human-Computer Interaction (HMI) innovation.Gender recognition, as a part of the entire computer vision system has acquired much attention in recent decades.This is due to gender recognition systems that can be utilized in many areas such as airport security [4], surveillance and security systems [5].Besides, it is also very helpful in rehabilitation whereby the computer interface can directly recognize the gender of a person, and consequently determine and set the input of the PDs like a current input associated with the strength level of men and women.This is considered as the visual surveillance or automated monitoring in the rehabilitation process.
In addition, a biometric recognition provides rigorous security by recognizing an individual based on the physiological characteristics [6].Various biometric modalities have been explored previously such as iris, fingerprint, face, and hand.Table 1 shows gender recognition based on the face in [7][8][9][10][11].In [12][13][14][15], a gender is recognized based on human gait, and it can also be recognized based on footwear, color information, and human speech [16][17][18][19].Nevertheless, these biometrics are not sufficiently robust against counterfeiting.For example, the iris can be duplicated by using contact lenses with duplicated iris features, the fingerprint can be imitated based on latex and face can be falsified using an artificial mask.Besides, eligible performance based on recognition accuracy can also not be provided by this biometrics.

Work Method
[7] • Sift features [8] • Combination of gabor filters and binary features [9] • Using hybrid of gabor filters and binary features [10] • Face_based gender recognition performance with a fuzzy inference system [11] • Using periocular biometric for gender classification in the wild [12] • From gait_based using mixed conditional random field [13] • Extraction of the hip joint data that was computed from Bio-vision hierarchical data [14] • From gait sequences with arbitrary walking directions [15] • Human gait based gender identification using Hidden Markov Model and Support Vector Machines [16] • Using footwear appearance [17] • Color information [18,19] • Speech recognition This research • Fusion of HRV and EMG signals during stepping exercise by stepper

Gender Recognition Using Physiological Signal
Meanwhile, other information from physiological biometrics that reflects neurological changes such as electromyography (EMG) and autonomic behavior such as heart rate variability (HRV) can also affect gender recognition.Gender recognition using new physiological biometrics (e.g., EMG, ECG, and EEG) is quite complicated compared to face, iris and fingerprint physiological biometrics.This is because the signal of ECG and EMG recordings can be changed or distorted due to the physical status of the subject, movements, powerline interference and certain medications [20,21].However, the EMG and ECG biometrics information are difficult to be stolen and impossible to imitate.So, it is well secured and confidential [22].
Much basic research on electrocardiogram (ECG) has been actively conducted.Several suggestions proposed the feasibility of utilizing the ECG as another biometric modality for identity identification [23,24].The fact that the physiological differences of the heart in various people show certain uniqueness in their ECG signals have supported why it is logical for biometric recognition [25].Gender recognition indicated by HRV is a standout amongst the most challenging issues in individual identification [26].
Furthermore, digital information about the physical muscles is provided by EMG.Although there a few analysis on EMG involving the lower limb [27], a research work on gender recognition during the stepping activity using EMG is yet to be investigated.Interestingly, there is a significant time delay difference between genders during the step [28], which indicates that there must be some relevant features in human muscles that help in differentiating gender.
Multimodal physiological biometrics are a combination of at least two physiological biometric features in order to handle certain limitations of utilizing them separately.There have been some previous attempts to fuse multiple biometrics such as combining the face, gait, fingerprint, palm print and geometry for gender recognition [29][30][31].Table 2 shows more related works for the fusion of physiological biometrics and the gap of this study.

Work Physiological Signals Fusion Application
ECG EMG EEG [32][33][34] Automatic seizure detection [35] hybrid-BCI [36] Epileptic event detection [37] Human movement intention [38] Automatic sleep stage classification [39] Gait intent detection [40,41] and this research Gender Identification for rehabilitation exercise Therefore, the bimodal physiological systems for automatic gender recognition were carried out in this study to develop the mechanism of physiotherapy for online monitoring.To guarantee the accurate signal of a single modality system, the fusion of two or more signals is significant to improve the performance of the system [39,[41][42][43][44][45][46][47][48].To achieve robust and discriminative performance for gender recognition, a fusion of EMG and HRV [40,41,49] is proposed before feeding into the interface system in order to control the level of MR valve stiffness of the stepper before the rehabilitation exercise can be emulated [40,41].Two modalities ECG (HRV) and EMG are selected to be combined because both modalities are closely related when performing exercise associated with lower limb.It seems that the gender factor motivates people differently, in performing the regular exercise for rehab.The reduction of cost and frequent monitoring are also the ultimate objectives in the improvement of rehabilitation process based on the gender physiological feedback control.Hence, the automated online monitoring system can be developed and a bimodal fusion system can be achieved.
The overview of this research is shown in Figure 2. Firstly, all the raw data are preprocessed for artifact corrections.Next, all the features are extracted and the most relevant features are selected based on the performance and the diversity between the two feature sets.Then, the Combinatorial Fusion Algorithm (CFA) (Hsu et al. [50][51][52]) is implemented to fuse the EMG and HRV features for automatic gender recognition during stepping which may help to isolate male and female subjects before undergoing the rehabilitation process.Finally, the gender classification based on the feature fusion correct rate (%) is obtained.However, the section outside the bold solid line (i.e., the interface section, control signal section and physiological device section) is out of the scope of this study.The organization of this paper is as follows.Section 2 describes the research steps involved in automatic gender recognition during the stepping activity using the MR-based stepper in rehabilitation.The steps include data acquisition and preprocessing of EMG and HRV.The HRV is quantified from the ECG signals and both EMG and HRV features are extracted.Section 3 represents the feature selection using performance and cognitive diversity.The implementation of the CFA to fuse the features is conducted in Section 4. The experimental results are analyzed and discussed in Section 4.2.Finally, the conclusion and the recommendation of future works resulted from this paper is suggested in Section 5.

Methodology
The proposed young gender recognition based on the EMG and HRV biometrics is illustrated in Figure 3.The proposed method is composed of a sequence of processing steps which are the preprocessing of the EMG and ECG signals, feature extraction, feature selection based on the performance and the diversity between EMG and HRV feature sets, and EMG and HRV feature fusion process based on the Combinatorial Fusion Approach (CFA) (Hsu, Chung and Kristal [51] and Hsu, Kristal and Schweikert [52]).In this paper, the feature selection and the feature fusion have been focused as shown in the bold dotted line.

Data Acquisition
The EMG and ECG data used in this study were acquired from 59 healthy, untrained young volunteers (i.e., 23 years old on average).All the subjects were considered free from any disease and have been briefed about the procedures involved, and informed consents have been obtained.An EMG channel was simultaneously recorded with ECG using the Porti system from the Twente Medical Systems International (TMSi) equipment.The Porti is a 32-channel ambulatory and stationary system for physiological research.An EMG channel was obtained from the vastus lateralis muscle (VL), placed according to the recommendations of Surface Electromyography for the Non-Invasive Assessment of Muscles (SENIAM) [53].In summary, 16 European expertise groups are working together with the SENIAM project for the development and applications of surface EMG (sEMG).The recommendation of the sEMG electrode placement is partly based on simulation outputs.The sampling frequency of raw EMG and ECG biosignals used in this study was about 2048 Hz.Higher sampling frequency was chosen to prevent a signal loss and to avoid too much noise during the stepping exercise.Plus, it is also to avoid jitter in the estimation of the R-wave fiducial point for ECG signal.Also, the stepper machine was used to carry out the stepping activity.In order to get the same stepping rate among the subjects, a metronome (i.e., 60 beats per minute) was used simultaneously with the stepping activity.Generally, a metronome is a device used by musicians that indicate the time at a selected rate by giving a regular tick.

ECG:
The Kubios HRV software [54] is utilized for ECG preprocessing to quantify the HRV.It is an advanced software and extremely easy to use for HRV analysis.It consists of a mechanism for artifact correction (i.e., powerline interference, baseline wander and other noise components), trend component removal, an adaptive QRS detection and analysis for sample selection.The QRS detection algorithm in the Kubios HRV software was developed based on the Pan-Tompkins algorithm [55].Figure 4a shows the patterns of the HRV.
EMG: All the EMG signals were processed using MATLAB.There was a low amplitude voltage offset present in the hardware.The direct current (DC) power was eliminated.Then a bandpass filter was applied to the raw data to remove any movement artifacts with 20 Hz and 300 Hz cutoff frequencies.Since the raw EMG signal had positive and negative values [56], the full-wave rectification was applied to make sure the raw EMG signal did not have nearly zero mean and to analyze positive values only.The EMG signals linear envelope was then computed for further processing.Figure 4b shows the patterns of the EMG.

Combinatorial Fusion Approach (CFA)
Combinatorial Fusion (CF) (Hsu et al. [50][51][52]) gives a useful piece of fusion method information in the analysis of a set of Multiple Scoring Systems (MSS) to obtain better accuracy and higher effectiveness of feature characteristics.Each system contains a score function, a rank function, and a Rank-Score Characteristic (RSC) function (see Figure 5).The scoring or ranking behavior is indicated by the RSC function and it is extremely easy to compute.Plus, the cognitive diversity of at least two scoring systems can be measured [52].Furthermore, the CFA method has been widely applied in some research areas such as visual cognition systems [66], ChIP-seq peak detection [67], video target tracking [68] and the prediction of protein structure [69] when the result was obtained by using an RSC graph to assess the scoring diversity [52].Meanwhile, the CFA was implemented in this work in order to demonstrate that the fusion performance of HRV and EMG for automatic gender recognition during the stepping activity can be improved by fusing score or rank combinations using the RSC function as an indicator.
Figure 5 illustrates the RSC function in a set of MSS.Let T = {t 1 , t 2 , ..., t n } be a set of subjects under study.Let N = [1, n] be the integers set from 1 to n and R be the set of real numbers.Next, let A be the scoring system.Each scoring system A is one of the q scoring systems A 1 , A 2 , ..., A q on the set T, contains two functions which are: The r A are derived by sorting the s A .A Rank-Score Characteristics (RSC) function, f A was defined (see [51,52]) as: f A : N → R while a score function, s A , and a rank function, r A are shown in Figure 5.There are two distinct forms of procedures that can be applied to integrate a set of q scoring systems namely: (a) Score Combination, sc.(b) Rank Combination, rc. and the equations are defined as follows: where t i is in T.
Moreover, the diversity between A and B, for a pair of two scoring systems, d(A, B) is defined as d(f A , f B ), between RSC functions f A and f B (see [51,52]).

Feature Selection Using Performance and Cognitive Diversity
The features from the EMG and HRV are viewed as a scoring system.The performance and the diversity between the feature sets in MSS have been carried out to select the relevant and non-redundant features, which can yield a better result when the features are fused.Thus the combination of the MSS can improve the result when the features of each of the individual scoring systems have relatively high performance and high diversity [52].Therefore the aim of our feature selection is to find the features with relatively good performance as well as relatively high diversity.
In our experiment, there are 32 features in total and 59 possible values for each feature.The correct rate is calculated to measure the performance and to make comparisons between the machine learning classifiers (see Figure 6).The classifiers used in this work are Naïve Bayes (NB), k-Nearest Neighbors (k-NN with k = 1, 3, 5), and Decision Tree (J48).As shown in Figure 6, the highest average correct rate is J48 classifier with 72.19% of accuracy.After that, the final performance of each feature from the J48 classifier is sorted in decreasing order as illustrated in Figure 7.It can be seen that the highest correct rate with a rate of 86.44% is generated from feature O (LF_power (ms 2 )) and feature d (maximum amplitude of EMG (ms 2 )) generates the lowest correct rate of with a rate of 66.1%.From previous work in [70], there are significant gender differences (p < 0.05) in the LF in male subjects compared to females.This implies that the male subjects are more active and dominant than the female subjects.Therefore, this is particularly suitable to be used to isolate male and female subjects in developing the rehabilitation devices, control system, and rehabilitation engineering.

The Diversity between MSS
Generally, the diversity of a feature pair {F 1 , F 2 } can be derived as in Equation ( 3), where f 1 and f 2 are the RSC function represented for F 1 and F 2 respectively in which they can be calculated based on the equation denoted in Figure 5.Both functions f 1 and f 2 have n score values in total and a rank sequence ranging from 1 to n.
The diversity of feature sets D = {F 1 , F 2 , ..., F n } is generated as in Equation (4).C(n, 2) depicts the total number of the combination of two features in the feature set D.
In addition, the diversity between the two feature sets D 1 and D 2 is derived as in Equation ( 5).Generally, |D 1 | and |D 2 | are the cardinal number of D 1 and D 2 .

Feature Selection Approach
The approach for feature selection based on the performance and the diversity between the two features (Deng, Wu, Chu, Zhang, and Hsu [46]) is illustrated in Figure 8. Figure 9 illustrates the modalities, performance, and diversity of the features.All 32 features are divided into two modalities according to the corresponding sensor types which are EMG and ECG.The feature performance in each of the EMG and ECG physiological signals is sorted in decreasing order.15 features with a bold font demonstrated in the figure is selected for further process and is denoted as the top features.While a number above the horizontal link between two modalities is denoted as the diversity between the two feature sets.Besides, a number above the curve within a node of the modality is the average of a feature set diversity.Generally, a better fusion performance is produced when the diversity between two modalities is larger [46].Apart from that, better individual feature performance also provides a better combination result [46].However, only two modalities were utilized in this work but the comparison of the diversity of the two modalities is not required.In this work, the diversity between EMG and HRV generates a better diversity of a feature set and the diversity between two feature sets.Hence, a better combination performance can be achieved.

Feature Selection Performances
In the feature selection process, the top-15 features which are O, V, A, F, C, G, M and I from the HRV and a, b, c, e, f, g, and i from the EMG are selected.All of these top-15 features are selected because of their performance and the diversity is higher than the other features.A total of four 5-feature sets (A), four 7-feature sets (C), four 9-feature sets (D) and four 11-feature sets (E) are then   With reference to Figures 10 and 11, the dotted lines show the average of 26 combined features.In this paper, the combined features that have the correct rate over an average of 26 combined features will be the better selected combined features for the automatic gender identification.In Figure 10a, the average correct rate is 95.49%, and it possesses the top-13 combined features which are OV, OA, OVA, OVF, OF, VA, OVAF, OAF, VF, VAF, AF, OVAa, and OVAFa.Next, in Figure 10b, the average correct rate is 93.53% and it possesses the top-13 combined features which are OV, OA, OVA, VA, OVAb, OVAc, OVb, OVc, OAb, OAc, VAb, VAc, and OVAbc.Then, in Figure 10c, the average correct rate is 93.73% and it possesses the top-13 combined features which are OV, OVF, OF, VF, OVFa, OVa, OVFb, OVb, OFa, OFb, VFa, Oa, and OVFab.Lastly, in Figure 10d, the average correct rate is 95.27% and it possesses the top-13 combined features which are OV, OA, OVA, OVF, OF, VA, OVAF, OAF, VF, VAF, AF, OVAFb, and OVAb.Thus, in short, all four 5-feature sets (A) have their own top-13 combined features that contribute to automatic gender identification during the stepping activity using the stepper.
Moreover, in Figure 11a, the average correct rate is 78.36% and it possesses the top-12 combined features which are IP, IU, IPU, Ic, IPc, IPUc, IUc, PU, IPd, IPUd, Pc, and PUc.Next, in Figure 11b, the average correct rate is 77.75% and it possesses the top-12 combined features which are HR, HRa, Ha, Ra, HRh, HER, HRah, EHRa, Hh, EH, Hah, and EHRh.Then, in Figure 11c, the average correct rate is 80.34% and it possesses the top-12 combined features which are HI.HIJ, IJ, HJ, HIc, HIJc, IJc, Ic, HJc, HIJd, Hc, and HId.Lastly, in Figure 11d, the average correct rate is 77.85% and it possesses the top-12 combined features which are OV, IQ, IS, IQS, QS, IQi, IQSi, Ii, ISi, IQSd, IQd, ISd, and Id.Thus, all the four 5-feature sets (B) have their own top-12 combined features.
All the figures from Figures 10 and 11, shows that the four 5-feature sets (A) have higher correct rate compared to the four 5-feature sets (B).Thus, the overall performance of the 5-feature sets (A) is considerably better than that of the 5-feature sets (B) for automatic gender identification during the stepping activity.
Besides that, the fusion of t-feature sets where t = 7, 9 and 11 is also calculated.7-feature sets have 120 combined features, 9-feature sets have 502 combined features and 11-feature sets have 2036 combined features in total.Since the total combination is large, the results were only listed in Table 5 in Section 4.2.

Fusion Results Comparison
For the feature fusion evaluation, six feature fusion algorithms which are Naïve Bayes (NB), k-Nearest Neighbors (k-NN with k = 1, 3, 5), Decision Tree (J48) and Support Vector Machine (SVM) are implemented.The comparisons of the correct rates for different 20 feature sets found in the fusion methods are shown in Table 5.The average correct rate of all the combinations's results for all the selected feature sets is denoted as the overall correct rate of CFA for automatic gender recognition during stepping.a face biometric and Hybrid Gabor filters and binary features of an image for gender recognition.Table 6 shows that our proposed method outperformed the work done in [8,9].

Conclusions and Future Work
In this paper, the function of CFA [51,52] was described for both features' selection and fusion in order to automatically recognize gender during the stepping exercise.The performance results demonstrated that the CFA provided a promising solution compared to other machine learning methods in combining relevant features from physiological biosignals.The CFA also provided a much better correct rate compared to Barani's [8] and Nazarloo's work [9].Generally, when the performance and diversity between the selected two features are relatively high, a better correct rate of the feature fusion can be attained.
The main contributions of this work are the physiological biometrics used for automatic gender recognition during stepping using stepper which are EMG and HRV, a feature selection technique based on the performance and the diversity between the feature sets, feature fusion technique using CFA, a comparison of CFA with another five machine learning classifiers, and a comparison of our proposed method with previous research for gender recognition.Our work demonstrated that the CFA is a good and efficient method for the feature fusion.
In future, the HRV features will be combined with bi-channel EMG features based on CFA.In addition, the decision level combination using CFA will be considered.

Figure 2 .
Figure 2. The need for this research.

Figure 3 .
Figure 3. Gender identification process based on electromyography-heart rate variability (EMG-HRV) feature fusion.The selected features of EMG and HRV are combined using Combinatorial Fusion Approach (CFA) to obtain the gender classification result.

Figure 4 .
Figure 4.The patterns of HRV and EMG.
(a) a score function, s A , and (b) a rank function, r A .(a) a score function, s A , and (b) a rank function, r A .

Figure 9 .
Figure 9. Feature classification using diversity and performance in two modalitites.

Figure 10 .
Figure 10.Performance of the combination of 5-features sets (A group) using CFA.

Figure 11 .
Figure 11.Performance of the combination of 5-feature sets (B group) using CFA.

Table 1 .
Most related works for gender identification.

Table 2 .
Most related work for the fusion of physiological signals.

Table 3 .
Description of the 32 extracted features.

Table 5 .
Comparison of performance among fusion method.

Table 6 .
Correct Rate (%) comparison of proposed method with previous work of gender recognition.