Next Article in Journal
Robotics in Endometrial Cancer Care
Previous Article in Journal
Special Issue on Intelligent Robots
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Robust Bio-Signal Based Control of an Intelligent Wheelchair

1
School of Automation Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China
2
Engineering Research & Development Center for Information Accessibility, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
*
Author to whom correspondence should be addressed.
Robotics 2013, 2(4), 187-197; https://doi.org/10.3390/robotics2040187
Submission received: 5 August 2013 / Revised: 23 September 2013 / Accepted: 25 September 2013 / Published: 30 September 2013

Abstract

:
In this paper, an adaptive human-machine interaction (HMI) method that is based on surface electromyography (sEMG) signals is proposed for the hands-free control of an intelligent wheelchair. sEMG signals generated by the facial movements are obtained by a convenient dry electrodes sensing device. After the signals features are extracted from the autoregressive model, control data samples are updated and trained by an incremental online learning algorithm in real-time. Experimental results show that the proposed method can significantly improve the classification accuracy and training speed. Moreover, this method can effectively reduce the influence of muscle fatigue during a long time operation of sEMG-based HMI.

1. Introduction

The deployment of electric powered wheelchairs (EPWs) has been increased rapidly for better quality of life for handicapped and elderly people over the last 20 years [1]. However, most of these EPWs are traditionally controlled by joysticks, which are not suitable for people with severe physical disabilities like spinal cord injury or hemiplegia. Therefore, alternative HMI methods for wheelchair operation have been studied, such as voice recognition [2], eye moment [3], head gesture [4] and biopotential signals interface [5,6,7,8]. Bio-potential signals such as Electrooculargraphic (EOG), electromyographic (EMG) and electroencephalographic (EEG) are also deployed to control such kind of smart service robot [9,10,11]. Compared to other signals, EMG signal is more easily obtained and has less signal interference, especially for the weak groups, such as patients with upper limb amputation and elderly people for which other methods are inconvenient for their daily life.
The EMG signal, which is sampled from the skin’s surface, can directly reflect the movement intention of human body and could be very suitable to serve as the controller. Englehart and Hudgins investigated a continuous segmentation or continuous classification by using different analysis window lengths [12]. Moon used a preset threshold value method to distinct the scapulae muscle signal at fairly high recognition rates [13]. Hiroki Tamura et al. obtained s-EMG signals from facial movements and six patterns were recognized to control the wheelchair movements [14]. Lai Wei combined forehead sEMG signals and color face image information to control the smart robot [5]. However, it suffers from poor reliability and robustness in long-term operations, which were mainly caused by muscle fatigue. These may arise not only because of peripheral changes at the level of the muscle, but also because the central nervous system fails to drive the motoneurons adequately [15]. Usually, the amplitude of the sEMG signals from muscles we used without fatigue is lower than 400 μV. However, after fatigue occurs in muscles, there are significant rises to 600 μV, which can be seen Figure 1. Qin and Mitsuhiro investigate a torque estimation method for muscle fatigue tracking, using stimulus evoked electromyography in the context of a functional electrical stimulation rehabilitation system [16]. However, how to choose the best signal evaluation parameter according to specific muscle condition and method of reducing the influence of muscle fatigue is still not very clear, and has been discussed during recent decades [17,18].
Figure 1. The amplitude of surface electromyography (sEMG) signals.
Figure 1. The amplitude of surface electromyography (sEMG) signals.
Robotics 02 00187 g001
To investigate the real time and long time performance of the proposed system in this paper, forehead sEMG signals are first obtained and analyzed by using three dry electrodes sensor. Then, a four-order autoregressive model parameter algorithm is deployed to acquire data compression and extract four coefficients of the signals. An incremental online support vector machine (SVM) is employed to classify sEMG patterns that have the systematic parameters corresponding to the changes of the signals during the running time. Furthermore, the improved incremental training algorithm is presented and applied on the HMI system.

2. The Research Objects and Methods

The sEMG signal controlled intelligent wheelchair should provide a high degree of intuitive and dexterous control, and offer a convenient and high level of performance for disabled people. The human face has plenty of facial muscles to make expressions. The frontalis muscle, located on the front of the head, is one of the largest muscles which can provide continual and distinct, recognizable signals. The main components of the system include sEMG signals collected by electrodes placed on the skin over a user’s forehead muscle, electrodes accompanied by miniature pre-amplifiers to differentiate small signals of interest, signal features detected after amplified, filtered, digitized via standard EMG instruments, and finally transferred to a wheelchair controller. Five subjects voluntarily participated in the experiment and have independent consciousness without facial disease, deformity or trauma. In addition, there was no muscle fatigue before the experiments were performed.
The experimental system consists of an intelligent wheelchair with two independently driven motors, a laptop computer for human-machine interface, and a data acquisition device. A Cyberlink or NIA system is used to acquire the sEMG signals of the subjects. The interface unit has an amplifier with a gain of 50,000, and a filter system provides a band pass filter with a bandwidth of 0.5 to 45 Hz. The weak bioelectricity signals from forehead surface skin are adopted from the user by wearing a sensory headband. The interface box processes the signals and connects to a computer through a serial or USB port. Figure 2 shows the headband devices we used in the experimental system.
Figure 2. The headband EMG device in the human-machine interaction (HMI) system.
Figure 2. The headband EMG device in the human-machine interaction (HMI) system.
Robotics 02 00187 g002
The muscle movements are defined to be control signals for a HMI system, namely single jaw click (SJC) and double jaw click (DJC). They are generated by the subjects contracting the masseter muscle and buccinator muscle with a jaw clenching and chewing-like movement. Every subject uses a designed sEMG pattern based HMI system to control an intelligent wheelchair, and the control signals are recorded at the same time. There are five control states designed for the wheelchair, namely: “Forward”, “Backward”, “Left”, “Right”, and “Stop”. Note that the first four states are scanned in turn. Users can use a single jaw click movement to switch the direction to whereas they want. If the wheelchair is in the control state of “Forward”, “Backward”, “Left”, or “Right”, a double jaw click movement will be used to switch the wheelchair to the “Stop” state. The complete structure of the HMI system is depicted in Figure 3.
Figure 3. The diagram of the proposed HMI system.
Figure 3. The diagram of the proposed HMI system.
Robotics 02 00187 g003

3. Feature Extraction

Myoelectric control systems can be divided into two groups. In the first group, the desired classes of functions are discriminated from signal patterns by classifiers, and the variety of functions depends directly on classification performance. In the second group, recognition-based controllers, which are mainly constructed on threshold control and/or finite state machines, are merely output limited and pre-defined control commands based on a sequence of input signal patterns [19]. The latter gives a direct sense sometimes but has a non-adaptive shortage. After the forehead sEMG signals are obtained by using the dry electrodes sensor, an autoregressive model parameter algorithm is deployed to extract the signal feature.
sEMG signals can be considered as the output signals of a linear sequence with zero mean white noise. The white noise reflects the randomness of signals, while the deterministic part (the order and parameters of the model) reflects the predictability of the process. The four-order AR model is chosen to analyze the sEMG signals; the mathematical expression is as follows:
Robotics 02 00187 i001
where x(n) is the current sampling value of sEMG signals. w(n) is the current incentive value, i.e., the residual of the white noise. p is the order of the model. ak is the kth coefficient of the AR model.
The meaning of Equation (1) is that x(n) is generated by the linear combination of several past values x(n - k) and the current incentive value w(n). The model function is defined as follows:
Robotics 02 00187 i002
For the AR model, the selection of p is critical. If the order is too low, like a high-order curve fitted with a low-order curve, it will produce a smooth result; if the order is too high, like a low-order curve fitted with a high-order curve, it will produce a dramatic change and vibration. Many experiments show that the analysis and recognition results of sEMG signals are the best when the order of AR model is 4, so the four-order AR coefficient vector A = [a1, a2, a3, a4] is the input to the SVM algorithm. Table 1 shows four groups of all four-order AR coefficients of Single jaw click and Double jaw click. The AR coefficients corresponding to two kinds of facial movements have obvious differences, so the variations of coefficients can be used as the basis for judging the changes of movement patterns. Meanwhile, it verifies whether the AR coefficients are reasonable as sEMG features.
Table 1. AR coefficients of Single jaw click and Double jaw click.
Table 1. AR coefficients of Single jaw click and Double jaw click.
1234
Single jaw clicka1−1.6459−1.4644−0.84630−1.6434
a20.776301.0654−0.443900.72990
a3−0.15580−1.1692−0.10020−0.11680
a40.0393000.577500.409000.048200
Double jaw clicka1−0.96300−0.99530−1.2525−0.98050
a2−0.22440−0.388800.12190−0.32100
a30.0873000.0886000.316800.22390
a40.127800.31810−0.167900.097000
Since EMG is a random signal, its power spectral density (PSD) should be estimated prior to calculating the time-course of the spectral variables. The mean (MNF) and median (MDF) spectral frequencies, which provide some basic information about signal spectral, have been the most popular such variables both in academic studies and in clinical practice, owing to their relevance to underlying physiological processes that control fatigue [20,21]. To measure muscle fatigue, the widely used Fast Fourier transform technique (FFT) is utilized to analyze mean frequency of sEMG.
Robotics 02 00187 i003
Robotics 02 00187 i004
where Pi is the ith line of the power spectrum; fi is the frequency variable; and M is the highest harmonic considered.
MNF and MDF were calculated as previously described [22]. Statistical analysis is applied to find meaningful differences of sEMG from different subjects during various periods. The five subjects are required to perform repetitive movements and the average MNF and MDF calculated by before muscle fatigue, run 20 min, 40 min, and 60 min is showed in Figure 4. Each period lasted 20 s. Both MNF and MDF will decrease with time when muscle fatigue takes place. The MNF and MDF of forehead muscle decreased 10.8%, 17.2% separately while the average err fluctuation was almost unchanged after 60 min muscle contractions. It is experimentally proven that the manifestation of fatigue in myoelectric signals is significant during long-term muscular activities.
Figure 4. Mean (MNF) and median (MDF) shifts during different periods. Each period lasted 20 s.
Figure 4. Mean (MNF) and median (MDF) shifts during different periods. Each period lasted 20 s.
Robotics 02 00187 g004

4. Adaptive Incremental Online SVM

sEMG patterns gradually change during long durations of muscle activities due to muscle fatigue, so the threshold method will lose its effectiveness. As we know, a SVM classifier constructs an optimal separating hyperplane in a high dimension feature space of training data that is mapped using a nonlinear kernel function [23,24]. The key ideas of SVM are: getting a maximized classification margin by solving dual optimization problem, and then obtaining an optimal separating hyperplane.
If the data is linearly separable, the decision function is:
Robotics 02 00187 i005
where {xi, yi}, i = 1,…, l, are the training samples, yi is the class label of the training sample xi, l is the number of the training samples, b is the offset.
When the data is nonlinearly separable, the original data is mapped into a high dimensional feature space by kernel function K(xi, x), in which the mapped data is linearly separable. Then, the decision function is as follows:
Robotics 02 00187 i006
Robotics 02 00187 i007
According to the Karush-Kuhn-Tucke (KKT) condition [25], there is only one solution for the optimization problem based on the quadratic optimization theory, and α = [α1, α2,…, αl] is the optimal solution for the dual problem. When it satisfies the KKT condition, the problem is divided into the following three situations:
Robotics 02 00187 i008
where α is Lagrangian dual, c is an upper bound for samples that lie on the wrong side of the hyperplane. Nonzero constant αi is the sample corresponding to support vector. Samples are located on the classification interval 0 < αi < c. When αi = c, samples lie in the classification interval.
Incremental support vector machines are instrumental in practical applications of online learning. The purpose of the incremental algorithm is to add the new sample data to the three sets and to delete the redundant data, while maintaining all other samples that still belong to the three sets. In the classical SVM, the hyperplane in m dimensions can be shown as wT∙x + b = 0. In our updating samples method, only the samples that are closest to the current boundary are chosen to train the classifier. There is a maximum distance of all update samples to boundary, which is defined as 2δ.
Robotics 02 00187 i009
If the new added training sample violates the KKT conditions, the Lagrange multiplier α that is obtained from the previous sample will not be the optimum solution. The proposed improved online incremental training algorithm with the updating method is as follows. The standard SVM is used to initialize the classification and a training set is obtained. The next step is choosing the most informative patterns that have the best possibility of becoming the support vectors in the in-put sEMG sequence, and checking whether these samples meet the KKT conditions. If not, try the next data until KKT conditions are satisfied. Then construct samples including the update data, and train samples online to form a new classifier. The process repeats until there is no in-put sEMG data.

5. Experiment Results

Traditional SVM, such as LIBSVM, employed a radial basis function kernel to classify data, with the same parameter (C = 1 and γ = 0.5) [26]. The data is normalized and limited to the range of −1 to 1. 800 samples are randomly selected from 2,000 pre-collected signals as a training set. After that, choose 800 samples in the rest samples randomly as a testing set. When the result is compared with the method given above, the classification accuracy is shown in Table 2. The training time was generated at 300 samples.
Table 2. Comparison real time performance of the LIBSVM and the Incremental online support vector machine (SVM).
Table 2. Comparison real time performance of the LIBSVM and the Incremental online support vector machine (SVM).
LIBSVMIncremental online SVM
Single jaw click73%90%recognition rate
92 ms58 mstraining time
Double jaw click81%93%recognition rate
103 ms62 mstraining time
Note that five subjects in the experiment are required to control the intelligent wheelchair running throughout a specified path in the lab. A map of the experiment environment with the designed path is given in Figure 5. The size of the map is 8,000 × 6,000 mm and the actual size of the wheelchair is 1,000 × 700 mm. “Forward”, “Backward”, “Left”, “Right”, and “Stop”: we tried to generate an average for each control state over two experiment days. For each experiment, the five subjects are required to control the intelligent wheelchair for 90 min without a rest.
Figure 5. The path designed for experiment.
Figure 5. The path designed for experiment.
Robotics 02 00187 g005
Figure 6 shows the average time consumptions of five subjects in repetitive experiments during deferent periods. The average consuming time is 103 s for one round at the beginning, but the average time increased to 154 s at the last period when muscle fatigue occurred. After using the new method, the average time at the last period could decrease to 110 s. Table 3 shows the average recognition rate of five subjects controlling the wheelchair using the proposed HMI system with the two online SVM in different periods. The recognition of jaw click, or what is called “single jaw click” and “double jaw click”, is the most important factor of the experiment.
Figure 6. The five subjects’ average time consumptions of repetitive experiments during different periods.
Figure 6. The five subjects’ average time consumptions of repetitive experiments during different periods.
Robotics 02 00187 g006
Table 3. Comparison long time performance of the LIBSVM and the Incremental online SVM.
Table 3. Comparison long time performance of the LIBSVM and the Incremental online SVM.
SubjectsRecognition rate for five state using LIBSVMRecognition rate for five state using incremental online SVM in seconds
0–20 min20–40 min40–60 min0–20 min20–40 min40–60 min
A86.580.173.892.688.486.7
B88.781.673.392.889.587.4
C83.179.270.986.782.879.3
D90.485.579.293.690.188.3
E80.875.370.184.981.678.4
Average85.980.3473.4690.1286.4884.02
Experimental results indicate that the performance of the sEMG based HMI system with LIBSVM gradually declines as time increases with the occurrence of muscle fatigue. The subjects control the intelligent wheelchair by this system using more time in different periods, as shown in Table 3. Also, one should note that, although the threshold EMG method has a pretty high recognition rate before muscle fatigue, the performance after that declines quickly. The incremental online SVM proposed showed good properties in both convergence and optimality.

6. Conclusions

Muscle fatigue can negatively influence the performance of human-machine interaction using sEMG signals. This paper presents a novel sEMG based HMI system for the hands-free control of an intelligent wheelchair both in real time and in long time operations. The system is used to control the simple movements of the intelligent wheelchair, i.e., forward, turn-left, turn-right, backward and stop. The experimental results show that the proposed HMI is easy to operate in real-time and has a high recognition rate. It could be deployed in many aspects of real world applications, such as helping the elderly and disabled, rehabilitation, and so on.
The speed and accuracy of pattern recognition of forehead sEMG signals are to be further improved in our next stage of research. By improving the incremental online SVM algorithm, we hope to reduce the false recognition rate, and obtain a fast response time. In addition, we will investigate how to effectively reduce the noisy signals from facial movements when a user is talking and looking around in the practical application. Moreover, Electrooculography (EOG) signals will be combined with EMG signals to generate the control signals for wheelchair motion control so that the control method is more flexible and robust.
Finally, it is worth remarking that Brain Computer Interface (BCI) based on the Electrooculography (EEG) signals have been used to control the electric-powered wheelchair during the last decade [17,18]. However, these EEG based HMIs are not performing well in the real world since they are non-stationary and low in signal to noise ratio. This was mainly caused by the subject’s brain conditions or dynamically changing environments. Therefore, it is necessary to carefully design signal processing algorithms for feature extraction, classification and adaptation. Alternatively, EEG signals could be fused with EMG and EOG signals for accurate human intension detection in real time.

Acknowledgments

This research is financially supported by International Science and Technology Cooperation Program of China, 2010DFA12160 and Science & Technology Research Project of Chongqing CSTC, KJ100516.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Nihei, M.; Fujie, M.G. Proposal for a New Development Methodology for Assistive Technology Based on a Psychological Model of Elderly People. In Assistive Technologies; InTech: New York, NY, USA, 2012. [Google Scholar]
  2. Pires, G.; Nunes, U. A wheelchair steered through voice commands and assisted by a reactive fuzzy-logic controller. J. Intell. Rob. Syst. 2002, 34, 301–314. [Google Scholar] [CrossRef]
  3. De Santis, A.; Iacoviello, D. Robust real time eye tracking for computer interface for disabled people. Comput. Methods Progr. Biol. 2009, 96, 1–11. [Google Scholar] [CrossRef]
  4. Jia, P.; Hu, H.; Lu, T.; Yuan, K. Head gesture recognition for hands-free control of an intelligent wheelchair. Int. J. Ind. Rob. 2007, 34, 60–68. [Google Scholar] [CrossRef]
  5. Wei, L.; Hu, H. A hybrid human-machine interface for hands-free control of an intelligent wheelchair. Int. J. Mechatron. Autom. 2011, 1, 97–111. [Google Scholar] [CrossRef]
  6. Galan, F.; Nuttin, M.; Lew, E.; Ferrez, P.W.; Vanacker, G.; Philips, J.; Millan, J.R. A brain-actuated wheelchair: Asynchronous and non-invasive Brain-computer interfaces for continuous control of robots. Clin. Neurophysiol. 2008, 119, 2159–2169. [Google Scholar]
  7. Rebsamen, B.; Guan, C.; Zhang, H.; Wang, C.; Teo, C.; Ang, M.J.; Burdet, E. A brain controlled wheelchair to navigate in familiar environments. IEEE Trans. Neural Syst. Rehabil. Eng. 2010, 18, 590–598. [Google Scholar] [CrossRef]
  8. Maskeliunas, R.; Simutis, R. Multimodal wheelchair control for the paralyzed people. Electron. Electr. Eng. 2011, 111, 81–84. [Google Scholar]
  9. Hashimoto, M.; Takahashi, K.; Shimada, M. Wheelchair Control Using an EOG-and EMG-Based Gesture Interface. In Proceedings of IEEE/ASME International Conference on Advanced Intelligent Mechatronics, Singapore, 14–17 July 2009; pp. 1212–1217.
  10. Santana, A.; Yang, C. Robotic control using physiological EMG and EEG signals. Adv. Auton. Rob. 7429, 449–450. [Google Scholar]
  11. Brando, A.S.; Felix, L.B.; Bastos-Filho, T.F.; Sarcinelli-Filho, M. Controlling devices using biological signals. J. Adv. Rob. Syst. 2011, 8, 22–33. [Google Scholar]
  12. Englehart, K.; Hudgins, B. A robust, real-time control scheme for multifunction myoelectric control. IEEE Trans. Biomed. Eng. 2003, 50, 848–854. [Google Scholar] [CrossRef]
  13. Moon, I.; Lee, M.; Chu, J.; Mun, M. Wearable EMG-Based HCI for Electric-Powered Wheelchair Users with Motor Disabilities. In Proceedings of the 2005 IEEE International Conference on Robotics and Automation (ICRA 2005), Barcelona, Spain, 18–22 April 2005; pp. 2649–2654.
  14. Gandevia, S.C. Spinal and supraspinal factors in human muscle fatigue. Physiol. Rev. 2001, 81, 1725–1789. [Google Scholar]
  15. Tamura, H.; Murata, T.; Yamashita, Y.; Tanno, K.; Fuse, Y. Development of the electric wheelchair hands-free semi-automatic control system using the surface-electromyogram of facial muscles. Artif. Life Rob. 2012, 17, 300–305. [Google Scholar] [CrossRef]
  16. Zhang, Q.; Hayashibe, M.; Fraisse, P.; Guiraud, D. FES-induced torque prediction with evoked EMG sensing for muscle fatigue tracking. ASME Trans. Mechatron. 2011, 16, 816–826. [Google Scholar] [CrossRef]
  17. VØllestad, N.K. Measurement of human muscle fatigue. J. Neurosci. Methods 1997, 74, 219–227. [Google Scholar] [CrossRef]
  18. Williams, C. Human Muscle Fatigue; Routledge: New York, NY, USA, 2009. [Google Scholar]
  19. Oskoei, M.; Hu, H. Myoelectric control systems: A survey. J. Biomed. Signal Process. Control 2007, 2, 275–294. [Google Scholar] [CrossRef]
  20. Nikooyan, A.A.; Zadpoor, A.A. Effects of muscle fatigue on the ground reaction force and soft-tissue vibrations during running: A model study. IEEE Trans. Biomed. Eng. 2012, 59, 797–804. [Google Scholar] [CrossRef]
  21. Alty, S.R.; Georgakis, A. Mean Frequency Estimation of Surface EMG Signals Using Filterbank Methods. In In Proceedings of the 19th European Signal Processing Conference (EUSIPCO 2011), Barcelona, Spain, 29 August–2 September 2011.
  22. Gerdle, B.; Elert, J. The temporal occurrence of the mean power frequency shift of the electromyogram during maximum prolonged dynamic and static working cycles. Int. J. Sports Med. 1994, 15, 32–37. [Google Scholar] [CrossRef]
  23. Lucas, M.F.; Gaufriau, A.; Pascual, S.; Doncarli, C.; Farina, D. Multi-channel surface EMG classification using support vector machines and signal-based wavelet optimization. J. Biomed. Signal Process. Control 2008, 3, 169–174. [Google Scholar] [CrossRef]
  24. Kawano, S.; Okumura, D.; Tamura, H.; Tanaka, H.; Tanno, K. Online learning method using support vector machine for surface-electromyogram recognition. J. Artif. Life Rob. 2009, 13, 483–487. [Google Scholar] [CrossRef]
  25. Vapnik, V. The Nature of Statistical Learning Theory; Springer: Heidelberg, Germany, 1999. [Google Scholar]
  26. Chang, C.C.; Lin, C.J. LIBSVM: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2011, 2, 27:1–27:27. [Google Scholar]

Share and Cite

MDPI and ACS Style

Xu, X.; Zhang, Y.; Luo, Y.; Chen, D. Robust Bio-Signal Based Control of an Intelligent Wheelchair. Robotics 2013, 2, 187-197. https://doi.org/10.3390/robotics2040187

AMA Style

Xu X, Zhang Y, Luo Y, Chen D. Robust Bio-Signal Based Control of an Intelligent Wheelchair. Robotics. 2013; 2(4):187-197. https://doi.org/10.3390/robotics2040187

Chicago/Turabian Style

Xu, Xiaodong, Yi Zhang, Yuan Luo, and Dongyi Chen. 2013. "Robust Bio-Signal Based Control of an Intelligent Wheelchair" Robotics 2, no. 4: 187-197. https://doi.org/10.3390/robotics2040187

Article Metrics

Back to TopTop