1. Introduction
Amyotrophic lateral sclerosis (ALS) attacks the motor neurons. Numerous ALS patients around the world report that ALS seriously reduces the quality of their lives. ALS critically reduces their motor abilities; they experience difficulties in performing daily activities, such as participating in entertainment activities. Therefore, in recent years, an extensive range of high technology assistive devices have been developed to fulfill the needs of these individuals. These assistive devices can help these patients perform daily activities such as eating, communicating, and moving from place to place. Although assistance in performing daily activities is essential, the psychological effects of nonessential patient activities such as entertainment should also be considered. Therefore, the development of a brain-computer interface (BCI) application for entertainment is worthwhile for subjects with ALS.
A BCI is used to give commands to a computer or other device through electroencephalography (EEG) measurements taken from sensors on a human scalp. A BCI is not necessary for the implementation of any sensor into the brain, so it can easily be applied to subjects with motor neuron disease [
1]. This technology can effectively and easily help subjects with ALS to communicate with the outside world [
2]. A BCI can be developed using P300 waves, motor imagery, event-related potential, visual evoked potential, and steady-state visual evoked potential (SSVEP) [
3,
4,
5,
6,
7,
8,
9]. A P300-based BCI is operated by asking the user to select a row and a column from a command matrix [
4]. BCI applications can integrate several methods, such as P300 and SSVEP [
5,
6]. A suitable BCI for subjects with ALS may enable those subjects to operate various devices, including entertainment devices.
SSVEP technology has been utilized in BCI applications by using visual stimuli to induce human brain waves and trigger brain potentials [
3]. Several boxes with different flickering frequencies can be used as stimuli for SSVEP applications [
7]. Other researchers have used several boxes that flickered at the same frequency but with different phases as stimuli for an SSVEP application [
8]. The light source that was used as a stimulus also varied. Some research has used liquid-crystal display (LCD) monitors as stimuli, and other research has used LEDs as stimuli. A SSVEP-based BCI is one of the easiest interfaces to implement for subjects with ALS.
Several studies have been proposed to develop BCI applications for entertainment. A BCI application was designed to help people with severe disabilities play tennis games on a computer [
10]. The use of consumer-grade equipment rather than research lab-grade equipment for BCI gaming applications can improve public awareness of BCI technology. SSVEP technology can be adopted to provide excellent control when applied in a tactical video game [
11]. Another SSVEP-based BCI application was used to control an avatar in a maze game. Four commands triggered by responses to stimuli, namely up, down, left, and right, controlled the movement of an avatar in a maze [
12]. Phase-tagging SSVEP has been implemented to control spacecraft movement to avoid obstacles. Two stimuli with 180° phase differences and frequencies in the 3–5 Hz range were used to produce inputs for controlling left or right movements [
13].
In general, most fuzzy recognition algorithms contain three processes: fuzzification, inference rules, and defuzzification. In fuzzy recognition algorithms, fuzzy sets and fuzzy logic are used for heuristic quantification of the meanings of linguistic variables, linguistic values, and linguistic rules that are specified by experts. The concept of a fuzzy set can be introduced by first defining a “membership function.” Fuzzy sets are used to quantify the information in the rule-base, and the inference mechanism operates on fuzzy sets to produce fuzzy sets; hence, the fuzzification process converts numeric inputs into fuzzy sets. The inference mechanism has two basic tasks: (1) determining the extent to which each rule is relevant to the current situation, as characterized by the inputs; and (2) drawing conclusions by using the current inputs and the information in the rule-base. The defuzzification process generates the output of fuzzy inference, which is a crisp value for a controlled system. The fuzzy recognition algorithm is simple and fast; thus, it is suitable for a real-time SSVEP-based BCI system.
EEG signals are small, varying, and complex, so accurately recognizing EEG features is extremely difficult. Fuzzy sets can easily be installed in single-chip microprocessors. Single-chip fuzzy inference systems have been widely used in numerous applications, especially for real-time systems [
14]. Lou and Loparo combined wavelet methods and fuzzy algorithms to implement a bearing fault detection system [
15]. An adaptive neurofuzzy inference system (ANFIS) was used to classify breast mass [
16]. Güler and Übeyli used ANFIS to detect electrocardiographic (ECG) changes in patients with partial epilepsy [
17] and classify EEG signals [
18]. However, an ANFIS must collect a training database for parameter estimation of a neural network. The performance of an ANFIS would be degraded when its user’s EEG data is not collected. Achieving real-time recognition for a BCI-based application is essential, as is adapting the system so that it can quickly fit a new user.
In this study, a fuzzy tracking and control algorithm is proposed for helping subjects with ALS use a BCI remote control system. For accurate representation of EEG signal characteristics, a fast Fourier transform (FFT) was applied to extract the power spectrum, which involves the features for recognition. To quickly track the changes of EEG signal characteristics, a fuzzy tracking algorithm was developed. To improve the accuracy and stability of this BCI system, a fuzzy control algorithm is proposed. To help subjects with ALS operate remote-controlled devices, a BCI-based air swimmer drone vehicle was developed.
The remainder of this paper is organized as follows.
Section 2 describes the fuzzy tracking and control algorithms and discusses visual stimulation and signal acquisition.
Section 3 presents the results of a series of experiments and evaluates the performance of our approach. Finally, conclusions are drawn in
Section 4, and possible improvements for the future development of this work are discussed.
3. Results and Discussion
In this study, the EEG signals were recorded at a sample rate of 1k Hz and a bit resolution of 22 bits. A second-order Butterworth band-pass filter was used to remove the signals with frequencies lower than 4 Hz and higher than 60 Hz. The window size of the Hamming window was 1000 points, which was then applied to obtain a sequence of trials. Subsequently, the power energies for each visual stimulus were estimated using an FFT with 4096 points and triangular windows with 2-Hz bandwidths.
3.1. Performance of the Fuzzy Tracking and Control Algorithm
This subsection reports that fifteen healthy subjects (11 males and four females) aged between 21 and 23 were asked to participate for evaluating the fuzzy tracking and control algorithm. To collect the training data, we developed a visual stimulation procedure comprised of five sets of stimulation sequences. Each set of stimulation sequence consisted of three stimulation frequencies that were randomly generated from the five given frequencies (6 Hz, 6.67 Hz, 10.00 Hz, 8.57 Hz, and 7.50 Hz). Each set of stimulation sequence consisting of three stimulation frequencies followed the following procedure: Each set began with a five-second countdown delay followed by a series of a ten-second visual stimulation and ten seconds of rest. Afterwards, one minute of compulsory rest time was provided for the subject after every set of stimulation sequence. The acquired EEG signals during every ten-second visual stimulation were then blocked into ten non-overlapping frames, and the duration of each frame was one second.
Because the membership functions of fuzzification and defuzzification can greatly affect the accuracy, validating the factors of membership functions such as their shapes and the ranges is crucial. For fuzzy-based approaches, the triangular function, trapezoidal function, generalized bell curve function, symmetric Gaussian function, and a two-sided version of the Gaussian are widely used [
22]. Therefore, these five functions were selected as the membership functions and denoted as TriaMF, TrapMF, GbellMF, GaussMF, and Gauss2MF, respectively. The range of each membership function in fuzzification and defuzzification was examined from 5 to 50, and the results are shown in
Figure 6. The results show that the recognition rates converged when the range of the membership function was greater than 20 for each membership function. When the range of the membership function did not suffice to model the features, the recognition rates were seriously degraded.
The optimal recognition rates for the five membership functions are shown in
Table 2. The results indicate that the recognition rates for different membership functions were similar. The computational time of the proposed approach was not affected by the RD or the RF, but it was greatly affected by membership functions with different levels of complexity. Since TrapMF, GbellMF, GaussMF, and Gauss2MF are second-order polynomial equations; their computational complexity are similar. However, the computational complexity of TraiMF was lower than those of the other membership functions. The difference between the optimal recognition rate (GaussMF) and the pessimal recognition rate (TriaMF) was only 0.6%; such a level of error can be ignored in numerous applications. Therefore, the TriaMF was very suitable for real-time applications and was selected in this study.
3.2. Comparison with CCA
For comparison with the proposed approach, the CCA was selected as the baseline system [
23]. For the
mth stimulus frequency
fm, a preconstructed reference signal set was formed using a series of sin-cosine waves, which can be written as follows:
where
.
H,
P, and
F are the number of harmonics, the number of points for an input signal, and the sampling rate, respectively. The number of harmonics,
H, was 4 in this experiment. The experimental results of the fifteen subjects are shown in
Figure 7. The recognition rates were 94.49% and 96.97% for the baseline and the fuzzy approach, respectively. The recognition rate of the proposed approach for each subject was clearly higher than that of baseline. To examine the experimental results, a Wilconxon's signed-rank test was adopted, and the two-tailed test
p-value was approximately 0.002. Thus, at a signification level of 0.01, the results we obtained surely indicate that the recognition rate of the proposed approach is higher than that of CCA. Therefore, the proposed approach is an excellent solution for correctly recognizing the frequency response of the visual stimulation.
To analyze the effects of the visual stimulus frequency, the experimental results for different stimulus frequencies are shown in
Figure 8. The results indicate that, when the stimulus frequency increased, the recognition rates were slightly degraded. To determine the cause, the normalized amplitude at the stimulus frequency, which reflects the SNR of the SSVEP, can be calculated as follows:
where
x is the preprocessed EEG data and
FFT(
x) is the fast Fourier transform of
x.
denotes summing over the total frequency points of the spectrum; thus, the sum of the amplitude spectrum is normalized to one [
24]. The results for one subject are shown in
Figure 9. The peak of the stimulus frequency can be clearly identified for that subject. However, the SNR of the peak value for a high stimulus frequency is lower than that for a low stimulus frequency. Therefore, if the stimulus frequency is increased, the performance is degraded. For the proposed approach, the adverse effects can be reduced by automatically adapting the threshold for making the decision.
3.3. Real-Time Application
In this subsection, the fuzzy tracking and control algorithm was applied to control an air swimmer drone vehicle for subjects with ALS in entertainment. The air swimmer drone vehicle (
Figure 10) could be controlled to swim through the air with incredibly smooth and life-like swimming motions. The BCI layout (
Figure 2) was displayed on a 19-inch LCD monitor (4:3), and the average distance between two centers of two adjacent flickering boxes was 13.22 cm. The range of the input triangle membership function was 40, and the range of the output triangle membership function was 28. An example of stimulus frequency response for a subject is shown in
Figure 11a. In this example, a subject was asked to control the air swimmer drone vehicle with a sequence of stimulation frequencies that correspond to the commands "dive" (6 Hz), "turn left" (7.5 Hz), "go forward" (8.57 Hz), "turn right" (6.67 Hz), and "elevate" (10 Hz). The subject was requested to focus on a single command for a duration of five seconds, and EEG data of the subject was acquired every second without any time interval in between two adjacent trials. The corresponding recognition results are shown in
Figure 11b. The results revealed that the proposed approach can successfully convert the SSVEP EEG signals into corresponding real-time control signals.
To evaluate the proposed approach in a real-time application, five subjects with ALS (four males and one female; age between 21 and 37) were asked to operate the air swimmer drone vehicle. For each subject, the rate of progression was measured using a revised ALS functional rating scale (ALSFRS-R) [
25]. The scale of ALSFRS-R is composed of 12 items graded from 0 (complete loss of function) to 4 (normal function), with a score range between 0 (unable to perform the tested functions) and produces a score between 48 (normal function) and 0 (severe disability). The average ALSFRS-R score of five subjects was 29.8. The participants gave informed consent, and the study was approved by Institutional Review Board of National Cheng Kung University Hospital.
Since the proposed application is to relax subjects with ALS by operating the air swimmer drone vehicle, the entertainment value is more important than the recognition rate. Thus, to quantify the entertainment value of the proposed system, the caregivers helped us to ask each subject to answer the question "Is the system interesting?" Then, each subject was asked to provide a mean opinion score (MOS) ranging from 5 (excellent) to 1 (unsatisfactory). The average MOS was 4.3. Thus, the subjects’ responses indicate that the proposed system can hold the interest of subjects with ALS.