Enhancing the Security of Pattern Unlock with Surface EMG-Based Biometrics

: Pattern unlock is a popular screen unlock scheme that protects the sensitive data and information stored in mobile devices from unauthorized access. However, it is also susceptible to various attacks, including guessing attacks, shoulder surﬁng attacks, smudge attacks, and side-channel attacks, which can achieve a high success rate in breaking the patterns. In this paper, we propose a new two-factor screen unlock scheme that incorporates surface electromyography (sEMG)-based biometrics with patterns for user authentication. sEMG signals are unique biometric traits suitable for person identiﬁcation, which can greatly improve the security of pattern unlock. During a screen unlock session, sEMG signals are recorded when the user draws the pattern on the device screen. Time-domain features extracted from the recorded sEMG signals are then used as the input of a one-class classiﬁer to identify the user is legitimate or not. We conducted an experiment involving 10 subjects to test the effectiveness of the proposed scheme. It is shown that the adopted time-domain sEMG features and one-class classiﬁers achieve good authentication performance in terms of the F 1 score and Half of Total Error Rate (HTER). The results demonstrate that the proposed scheme is a promising solution to enhance the security of pattern unlock.


Introduction
Mobile usage has grown tremendously in recent years. Mobile usage is surpassing desktop usage, as mobile devices are being widely used for a variety of online tasks, including web surfing, online shopping, mobile banking, and social media [1]. Fast-growing mobile usage has been leaving lots of sensitive private data and information in mobile devices. These data and information need to be protected against unauthorized access of the devices.
Screen unlock is a popular mechanism used to secure the mobile device from unauthorized access. A number of screen unlock schemes have been provided by popular mobile platforms (iOS and Android), including the slide, PIN, password, pattern, fingerprint, and face. Among them, pattern unlock provided by the Android platform has been widely adopted by mobile users. It was shown in a recent study [2] that about 40% of Android users use pattern unlock to secure their devices. Another study showed that pattern unlock was preferable over the password and PIN by mobile users, although the PIN outperforms patterns in terms of input speed and error rate [3].
Although pattern unlock is a convenient scheme for protecting the user's device, it is also susceptible to a number of attacks, including guessing attacks [4], shoulder surfing attacks [5], smudge attacks [6], and side-channel attacks [7,8]. It was shown in [6] that smudge attacks were able to break the pattern up to 68% of the time. The success rate of guessing attacks can be greatly improved from 13.33% to 74.17% when assisted by smudge attacks [4]. The study of [5] showed that patterns are more memorable and easier to be spied on through shoulder surfing attacks compared with PINs.
Ye et al. [8] developed a computer vision algorithm that can track the user's finger movement to infer the pattern drawn on the screen. Their approach can break over 95% of the patterns in five attempts before the device is automatically locked by the Android system. Thus, there is a great need to enhance the security of pattern unlock to defend the attacks.
In this paper, we propose a new two-factor screen unlock scheme that incorporates biometric information with patterns to achieve improved security. The proposed scheme not only verifies "what you possess" (pattern), but also "who you are" (unique biometric information) to authenticate the mobile user. Specifically, we use surface electromyography (sEMG)-based biometric information to enhance the security of pattern unlock. sEMG signals are generated by the electric activities of muscles and are recorded by the electrodes placed on the skin's surface. Since the sEMG signals are electric potentials generated by muscle cells, the same movements of different people will generate different electric potentials. Compared with existing biometric information used for screen unlocking, such as the fingerprint and face, sEMG signals are much harder to be forged, which makes the proposed scheme secure against attacks. Even the attackers can break the patterns using the aforementioned attacks, and they will not be able to forge the sEMG signals to gain access of the mobile device.
The rest of this paper is organized as follows. The related work on improving the security of pattern unlock and biometric-based mobile user authentication is described in Section 2. Section 3 introduces the background information of pattern unlock and EMG signals. The proposed method and experiment procedure are provided in Section 4. The experimental results are presented in Section 5. Finally, Section 6 concludes the paper.

Improving the Security of Pattern Unlock
There are a number of schemes which have been proposed up to now to improve the security of pattern unlock. To defend against smudge attacks, Kwon et al. [9] proposed TinyLock, an affordable defense against smudge attacks. They made the size of the pattern grid very small and introduced a virtual wheel to wipe off the actual smudge. However, due to the tiny size grid, it has several drawbacks. Because the grid dots are not visible to the users, the error rate is very high, and it is very difficult to draw complex patterns. Concepts where the position of the pattern grid is translated to a different position, scale, and rotation on the screen for each unlock attempt have been explored by Schneegass et al. [10] and von Zezschwitz et al. [11]. However, they reported that users experienced difficulty in locating the grid due to its varying location. M-Pattern is a scheme which was proposed in [12] that employs multiple patterns to unlock the screen. The system randomly pick a pattern that can be inferred by the user from a visual cue, such as a background image. Since the smudges created by M-Pattern overlapped with each other, it was much harder for the attackers to recover the patterns compared with pattern unlock. The usability of the M-Pattern is compromised due to the memorization of multiple patterns by the user.
Besides smudge attacks, there are schemes proposed to defend against other attacks. Higashikawa et al. [13] proposed a shoulder-surfing-resistant scheme that uses a pass pattern of pattern unlock. The scheme utilizes challenge-and-response authentication and the user's short-term memory. In [14], a scheme called Pass-O was proposed, which employs a circular layout of the grid. Pass-O has a pattern space almost 2.5 times larger than pattern unlock, which makes it harder for guessing attacks. Chiang et al. [15] proposed a multi-layered drawing lock mechanism which greatly increases the pattern space compared with pattern unlock. Warp cells at the corners of the grid enable more complex patterns by using multiple layers. For example, when a warp cell is touched as part of pattern entry, a second empty grid layer is displayed, obscuring the original grid layer, on which the pattern entry can continue.
Some other schemes were also proposed to improve the security by encouraging users to create more complex patterns. Sun et al. [16] proposed a formula to calculate the pattern strength quantitatively.
The quantitative strength score is displayed through a pattern strength meter when the user creates a pattern. Their study showed that users created more complex patterns with the help of a pattern strength meter. TinyPal is an enhanced interface proposed in [17] for pattern unlock, which highlights the set of available dots that can be connected from the currently connected dot. TinyPal helps users create patterns containing more visual features, like overlaps and knight moves, to resist attacks. Their experimental results showed that users created more secure patterns using TinyPal than those using the original interface.

Biometrics-Based Mobile User Authentication
Biometrics have been widely used in the user authentication of mobile devices. The Apple Touch ID utilizes fingerprints to unlock the user's device and authenticate Apple Pay. However, it was shown in [18] that fingerprints can be easily forged. Face recognition is used by Apple Face ID and Android Face Unlock to unlock the screen, which is susceptible to attacks, such as using a photo or video of the authorized user.
Besides the traditional biometrics like the fingerprint and face, behavioral biometrics have also been used for user authentication of mobile devices. In [19], keystroke dynamics were combined with the PIN as multifactor authentication for the mobile user. Data from the motion sensors, such as the accelerometer, gyroscope, and rotation was included to improve the performance of authentication. Lamiche et al. [20] proposed a scheme that uses the gait pattern from the accelerometer and keystroke dynamics of text input for continuous authentication. The scheme was proved to be secure against the zero-effort attack and minimal-effort mimicking attack. SwipeVLock is a screen unlock mechanism proposed in [21] based on swipe dynamics. Touch features, such as the coordinates of location, touch pressure, touch size, touch time, and touch speed were used to model the swipe behavior.
Recently, the use of physiological biometrics for mobile user authentication has increased significantly. Huang et al. [22] proposed a privacy-preserving ECG-based authentication scheme that utilizes the ECG signal acquired by a medical IoT for ECG monitoring to authenticate the patient. A mobile EEG-based biometric authentication system was developed in [23] that combines EEG recordings with other techniques, such as nearfield communication (NFC) and face recognition. Kumar et al. [24] proposed a framework to enhance the pattern-based authentication with EEG signals. The Hidden Markov Model (HMM) was applied to model the EEG signals recorded during pattern drawing. The Support Vector Machine (SVM) classifier was then used to verify the authenticity of the drawn pattern. In the studies of [25,26], sEMG signals generated from a list of gestures were used for mobile user authentication. A list of gestures is used as the password, which is combined with the features extracted from the sEMG signals to defend against shoulder-surfing attacks.
In this paper, we propose to enhance the security of pattern unlock with sEMG-based biometrics. To the best of our knowledge, the proposed scheme is the first work that incorporates sEMG-based biometrics with pattern unlock for mobile user authentication.

Background
In this section, we provide the background information of pattern unlock and EMG signals.

Pattern Unlock
Pattern unlock was introduced by Google in 2008 as a graphic-based screen unlock scheme for Android devices. Instead of entering a PIN or typing a password, the user draws a pattern by connecting dots in a 3 × 3 grid. Figure 1 shows the pattern unlock grid with the dots labeled from 1 to 9. A pattern is represented as a sequence of connected dots, such as the pattern shown in Figure 1 is A pattern has to follow the following rules to be valid: (1) It has to connect at least 4 dots; (2) each dot can only be connected once; (3) a pattern always connects the first unconnected dot along its path; (4) a pattern can go through a previously connected dot to connect an unconnected dot. Based on these rules, it has been shown that the total number of valid patterns is 389,112 [6,16], which is less than the number of possible passwords for a four-character, case-insensitive alphabetic password. This indicates that pattern unlock is relatively weak in security compared with other schemes, such as the PIN and password, in terms of pattern space.

EMG Signals
EMG signals record the electric activities produced by skeletal muscles, which usually have a potential difference. Therefore, when recording EMG signals, at least one pair of electrodes are needed. Sometimes a more complex array of multiple electrodes is used to record the activities of more than one muscle.
There are two types of EMG: surface EMG (sEMG) and intramuscular EMG (imEMG). sEMG signals are obtained by measuring muscle activities on the skin surface above the muscle. On the contrary, imEMG signals are recorded from the muscle tissue which are acquired by percutaneous wire needle electrodes inserted into muscle. Compared with imEMG, sEMG is convenient to acquire and is non-invasive. In this study, we used sEMG signals recorded from the forearm muscle as the biometric information for user authentication.

Materials and Methods
In this paper, we propose a new two-factor screen unlock scheme that utilizes sEMG-based biometric information to enhance the security of pattern unlock. In the following, we first provide an overview of the proposed scheme. The subjects recruited for this study are then introduced, followed by the description of the experiment protocol. The experiment serves as a proof-of-concept for the proposed scheme. Next, we present the features extracted from the sEMG signals. Finally, the one-class classification algorithms for user classification are introduced.

Overview of Proposed Scheme
The workflow of the proposed two-factor screen unlock scheme is illustrated in Figure 2. During the screen unlocking process, the user draws the unlock pattern on the screen of the device while the sEMG signals of the user are simultaneously recorded. The pattern drawn on the screen is first checked regarding whether it is the same as the one set by the authorized user or not. The access is denied if it is the wrong pattern. Otherwise, the recorded sEMG signals will further be used to verify the user. The verification is performed by extracting the time domain features from the recorded sEMG signals first. The features are then used as the input of a one-class classifier to classify the user as legitimate or not. Note that the one-class classifier is trained only using the data from the authorized user.

Subjects
This study was approved by the Institutional Review Board (IRB) of New Mexico Institute of Mining and Technology. We recruited 10 subjects from the school (7 males and 3 females, age = 23.8 ± 2.5 years, height = 173.9 ± 9.7 cm, weight = 70.0 ± 15.6 kg, all right-handed) for this study. All participants reported that they did not have upper limb musculoskeletal and nervous system diseases.

sEMG Signal Acquisition and Pre-Processing
The experiment setting is shown in Figure 3. The raw sEMG signal is recorded by a gold cup electrode (OpenBCI) placed on the FDS (Flexor Digitorum Superficialis) muscle of the forearm, which is connected to a Cyton biosensing board (OpenBCI). It is known that the activation of FDS corresponds to finger flexion [27]. The electrode placement of the experiment is shown in Figure 4. The sampling rate of the Cyton board is 250 Hz. The acquired sEMG signal is then sent from the Cyton board to the Cyton Dongle (OpenBCI), a Bluetooth adaptor plugged into a laptop. The OpenBCI GUI software installed on the laptop then records the acquired data in the local storage.
The quality of the recorded raw sEMG signals was affected by noises, such as direct current offset, environment noises, and artifact noises. We applied a 5Hz high-pass filter to eliminate the direct current offset, baseline drift due to the movement in recording, and perspiration. A 60Hz notch filter was then applied to filter out the power-line noise. Figure 5 shows an example of recorded sEMG signal after pre-processing.

Experiment Procedure
During the experiment, each subject was comfortably seated on a chair before a table. The electrodes were placed on the subject's forearm, as shown in Figure 4. Before electrodes were placed, an alcohol pad was used to clean the skin of the electrode sites. The electrode cap gel was applied to the cleaned skin's surface to reduce electrode-skin impedance for better recording.
After the setting was done, the subject was instructed to draw the two patterns shown in Figure 6. Each pattern was repeated for 20 trials. sEMG signals were recorded for each trial. The subject was instructed to rest the arm on the table for one minute before the next trial to avoid muscle fatigue. The whole process for a subject lasted about one hour and twenty minutes, which consisted of 40 trials for the two patterns.

sEMG Feature Extraction
There are three types of features that can be extracted from sEMG signals: time domain features, frequency domain features, and time-frequency domain features [28]. In our study, we concentrated on using features extracted from the time domain, which are widely used in studies and practices due to their low computational complexity compared with frequency domain and time-frequency domain features and performance in low-noise environments [29][30][31][32]. We selected the following 11 time domain features in this study.

Mean Absolute Value (MAV)
MAV is one of the most popular time domain features for EMG signal analysis [28]. It has been widely used as an onset detection index for applications like prosthetic limb control [33]. MAV is calculated as the mean absolute value of the EMG signal in a segment, as shown in Equation (1), where N is the number of samples in the segment, and X i is the ith sample of the segment.

Variance (VAR)
VAR is a measure of the power of the EMG signal [34]. Since the mean value of the EMG signal is close to zero, VAR is defined as:

High-Order Temporal Moment (TM)
The temporal moment (TM) was proposed in [35] as a statistical analysis of EMG signals for prosthetic arm control. The first-order TM is the same as MAV, and the second-order TM is similar to VAR. High-order TMs from Order 3 to Order 5 (TM3, TM4, and TM5) used in [35] were employed in our study. They are defined in Equations (3)- (5). Note that for the calculation of odd moments, the absolute value was taken to reduce within-class separation [35].

Mean Square Root (MSR)
The Mean Square Root (MSR) is a time-domain feature proposed in [31] for limb motion classification. The feature provides an estimation of the total amount of activity.

Root Mean Square (RMS)
The Root Mean Square (RMS) is another popular feature for EMG pattern recognition [30,32], which is related to constant force and non-fatiguing contraction. RMS is similar to the standard deviation of the EMG signal, as the mean of the signal is close to zero.
4.4.6. Log Detector (LD) The Log Detector (LD) is a non-linear detector that provides an estimation of the muscle contraction force [34]. LD is defined in Equation (8) where exp(·) is the exponential function.

Waveform Length (WL)
Waveform Length (WL) is a measure of the waveform complexity of the EMG signal [33], which is defined as the cumulative length of the waveform over the segment, as shown in Equation (9).

Difference Absolute Standard Deviation Value (DASDV)
The Difference Absolute Standard Deviation Value (DASDV) is a feature similar to RMS, which uses the difference value between two adjacent samples instead of the sample value [36].

Number of Zero Crossing (NZC)
Number of Zero Crossing (NZC) is a time-domain feature that measures the frequency information of an EMG signal [33]. It is defined as the number of times the signal crosses zero. A threshold, T, was included in the calculation to reduce the effect of noise-induced zero-crossings [33].

One-Class Classification Algorithms
For the purpose of user authentication, all users were considered as abnormal except for the authorized user. One-class classification algorithms are suitable for the task, as they are trained only using the data from the authorized user. In this study, two popular one-class classification algorithms were employed: the one-class SVM (OCSVM) and local outlier factor (LOF).

OCSVM
SVM is a widely used supervised learning algorithm for multi-class classification. It was extended for the one-class classification problem in [37,38]. OCSVM looks for a non-linear decision boundary by mapping the training samples x i (i = 1, 2, ..., n) into a higher dimension feature space using a non-linear kernel map function Φ(·). A hyperplane was then calculated to separate the mapped training samples from the origin with maximal margin, which will be used as the decision boundary. The quadratic programming minimization function used to separate the training samples from the origin is defined below, where ω is the normal vector to the hyperplane, ξ is for penalization, ρ is the margin, and ν ∈ (0, 1) is a parameter to control the ratio of anomalies in the training samples.
After solving the problem with ω and ρ, the decision function is defined in Equation (13). A new sample y was classified as an anomaly if f (y) was negative.
There are several choices of the kernel function for OCSVM. In this study, the radial basis function (RBF) kernel was chosen, which produced the best results among all kernels. The RBF kernel is defined in Equation (14), where ||x − x || 2 is the squared Euclidean distance between two data points x and x , and σ is a parameter related to the decision region. A smaller σ leads to a broader decision region.

LOF
LOF is a popular anomaly detection algorithm proposed in [39]. LOF is based on the idea of local reachability density (LRD) which is calculated as the inverse of the average distance from a sample to its k nearest neighbors. The LOF score of a sample x is then calculated as the degree of x isolated from its k nearest neighbors, as shown in Equation (15). A LOF score larger than one indicates that the sample is an anomaly.
where N k (x) is the set of k nearest neighbors for x.

Dataset
During the experiment, we collected the sEMG signals of 10 subjects when they were drawing unlock patterns. For each subject, 20 trials of sEMG signals were recorded for each of the two patterns shown in Figure 6. A 6s signal segment was extracted from each trial, starting from the time when the subject started the drawing of the unlock pattern. Each segment was further divided into a series of overlapping analysis windows with a window length of 2 s and a 1 s overlapping, which is shown in Figure 7. The 11 time domain features described in Section 4.4 were then calculated for each analysis window. A total of 55 features were extracted for each segment, which is an instance of a dataset. Finally, we obtained two datasets for the two patterns. Each dataset contains 200 instances for the 10 subjects.

Performance Metrics
The proposed scheme was evaluated using two popular metrics for measuring the performance of authentication systems: F 1 score and Half of Total Error Rate (HTER). Here, the patterns from the authorized user or client were treated as negatives, while the patterns from the unauthorized users or impostors were positives. The two metrics were calculated from the four elements of the confusion matrix, true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN). TP and TN are the numbers of correctly classified patterns from the impostors and client, respectively. FP and FN are the numbers of misclassified patterns from the client and impostors, respectively. F 1 score is a widely used metric for evaluating the performance of classifiers which is defined as the harmonic mean of precision and recall, as shown in Equation (16).
HTER has been used for evaluating the performance EEG-based authentication systems [24,40]. HTER is defined as the average of the False Acceptance Rate (FAR) and False Rejection Rate (FRR): FAR is the ratio of the number of false acceptances to the number of attempts from the impostors. FRR is the ratio of the number of false rejections to the number of attempts from the client.

Results
We tested the authentication performance using sEMG-based biometrics for each of the 10 subjects as the client. Subjects other than the client were considered as impostors for the test. For a dataset obtained in Section 5.1, we randomly selected 10 instances from the client for training. Other instances of the dataset were used for testing. The data was normalized using min-max normalization. The process was repeated 10 times for each subject. Python and the scikit-learn library [41] were used for the experiment. Tables 1 and 2 show the test results for unlock patterns 1 and 2, respectively. The mean and standard deviation of the results obtained from the 10 repeated tests on a subject are reported in the tables. It can be seen that both OCSVM and LOF achieve good performance in terms of the F 1 score, indicating that they are good at finding attempts from impostors. On the other hand, LOF has significantly better performance than OCSVM in terms of HTER, which is due to the high FRR obtained by OCSVM. Thus, LOF is a better classifier than OCSVM for our task. We also observe that the results obtained from the two patterns are similar. The performance evaluation demonstrates that using LOF with time domain features extracted from sEMG signals for user identification is a viable solution for enhancing the security of pattern unlock.

Conclusions
Pattern unlock is a popular screen unlock mechanism, but has relatively weak security compared with other mechanisms, such as the PIN and password. It is susceptible to various attacks, including guessing attacks, over-the-shoulder attacks, smudge attacks, and side channel attacks. In this paper, we proposed a new two-factor authentication scheme that utilizes sEMG-based biometrics to enhance the security of pattern unlock. The proposed scheme uses time domain features extracted from recorded sEMG signals and one-class classifiers to determine the authenticity of the user. To test the effectiveness of the proposed scheme, 10 subjects were recruited for the study. The results show that the proposed scheme achieves good authentication performance in terms of F 1 score and HTER. In addition to mobile devices, the proposed scheme has the potential to be applied for the vehicle access control system of electrical vehicles (EVs) which is projected to be a 22.6 billion market by 2027 [42]. There are several limitations of the current research. First, the sEMG acquisition system used in the experiment was for the purpose of proof-of-concept, which is not intended for real-world applications. Wearable wireless sEMG acquisition devices, such as the MyoWare Muscle Sensor [43] or Myo Armband [44] with better usability should be considered in those scenarios. The second limitation is the relatively small number of subjects recruited for the experiment. We plan to perform larger-scale testing of the proposed scheme in the future. Finally, the experiment conducted in the paper only considered the scenario of using both hands to unlock the device. In future studies, we will also consider the scenario of using only one hand to unlock the device.