Multi-Session Electrocardiogram–Electromyogram Database for User Recognition

Abstract

Current advancements in biosignal-based user recognition technology are paving the way for a next-generation solution that addresses the limitations of face- and fingerprint-based user recognition methods. However, existing biosignal benchmark databases (DBs) for user recognition often suffer from limitations, such as data collection from a small number of subjects in a single session, hindering comprehensive analysis of biosignal variability. This study introduces CSU_MBDB1 and CSU_MBDB2, databases containing electrocardiogram (ECG) and electromyogram (EMG) signals from diverse experimental subjects recorded across multiple sessions. These in-house DBs comprise ECG and EMG data recorded in multiple sessions from 36 and 58 subjects, respectively, with a time interval of more than one day between sessions. During the experiments, subjects performed a total of six gestures while comfortably seated at a desk. CSU_MBDB1 and CSU_MBDB2 consist of three identical gestures, providing expandable data for various applications. When the two DBs are expanded, ECGs and EMGs from 94 subjects can be used, which is the largest number among the multi-biosignal benchmark DBs built by multi-sessions. To assess the usability of the constructed DBs, a user recognition experiment was conducted, resulting in an accuracy of 66.39% for ten subjects. It is important to emphasize that we focused on demonstrating the applicability of the constructed DBs using a basic neural network without signal denoising capabilities. While this approach results in a sacrifice in accuracy, it concurrently provides substantial opportunities for performance enhancement through the implementation of optimized algorithms. Adapting signal denoising processes to the constructed DBs and designing a more sophisticated neural network would undoubtedly contribute to improving the recognition accuracy. Consequently, these constructed DBs hold promise in user recognition, offering valuable research for future investigations. Additionally, DBs can be used in research to analyze the nonlinearity characteristics of ECG and EMG.

1. Introduction

In today’s advanced society, user recognition technologies are increasingly important for safeguarding personal information. Among these technologies, biosignal-based user recognition stands out as a solution to the shortcomings of conventional methods, such as facial, fingerprint, and iris recognition, which are susceptible to replication. This technology is actively researched as the next-generation approach to user recognition [1]. Biosignals encompass information that measures the microcurrents generated by human physical activity, including electrocardiogram (ECG), electromyogram (EMG), and electroencephalogram (EEG) signals. As biosignals exhibit the unique physiological characteristics of an individual, they remain imperceptible to the naked eye from the outside. Leveraging the advantages of being unforgeable and variable, biosignals address the challenges of conventional user recognition methods [2].

To measure such biosignals, a sensor must be attached to the body, as shown in Figure 1 [3]. ECG is a biosignal originating from the heartbeat, producing a signal composed of PQRST waves. ECG signals can be acquired from both hands and both feet following the standard 12-lead method, as shown in Figure 1a. EEG is a biosignal generated by brain activity and can be measured at specific locations using the international 10–20 system, positioned based on front–back or left–right distance on the skull, as shown in Figure 1b. EMG is a signal that measures the microcurrents generated when a muscle moves, and the signal can be acquired by attaching a sensor to a muscle in the body, as shown in Figure 1c. Since biosignals are measured by attaching sensors to the body, subjects may experience discomfort and repulsion when constructing the database (DB). To conduct user recognition research using biosignals, a substantial DB (comprising a large number of subjects and repetitions) is essential. However, open access benchmark biosignal DBs used in previous studies have typically featured a limited number of subjects and repetitions. Furthermore, despite the fluctuation in biosignals over time, data in most benchmark biosignal DBs are recorded in a single session (usually one day or less), posing the challenge of limited analyzability.

Figure 1. Common biosignal measurement methods. (a) ECG measurement method. (b) EEG measurement method. (c) EMG measurement method.

To solve this issue, we present two large Chosun University Biosignal Databases (CSU_BIODBs) designed for user recognition in this study. These databases, referred to as CSU_MBDB1 and CSU_MBDB2, encompass datasets that concurrently acquire both ECG and EMG signals. CSU_MBDB1 comprises ECG and EMG data recorded over multiple sessions (2 days or more) while 36 subjects performed six hand gestures at intervals exceeding one day. Similarly, CSU_MBDB2 includes ECG and EMG data recorded across multiple sessions with 58 subjects executing six hand gestures at intervals exceeding one day. To evaluate the effectiveness of constructed DBs, we conducted a user recognition experiment.

The paper’s structure is outlined as follows: Section 2 analyzes open access benchmark DBs used in conventional biosignal-based user recognition research. Section 3 presents CSU_MBDB1 and CSU_MBDB2, two extensive multi-session electrocardiogram–electromyogram DBs introduced in this study. Section 4 analyzes the usability of the constructed benchmark biosignal DBs, and Section 5 concludes the paper.

2. Related Work

In open access benchmark DBs, biosignal types include ECG, EMG, and EEG signals. Among them, ECG and EMG DBs are prominently used in user recognition research. However, the majority of ECG DBs have focused on healthcare applications, such as disease detection, using data acquired within a single session. Conversely, EMG DBs have been constructed with a limited number of subjects, geared towards applications in human–computer interfaces (HCIs) and motion recognition.

2.1. Benchmarking ECG DBs

The MIT-BIH Normal DB [4,5] comprises ECG signals recorded from 18 subjects, all in good health. The MIT-BIH Arrhythmia DB [6,7] comprises ECG signals recorded from 47 subjects. The Arrhythmia DB was developed for arrhythmia detection. ECG data were recorded in two channels while the subjects were walking. The MIT-BIH ST Change DB [8] comprises ECG signals recorded from 28 subjects over varying time spans. To create a DB with signals recorded over extended periods, data from five of these subjects were recorded during an exercise stress test.

The QT DB [9] comprises ECG signals representing various QRS and ST changes. This DB encompasses 105 records of ECG signals sourced from a total of seven DBs, including 15 MIT-BIH Arrhythmia records, 6 MIT-BIH ST Change records, and 10 MIT-BIH Normal Sinus Rhythm records. The Abdominal and Direct Fetal ECG DB [10] contains 5 min signals recorded from five pregnant women. The signals were recorded at a sampling rate of 1 kHz, with each record containing four abdominal signals from a pregnant woman and one simultaneous recording of fetal ECG signals from the fetal scalp.

The PTB Diagnostic DB [11] contains data obtained from Frank XYZ leads in addition to the standard 12-lead positions, totaling 14 leads. ECG data were recorded at a maximum sampling rate of 10 kHz, featuring 290 subjects inclusive of both able-bodied subjects and heart disease patients. The ECG-ID DB [12] contains 310 lead I records gathered from 90 subjects. The signals were generated at a sampling rate of 500 Hz. The number of signal measurements for each subject ranged from 2 (collected on a single day) to 20 (collected periodically over six months).

The CSU_ECG DB [13] comprises ECG signals recorded in both static and dynamic situations. ECG data were obtained from 506 subjects engaged in four static and three dynamic situations, and the DB was established by acquiring data across three sessions. ECG lead I was recorded at a sampling rate of 2000 Hz using a Biopac MP160 instrument (Biopac Systems Inc., Goleta, CA, USA). Table 1 shows a summary of the benchmarking ECG DBs. In Table 1, the benchmark DBs built for user recognition research include ECG-ID and CSU_ECG DB.

Table 1. Benchmarking ECG DB description.

Category	DB Name		Channels	Session Type	No. of Subjects
1	MIT-BIH	Normal [4,5]	2	Single	18
		Arrhythmia [6,7]	2	Single	47
		ST Change [8]	2	Single	28
2	QT [9]		2	Single	105
3	Abdominal and direct fetal ECG [10]		5	Single	5
4	PTB Diagnostic [11]		14	Single	290
5	ECG-ID [12]		1	Multi	90
6	CSU_ECG [13]		1	Multi	506

2.2. Benchmarking EMG DBs

The sEMG Basic Hand Moves Upatras [14,15] is an EMG DB comprising data recorded from two channels in the extensor carpi radialis and flexor carpi ulnaris muscles. This DB contains records of EMG signals generated when subjects aged 20 to 22 performed six hand gestures. It consists of two datasets in total. In Set 1, five subjects repeated each gesture 30 times within a single session, while in Set 2, one subject repeated each gesture 100 times across multiple sessions. For each set, band pass filter (BPF) and notch filter (NF) were employed to eliminate noise present in the signals.

NinaPro comprises subdivided DBs. DB1 [16] was created by collecting data from 27 subjects who repeatedly executed 52 gestures, each performed 10 times. Each gesture lasted five seconds with a 3 s rest between gestures. EMG was measured by placing electrodes at the height of the radio-humeral joint, utilizing eight channels around the forearm and one channel each on the flexor digitorum superficialis and extensor digitorum superficialis. The DB consists of three sets: basic finger gestures, hand and wrist gestures, and grip and functional gestures. DB2 is a DB constructed with 12 channels of EMG, featuring the participation of 40 subjects who performed 49 gestures. Sets 1 to 2 of DB2 [17] correspond to Sets 2 to 3 of DB1, while Set 3 contains data acquired during the execution of finger gestures. Each gesture was repeated six times, lasting 5 s. DB5 [18] is a DB constructed with 16 channels of EMG, involving 10 subjects who performed 52 gestures. In DB5, all gestures mirrored those in DB1, with each gesture being repeated six times and lasting 5 s.

CapgMyo [19,20] is a DB of high-density sEMG (HD-sEMG) recorded from the skin using a two-dimensional array. The EMG data were acquired through 128 channels using an 8 × 16 electrode array. The DB comprises two sets, where DB-a contains records of EMG data obtained from 18 subjects performing eight isometric and isotonic hand gestures (gestures 13–20 of Ninapro DB1). In DB-b, 10 subjects performed the same gestures as in DB-a, and EMG measurements were taken in two sessions separated by at least 1 week. Each gesture was maintained for 3 to 10 s, repeated 10 times.

The Anglese EMG DB [21] was designed to predict future knee flexion angles during gait, using knee flexion angles and knee muscle EMG signals. The EMG data were obtained using 12 electrodes placed on the tensor fasciae latte, rectus femoris, vastus medialis, vastus lateralis, biceps femoris, and semitendinosus muscles on both thighs. Recorded at a sampling rate of 1111 Hz, the EMG data involved ten subjects, who repeated the gait gesture of walking a distance of 20 feet 15 times. Motion artifacts (less than 20 Hz) and high-frequency aliasing effects (greater than 500 Hz) were eliminated using a Butterworth filter.

ISRMyo-I [22] involved six subjects who participated in data collection across five sessions. Each session was repeated twice a day with a 30 min interval. Throughout the same session, the electrode positions remained unchanged, arranged in the form of two lines with eight channels each in the upper and lower positions of the forearm muscles. The subjects maintained a total of 12 gestures, each held for 10 s, and the EMG data were obtained at a sampling rate of 1000 Hz.

GrabMyo [23] is an EMG signal DB recorded using 28 channels (16 channels on the forearm and 12 channels on the wrist) in 43 subjects. Forearm sensors were positioned at one-third of the forearm length from the elbow, and wrist sensors were located 2 cm from the ulnar styloid. Each participant repeated seven cycles of performing all 16 gestures, consisting of finger and wrist gestures, once, with a 10 s rest between cycles. The EMG signals were measured at a sampling rate of 2048 Hz, and a 10–500 Hz BPF was applied to the recorded EMG to eliminate noise from the signals.

CSU_sEMG [24] is a DB of EMG data collected from the right arms of 200 subjects. The signals were recorded across three sessions with intervals of at least one day. EMG sensors were attached to the palmaris longus and extensor digitorum, and the signals were recorded at a sampling rate of 2000 Hz using Biopac MP160. Twelve gestures were performed, comprising seven static gestures with a single movement and five dynamic gestures involving continuous movement. Subjects were instructed to sustain each gesture for at least 1 s, and a total of 30 signals were recorded. Table 2 shows a summary of the benchmarking EMG DBs. In Table 2, the benchmark DB built for user recognition research is CSU_sEMG DB.

Table 2. Benchmarking EMG DB description.

Category	DB Name		Channels	Session Type	No. of Subjects	No. of Gesture
1	sEMG Basic Hand Movements Upatras [14,15]		2	Single	5	6
			2	Multi	1	6
2	Ninapro [16,17,18]	DB1	10	Single	27	52
		DB2	12	Single	40	49
		DB5	16	Single	10	52
3	CapgMyo [19,20]	DB-a	128	Single	18	8
		DB-b	128	Multi	10	8
4	Anglese EMG [21]		12	Single	10	1
5	ISRMyo-I [22]		16	Multi	6	12
6	GrabMyo [23]		28	Multi	43	16
7	CSU_sEMG [24]		2	Multi	200	12

2.3. Benchmarking Multi-Biosignal DBs

MeganePro MSD1 [25] involved 15 subjects with transradial amputation and 30 able-bodied subjects, all performing 10 different grasping tasks. During these tasks, data on EMG, video, and gaze tracking were recorded. EMG signals were recorded at a sampling rate of 148 Hz, with eight sensors placed equidistantly from the radio-humeral joint and four sensors positioned 45 mm away. Video and gaze tracking was recorded through eye-tracking glasses.

The DEAP dataset [26] captured EEG, electrooculogram (EOG), and EMG signals while 32 subjects watched 40 one-minute videos. Facial images were additionally recorded for twenty-two of the subjects during video observation. The data were obtained at a 512 Hz sampling rate, with 2 channels for EOG sensors positioned vertically and horizontally, 2 channels for EMG sensors on the trapezius and zygomaticus major, and 32 channels for EEG. The recorded biosignals were down-sampled to 128 Hz, and noise was eliminated by applying a 4–45 Hz BPF.

The DREAMER dataset [27] is a DB constructed for emotion recognition using EEG and ECG signals. Biosignals from 25 subjects were recorded in response to audio–visual stimuli generated through 18 videos, which portrayed nine different emotions, such as joy, excitement, and happiness, with varying durations ranging from 1 to 393 s. EEG was measured using an Emotiv EPOC wireless EEG headset (EMOTIV, San Francisco, CA, USA), covering 14 channels, while ECG was measured using Shimmer2 ECG sensors. In the constructed DB, data from two subjects were omitted due to inappropriate content.

Stress Recognition in Automobile Drivers [28] is a DB designed for detecting stress in driving situations. Various driving situations were incorporated, including a rest period (low stress), highway driving (moderate stress), and city driving (high stress). During these scenarios, subjects had their ECG, EMG, and electrodermal activity (EDA) recorded while refraining from listening to the radio. Each subject engaged in driving for durations ranging from 50 to 90 min, with ECG sensors attached to the Lead II positions, the EMG sensors attached to the trapezius, and EDA sensors attached to the left hand and foot. While six subjects had their biosignals recorded during a single driving session, three subjects underwent repeated biosignals over several days.

The MAHNOD implicit-tagging DB [29] captures user responses to multimedia content. Thirty subjects wore six video cameras, a head-mounted microphone, a gaze tracker, and sensors measuring ECG, EEG (32 channels), respiration, and skin temperature while watching movies and images. Two experiments were conducted. In the first experiment, participants viewed short videos from movies and were instructed to tag their emotional state using valence and arousal. In the subsequent experiment, images or videos were shown alongside tags, and subjects were prompted to press the green button if they agreed with the displayed tag and the red button if they disagreed. Table 3 shows a summary of the aforementioned benchmarking multi-biosignal DBs. The benchmark DBs written in Table 3 were built for research on prosthesis control, emotion recognition, etc., not for user recognition research.

Table 3. Benchmarking multi-biosignal DB description.

Category	DB Name	Biosignal Types	Session Type	No. of Subjects
1	MeganePro MDS1 [25]	EMG, video, gaze tracking, etc.	Single	45
2	DEAP dataset [26]	EEG, EOG, EMG, etc.	Single	32
3	DREAMER dataset [27]	EEG, ECG	Single	25
4	Stress Recognition in Automobile Drivers [28]	ECG, EMG, EDA, etc.	Multi	9
5	MAHNOD-implicit-tagging [29]	ECG, EEG, gaze tracking, etc.	Single	30

Due to the inconvenience of attaching sensors to the body for biosignal recording, conventional biosignal DBs used in benchmarking for user recognition involve a limited number of subjects, as shown in Table 1, Table 2 and Table 3. Furthermore, a significant issue arises from the inability to analyze the variability in biosignals due to the data being recorded in a single session. To solve these problems, this study presents extensive multi-session ECG-EMG DBs (CSU_MBDB1 and CSU_MBDB2).

3. Measured Method of Multi-Session Biosignal Benchmarking DBs

The multi-biosignal DB for user recognition research was constructed by simultaneously measuring ECG and EMG in subjects. During the execution of a specific hand gesture by each subject, the EMG signal from the muscle was recorded using two channels, while the ECG signal generated from the heart was recorded using one channel. Given that the constructed multi-biosignal DB contains simultaneously recorded signals, we have the flexibility to use the signals either in the form of multi-biosignal data or individually for each signal. The benchmarking multi-biosignal DB introduced in this study comprises two datasets (CSU_MBDB1 and CSU_MBDB2). CSU_MBDB1 includes ECG and EMG data recorded as 36 subjects performed six hand gestures, while CSU_MBDB2 encompasses ECG and EMG data recorded as 58 subjects performed the same six hand gestures.

3.1. Multi-Biosignal Measurement Method

In the case of CSU_MBDB1, signals were recorded during the performance of six hand gestures (Table 4) by each participant. These gestures include the following: (1) clenching the fist, (2) pressing the index finger with the thumb while clenching the fist, (3) simultaneously flexing the index, middle, and ring fingers, (4) flexing the wrist, (5) extending the wrist outward, and (6) rotating the wrist 90 degrees to the left. The construction of CSU_MBDB1 involved the active participation of 60 subjects.

Table 4. Six hand gestures from CSU_MBDB1.

1. Clenching the fist	2. Pressing the index finger with the thumb while clenching the fist
3. Simultaneously flexing the index, middle, and ring fingers	4. Flexing the wrist
5. Extending the wrist outward	6. Rotating the wrist 90 degrees to the left

Each hand gesture followed a rest period–gesture period–rest period sequence, repeated 10 times within a single session. The gesturing method for each gesture was specifically standardized in certain steps to ensure consistency across all subjects. Figure 2 shows an example of the rest period–gesture period–rest period situation during a single execution of the gesture. The ECG and EMG were measured across two sessions, with the DB’s construction including a minimum one-week interval between the sessions to facilitate the analysis of biosignal variability.

Figure 2. Examples of DB recording method (an example of the biosignal recording procedure for the fist-clenching gesture).

We used the Biopac MP160 as the biosignal construction equipment, measuring ECG with one channel and EMG with two channels. When ECG and EMG were acquired simultaneously, interference occurred between the ECG sensor and the EMG sensor. Therefore, we attached the EMG sensor to the position where the muscles are activated while performing hand gestures, changed the ECG sensor location, as shown in Figure 3, and checked for interference between sensors. The location where the final ECG and EMG sensors were attached was where interference between the sensors was minimized. ECG sensors were positioned under the left and right biceps brachii, with the GND attached above the left biceps brachii. For EMG, Ch1 was attached to the flexor carpi radialis, Ch2 to the extensor indicis proprius, and GND to the outside of the upper arm. Figure 4 shows the ECG and EMG sensor positions. The signal bandwidth was set to 0.5–35 Hz, the sampling rate to 2000 Hz, and the ADC resolution to 16 bits.

Figure 3. Signal measurement position experiment for electrode positions.

Figure 4. Electrode positions for CSU_MBDB1.

For CSU_MBDB2, signals were recorded as each subject performed six hand gestures, outlined in Table 5: (1) clenching the fist, (2) flexing the wrist, (3) extending the wrist upward, (4) rotating the wrist 90 degrees to the left, (5) rotating the wrist 90 degrees to the right, and (6) raising the cell phone 90 degrees. The DB was designed for expansion since CSU_MBDB2 has the same three hand gestures as CSU_MBDB1 (CSU_MBDB1: gesture no. 1 and CSU_MBDB2: gesture no. 1; CSU_MBDB1: gesture no. 4 and CSU_MBDB2: gesture no. 2; CSU_MBDB1: gesture no. 6 and CSU_MBDB2: gesture no. 4). Each hand gesture was repeated 10 times in one session, following the same rest period–gesture period–rest period procedure as in CSU_MBDB1. Signals were recorded across two sessions with an interval of at least one week. A hundred subjects participated in the construction of CSU_MBDB2. EMG ch2 was attached to the extensor carpi radialis longus of the right arm. EMG ch1, ECG sensor, signal bandwidth, and sampling rate were set the same as CSU_MBDB1.

Table 5. Six hand gestures from CSU_MBDB2.

1. Clenching the fist	2. Flexing the wrist
3. Extending the wrist upward	4. Rotating the wrist 90 degrees to the left
5. Rotating the wrist 90 degrees to the right	6. Raising the cell phone 90 degrees

3.2. Multi-Biosignal DB Verification and Segmentation

In order to minimize the variability in the sensor location while proceeding with the measurement protocol, hand gestures were performed when attaching the EMG sensor to confirm the location where the muscles were activated (moved). Additionally, in order to maintain the same force and duration, the DB construction procedure was explained to the subjects, which they had to learn. Information such as the duration and amplitude of ECG and EMG measured newly (Day 2) from the same subject was reviewed by comparing similarity with signals from other sessions (Day 1). However, due to technical problems, an incomplete signal was confirmed. Various types of incomplete signals were observed, including (1) Bluetooth communication instability between the measurement PC and Biopac MP160 during the gesture period, (2) interference between EMG and ECG sensors during the gesture period, (3) resistance in the connection cable between the sensor and the measurement device during the gesture period, and (4) destruction of the signal waveform due to body movement during the rest period. In CSU_MBDB1, data from 24 subjects are incomplete, meaning that data from 36 subjects can be used. In CSU_MBDB2, data from 42 subjects is incomplete, so data from 58 subjects can be used. Therefore, the final CSU_MBDB1 consists of the ECG and EMG signals obtained from 36 subjects, while CSU_MBDB2 consists of those obtained from 58 subjects.

The ECG and EMG data underwent visual inspection, and division was carried out to ensure the preservation of the PQRST waveform in the ECG after the gesture was performed. Figure 5 shows the segmented ECG and EMG waveforms. Data segmentation occurred only for the segments longer than or equal to 0.5 s, considering both before/after data acquisition. The introduced benchmark DBs, CSU_MBDB1 and CSU_MBDB2, consist of raw signals, including R-peak positions and division points that facilitate the division process, as shown in Figure 5.

Figure 5. Examples of dividing the ECG and EMG waveforms in CSU_MBDB1 and CSU_MBDB2. (a) Examples of EMG_ch1 waveforms. (b) Examples of EMG_ch2 waveforms. (c) Examples of ECG waveforms.

Table 6 shows a summary description of the constructed CSU_MBDB1 and CSU_MBDB2. In contrast to Table 3, which summarizes existing multi-biosignal DBs, CSU_MBDB1 and CSU_MBDB2 were designed as multi-session data with a substantial number of subjects. An advantageous feature of CSU_MBDB1 and CSU_MBDB2 is their expandability, as they share three identical gestures, resulting in multi-session databases with a total of 94 subjects. This expansion allows for more subjects compared to the benchmark DBs of ECG (Table 1) and EMG (Table 2), excluding CSU_ECG and CSU_sEMG. The databases can be actively used for research on biosignal variability and user recognition.

Table 6. Description of CSU_MBDB1 and CSU_MBDB2.

DB Name	CSU_MBDB1	CSU_MBDB2
Biosignal types	ECG, EMG	ECG, EMG
Session type	Multi	Multi
No. of participants/construction subjects	60/36	100/58
No. of channels	ECG 1ch, EMG 2ch	ECG 1ch, EMG 2ch
No. of gestures	6	6
Sampling rate	2000 Hz	2000 Hz
Remark	In the form of raw signals. The R-peak positions and the divide points that facilitate the signal division are provided.

4. User Recognition Method, Results, and Discussion

A user recognition experiment was performed to confirm the usability of CSU_MBDB1 and CSU_MBDB2, the benchmark DBs constructed in this study. The experiment employed a pre-existing designed network [30], depicted in Figure 6. This designed network consists of two sub-networks, with each sub-network featuring six convolutional layers and two max-pooling layers. Within the convolutional layers, features are extracted by configuring 8, 16, and 32 filters of size [1 × 3]. After the convolutional layer, batch normalization is performed. The pooling layer is a filter of size [1 × 2] and reduces the feature dimension with padding 0 and stride [1,2]. Sub-network 1 utilizes as input data a size of [1 × 5000], representing an ECG signal, while sub-network 2 uses an input data size of [1 × 5000], representing an EMG signal. Both sub-networks share the weights of the convolutional layers during training. The features calculated from sub-network 1 and sub-network 2 are concatenated and input to the fully connected layer. The fully connected layer consists of 2048, 256, and 36 (number of classes) to recognize users. The experiment is conducted using a batch size of 256, epoch 100, learning rate of 0.001, and activation function rectified linear unit (ReLU).

Figure 6. Networks used for user recognition.

In the experiment, the multi-session signals are divided into Each Data and Cross Data. Each Data are used to confirm the variability in biosignal analysis. As shown in Figure 7, this method uses signals recorded in different sessions for both training and testing. Specifically, the Each Data method uses the ECG and EMG data recorded on Day 1 for training and ECG data recorded on Day 2 for testing. However, Cross Data are used to analyze the validity of the constructed DB. As shown in Figure 7, this method divides signals recorded in different sessions in a 70:30 ratio using them as training and test data, respectively. The Cross Data method uses 70% of the ECG and EMG signals recorded on Days 1 and 2 as training data and 30% of the ECG and EMG signals recorded on Days 1 and 2 as test data. The experiment was conducted with 10 and 36 people of CSU_MBDB1 and 10 and 58 people of CSU_MBDB2. To analyze hand gestures suitable for user recognition in the two constructed databases, the same number of subjects (10 subjects) was used.

Figure 7. Structures of Each Data and Cross Data.

Table 7 shows the experimental results of Each Data and Cross Data using CSU_MBDB1. The experiment involved ECG and EMG data from 10 and 36 subjects. The results from the Each Data showed an accuracy of 45.11% for the 10-subject dataset and an accuracy of 35.28% for the data of 36 subjects. In the Cross Data experiment, the accuracy reached 63.42% for the 10-subject datasets and 45.37% for the 36-subject datasets. The experimental results indicate that the user recognition accuracy of the Cross Data method surpassed that of the Each Data method. The heightened performance of the Cross Data method, using biosignal data from two sessions, underscores the significance of biosignal variability, specifically in ECG and EMG signals. This underscores the necessity of constructing a large DB spanning multiple sessions to effectively analyze the variability for these biosignals.

Table 7. User recognition result using CSU_MBDB1.

Category		Round 1 (%)	Round 2 (%)	Round 3 (%)	Mean (%)
Each Data	10 subjects	46.62	43.67	45.05	45.11
	36 subjects	34.18	36.29	35.37	35.28
Cross Data	10 subjects	62.44	62.22	65.61	63.42
	36 subjects	46.01	45.59	44.52	45.37

Table 8 shows the experimental results of Each Data and Cross Data using CSU_MBDB2. The experiment involved ECG and EMG data from 10 and 58 subjects. In the Each Data experiment, the accuracy was 48.44% for the 10-subject datasets and 27.71% for the 58-subject datasets. Meanwhile, the Cross Data experiment yielded an accuracy of 66.39% for the 10-subject dataset and 49.42% for the 58-subject dataset. Similar to CSU_MBDB1, the experimental results showed that the user recognition accuracy of the Cross Data method in CSU_MBDB2 surpassed that of the Each Data method. In addition, it was confirmed that the three hand gestures performed in CSU_MBDB2 were suitable for user recognition compared to the three hand gestures performed in CSU_MBDB1 (excluding the three identical hand gestures).

Table 8. User recognition result using CSU_MBDB2.

Category		Round 1 (%)	Round 2 (%)	Round 3 (%)	Mean (%)
Each Data	10 subjects	48.33	47.17	49.83	48.44
	58 subjects	29.14	26.43	27.56	27.71
Cross Data	10 subjects	67.67	65.17	66.33	66.39
	58 subjects	50.19	49.49	48.59	49.42

Figure 8 shows the data distribution using CNN features, referring to a previous study [31], using the ECG and EMG of 10 CSU_MBDB2 subjects. In Figure 8, it can be seen that the biosignals of subject 1, subject 2, subject 3, and subject 7 are well clustered. However, it can be seen that many feature areas overlap when the data distribution of four subjects (subject 5, subject 8, subject 9, subject 10) uses the CNN features from Figure 6. These results are a result of the variability in ECG and EMG constructed in a multi-session and indicate the need for research on feature extraction technology for user recognition using CSU_MBDB1 and CSU_MBDB2.

Figure 8. Three-dimensional plot of CNN features from 10 subjects (CSU_MBDB2).

Table 9 shows the experimental results of the extended DB using the same three hand gestures (CSU_MBDB1: gesture no. 1 and CSU_MBDB2: gesture no. 1; CSU_MBDB1: gesture no. 4 and CSU_MBDB2: gesture no. 2; CSU_MBDB1: gesture no. 6 and CSU_MBDB2: gesture no. 4) from CSU_MBDB1 and CSU_MBDB2. The experiment used ECG and EMG from 94 subjects as Cross Data. As a result of the experiment, 94 people were recognized with an accuracy of 27.05%. In addition, to show the development potential of the constructed DB, an experiment was conducted using filtering (noise removal), which is widely used in existing research. ECG noise was removed using BPF with a bandwidth of 0.5 to 40 Hz. EMG noise was removed using BPF with a bandwidth of 5 to 500 Hz. The experimental results showed an accuracy of 30.94%, confirming that the accuracy was improved by 3.89% compared to before removing noise (with noise).

Table 9. User recognition result using CSU_MBDB1 and CSU_MBDB2.

Category		Round 1 (%)	Round 2 (%)	Round 3 (%)	Mean (%)
Cross Data (94 subjects)	With noise	25.89	27.42	27.84	27.05
	Without noise	30.14	31.86	30.83	30.94

The user recognition experiment using the constructed biosignal DB showed a relatively low accuracy of 66.39%. Notably, the ECG signals displayed a significant amount of noise, as shown in Figure 9 (red box), due to the simultaneous measurement of ECG and EMG during gesture performance. It is crucial to note that the user recognition experiment in this study was conducted using raw signals without noise removal or relatively simple noise removal. The network employed for user recognition utilized a straightforward model, contributing to the observed low accuracy. The presented CSU_MBDB1 and CSU_MBDB2 have limitations as they involve the measurement of ECG and EMG exclusively in healthy subjects, rendering them unsuitable as DBs for patient user recognition. Furthermore, given the focus on user recognition during the DB construction, ECG (1ch) and EMG (2ch) were measured with a limited number of channels, posing a drawback in terms of the analysis of various biosignals’ information (e.g., spatio-temporal analysis of HD-sEMG). Nevertheless, by expanding the DB using the same three hand gestures from CSU_MBDB1 and CSU_MBDB2, it was shown that user recognition experiments were possible with ECG and EMG for 94 subjects acquired in multi-sessions, and performance was improved by performing noise removal (BPF). The two built DBs (CSU_MBDB1 and CSU_MBBD2) have the advantage of being able to acquire ECG and EMG in multi-sessions and analyze the nonlinearity characteristics of biosignals that occur over time.

Figure 9. Examples of noise occurring in CSU_MBDBs.

Table 10 shows the calculation of consistency determination to analyze the efficiency and comfort of the constructed DB, referring to existing research [32]. If the consistency determination is higher than 0.5, it is an appropriate gesture for user recognition, and if it is lower than 0.2, the gesture needs improvement. As a result of the experiment, all gestures from CSU_MBDB1 and CSU_MBDB2 showed values close to 0.5, confirming their suitability for user recognition. Furthermore, when comparing the consistency determination of the three hand gestures from CSU_MBDB1 (gesture no. 1, gesture no. 4, and gesture no. 6) with those of CSU_MBDB2 (gesture no. 1, gesture no. 2, and gesture no. 4), CSU_MBDB1 yielded an average of 0.4447, while CSU_MBDB2 produced an average of 0.4606. Therefore, it was confirmed once again that the three hand gestures from CSU_MBDB2 are more suitable for user recognition than the three hand gestures from CSU_MBDB1.

Table 10. Consistency determination of CSU_MBDB1 and CSU_MBDB2.

DB Name	Gesture No. 1	Gesture No. 2	Gesture No. 3	Gesture No. 4	Gesture No. 5	Gesture No. 6
CSU_MBDB1	0.4766	0.4555	0.4462	0.4524	0.4324	0.4718
CSUMBDB2	0.4607	0.4492	0.4583	0.4574	0.4656	0.4580

5. Conclusions

Various benchmarking biosignal DBs have been established for user recognition research using biosignals. However, these DBs encountered limitations as the variability in signals could not be adequately analyzed, either due to their construction based on a limited number of subjects or the recording of biosignals in single sessions. This study address this gap by introducing CSU_MBDB1 and CSU_MBDB2, constructed by recording biosignals from numerous subjects across multiple sessions. The participant count for constructing the DBs was 60 and 100, respectively, with sensors attached to both the left and right arms for recording ECG and EMG signals during the execution of six hand gestures. Despite technical challenges in the measurement protocol, the DBs ultimately comprised data from 36 and 58 subjects. Since these two benchmarking biosignal DBs share three identical hand gestures, they can be expanded into a combined DB covering 94 subjects. To analyze the usability of the constructed DBs, we conducted a user recognition experiment using a neural network designed in a previous study. Following the experiment, the user recognition accuracy stood at 66.39% (Cross Data and 10 subjects from CSU_MBDB2). This relatively lower accuracy is attributed to the use of a straightforward neural network without the removal of noise from the signals. The introduced DBs (CSU_MBDB1 and CSU_MBDB2) hold potential as benchmark DBs for user recognition research, given their recording of ECG and EMG signals in multiple sessions. By expanding the DB using three gestures from the multi-session DB for user recognition, we showed that user recognition experiments were possible with ECG and EMG from 94 subjects acquired simultaneously. Additionally, it was confirmed that performance improved when using a preprocessed signal (without noise) rather than a raw signal (with noise). There is a prospect of improving user recognition accuracy by implementing noise removal techniques on the constructed multi-session biosignal DBs and designing an optimal neural network. Lastly, by measuring ECG and EMG in multi-sessions, the nonlinear characteristics of biosignals that occur over time can be analyzed, and through this, user recognition research that is robust to volatility can be conducted. Future research endeavors will include investigating the variability analysis of the multi-session biosignal DBs and exploring techniques for noise removal from biosignals to further improve user recognition accuracy.