Advances in Modeling and Interpretability of Deep Neural Sleep Staging: A Systematic Review

Abstract

Sleep staging has a very important role in diagnosing patients with sleep disorders. In general, this task is very time-consuming for physicians to perform. Deep learning shows great potential to automate this process and remove physician bias from decision making. In this study, we aim to identify recent trends on performance improvement and the causes for these trends. Recent papers on sleep stage classification and interpretability are investigated to explore different modeling and data manipulation techniques, their efficiency, and recent advances. We identify an improvement in performance up to 12% on standard datasets over the last 5 years. The improvements in performance do not appear to be necessarily correlated to the size of the models, but instead seem to be caused by incorporating new architectural components, such as the use of transformers and contrastive learning.

1. Introduction

Sleep disorders can impair attention, long-term memory, decision making, and overall negatively affect cognitive performance [1,2]. Diagnosing sleep-related disorders early is therefore important for providing timely treatment, but the task is time-consuming and can be biased by the clinician. Due to the repetitiveness of the task, deep learning techniques can be applied to assist clinicians in sleep scoring and in diagnosing illnesses. Additionally, polysomnographic sleep studies are typically conducted in the clinic using numerous sensors and electrodes, which can be uncomfortable, potentially reducing the quality of sleep data recorded. Besides enabling inference, deep learning techniques may also be applied to reduce the number of sensors needed and to enable the use of wearable sensors for at-home care and preliminary diagnoses.

A polysomnogram (PSG) captures brain waves, oxygen levels, eye and leg movements, heart rate, and breathing throughout the patient’s sleep cycle, segmented into 30 s segments referred to as epochs [3]. The American Academy of Sleep Medicine (AASM) and the Rechtschaffen and Kales (RK) manuals are used to define sleep stages. In this paper, we will use the AASM manual definitions. The AASM [4] manual defines five stages of sleep (Wake, N1, N2, N3, REM). Each of these stages has its own characteristics, as shown in

Table 1. AASM EEG Frequency Definitions.

Rhythm	Frequency Band (Hz)	Target Stages
Delta	1–4	N3
Theta	4–8	N1, REM
Alpha	8–12	N1, Wake
Sigma	12–16	N2
Beta	16–32	Wake, REM

. It can be seen in Table 1 that each stage has its own frequency bands and specific rhythm. N3 occupies the 1–4 Hz band and consists of Delta rhythm. N1 is presented in 4–12 Hz and has Theta and Alpha rhythms. REM is defined by the 4–8 and 16–32 Hz bands. The wake stage is present in 8–12 and 16–32 Hz while N2 is present just in the 12–16 Hz band. The N2 stage typically consists of the K-complex and spindles. The REM stage is typically composed of saw-tooth waves. Atypical patterns in sleep stages and progression can help diagnose sleep disorders, provide indicators for sleep quality, and provide additional details about the patient’s health. Given the large amount of data in a single overnight PSG, clinicians expend significant effort to annotate the data. Furthermore, PSGs are highly variable with multiple sources of aleatoric and epistemic uncertainty, including age, pathology, electrode placement, and annotator bias. This causes even expert human annotators to disagree on epoch labels. With this in mind, machine learning algorithms will never achieve perfect alignment with human annotators because of the level of human disagreement in sleep score labels. At best, we can expect machine learning models to agree with human annotators to the extent that human annotators agree with each other.

Automating sleep staging can remove human subjectivity in decision making and reduce the labor considerably by automatically assigning labels based on models trained with multiple annotators. Deep learning models, to a much greater extent than classical models, are capable of capturing information from large datasets. Despite reaching near-human-level performance, deep learning models are opaque, obfuscating the model’s decision-making process [5]. Opacity in modeling limits the usefulness of these techniques in the medical community. However, ongoing efforts to make deep learning models more transparent and explainable will enable automatic sleep scoring to be more widely adopted. There have been some efforts in recent years to summarize the different methodologies on sleep studies [5,6,7]. However, these papers do not detail the modeling efforts (e.g., the reasoning behind the choices of architectures) and do not discuss their nuances and how they affect model performance.

In this review, the main application discussed will be sleep stage scoring, defined as the annotation of sleep signals. We provide an overview of recent research papers between 2018 and 2023 on sleep staging using deep learning. We provide an overview and survey of the sleep classification process and describe in detail the state-of-the-art (SOTA) techniques for sleep staging, including convolutional and recurrent models, transformer-based models, contrastive learning, transfer learning, domain adaptation, and interpretability. We aim to identify recent trends in performance improvement and the causes of these trends. Additionally, we investigate recent models to explore data manipulation techniques, their efficiency, and recent advances.

The rest of the paper is organized as follows. In Section 2, the criteria for the process with which we chose the articles is presented together with a description of the metrics used. In Section 3, we go over sleep-related datasets and some commonly used techniques and algorithms to standardize the signals. In Section 4, different types of modeling schemes are discussed in detail. In Section 5, the explainability for deep neural models is discussed. In Section 6, different models are compared and analyzed. Lastly, we summarize our findings in Section 7.

2. Method

2.1. Design

The literature search for this review was performed according to the PRISMA guidelines [8]. Only papers published between 1 January 2018 and 23 March 2023 were considered. A corpus of machine learning papers were collected from Google Scholar and Web of Science using terms “Deep learning”, “Sleep staging”, “electroencephalogram (EEG)”, “electrooculogram (EOG)”, and “electromyogram (EMG)”. The same exact set of terms were used for both search engines. We aimed to exclude papers that focus on hardware design, Internet of Things (IOT), and wearables. We consider only those papers that focus on the modeling and algorithms. To achieve this, we added some exclusion criteria, such as “wearable”, “IOT”, “artifact”, “hardware”, and “mobile” to remove the papers that are not relevant to this study. Of these papers, only publications that were cited by multiple researchers (at least 5 citations) or that presented a novel idea in the field were considered for review. The papers that were removed due to low citation did not have novel modeling, while some recent papers with zero citation that presented new techniques were included in the study. These inclusion and exclusion criteria were chosen carefully to make sure all the relevant studies were included. The PRISMA flowchart for article selection is shown in Figure 1. The most commonly used datasets were selected from these papers and are characterized in Section 3.

2.2. Metrics

To better contextualize the performances of these models, here we define the most useful metrics for the task. Cohen’s kappa is a measure of reliability between two annotators rating the same thing, correcting for the probability that two raters agree by chance. The metric is defined as:

$$\kappa =\frac{p_o-p_e}{1-p_e}$$

where $p_o$ is the relative observed agreement among raters, and $p_e$ is the probability of agreement by chance. In other words, $p_o$ is the joint probability of rater agreement, and $p_e$ is the marginal probability of the annotation choices. For reference, to achieve human-level performance, an automated sleep stage scoring algorithm should achieve a Cohen’s kappa of 0.76 [9], where 0.76 is the inter-rater agreement between trained scorers using the 2007 AASM scoring rules [10]. However, in the case of unbalanced datasets, Cohen’s kappa may be an inadequate measure of classification performance, penalizing judges with different marginal probabilities for annotations [11]. Many papers reviewed here do not provide Cohen’s kappa and instead provide an F1 score or an accuracy score.

The F1 score is the harmonic mean of precision and recall, providing a balanced performance measure that takes into account class imbalance. Simple accuracy works well when the classes are balanced in the dataset, but is a less informative metric when used for a dataset with highly imbalanced classes. Sleep datasets are often imbalanced, so papers often pair an accuracy metric with an F1 metric or another balance agnostic metric. Additionally, given the disparity between modalities represented in each of the datasets, comparisons between models will be made based on what dataset they use and the modalities used in the model for the sleep staging task. Furthermore, we will examine the complexity of the models and analyze the complexity of the model versus the performance.

3. Datasets

In this section, the datasets used in the papers reviewed are described. In particular, this review uses the AASM guidelines and makes conversions where necessary. In

Table 2. Percentage of Epochs Assigned to each Target Stage Label in the Different Datasets.

Dataset	Wake%	N1%	N2%	N3%	REM%	Total
Sleep EDF	19.50	6.60	42.07	13.48	18.24	42,308
MASS	9.88	7.88	50.90	13.18	18.15	57,395
SHHS	23.08	4.01	44.20	14.64	15.07	5,421,338
ISRUC (Subgroup 1)	23.62	11.54	32.74	18.69	13.41	90,123
ISRUC (Subgroup 2)	17.18	14.88	33.98	18.80	15.16	14,207
ISRUC (Subgroup 3)	19.53	13.44	30.96	22.79	13.28	8883
UCD	22.60	16.30	33.60	12.80	14.50	20,774

, the number of epochs containing each sleep stage as defined by the AASM are presented. As can be seen in the table, the classes are imbalanced, posing additional difficulties for automatic sleep scoring. The effects of this disparity can be seen throughout the presented papers.

3.1. Data Description

In this section, we briefly introduce popular datasets for sleep staging. Private or niche datasets are not enumerated here.

• SleepEDF (all versions): SleepEDF [12] is a publicly available dataset of physiological signals with corresponding sleep annotations. The initial version of this dataset was small and was originally published in 2002. In 2013, version 1 was published, which was greatly expanded to contain 61 PSGs with accompanying hypnograms. In March 2018, the database was further expanded to version 2 containing 197 PSGs with accompanying hypnograms. Each subject wore a modified Walkman-like cassette-tape recorder to record their normal activities. For each subject, two PSGs of about 20 h each were recorded during two subsequent day–night periods at the subjects’ homes. PSG recordings include EEG from Fpz-Cz and Pz-Oz electrode locations, EOG, and submental chin EMG signals. Corresponding hypnograms (sleep patterns) for these recordings were manually annotated by well-trained technicians according to the Rechtschaffen and Kales manual, and are also available. Annotation consists of sleep stages W, R, 1, 2, 3, 4, M (Movement time), and “not scored”. The Sleep EDF dataset has been used in numerous studies to develop and evaluate algorithms for automatic sleep staging recognition problems, as well as to investigate various aspects of sleep and its disorders. Papers using this dataset include [13,14,15,16,17,18,19,20,21,22].
• MASS: This is an open-access and collaborative database of laboratory-based PSG recordings [23]. The aim of the dataset is to offer a consistent and readily available data source for evaluating different systems developed to automate sleep analysis. The MASS dataset is unique in its size and scope, with the cohort consisting of 200 complete nights of polysomnograms, recorded from a diverse group of individuals (97 males and 103 females) with an average age of 40.6 years (ranging from 18 to 76 years). The dataset provides a valuable resource for researchers and practitioners working in the field of sleep analysis, with a focus on benchmarking and improving automated sleep analysis systems. This dataset includes polysomnographic recordings of sleep patterns, as well as related data such as participants’ medical histories and sleep questionnaires. All recordings feature a sampling frequency of 256 Hz and an EEG montage of 4–20 channels plus standard EOG, EMG, ECG, and respiratory signals. Papers using this dataset include [14,15,24,25].
• SHHS: The Sleep Heart Health Study (SHHS) dataset [26] was collected as part of a study aimed at investigating the relationship between sleep-disordered breathing and cardiovascular disease. This dataset collected data from over 6441 healthy participants without treatment of sleep apnea, with an age over 40 years for the first round (SHHS-1). A second polysomnogram (SHHS-2) was obtained from 3295 of the participants. The dataset includes PSG recordings, as well as data on demographics, medical history, and sleep habits. The PSG recordings were scored by certified technicians using standard criteria for sleep staging, respiratory events, and other sleep-related events. The dataset also includes information on the presence and severity of sleep-disordered breathing, such as apnea and hypopnea. The SHHS dataset has been used in numerous studies investigating various aspects of sleep and sleep-disordered breathing. The dataset is publicly available through the National Sleep Research Resource. Papers using this dataset include [9,18,27,28,29,30,31].
• CAP: The Cyclic Alternating Pattern (CAP) is a periodic EEG activity observed during non-REM (NREM) sleep. The CAP Sleep dataset [32] is comprised of 108 polysomnographic recordings collected at the Sleep Disorders Center of the Ospedale Maggiore in Parma, Italy. These recordings encompass a minimum of three EEG channels, EOG, EMG of the submentalis muscle, bilateral anterior tibial EMG, respiratory signals, and EKG. The CAP Sleep dataset includes 108 polysomnographic recordings from the Sleep Disorders Center of Ospedale Maggiore in Parma, Italy. The data include recordings from 16 healthy subjects and 92 pathological recordings with various sleep disorders, such as Nocturnal Frontal Lobe Epilepsy (NFLE), REM Sleep Behavior Disorder (RBD), Periodic Limb Movement (PLM), insomnia, narcolepsy, Sleep-Disordered Breathing (SDB), and bruxism. The recordings contain data from multiple sources including EEG, EOG, and ECG, along with muscle and respiration signals.
• ISRUC: The ISRUC-Sleep dataset [33] is a comprehensive polysomnographic (PSG) resource aimed at aiding sleep research. This dataset features data from adults, including healthy individuals and those suffering from sleep disorders, with some on sleep medication. The dataset is organized to accommodate different research goals, with information from 100 subjects each with one recording session, eight subjects with two sessions each for tracking changes over time, and 10 healthy subjects in a single session for comparison studies. Each PSG recording contains electrophysiological and pneumological signals, among other contextual details, all of which have been visually scored by two human experts. Papers using this dataset include [16,24].
• MESA: The Multi-Ethnic Study of Atherosclerosis (MESA) [34,35] is a long-term study sponsored by the National Heart Lung and Blood Institute (NHLBI), focusing on the development and progression of subclinical cardiovascular disease in over 6800 ethnically diverse individuals. Between 2010 and 2012, a subset of participants undertook a Sleep Exam (MESA Sleep), which studied how sleep variations and disorders correlate with subclinical atherosclerosis. The raw sleep data, including polysomnography and actigraphy, are publicly accessible for further research. Papers using this dataset include [27].
• Bonn database: The Bonn-Barcelona micro- and macro- EEG database [36] comprises 960 multichannel EEG signals, each with a duration of 32 s, extracted from long-term EEG data. The selection process did not involve any clinical criteria, such as the presence or absence of epileptiform activity, and all data are de-identified. Papers using this dataset include [37].
• UCD: The St. Vincent’s University Hospital/University College Dublin Sleep Apnea Database [38] consists of 25 full overnight polysomnograms, accompanied by a simultaneous three-channel Holter ECG, collected from adult subjects suspected of having sleep-disordered breathing. The subjects, aged between 28 and 68 years with no known cardiac disease or autonomic dysfunction, were randomly selected over six months from patients referred to St Vincent’s University Hospital’s Sleep Disorders Clinic. Polysomnograms were obtained using the Jaeger–Toennies system, capturing a variety of signals including EEG, EOG, EMG, ECG, and body position, among others. Additionally, three-channel Holter ECGs were recorded using a Reynolds Lifecard CF system. All data are available in EDF format. Papers using this dataset include [39,40].
• MIT-BIH Polysomnographic: The MIT-BIH Polysomnographic Database [41] contains over 80 h of four-, six-, and seven-channel polysomnographic recordings. Each recording includes a beat-by-beat annotated ECG signal, as well as EEG and respiration signals annotated in relation to sleep stages and apnea. These data are collected in Boston’s Beth Israel Hospital Sleep Laboratory. Papers using this dataset include [29].
• Apnea-ECG: Apnea-ECG [42] consists of 70 records, divided into a learning set of 35 records, and a test set of 35 records. Each recording includes a continuous digitized ECG signal, a set of apnea annotations and machine-generated QRS annotations. Papers using this dataset include [40].
• DREAMS: The DREAMS Databases [43] consist of recordings annotated in microevents or in sleep stages by several experts. They were acquired in a sleep laboratory of a Belgium hospital using a digital 32-channel polygraph (BrainnetTM System of MEDATEC, Brussels, Belgium). This database is split into 8 databases according to the annotation carried out. Papers using this dataset include [20,21].
• You Snooze You Win: The PhysioNet/Computing in Cardiology Challenge 2018: This dataset was contributed by the Massachusetts General Hospital’s (MGH) Computational Clinical Neurophysiology Laboratory (CCNL) and the Clinical Data Animation Laboratory (CDAC) [44]. The dataset consists of 1985 subjects and the sleep stages were annotated by clinical staff based on the American Academy of Sleep Medicine manual with six sleep stages noted in 30 s intervals: wakefulness, stage 1, stage 2, stage 3, rapid eye movement (REM), and undefined. Certified technologists also annotated waveforms for the presence of arousals interrupting sleep, classifying them into various categories such as spontaneous arousals, respiratory effort related arousals (RERA), bruxisms, and others. The physiological signals recorded during the subjects’ sleep include EEG, EOG, EMG, ECG, and oxygen saturation (SaO${}_2$). All signals, excluding SaO${}_2$, were sampled at 200 Hz and measured in microvolts. SaO${}_2$ was resampled to 200 Hz and measured as a percentage. Papers using this dataset include [31].

A summary of selected datasets with their respective modalities is presented in

Table A1. Different modalities present in selected datasets used in Table 3.

Dataset	EEG	EOG	EMG
SleepEDF	✓	✓	✓
MASS	✓	✓	✓
SHHS	✓	×	×
CAP	✓	✓	✓
ISRUC	✓	✓	✓
DRM-SUB	✓	✓	✓
SVUH-UCD	✓	✓	✓

.

3.2. Data Preprocessing

To use any of these datasets for automatic scoring, disorder diagnoses, and other medical applications, the data must be preprocessed into a form more amenable for use with deep learning. In this section, an overview of commonly used preprocessing techniques are presented.

Many authors [28,45,46,47] preprocess the data by removing frequency components outside of the accepted ranges of brainwave activity. In EEG, this is usually performed with a Butterworth band-pass filter tuned to 0.4–30 Hz. This removes outside noise from the input signal. Additionally, the data is often normalized using a variety of techniques. In [47], the EEG data is normalized using the 5th and 95th quantiles of each PSG individually. More commonly, the EEG is normalized using standard methods like computing z-scores based on PSG statistics. Statistical and handcrafted features [21] are also commonly extracted from PSG recordings as a pre-processing step.

Some authors [45,48,49] provide semantic structure to EEG data by segmenting the input signals into representative sub-bands containing the alpha, beta, delta, and theta waves. These bands were used as direct inputs to deep learning methods or used to generate handcrafted statistical features for analysis. Methodologies like these inject human insight into the modeling schemes which can improve results by simplifying the task. The raw EEG signals may be transformed using spectrograms [50], empirical mode decomposition, and discrete cosine transforms [51] to add additional structure to the data, allow for transfer learning techniques to be used, or for interpretability reasons.

3.3. Addressing Dataset Deficiencies

In this section, we investigate different techniques that have been used in the literature to overcome label imbalance, poor data quality, and shortage. For classification, these dataset deficiencies can reduce final performance and model reliability. To address deficiencies, techniques for data augmentation have been used, such as techniques for minimizing these issues through data augmentation from statistical methods, generative adversarial methods [52], resampling, etc. Data augmentation of EEG signals should not alter the semantic meaning of the signal, limiting the options available for augmentation.

A popular augmentation method is to add stochasticity to data [47,50,53,54,55,56,57,58,59]. Jittering, masking, flipping, scaling, and spectrogram reflection are some augmentation methods that have been used in the literature. In jittering, a random uniform noise is added to the physiological signal. With masking, parts of the signal will be randomly set to a pre-defined value. In [60], the authors proposed four different augmentation techniques, three of which we will explain since one of the methods (horizontal flipping) has already been described. The first proposed method is Epoch Overlapping. In this method, the authors start the current epoch, $x_i$, by using $2/3$ or $4/5$ of the previous epoch, $x_{i-1}$, as the starting point to include 30 s worth of recordings, with the label of the current epoch. The second approach is called Mixup. In this setup, two randomly selected signals ($x_i,x_j$) with their corresponding labels ($y_i,y_j$) are linearly mixed as follows:

$$x_{mix}=\lambda x_j+(1-\lambda )x_i$$

$$y_{mix}=\lambda y_j+(1-\lambda )y_i$$

where $\lambda \sim Beta(\alpha ,\alpha )$ with $\alpha =0.25$. The last method is called Random Cutout. In this method, the signal will be randomly zeroed out for 6 s in each epoch. This is used to simulate sensor error, such as the times when the leads have been detached from the skin.

While these methods add variety to the dataset, they do not address underlying issues with class imbalance. Common methods for dealing with class imbalances include resampling, bootstrapping [61], and deep learning techniques. In [62], the authors proposed using two methods: (1) generative adversarial network (GAN) and Gaussian white noise (GWN) to increase N1 stage samples; (2) balance the relationship between the trained model and the original imbalanced dataset by having different weights for different classes. The authors argue that the reason for adding GWN to the signals is twofold. The first reason is that this type of noise is present in EEG acquisition. So adding this noise with different intensities will imitate the true signal. The second reason is that the new data with noise addition can provide new features that help with generalization. Mitigating class imbalance through modeling rather than data augmentation (DA) would be more natural since the imbalance exists in the true distribution of the sleep signals for different classes. Other deep techniques used for balancing class imbalances include modifications to the loss [16,51], which reduce the impact of over-represented classes and increase the impact of underrepresented classes while preserving as much information from the data as possible.

4. Modeling

Deep learning models have revolutionized how time series data are modeled, capturing complex non-linear relationships that are not representable with classical modeling schemes. For sleep studies, models include convolutional neural networks (CNN), recurrent neural networks (RNN), long-short-term memory models (LSTM), graph neural networks (GNNs), and transformers. More recent efforts have made use of contrastive learning to further enforce feature differentiation via instance-level comparisons, allowing models to learn inter- and intra-group descriptors simultaneously. In this section, we explore the types of models used, training schemes, and learning techniques applied.

4.1. Transfer Learning and Domain Adaptation

In recent years, large models have been developed and have shown very promising results in different fields such as computer vision (CV) and natural language processing (NLP). In particular, in NLP, large models such as BERT [63] and GPT [64] have shown significant performance boosts on downstream tasks such as text classification or summarization by having millions, if not billions, of parameters. The base of such popular models is a transformer [65], which revolutionized the overall field of NLP. Since not all researchers have access to large clusters of GPUs to train big models, transfer learning is becoming more and more popular. By having large models as a feature extractor or an embedding space, researchers can perform downstream tasks having to train the model from scratch. This method has been adapted for physiological signals as well. In recent years, researchers have been utilizing deep learning models to perform classification for sleep staging. Most models are trained on a specific dataset and they may not generalize well to other datasets, which shows some of the limitations and challenges that we will explain in subsequent sections. In this section, we review the papers where the authors study transfer learning for sleep studies, and also, we study domain adaptation to investigate the generalizability of the models against different datasets.

The purpose of transfer learning is to leverage models learned in another domain for a different, related domain, providing a stronger initialization for the model in the target domain. Domain, here, can refer to a difference in tasks, datasets, or modalities. This provides numerous advantages than just training on the target domain including faster convergence, greater performance, or a reduction in computational capacity required for training, especially if the target dataset is size-constrained [66]. Furthermore, a source domain for transfer learning need not be directly related to the target domain; models in computer vision and natural language processing have successfully been used to provide strong starting points in seemingly unrelated domains including sleep stage scoring [66,67]. In this section, techniques for and applications of transfer learning and domain adaptation in sleep stage scoring are explored.

Earlier efforts in transfer learning focused on using models trained on large public sleep datasets to improve model performance for small datasets. In [66], the authors trained DeepSleepNet [68] on large public sleep scoring datasets and fine-tuned the model on a small private sleep scoring dataset, reporting significant improvement over training on the small dataset alone. Transfer learning can also be performed in the form of knowledge distillation [69] for model compression or wearable use cases.

Performance gain from using transfer learning can be limited due to loss of local domain information, different data distributions, changes in class information, etc. Even in the simplest cases, such as using different source datasets with the same task for transfer, the differences in source and target datasets can result in limited performance gains. In [70], the transferability of sleep stage scoring datasets with TinySleepNet [15] were assessed using a relative performance improvement ranking metric. Datasets were differentiated by their recording characteristics, the health of the patients represented in the dataset, and the recording environments. They found that, the greater differences between dataset characteristics, the worse the transfer. To reduce the performance loss from domain mismatch, researchers have proposed multiple techniques for adapting source to target domains.

The researchers in [71] used an adversarial learning scheme to reduce loss of domain-specific information from sharing a model between domains and to reduce the classification accuracy degradation. First, they proposed an unshared attention mechanism to maintain domain-specific information for target and source domains. Second, they designed an iterative self-training algorithm to enhance classification accuracy on the target domain by using pseudo labels from the target domain. They also proposed dual distinct classifiers to increase the robustness and quality of the pseudo labels. Other adversarial domain adaptation techniques have been proposed and applied by [60,72,73,74], dealing with various pain points including cross-subject adaptation and population health differences. A domain mismatch may also result from data-distribution-dependent model parameters like batch normalization. The authors of [75] deal with this by learning the batch normalization characteristics of the target dataset and replacing them in the target model.

By training the model on multiple heterogeneous datasets or on a single large, diverse dataset [76,77], the researchers were able to demonstrate that deep learning models are capable of adapting to the differences in the datasets in the form of differing input shapes, electrode positioning differences, and distribution shifts.

4.2. Convolutional and Recurrent Models

In this section, we explore different types of architectures that involve CNNs, RNNs, and GNNs. We will start by investigating the most influential papers.

In [68], the authors proposed a model named DeepSleepNet with publicly available code. This model consists of two different filter sizes that learn local time-invariant features at different resolutions to have a richer representation space. A bidirectional LSTM is appended to the CNN layer to encode the temporal information and sleep transition in each epoch of data. This model is trained in two steps. In the first step, the CNN layers are trained to extract the representation for the LSTM layer. This helps the model to be better conditioned on the data itself, rather than random initialization for the second step of the training. In the second step, the whole model is fine-tuned by using the pre-trained layer.

In [78], the authors proposed XSleepNet, which is a sequence-to-sequence sleep staging model capable of learning joint representation from multi-modal raw signals and time-frequency images (also known as a spectrogram), as shown in Figure 2. The model blocks are a combination of CNNs, RNNs, and attention mechanisms. In particular, ${\mathcal{F}}_1$ is a fully-connected CNN which extracts features from raw inputs, and ${\mathcal{F}}_2$ is an attention-based RNN with learnable filterbank layers to convert spectrograms into high-level features. In the last layer, there are three branches for raw, spectrogram, and a mixture of them to calculate the loss. Authors argue that different views (raw and spectrogram) may generalize or overfit at different rates. For more information, please refer to Appendix A.1.

In [79], the authors proposed a new architecture consisting of a modified ResNet-50 layer and a two-layered bidirectional LSTM named IITNet. In this model, a feature representation is extracted at sub-epoch level (less than 30 s) by using residual neural networks and extracting intra- and inter-epoch temporal context from the time series by using bidirectional RNNs. The latter helps to learn the transition rules between different epochs. If only the target epoch is used as an input to the model, the intra-epoch features will be learned and utilized for decision making. In order to capture intra- and inter-epoch temporal context, the target epoch and its previous neighboring epochs are fed to the model. This architecture mimics real-time sleep scoring where we cannot use any epoch from future epochs.

A mixture of convolution and attention layers were proposed in [80]. The CNN layer extracts local features and the multi-layer attention module is used to learn salient intra- and inter-epoch features, as shown in Figure 3. For the unbalanced dataset, the authors proposed to use a weighted loss function during training to improve model performance on minority classes. In order to learn some short-term features, such as K-complex and spindles, which last around 0.5–1.5 s, each epoch is divided into multiple windows (29 in the paper), as shown in Figure 3a. In Figure 3b, more details on how each window is treated are shown, and Figure 3c shows the attention module.

The authors of [81] proposed a novel framework based on local pattern transformation (LPT) methods and CNNs for automatic sleep stage scoring. Unlike previous works in other fields, these methods were not employed for manual feature extraction, which requires expert knowledge and the pipeline behind it might bias results. The transformed signals were directly fed into a CNN model (called EpochNet) that can accept multiple successive epochs. The model learns features from multiple input epochs and considers inter-epoch context during classification. The study investigates the role of LPT methods prior to the classification. Four LPT methods are used for both single and multi-channel setup. More details on this model can be found in Appendix A.2.

In [82], deviating from the previous architectures, the authors proposed a model called U-Time. This model is a fully convolutional encoder–decoder network based on U-Net architecture [83], which originally was used for image segmentation. U-Time adopts key concepts from U-Net for 1D time-series segmentation and maps sequential inputs of arbitrary length to sequences of class labels. To achieve this, every individual time-point in the signal is classified and aggregated over fixed intervals to reach the final prediction. This model is easier to train when compared to [68,84], it is more robust due to not using an RNN layer, and it has a smaller number of parameters.

The authors of [85] proposed to use a Pearson-correlation-based graph attention network (PearNet) to address the external relationship of electrodes in different regions of the brain and internal relationships between segments of electrodes within specific brain region problems. Graph nodes are produced by spatial-temporal convolutional layers with SE blocks followed by Pearson correlation block. The graph is then passed on to another module to adaptively learn the node connection for graph structure. The whole structure is shown in Figure 4. The spatial and temporal convolution networks are depicted in Figure 5 and Figure 6, respectively. The VIF (variance inflation loss) determines the degree of multi-collinearity [86].

In [87], the proposed model (SleepExpertNet) aims to classify sleep stages in a similar way to how the experts evaluate PSG recordings by using the relationship between the EEG signals and the patterns of changes in sleep stages by combining a time series-based multi-head attention mechanism and bidirectional LSTM. To extract the temporal-frequency features from raw EEG signals, a bandpass filter is applied to the single-EEG channel in various frequency bands. Embedded vector generation, including a novel CNN-based architecture, is proposed to consider the neurophysiological characteristics of EEG signals. To solve the class imbalance problem, the cost-sensitive learning of focal loss and a novel weighted random sampling method are applied in the proposed model structure. To improve the classification performance, an ensemble decision algorithm, which can be used in the proposed model without additional training, is developed. The model structure is shown in Figure 7.

In [88], a novel multi-view fusion model named MVF-sleepNet was proposed. This model accepts EEG, ECG, and EOG signals as input. In order to properly encode the relationship between different modalities, the authors proposed constructing two views of time-frequency images (TF images) and graph-learned graphs (GL graphs). In Figure 8, the model framework is shown. This model consists of two streams, one for TF images and one for graph features. For sequentially timed TF images, a combination of VGG-16 and GRU with attention mechanism networks is used for spectral and temporal representation, respectively. Furthermore, for graphs, a combination of Chevyshev graph convolution and temporal-spatial attention mechanism, and temporal convolutional network is employed to learn spatial-temporal representation sequentially timed GL graphs.

In [89], the authors proposed a sleep stage classification method using orthogonal convolutional neural networks. They preprocessed the data with a Hilbert Huang transform. Their model consists of three blocks, with each block consisting of a convolutional layer connected to a squeeze excitation network and a rescaling operation. The convolutional network also has a skip connection to the scaling operation. The convolutional layers are initialized with orthogonal weights, and additional regularization is performed to ensure the orthogonality of the weights throughout training.

So far, we have explored models based on CNN and RNN architectures. There are many other papers that, in general, used these types of architecture, with subtle differences in their modeling through loss function and some prepossessing on the data [13,15,22,27,37,46,58,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123]. In [68], the authors showed the power of multi-scale modeling to capture fast and slow varying features. To improve upon this approach, many papers used different variations of the multi-scale approach to better capture the features for downstream tasks [14,18,124,125,126,127,128,129,130,131,132,133,134,135,135]. Some papers followed the direction in [84] and added attention modules to their models to improve performance [18,30,80,84,126,136,137,138,139,140,141,142,143,144,145,146,147,148,148]. Some authors applied post processing algorithms based on the hidden Markov model (HMM), conditional random fields (CRFs), soft-voting, etc. to improve model performance and correct unreasonable sleep stage transitions [20,133,140,149,150,151,152,153]. Some researchers used time-frequency information to model the data [29,31,40,131,153,154,155,156,157,158]. The time-frequency information was either solely used as the input to the model or fused with the raw time series in the latent space. In [40,159], the authors used empirical mode decomposition (EMD) [160] to extract intrinsic features from the input signal by outputting different signals for different frequency ranges which can correspond to different sleep stages. In [161], the authors proposed the multi-modal physiological-signals-based Squeeze-and-Excitation Network with Domain Adversarial Learning (SEN-DAL) to capture the features of EEG and EOG for sleep staging. The SEN-DAL consists of a Multi-modal Squeeze-and-Excitation feature fusion module for adaptively utilizing the multi-modal signals and a Domain Adversarial Learning module to extract subject-invariant sleep features.

The current SOTA approaches make extensive use of contrastive learning paradigms, allowing for unsupervised learning mechanism to encode sample-wise and class-wise, by extension, similarity in the underlying feature vector. Recent efforts [17] have also seen success in using contrastive methods as a training step. Advancements in this field also include modifications to the loss function to either enforce certain model behaviors or to address deficiencies in the data, including class imbalance [51,162,163]. Other techniques have also been proposed including manifold-based techniques [164] and incorporating spatio-temporal features.

4.3. Transformer-Based Models

Another class of modeling includes transformers [65] to perform sleep scoring. Because of transformers’ tremendous success in NLP, and the fact that texts are considered time series, they attract attention in the physiological set up as well. In the following section, the papers with transformers architecture are investigated.

The authors of [165] proposed a masked-transformer-based model to learn a signal’s features in a self-supervised manner. They found that, although their method helps to improve the performance compared to the baseline, this can be due to the effect of the human subjects that they sampled from. An interesting observation from this paper is the effect of masked length on downstream tasks. They reported that a longer mask will result in better performance in the downstream task. They argue that this could be due to the fact that longer masks force the model to learn more details and nuances about the signal.

In [166], the authors proposed a transformer-based modeling, SleepTransformer, for sleep staging. This modeling allows one to add interpretability to the model through the attention layers to have more confidence in decision making and also to be integrated in clinical settings. At epoch level, through attention, the model learns to pick the most relevant parts of the signal. Since this model can be applied at sequence level, it can measure the influence of the neighboring epochs on the target epoch for decision making. We will explain more about the interpretability in Section 5. The model architecture is shown in Figure 9. This model takes time-frequency images as the input, and processes each epoch by a standard transformer block. The output for each epoch is then concatenated and fed into another transformer for fusion. Then, multiple FC layers for sleep staging are utilized.

Another transformer-based model is composed of a multi-Scale CNN block with intra-modal and cross-modal attention [167]. The main difference (besides the model architecture) between this work and ref. [166] is the cross-modal relationship, which ref. [166] fails to examine. The multi-scale CNN block learns optimal feature representations by considering local and global features. The authors proposed a novel cross-modal transformer model encoder architecture to learn intra-modal temporal attention between time steps within a feature embedding of a modality and cross-modal attention to capture relevant information between each modality. In case of the sequence-based classification, the cross-modal transformer has another block to learn inter-epoch relations. The sequence-based model is depicted Figure 10. More details about this model can be found in Appendix A.3.

In [168], the authors proposed a multi-modal attention network (MMASleepNet) to perform sleep staging. This model consists of multi-branch feature extraction (MBFE) and an attention-based feature fusion (AFF),and classification layer, as shown in Figure 11. The MBFE takes different modalities and extracts features by simple convolutional layers. The AFF module contains a modal-wise Squeeze-and-Excitation [169] block to adjust the weights of modalities with more discriminative features and a transformer encoder (TE) to generate attention matrices and extract the inter-dependencies among multi-modal features.

In [170], the authors introduced ENGELBERT. This model introduces overlapping attention mechanisms, which helps with efficient matrix formulation, aggregation, and reduces the critical quadratic computational complexity to linear. As a regularization mechanism, the authors applied non-linearity in a dimensionally compressed space for each attention layer. The model architecture is shown in Figure 12. The model has a CNN feature extraction layer that is followed with linear dimensionality expansion to be fed to the attention-based feature fusion layer, which helps increase the representation capacity. One major difference compared to a regular transformer encoder is that the arrangement of the expansion and compression blocks are reorganized, as shown in Figure 13.

4.4. Contrastive Learning

Another class of modeling for sleep classification is contrastive learning. In contrastive learning, the goal is to learn a proper representation of the signal by itself (self-supervised) or by having access to partial labels (semi-supervised) to optimally perform downstream tasks such as classification, anomaly detection, etc.

In [171], the authors proposed a novel methodology for fusing multi-channel EEG to provide salient information to a sleep staging model. The logic behind their methodology is that each electrode used for PSG captures a portion of brain activity from different regions of the brain. By structuring the channels as an undirected graph and learning adjacencies, they are able to find strongly related brain activity in terms of both spatial and temporal features. In this paper, the model learns the adjacency matrix, which is utilized to learn graph parameters. More details about the approach are presented in Appendix A.4. The main contributions of this paper are the adaptive adjacency matrix learning and the use of spatial and temporal features for sleep staging.

In [172], a contrastive representation learning model named SleepPriorCL based on prior knowledge is proposed. The goal of this model is to learn a meaningful semantic representation for similar samples in self-supervised manner, without using any labels. It is known that each sleep stage occupies a certain frequency range. The authors leveraged this fact and calculated the energy for each of these frequency packets, and used them as prior knowledge for training. Since similar signals have similar energy, the absolute value of their difference can serve as a similarity measure. The model has to put more weight towards similar samples and less towards dissimilar ones. This is achieved by using the absolute value of the difference between energies of the sample to calibrate the temperature in a contrastive loss function. Moreover, to ensure there are multiple positive (samples) in each mini-batch, an augmentation on each signal is performed in the paper. It should be mentioned that this formulation through prior knowledge helps the model not be biased to positive samples that are just generated through augmentation, and also considers other positive samples in the mini-batch that are not the result of augmentation.

The authors of [173] proposed a new model, called CoSleep, to extract semantic representation for physiological signals through multi-view modeling. The architecture is shown in Figure 14. The CoSleep technique focuses on uncovering connections among instances within the same semantic class, regardless of their temporal proximity. It achieves this by attracting multiple positive instances simultaneously, enhancing representation modeling at the semantic-class level. The approach employs a multi-view co-training mechanism to identify positive instances, drawing from both temporal and spectral perspectives. The technique extends the Dense Predictive Coding (DPC) framework with a dedicated memory module, which enlarges the pool of negative candidates. This expansion improves DPC training and the extraction of positive instances. More details about model are presented in Appendix A.6.

In [174], the authors proposed SleePyCo for EEG-based sleep stage scoring. The model uses a multi-scale feature pyramid and contrastive learning to learn intra-class and inter-class features, using them to better separate sleep stages in feature space. The feature pyramid consists of CNN layers with varying kernel sizes. These layers are connected laterally to linear projectors to match the dimensions of all of the feature vectors. The resulting feature matrix is then fed to a transformer network to perform the classification. The backbone is pre-trained using a contrastive learning strategy. The second stage is learned through multi-scale temporal context learning, freezing the pyramid network during training to ensure the retention of features learned during the contrastive learning stage. Early stopping was used in both stages to ensure parsimony.

The authors of [175] proposed a multi-task contrastive learning strategy for semi-supervised sleep recognition (MtCLSS). In Figure 15, there are two similar branches, one for transformed signals and the other for the original signals. The model has two architecturally similar branches like a Siamese network. Branch 1 receives the raw EEG signal and Branch 2 receives a transformed EEG signal. Branch 2 predicts sleep stages and transformations. Branch 1 only predicts sleep stages. The features of the two branches are used to perform contrastive learning. The authors argue that the transformation identification helps with exploring the unlabeled data better and also improves the discriminative abilities and robustness of the model.

In [176], the authors proposed a semi-supervised sleep staging approach called co-attention meta sleep staging network (CMS2-Net). This framework aims to mitigate inter-class disparity and intra-class selection in sleep scoring. To resolve the inter-class problem, the authors proposed a co-attention module with second-order statistics to improve latent feature representation. To solve the intra-class problem, a triple-classifier is introduced to resolve the domain shift and re-parameterization.

The authors of [177] proposed to use both expert features and DNN features. They proposed a contrastive-based cross-attention model to predict sleep apnea through ECG signals. The cross-attention module is used to fuse expert knowledge and deep features, which helps with weighting the important features. The authors also performed classification with the contrastive task, which helps with better separation in feature space to enhance the downstream classification task. In order to achieve the goal above, a new hybrid loss has been proposed, which is composed of contrastive loss and cross-entropy. In Figure 16, it can be seen how both raw signal and expert features are fed to their corresponding models to extract features to input to the cross-attention layer to create a fused feature vector. Then, the fused feature vector goes through the projection for contrastive loss and it is passed to the classifiers for apnea detection.

In [178], the authors proposed the SCL-SSC (Supervised Contrastive Learning for Sleep Stage Classification) model, which combines contrastive learning with classification tasks shown in Figure 17. This task is performed in two steps: (1) feature learning and (2) classification. The feature learner block is trained separately to disentangle positive and negative samples from each other on the representation space, which translate to having more distance between samples from different classes and less distance between samples from the same class in Euclidean space. This is achieved by using the following triplet loss

$$\begin{array}{c}\hfill L(R_a,R_p,R_n)=max(0,\alpha +d(R_a,R_p)-d(R_a,R_n))\end{array}$$

where $R_a,R_p,$ and $R_n$ are anchor, positive, and negative feature representations, respectively. The function d is the Euclidean distance. The variable $\alpha$ is a bias term to minimize the distance between the same sleep stages and maximize the distance otherwise. It is expected to have $d(R_a,R_p)>d(R_a,R_n)$ for a meaningful learning. The authors used a similar architecture as in DeepSleepNet [68] for the feature block ($f_{\theta }$). For the classification portion, the authors used an encoder–decoder model with an attention mechanism, shown in Figure 18. This model takes in the features from the pre-trained feature block and feeds it to a sequence of RNNs to build a context vector to be fed to another RNN layer, which then goes through an attention block. This sequential modeling would encapsulate information from adjacent epochs to improve embedding space and overall model performance. The authors used the loss function as in [84] for classification.

In [56], the authors proposed a novel multi-view self-supervised method (mulEEG) for unsupervised EEG representation learning. The goal is to design a setup where the model can effectively utilize the complementary information in multiple views to improve the representation space. To achieve this goal, the authors introduced the diverse loss to encourage the model to use complementary information from different views. The model is shown in Figure 19. Apart from some augmentation techniques used in this paper, the main difference compared to CoSleep [173] is how the authors set up their loss function to impose similarity on the positive and negative samples. More details about the model can be found in Appendix A.5.

5. Interpretability

In Section 4, we mentioned that models normally have thousands, if not millions, of parameters. On top of a big number of parameters, non-linear activation functions add more complexity. Due to these facts, deep learning models are referred to as black-box models because there is no direct way to interpret how exactly information is processed through layers of a network. This is one of the major drawbacks of deep learning models. Another major issue with these models is their vulnerability to adversarial attacks, which can affect the model’s output drastically and lead to wrong results. Therefore, it is crucial to study these models thoroughly to reduce the uncertainty in their decision making and also make them more robust towards adversarial attacks. Especially in medical settings where clinicians work with patients, the explainability of the models and how the information goes from the input to the output becomes very important for reliable and trustworthy decision making. Interpretability will help the medical society to accept these models with more trust and certainty. Moreover, explainability can help with diagnosing the portions of the model that contribute the most to wrong decision making, which can later be fixed. We will go through the proposed models that provide some form of interpretability for sleep staging models.

The authors of [179] proposed Sleep staging via Prototypes from Expert Rules (SLEEPER). SLEEPER combines deep learning models with expert-defined rules via a prototype learning framework to generate simple interpretable models such as shallow decision trees and logistic regression models. In particular, SLEEPER utilizes sleep scoring rules and expert defined features to derive prototypes which are embeddings of polysomnogram (PSG) data fragments via convolutional neural networks. The final models are still simple interpretable models like a shallow decision tree or logistic regression defined over those phenotypes. The SLEEPER framework is shown in Figure 20.

In [180], the authors proposed an ablation-based interpretability for a multi-modal CNN-based model. This approach, instead of zeroing out the modalities (masking with zeros), replaces the signals with noise that normally appears in the measurements. They used a 40 Hz sinusoid and a Gaussian noise with mean 0 and standard deviation of 0.1. Before performing these modifications, the F1 score is recorded when all the modalities are intact. After replacing the modalities with the noise mentioned above, the difference in F1 score is reported, as shown in Figure 21. The authors also added a zeroing-out method for comparison.

Figure 21 shows the change in F1 score over all and individual classes. For F1 over all the classes, the EEG shows the most significance. This change can be associated with NREM2 and NREM3, which have the most change. EEG plays an important role for the NREM2, NREM3, and Awake stages. EOG and EMG have more importance for the NREM1 and REM stages. For REM class, both EEG and EOG play an important role in high F1 scores. In general, the EMG does not seem to contribute as much as other modalities. The authors mentioned that the model is originally designed for an EEG signal, which might not be able to properly show the effect of EMG on the classification accuracy.

In [166], the authors proposed SleepTransformer (explained in Section 4.3). They introduced two mechanisms for interpretability: (1) EEG attention map, in which the attention score for an epoch is collected and the effect of different regions of an EEG signal can be observed. The goal is to see where the model pays more attention to achieve high accuracy; (2) Epoch influence bar chart, in which the model has access to multiple epochs. The influence of each epoch on neighboring epochs can be quantitatively studied. This resembles how clinicians make decisions for sleep scoring. In Figure 22, the heat map for epoch attention score and sequence-level distribution for attention scores is shown. A sequence of 11 epochs is presented for analysis. Figure 22a refers to the epoch influence bar chart, where the influence of other epochs on the target epoch is shown. The arrow at the bottom shows the current epoch. Figure 22b shows the learned EEG features for epoch representation (the time signal is reconstructed by Inverse STFT). In Figure 22c, the epoch-level attention score is depicted on the ground-truth EEG signal, and Figure 22d shows the attention score for the spectrogram.

It can be seen at the epoch level that the model pays attention to the key parts of the signal. These attention scores can be traced back to each sleep stage frequency range in which they are present. At sequence level, the attention weights control the contribution of each epoch to perform the classification. In Figure 22b, for stage N2, all the sequences to the left of the target signal contribute the most to the final decision. Interestingly, for the wake stage, the concentration is in the middle. Lastly, for the N1 stage, the attention scores disperse, which can be related to the fact that this signal resembles the N2 and Wake stages. For more examples, please refer to [166].

The authors of [167] proposed a cross-modal transformer, which enables them to use the attention to learn and interpret: (1) intra-modal relationships, (2) cross-modal relationships, and (3) inter-epoch relationships. The intra-modal relations are similar to [166], with a difference, where they introduce the $CL{S}_c$ token, as discussed in Section 4.3. In a similar procedure, the relationships between modalities are interpreted by the scaled dot-product attention of the representations in the output of the cross-modal attention block corresponding to $CL{S}_c$ of each modality. Finally, the inter-epoch relationships are calculated by scaled dot-product attention between the $CL{S}_{cross}$ of each epoch.

In Figure 23, the visualization for interpretability is demonstrated. Five epochs are present in this experiment. Inter-epoch attention plots show the relationships between the five epochs (similar to [166]) and how much impact they have on the current epoch for decision making. The cross-modal attention plot shows how much contribution each modality is providing for decision making. It can be observed in some cases that EEG has more relevance, while in others, EOG dominates the importance for decision making. Intra-modal attention plots are shown as a heat map on the original signals. The darker the shade, the more the model pays attention to that time section. It can be observed that the model highlights the important parts of the signal, which indicates which parts are most relevant to that sleep stage. Please refer to [167] for more examples.

In [159], the authors used empirical mode decomposition (EMD) [160], which is a data-driven algorithm that decomposes the signal into intrinsic mode functions (IMFs). The IMFs are calculated by the noise-assisted bivariate empirical mode decomposition (NA-BEMD) algorithm described in [159]. The EMD is applied to the model input in the first layer, followed by an attention module. The rest of the model is composed of CNN and RNN layers. Each IMF represents a frequency band that is helping with interpretability of the model by calculating the attention score. Higher attention score to the IMF shows that components of the frequency band of that IMF have more significance towards the sleep staging. The authors showed that IMF2 contains beta and alpha waves, IMF3 alpha and theta waves, IMF4 little alpha, theta, and delta waves, and IMF5 and IMF6 mostly delta waves. In Figure 24, the mean and standard deviation of the attention weights of each IMFs for sleep stages are depicted. IMF2 has the widest spectrum (as expected) and highest mean of attention weight in all stages. IMF2 is the largest for the Wake stage. The attention weights for IMF3 and IMF4 decreased for the Wake stage, which corresponded to low frequency being less present. It can be observed that there are meaningful correlations between the IMFs and the attention weights for each sleep stage.

As shown above, researchers are using different means to be able to interpret the results from the model’s output. Next, we will discuss a few other approaches. In [181], the authors used the LIME library [182] on top of their model for interpretability. In [127], the authors proposed something similar to the layout in [68] for model explainability. The model is trained in a semi-supervised sense, which is different from [68], which is completely supervised. In [122], a methodology using linear models was used, achieving similar performance to SOTA deep learning. The techniques used are easily interpreted, yielding greater insight on what factors are most relevant to the analysis. In [183], a decision-tree-based method is proposed for performing sleep stage classification with reasonably accurate results. It is easy to see what factors contribute the most to the classification. This along with [46,122] are examples of verifiable and interpretable techniques. As presented, most of the interpretable approaches such as [24,108], which show significant insight on how the model perceives the data, can help the clinicians trust DNNs to be incorporated in clinics for sleep disorder detection.

6. Discussion

6.1. General Observations

In this section, as shown in Table 3, we summarize the SOTA models from 2018 to 2023 in chronological order with the corresponding reported results for different datasets. In Table 4, we summarize the backbone architectures, number of parameters, and whether interpretability is incorporated in their scheme for all the models presented in Table 3.

Of the methods and techniques described in prior sections, ENGELBERT [51,170], SleepExpertNet [87], MaskSleepNet [163], Xsleepnet [78], and RobustSleepNet [77] achieve highest performance in terms of their respective performance score for most popular datasets, i.e., EDF-20, EDF-78, MASS, SHHS, and CAP, respectively. Techniques using attention and transformer-style models achieve high performance in this domain, demonstrating the importance of context modeling in classification for this task. The top performing techniques are enumerated in Table 3. Aside from performance, advancements have been made in terms of improving the trustworthiness of these models through interpretability, which is one of the major limiting factors to adoption of these techniques in clinical settings.

From Table 3, we can see that different types of architecture and modeling techniques have been proposed. In early attempts, CNNs and RNNs were utilized for sleep stage classification. In the following years, new modeling and techniques, such as transformers, GNNs, and contrastive learning were explored. Due to the abundance of data, learning how to capture proper features for downstream tasks is critical. The contrastive learning and SSL show strong evidence that these techniques, if applied correctly, can be extremely efficient in feature extraction, as demonstrated in [174,176].

The model performances can vary between different datasets, which can be attributed to data size, data quality, age of the participants, health, and other factors that can affect the data collection. In Table 2, we show the data size for some of the popular datasets. For the SleepEDF dataset, most models have relatively good performance, which can be due to the sufficient size and the quality of the data. The same goes for the MASS and SHHS datasets, which include a large number of high quality recordings. But this is not always the case, as we can see for the CAP dataset. This dataset is very large and it contains many recordings for long hours (please refer to Section 3.1 for more information). Modeling such a dataset is very challenging due to the variability of the data for each participant, which can be directly correlated to each participant’s bio-markers that may not be present in other participants, and due to data imbalance. In Section 6.2, we can observe a quantitative comparison between the top performing models for the EDF-20 and EDF-78 datasets.

Another major difference in performance can be attributed to the models’ design. For the SleepEDF (EDF-78) datasets, SleepExpertNet [87] has the best performance. As described in Section 4.2, this model is consists of different modules to efficiently extract temporal and sequential information. The first layer is a temporal CNN, which extracts the temporal feature that is fed to a positional encoder. Then, the sequential information is fed to an attention layer to capture the relationship between feature components. The output is processed via LSTM layers for classification. MaskSleepNet [163] has the next best model performance. This model has a multi-scale CNN layer followed by a multi-head attention layer. This approach uses modality masking for training to adapt the model to different scenarios where all modalities, such as EEG, EOG, and EMG, might not be available all at once. This training pushes the model to learn salient information and relationships between different modalities implicitly. Both these models can handle multiple modalities at the same time. The major difference between these two is in their architecture and how they are trained. It should be mentioned that MaskSleepNet [163] has the best performance for the MASS dataset. The second best performing model is BSTT [184]. This is a transformer-based approach that utilizes the Bayesian inference method. This model utilizes two transformer blocks to encode temporal and spatial information based on a raw EEG signal as input. XSleepNet [78] is the best performing model for SHHS dataset. This model, as described in Section 4.2, is a sequential multi-view model that takes in spectrogram and raw EEG signals as the input. The authors proposed to use gradient blending to balance the generalization/overfitting trade-off for classification. The second best model is SeqSleepNet [25], which is a sequential hierarchical RNN model. This model accepts a spectrogram as input to perform classification. These two models have many similarities, but the most prominent difference is that XSleepNet has a completely different training technique.

In Figure 25, for four datasets, the number of parameters (x-axis) and their respective reported metric (y-axis) are plotted. These plots do not contain all the models reported in Table 3 and Table 4, because we just know the complexity of some of the models. It should be mentioned that not all the reported metrics in the figure are the same; they are F1-score and kappa score. We can observe that smaller models can perform on par with bigger models or even better. This shows that better training schemes, as we explained above in the modeling sections, can be very efficient.

6.2. Quantitative Observations

Since EDF-20 and EDF-78 are the most commonly used datasets for reporting performance, we provide a more quantitative discussion about the models reporting on these datasets. Furthermore, the kappa score was utilized for most models. Hence, we focus on this metric. Given that $s_m$ is the kappa score for model m, $s^{top}$ is the top score for a dataset, and $s^{base}$ is the score for a baseline model, we define the percentage of improvement of the top model over a model m as:

$$p_m^{top}=\frac{(s^{top}-s_m)}{s_m}\times 100\%,$$

(1)

and the percentage of improvement of model m over the baseline model as:

$$p_m^{base}=\frac{(s_m-s^{base})}{s^{base}}\times 100\%.$$

(2)

We consider the SeqSleepNet [25] model as our baseline since this is the earliest model (published in 2018) that was considered.

For the EDF-20 dataset, we observe that the top ranking model is the ENGELBERT [170]. This model had a $p_m^{base}=1.7\%$ improvement over the baseline. The second best model was CoSleepNet [51] with a $p_m^{base}=1.1\%$. The improvement of the top model over CoSleepNet [51] is $p_m^{top}=0.6\%$. ENGELBERT also reported results for EDF-78 with an improvement of $p_m^{base}=2.3\%$. Unfortunately, the CoSleepNet [51] model did not report results for the EDF-78 dataset, so it could not be compared against the other models. All other models had either a negligible (around 0.1%) or negative improvement over the baseline.

For the EDF-78 dataset, we observe that the top ranking model is SleepExpertNet [87]. This model had a $p_m^{base}=12\%$ improvement over the baseline. The second best model was the MaskSleepNet [163] with a $p_m^{base}=9\%$. The improvement of the top model over MaskSleepNet [163] is about $p_m^{top}=3\%$. Unfortunately, the MaskSleepNet [163] and SleepExpertNet [87] models did not provide results for the EDF-20 dataset so they could not be compared to the top ranked models in this dataset. Nevertheless, MaskSleepNet [163] did achieve the top ranking for the MASS dataset. The third highest score model is NAMRTNet [185] with $p_m^{base}=4\%$ improvement over the baseline. Other models (Cross-Model [167], SleepCo [174], SleepTransformer [166], and IITNet [79]) had improvements over the baseline on the EDF-78 dataset between 1% and 2%.

These results (and the ones for EDF-20) indicate that there is not a single model that outperforms the rest in all datasets. However, the models do show an improvement between 1% and 12% over the last five years.

Overall, we can observe that, despite different model architectures and numbers of parameters, there is no guarantee that the models would perform the best over all the datasets. It seems training and data manipulation techniques for models to extract and adapt to the dataset have a bigger role in model efficiency in terms of performance than other factors. This evidence can be seen in Figure 25 and Table 3 and Table 4. In the subsequent section, limitations of current approaches are discussed.

Table 3. Comparing SOTA models. The main metric in this table is kappa score. Superscripts a and f refer to accuracy and macro-F1 score when the kappa score was not reported. The highest scores in terms of macro-F1 and kappa scores are highlighted in bold.

Model	Year	EDF-20	EDF-78	MASS	SHHS	CAP	DRM-SUB	ISRUC	SVUH-UCD	WSC	Other
SeqSleepNet [25]	2018	0.809	0.776	0.815	0.838	–	–	–	–	–	–
U-Time [82]	2019	0.79 ${}^f$	0.76 ${}^f$	0.80 ${}^f$	0.80 ${}^f$	0.68 ${}^f$	–	0.77 ${}^f$	0.73 ${}^f$	–	0.85 ${}^f$
IITNet [79]	2019	0.78	0.79	–	0.81	–	–	–	–	–	–
GraphSleepNet [171]	2020	–	–	0.834	–	–	–	–	–	–	–
Xsleepnet [78]	2020	0.813	0.778	0.823	0.847	–	–	–	–	–	–
RobustSleepNet [77]	2021	0.817 ${}^f$	0.779 ${}^f$	0.825 ${}^f$	0.80 ${}^f$	0.738 ${}^f$	–	–	–	–	–
CCRRSleepNet [132]	2021	0.78	–	–	–	–	–	–	–	–	–
SalientSleepNet [147]	2021	0.83 ${}^f$	0.795 ${}^f$	–	–	–	–	–	–	–	–
SleepTransformer [166]	2021	–	0.789	–	0.828	–	–	–	–	–	–
Cross-Modal [167]	2022	–	0.785	–	–	–	–	–	–	–	–
MtCLSS [175]	2022	0.80	–	–	–	–	–	–	–	–	0.74
MAtt [146]	2022	-	–	–	–	–	–	–	–	–	0.7471 ${}^a$
PearNet [85]	2022	0.793 ${}^f$	0.753 ${}^f$	–	–	–	–	–	–	–	–
CMS2-Net [176]	2022	–	–	–	–	–	–	–	–	–	0.71
SleepContextNet [59]	2022	0.79	0.76	–	0.81	0.71	–	–	–	–	–
mulEEG [56]	2022	–	0.6850	–	0.7366	–	–	–	–	–	–
Liu et al. [18]	2022	0.862 ${}^f$	0.852 ${}^f$	–	0.835 ${}^f$	–	–	–	–	–	–
MVF-SleepNet [88]	2022	–	–	–	–	–	–	0.795	–	–	–
TrustSleepNet [30]	2022	–	–	–	–	–	–	–	–	–	0.82
SoftVotingSleepNet [152]	2022	0.81	–	–	–	–	–	–	–	–	–
Kim et al. [128]	2022	0.838 ${}^f$	–	–	–	–	–	–	–	–	–
CRFs [149]	2022	0.79	–	–	–	–	0.76	–	0.66	–	–
IDNN [159]	2022	0.81	0.76	–	–	–	–	–	–	0.75	–
CAttSleepNet [138]	2022	0.78	0.74	–	–	–	–	–	–	–	–
ISENet [140]	2022	–	0.79	–	–	–	–	–	–	–	–
Bi-RNN [102]	2022	0.8404 ${}^a$	–	–	–	–	–	–	–	–	–
S2MAML [186]	2022	–	–	–	–	–	0.821 ${}^f$	0.888 ${}^f$	0.904 ${}^f$	0.863 ${}^f$	–
Pei et al. [103]	2022	–	–	–	0.76	–	–	–	0.58	–	–
SleepExpertNet [87]	2022	–	0.87	–	–	–	–	–	–	–	–
CDNN [96]	2022	–	–	–	–	–	–	0.854	–	–	0.734
Van Der Donckt et al. [122]	2022	0.812	0.766	0.803	–	–	–	–	–	–	–
SleepyCo [174]	2022	–	0.787	0.811	0.83	–	–	–	–	–	–
ENGELBERT [170]	2022	0.823	0.794	0.799	–	–	–	–	–	–	–
CoSleepNet [51]	2023	0.8181	–	–	–	–	0.7693	0.715	–	–	0.674
MaskSleepNet [163]	2023	–	0.847	0.847	–	–	–	–	–	–	0.812
SHNN [127]	2023	–	0.7051	–	–	–	0.6955	–	–	–	–
Siamese AE [187]	2023	0.79	–	0.81	–	–	–	–	–	–	–
BSTT [184]	2023	–	–	0.8437	–	–	–	0.7678	–	–	–
NAMRTNet [185]	2023	–	0.808	–	–	–	–	–	–	–	–

Table 3. Comparing SOTA models. The main metric in this table is kappa score. Superscripts a and f refer to accuracy and macro-F1 score when the kappa score was not reported. The highest scores in terms of macro-F1 and kappa scores are highlighted in bold.

Model	Year	EDF-20	EDF-78	MASS	SHHS	CAP	DRM-SUB	ISRUC	SVUH-UCD	WSC	Other
SeqSleepNet [25]	2018	0.809	0.776	0.815	0.838	–	–	–	–	–	–
U-Time [82]	2019	0.79 ${}^f$	0.76 ${}^f$	0.80 ${}^f$	0.80 ${}^f$	0.68 ${}^f$	–	0.77 ${}^f$	0.73 ${}^f$	–	0.85 ${}^f$
IITNet [79]	2019	0.78	0.79	–	0.81	–	–	–	–	–	–
GraphSleepNet [171]	2020	–	–	0.834	–	–	–	–	–	–	–
Xsleepnet [78]	2020	0.813	0.778	0.823	0.847	–	–	–	–	–	–
RobustSleepNet [77]	2021	0.817 ${}^f$	0.779 ${}^f$	0.825 ${}^f$	0.80 ${}^f$	0.738 ${}^f$	–	–	–	–	–
CCRRSleepNet [132]	2021	0.78	–	–	–	–	–	–	–	–	–
SalientSleepNet [147]	2021	0.83 ${}^f$	0.795 ${}^f$	–	–	–	–	–	–	–	–
SleepTransformer [166]	2021	–	0.789	–	0.828	–	–	–	–	–	–
Cross-Modal [167]	2022	–	0.785	–	–	–	–	–	–	–	–
MtCLSS [175]	2022	0.80	–	–	–	–	–	–	–	–	0.74
MAtt [146]	2022	-	–	–	–	–	–	–	–	–	0.7471 ${}^a$
PearNet [85]	2022	0.793 ${}^f$	0.753 ${}^f$	–	–	–	–	–	–	–	–
CMS2-Net [176]	2022	–	–	–	–	–	–	–	–	–	0.71
SleepContextNet [59]	2022	0.79	0.76	–	0.81	0.71	–	–	–	–	–
mulEEG [56]	2022	–	0.6850	–	0.7366	–	–	–	–	–	–
Liu et al. [18]	2022	0.862 ${}^f$	0.852 ${}^f$	–	0.835 ${}^f$	–	–	–	–	–	–
MVF-SleepNet [88]	2022	–	–	–	–	–	–	0.795	–	–	–
TrustSleepNet [30]	2022	–	–	–	–	–	–	–	–	–	0.82
SoftVotingSleepNet [152]	2022	0.81	–	–	–	–	–	–	–	–	–
Kim et al. [128]	2022	0.838 ${}^f$	–	–	–	–	–	–	–	–	–
CRFs [149]	2022	0.79	–	–	–	–	0.76	–	0.66	–	–
IDNN [159]	2022	0.81	0.76	–	–	–	–	–	–	0.75	–
CAttSleepNet [138]	2022	0.78	0.74	–	–	–	–	–	–	–	–
ISENet [140]	2022	–	0.79	–	–	–	–	–	–	–	–
Bi-RNN [102]	2022	0.8404 ${}^a$	–	–	–	–	–	–	–	–	–
S2MAML [186]	2022	–	–	–	–	–	0.821 ${}^f$	0.888 ${}^f$	0.904 ${}^f$	0.863 ${}^f$	–
Pei et al. [103]	2022	–	–	–	0.76	–	–	–	0.58	–	–
SleepExpertNet [87]	2022	–	0.87	–	–	–	–	–	–	–	–
CDNN [96]	2022	–	–	–	–	–	–	0.854	–	–	0.734
Van Der Donckt et al. [122]	2022	0.812	0.766	0.803	–	–	–	–	–	–	–
SleepyCo [174]	2022	–	0.787	0.811	0.83	–	–	–	–	–	–
ENGELBERT [170]	2022	0.823	0.794	0.799	–	–	–	–	–	–	–
CoSleepNet [51]	2023	0.8181	–	–	–	–	0.7693	0.715	–	–	0.674
MaskSleepNet [163]	2023	–	0.847	0.847	–	–	–	–	–	–	0.812
SHNN [127]	2023	–	0.7051	–	–	–	0.6955	–	–	–	–
Siamese AE [187]	2023	0.79	–	0.81	–	–	–	–	–	–	–
BSTT [184]	2023	–	–	0.8437	–	–	–	0.7678	–	–	–
NAMRTNet [185]	2023	–	0.808	–	–	–	–	–	–	–	–

Table 4. Building blocks (CNN, RNN, GNN, transformer and contrastive learning) for each of the models in Table 3, number of parameters (No. Par.), and whether interpretabilty (Intr.) is introduced or not. indicates the presence of aforementioned features.

Model	Year	CNN	RNN	GNN	Transf.	Contr. Lear.	Interp.	#No. Par.
SeqSleepNet [25]	2018	–	✓	–	–	–	–	∼0.2 M
U-Time [82]	2019	✓	–	–	–	–	–	∼1.2 M
IITNet [79]	2019	✓	✓	–	–	–	–	–
GraphSleepNet [171]	2020	–	–	✓	–	–	–	–
Xsleepnet [78]	2020	✓	✓	–	–	–	–	∼5.8 M
RobustSleepNet[77]	2021	–	✓	–	–	–	–	∼0.18 M
CCRRSleepNet [132]	2021	✓	✓	–	–	–	–	–
SalientSleepNet [147]	2021	✓	–	–	–	–	–	∼0.9 M
SleepTransformer [166]	2021	–	–	–	✓	–	✓	∼3.7 M
Cross-Modal  [167]	2022	✓	–	–	✓	–		∼4.05 M
MtCLSS [175]	2022	✓	–	–	–	✓	–	–
MAtt [146]	2022	✓	–	–	–	–	–	–
PearNet [85]	2022	✓	–	✓	–	–	–	–
CMS2-Net [176]	2022	✓	–	–	–	✓	–	–
SleepContextNet [59]	2022	✓	✓	–	–	–	–	–
mulEEG [56]	2022	✓	–	–	–	✓	–	∼0.6 M
Liu et al. [18]	2022	✓	–		–	–	–	–
MVF-SleepNet [88]	2022	–	–	–	–	–	–	–
TrustSleepNet [30]	2022	–	–	–	–	–	–	–
SoftVotingSleepNet [152]	2022	✓	–	–	–	–	–	∼0.79 M
Kim et al. [128]	2022	✓	–	–	–	–	–	–
CRFs [149]	2022	✓	✓	–	–	–	–	∼0.76 M
IDNN [159]	2022	✓	✓	–	–	–	✓	–
CAttSleepNet [138]	2022	✓	✓	–	–	–	–	–
ISENet [140]	2022	–	–	–	–	–	–	–
Bi-RNN [102]	2022	✓	✓	–	–	–	–	–
S2MAML [186]	2022	✓	–	–	–	–	–	∼0.6 M
Pei et al. [103]	2022	✓	✓	–	–	–	–	–
SleepExpertNet [87]	2022	✓	✓	–	–	–	–	–
CDNN [96]	2022	✓	–	–	–	–	–	–
Van Der Donckt et al. [122]	2022	–	–	–	–	–	✓	–
SleepyCo [174]	2022	✓	–	–	–	✓	–	–
ENGELBERT [170]	2022	✓	–	–	✓	–	–	∼0.49 M
CoSleepNet [51]	2023	✓	✓	–	–	–	–	–
LPT-Based [81]	2023	✓	–	–	–	–	–	∼0.16 M
MaskSleepNet [163]	2023	✓	–	–	–	–	–	–
SHNN [127]	2023	✓	✓	–	–	–	–	–
Siamese AE [187]	2023	✓	✓	–	–	–	–	–
BSTT [184]	2023	–	–	✓	✓	–	–	–
NAMRTNet [185]	2023	✓	–	–	–	–	–	–

Table 4. Building blocks (CNN, RNN, GNN, transformer and contrastive learning) for each of the models in Table 3, number of parameters (No. Par.), and whether interpretabilty (Intr.) is introduced or not. indicates the presence of aforementioned features.

Model	Year	CNN	RNN	GNN	Transf.	Contr. Lear.	Interp.	#No. Par.
SeqSleepNet [25]	2018	–	✓	–	–	–	–	∼0.2 M
U-Time [82]	2019	✓	–	–	–	–	–	∼1.2 M
IITNet [79]	2019	✓	✓	–	–	–	–	–
GraphSleepNet [171]	2020	–	–	✓	–	–	–	–
Xsleepnet [78]	2020	✓	✓	–	–	–	–	∼5.8 M
RobustSleepNet[77]	2021	–	✓	–	–	–	–	∼0.18 M
CCRRSleepNet [132]	2021	✓	✓	–	–	–	–	–
SalientSleepNet [147]	2021	✓	–	–	–	–	–	∼0.9 M
SleepTransformer [166]	2021	–	–	–	✓	–	✓	∼3.7 M
Cross-Modal  [167]	2022	✓	–	–	✓	–		∼4.05 M
MtCLSS [175]	2022	✓	–	–	–	✓	–	–
MAtt [146]	2022	✓	–	–	–	–	–	–
PearNet [85]	2022	✓	–	✓	–	–	–	–
CMS2-Net [176]	2022	✓	–	–	–	✓	–	–
SleepContextNet [59]	2022	✓	✓	–	–	–	–	–
mulEEG [56]	2022	✓	–	–	–	✓	–	∼0.6 M
Liu et al. [18]	2022	✓	–		–	–	–	–
MVF-SleepNet [88]	2022	–	–	–	–	–	–	–
TrustSleepNet [30]	2022	–	–	–	–	–	–	–
SoftVotingSleepNet [152]	2022	✓	–	–	–	–	–	∼0.79 M
Kim et al. [128]	2022	✓	–	–	–	–	–	–
CRFs [149]	2022	✓	✓	–	–	–	–	∼0.76 M
IDNN [159]	2022	✓	✓	–	–	–	✓	–
CAttSleepNet [138]	2022	✓	✓	–	–	–	–	–
ISENet [140]	2022	–	–	–	–	–	–	–
Bi-RNN [102]	2022	✓	✓	–	–	–	–	–
S2MAML [186]	2022	✓	–	–	–	–	–	∼0.6 M
Pei et al. [103]	2022	✓	✓	–	–	–	–	–
SleepExpertNet [87]	2022	✓	✓	–	–	–	–	–
CDNN [96]	2022	✓	–	–	–	–	–	–
Van Der Donckt et al. [122]	2022	–	–	–	–	–	✓	–
SleepyCo [174]	2022	✓	–	–	–	✓	–	–
ENGELBERT [170]	2022	✓	–	–	✓	–	–	∼0.49 M
CoSleepNet [51]	2023	✓	✓	–	–	–	–	–
LPT-Based [81]	2023	✓	–	–	–	–	–	∼0.16 M
MaskSleepNet [163]	2023	✓	–	–	–	–	–	–
SHNN [127]	2023	✓	✓	–	–	–	–	–
Siamese AE [187]	2023	✓	✓	–	–	–	–	–
BSTT [184]	2023	–	–	✓	✓	–	–	–
NAMRTNet [185]	2023	✓	–	–	–	–	–	–

7. Conclusions

In this paper, we trace the evolution from early attempts using CNNs and RNNs to recent exploration of methods like transformers and contrastive learning. Proper feature extraction is crucial, with self-supervised learning and contrastive techniques proving efficient. Model performance variations across datasets are attributed to factors such as data size, quality, and participant characteristics. Notably, SleepEDF, MASS, and SHHS datasets yield better results, while the challenging CAP dataset exhibits high participant variability and data imbalance. Model designs greatly affect performance differences, for example, SleepExpertNet [87], while XSleepNet [78] balances generalization using multi-modal input. SleepTransformer [166] offers interpretability and is strong for SHHS dataset, while MaskSleepNet [163] adapts through modality masking for MASS dataset. A parameter–performance plot, Figure 25, demonstrates recent models achieving higher or closer performance to bigger models with fewer parameters. We highlighted evolving techniques, dataset influence, model design impact, and the trend of smaller models yielding improved results.

We investigated sleep studies from various aspects, such as data manipulation, modeling, transfer learning, and interpretability. Not all of these aspects have been thoroughly investigated. It can be seen in Section 3 that different methods have been utilized for data preprocessing and augmentation. Although some of these techniques, such as adding noise, flipping, stretching, etc., show some promise, to the best of the authors’ knowledge, there are no studies to properly study these mechanisms and the changes they cause to sleep profiles in the sense of time and frequency features. In [52], some effort to evaluate the DA methods has been performed, but it needs more proper investigation of feature preservation in both time and frequency for reliable data generation. It should be mentioned that some attempts have been made to properly learn time and frequency features to properly perform data manipulation [50,52,57,58]. The current SOTA models use many parameters (trainable parameters). Since these models are supposed to be used in medical settings, some guarantees will help physicians to be inclined to use the models. As discussed in Section 5, some efforts have been conducted to add explainability to the models to describe their behavior. These efforts are normally restricted to visualizing attention layers, using masking techniques, or feature extraction to be able to understand how the information flows from input to output. Despite the methods mentioned in Section 5, more investigation can be performed to quantitatively measure the interpretability of a model and explore new techniques that are more generalizable and do not rely on specific model architectures.

In summary, we observe that there has been a trend of improvement on model performance (e.g., we observed a 1.7% and 12% improvement on the EDF-20 and EDF-78 datasets over the last five years, respectively). However, these improvements are not necessarily due to models with a higher number of parameters (since we did not see a correlation between number of parameters and performance), but instead due to the inclusion of more sophisticated architectures and data augmentation techniques (e.g., using transformers and contrastive learning).