A Turkish Validity and Reliability Study of the Swiss Narcolepsy Scale

Abstract

Background: The clinical evaluation of a patient complaining of excessive daytime sleepiness is of crucial importance for the diagnosis of narcolepsy. The Swiss Narcolepsy Scale (SNS) was developed in 2004 as a screening tool for patients with narcolepsy and shown in three different studies to have a high sensitivity and specificity for narcolepsy type 1 (NT1). The aim of this study was to assess the validity and reliability of the Turkish version of SNS (SNS-TR). Patients and Methods: Twenty-one healthy controls, 26 patients with idiopathic hypersomnia, and 27 patients with narcolepsy were recruited from five accredited sleep centers in Turkey. Pearson’s correlation coefficient and degree of freedom were used to determine the validity of each question. Cronbach’s alpha was calculated to assess the internal consistency or reliability of Likert-type questions. The inter-rater reliability was tested using Cohen’s kappa analysis, and the intra-class correlation coefficient (ICC) was used to evaluate the validity and reliability between two evaluations with a one-month interval. Results: Sensitivity and specificity of SNS-TR were 90.5% and 100%, respectively, for diagnosing NT1. Cronbach’s alpha was 0.976, showing a highly reliable level of internal consistency. The inter-rater reliability of the questions and the validity and reliability between two evaluations were moderate or above. Conclusion: This study provides evidence for the validity and reliability of SNS-TR in diagnosing and discriminating NT1 from other disorders of hypersomnolence with a very high sensitivity and specificity.

1. Introduction

Narcolepsy is one of the central disorders of hypersomnolence, with a prevalence varying between 25 and 50 per 100,000 people worldwide [1]. Disease-related symptomatology usually appears in adolescence and young adulthood, although patients with pediatric- or geriatric-onset disease have also been reported [2]. The pathophysiology of narcolepsy is multifactorial, including genetic, immunologic, and environmental factors [3]. Quality of life, characterized by levels of psychosocial and occupational health, is greatly impaired and the risk of having traffic and occupational accidents is increased in these patients—mainly because of severe excessive daytime sleepiness (EDS) and cataplexy attacks. In light of these data, it is important to support the early diagnosis of narcolepsy and initiate treatment at an early stage to resolve the symptoms, increase patients’ quality of life, and decrease/eliminate the risks to public health.

The diagnosis of narcolepsy is made based on the latest version (3rd edition, text revision) of the International Classification of Sleep Disorders (ICSD) defined by the American Academy of Sleep Medicine (AASM) [1]. The clinical evaluation of a patient complaining of EDS is of crucial importance as the first step in diagnosing narcolepsy, as daily periods of irrepressible need to sleep or daytime lapses into sleep occurring for at least three months are mandatory in the ICSD-3-TR [1]. Asking the right questions and ensuring the accurate integration of clinical data in an efficient manner will provide a better selection of patients requiring further investigation. The tests used in the evaluation of EDS are scant; the most commonly used and practical test worldwide is the Epworth Sleepiness Scale (ESS), although it is not specific for narcolepsy [4].

Sturzenegger and Bassetti [5] developed a simple, easy-to-perform, self-administered test specific to the diagnosis of narcolepsy with cataplexy—the Swiss Narcolepsy Scale (SNS). This new test includes five questions assessing the following parameters: (i) the difficulty in initiating sleep; (ii) the presence of unrefreshed sleep in the morning; (iii) the need for having a nap at noon; (iv) the attacks of weakness in the knees or buckling sensation triggered by the emotional stimuli such as laughing or anger; and (v) the attack of sagging of the jaw triggered by emotional stimuli. The SNS was shown in three different populations to have high sensitivity and specificity for the diagnosis of narcolepsy with cataplexy (narcolepsy type 1, NT1). In the first study conducted in 57 narcoleptics with cataplexy, 56 patients with non-narcoleptic hypersomnia, and 40 normal controls, it was demonstrated that the SNS is an accurate test to screen/diagnose patients with NT1 with the identification of hypocretin-1-deficient patients [5]. In the second study conducted in 80 narcoleptics with cataplexy and 111 non-narcoleptic patients, SNS was demonstrated to be superior to other tests, including the ESS or the Ullanlinna Narcolepsy Scale (UNS) [6]. In the third study conducted in a dataset of 299 patients, an updated form of the SNS [7] included new scoring coefficients and an optimal cut-off point and confirmed the discriminating power of the SNS and its short-form for the diagnosis of NT1 against NT2 and other types of hypersomnolence.

In Turkey, there is no study showing the prevalence of narcolepsy in the general population, though a nation-wide study has shown a prevalence of EDS of 5.4% [8]. An estimated prevalence of 30–35 per 100,000 people may be suggested, considering its geographical location. Because the measurements of the cerebrospinal fluid (CSF) hypocretin levels are not available in our country, the clinical anamnesis, together with polysomnography (PSG) and the multiple sleep latency test (MSLT), constitute the mainstay in the diagnosis of narcolepsy and its differentiation from the other types of central hypersomnolence [1]. As there are only a limited number of specialized sleep centers for narcolepsy (about 12 centers), a practical screening tool is needed, especially for non-sleep experts, to decrease the delay and increase the accuracy of the referral of patients with hypersomnolence to these specialized sleep centers for narcolepsy-specific diagnostics. As there are differences with the pediatric population, both in terms of clinic [3] and of narcolepsy assessment scales [9], we here aimed to investigate the validity and reliability of the Turkish version of the SNS in a cohort of adult participants with central hypersomnolence disorder.

2. Materials and Methods

2.1. Development of the SNS-TR Version

SNS is composed of five questions answered by the participants, including the frequency of having difficulty falling asleep, being rested in the morning, having short-lasting naps during the day, and experiencing a buckling sensation of the knees or sagging of the jaw triggered by the emotions. Each question is scored from 1 to 5, where 1 point represents the absence of the symptoms and 5 points denotes the frequent (almost always/daily) occurrence of the symptoms. A negative result of the equation points to narcolepsy based on the following formula:

Narcolepsy score = [6 × (Question 1)] + [9 × (Question 2)] − [5 × (Question 3)] − [11 × (Question 4)] − [13 × (Question 5)] + 20

Members of the Turkish Neurology Society’s Sleep Medicine Study Group planned to conduct a Turkish validity and reliability study of the SNS. On obtaining written consent from the developers of the scale, the SNS was translated from English to Turkish (SNS-TR) based on international ISPOR criteria [10]. An independent researcher with an interpreting certificate in English translated the SNS-TR back to English, and revisions were needed in three questions (numbers 1, 4, and 5), especially in the verbs of the sentences. The latest Turkish form was re-translated into English, re-evaluated, and confirmed. On the basis of international ISPOR criteria [10], upon the completion of preparation, forward translation, reconciliation, back translation, back translation review, harmonization, and cognitive debriefing, proofreading was carried out, and the Turkish version of the instrument was finalized.

2.2. Cohort of the Study

Upon completion of developing the SNS-TR version, researchers in five different cities in Türkiye performed the test. The selection of the centers was based on the presence of accreditation by the Turkish Sleep Medicine Society, the presence of a Sleep and Disorders Unit and outpatient clinic for patients with narcolepsy and other central disorders of hypersomnolence, and the location of the cities in different districts of Türkiye. Considering the low prevalence of narcolepsy and the number of the items (n = 5) used in the questionnaire, power curve analysis resulted in the recommendation that at least 25 participants per group should be included (confidential range [CR] 90% and confidential interval [CI] 5). For this reason, the inclusion of at least 15 participants in every center was planned, including five healthy participants, five patients with a diagnosis of idiopathic hypersomnia, and five patients with narcolepsy (NT1 and/or NT2).

2.3. Participants and Procedures

All participants were consecutively enrolled in the study during a study period of 2 months; demographic data (including gender, age, and education) and ESS were noted. Detailed clinical data were obtained from every participant, and a full-night PSG test with the MSLT was performed the next day. Patients with any other sleep-related disorders, such as sleep apnea or periodic limb movement disorder, were excluded. The diagnosis of central hypersomnolence disorders was made based on the latest criteria defined by the AASM [1]. Secondary or symptomatic narcolepsy or patients with daytime sleepiness due to other possible causes than central disorders of hypersomnolence were not included, which were investigated on clinical grounds with laboratory and neuroimaging investigations. The differentiation of NT1 from NT2 was made based on the presence of typical cataplexy, and the differentiation of NT2 and idiopathic hypersomnia was made based on the number of sleep-onset rapid eye movement (REM) periods (SOREMPs) in the MSLT or nocturnal PSG [1]. Measurements of the CSF hypocretin level are lacking in our study. The typing for HLA-DQB1*0602 was available in 12 patients and positive in 9 patients (75%), all of which were diagnosed as having NT1; of the other three patients, two had NT1 and one had NT2.

A healthy control group consisted of participants admitted to the sleep disorders unit complaining of subjective EDS, whose PSG and MSLT results were normal. All patients and participants in the healthy control group were consecutively enrolled in the study during the study period. Participants without PSG or MSLT were excluded; among other exclusion criteria were the presence of other sleep-related disorders, other medical conditions associated with EDS, use of drugs and/or substances that could otherwise impact EDS or night-time sleep architecture, and refusal to participate. None of the individuals in the healthy control group showed more than one SOREMP in the MSLT [11,12].

To evaluate inter-cultural differences, the SNS-TR was administered to all participants once and repeated one month later by the same researcher to avoid bias in the interviews. The participants were asked to make no changes in their lifestyles, social activities, or drug regimens during the study period.

The study was approved by the Local Ethics Committee of Uludag University Faculty of Medicine, Bursa (2011-KAEK-26/252), and written informed consents were obtained from all participants. Some of the results of this paper were presented at the 8th Congress of the European Academy of Neurology in 2022, and this is an extension of the conference paper [13].

2.4. Statistical Analysis

All data obtained in the study were gathered and analyzed using IBM Statistical Package for the Social Sciences (SPSS) version 20.0. The Kolmogorov-Smirnov test was used to analyze the distribution of the continuous data. Nominal data were analyzed using the chi-square test, continuous data without normal distribution or ordinal data were analyzed using the Kruskal–Wallis test, and continuous data with normal distribution data were analyzed using the Student t-test. The multiple comparisons were adjusted for the demographic variables, including age, gender, and level of education. The degree of freedom of Pearson’s correlation coefficients was used to determine the validity of each question. The sensitivity and specificity of five questions were also assessed by using the receiver operating characteristic (ROC) and the area under the ROC curve analyses. The construct validity of the SNS-TR was assessed by its association with the external criterion defined in the ICSD by the AASM, and Cronbach’s alpha was calculated to assess the internal consistency or reliability of the scale with Likert-type questions. The inter-rater reliability of the SNS-TR among the researchers was tested using Cohen’s kappa statistical analysis [14]. The intraclass correlation coefficient (ICC) was used to evaluate the validity and reliability between two evaluations with a 1-month interval. The level of statistical significance was considered to be a p-value < 0.05. The multiple comparisons were corrected by using the Benjamin–Hochberg procedure (with a false discovery rate of q = 0.05).

3. Results

3.1. Characteristics of the Study Population

Of the seventy-eight participants enrolled in the study, 74 completed the study protocol: 21 participants were healthy controls, 26 had idiopathic hypersomnia, and 27 had narcolepsy. Of patients with narcolepsy, 21 had NT1 (77.8%) and 6 had NT2 (22.2%). Comparisons of the demographic and clinical data among all groups are given in

Table 1. The comparison of the demographic and clinical data among the study groups.

Parameters	Healthy Controls (n = 21)	Idiopathic Hypersomnia (n = 26)	Narcolepsy (n = 27)		p Values 1
			Type 1 (n = 21)	Type 2 (n = 7)
Males/Females (%)	38.1/61.9	38.5/61.5	52.4/47.6	33.3/66.7	n.s. *,#,§,£
Age (years)					n.s. *
(mean + sd)	32.0 + 9.2	34.8 + 11.0	29.4 + 8.0	28.2 + 9.1	0.038 #
[median, 25–75% limits]	[33.0, 25.0–38.0]	[37.5, 23.0–44.25]	[29.0, 21.0–37.0]	[31.0, 18.0–36.25]	n.s. §,£
Level of education (years)					0.002 *
(mean + sd)	13.9 + 3.0	10.4 + 3.7	11.2 + 3.0	10.6 + 2.6	0.009 #
[median, 25–75% limits]	[15.0, 11.0–15.0]	[11.0, 5.0–13.5]	[11.0, 11.0–13.0]	[11.0, 8.0–12.0]	n.s. §,£
ESS points					<0.001 * <0.001 # 0.033 § n.s. £
(mean + sd)	6.2 + 3.4	13.4 + 3.6	18.2 + 3.5	15.5 + 5.6
[median, 25–75% limits]	[4.5, 4.5–10.0]	[12.5, 11.0–15.25]	[18.0, 16.0–21.5]	[15.5, 10.75–21.25]
Sleep latency in MSLT (min)					<0.001 * <0.001 # <0.001 § n.s. £
(mean + sd)	14.4 + 2.6	4.3 + 2.6	2.0 + 2.4	2.3 + 2.2
[median, 25–75% limits]	[14.0, 13.0–16.0]	[5.15, 2.275–6.25]	[1.18, 0.62–2.2]	[1.85, 0.375–4.2]
Initial SNS-TR points					n.s. * <0.001 # <0.001 § <0.001 £
(mean + sd)	27.5 + 18.5	25.0 + 15.4	−39.5 + 37.8	24.2 + 17.3
[median, 25–75% limits]	[33.0, 10.5–42.5]	[22.0, 16.5–33.5]	[−33.0, −72.5–−13.5]	[22.0, 9.75–44.0]
Second SNS-TR points					n.s. * <0.001 # <0.001 § <0.001 £
(mean + sd)	26.2 + 14.0	23.3 + 14.8	−36.4 + 34.2	25.3 + 16.0
[median, 25–75% limits]	[27.0, 13.5–36.0]	[22.0, 13.0–31.5]	[−33.5, −72.5–−13.0]	[23.0, 11.0–44.0]

¹ p values between the control group and the patients with idiopathic hypersomnia *; between the control group and the patients with narcolepsy ^#; between the patients with idiopathic hypersomnia and narcolepsy ^§; between the patients with NT1 and NT2 ^£.

. Patients with narcolepsy were younger than those in the control group (p = 0.038). The level of education was higher in healthy controls than in those with idiopathic hypersomnia and narcolepsy, but there was no significant difference between patients with idiopathic hypersomnia and narcolepsy. ESS scores were higher in patients with narcolepsy than those in all other groups, and they were higher in patients with idiopathic hypersomnia than in healthy controls (Table 1). The evaluation of MSLT revealed that the mean sleep latency was lowest in patients with narcolepsy and highest in controls; these differences were highly significant (Table 1).

3.2. Validity and Reliability of the SNS-TR

All participants were successfully evaluated by the SNS-TR at baseline and at the end of 1 month, with no missing patients. The sample size of the study constituted 74 participants, and the degree of freedom (DF) was set at 72. The critical values at 72 DF for the two-tailed proportions were 0.232 for a p-value < 0.05 and 0.303 for a p-value <0.01 in the table for the critical values for Pearson’s correlation coefficients. In this calculation, the construct validity of the SNS-TR was measured and quantified, showing the reliability and validity of the criterion variables. The obtained values for all five questions were significantly above the critical value (ranging between 0.315 and 0.884), demonstrating that every question was valid (

Table 2. Pearson’s correlation coefficients of the five questions in SNS-TR.

Questions	Initial Application	Second Application after One Month
Q1	0.477 *	0.457 *
	<0.001 #	<0.001 #
Q2	0.558 *	0.502 *
	<0.001 #	<0.001 #
Q3	0.427 *	0.315 *
	<0.001 #	0.007 #
Q4	0.740 *	0.820 *
	<0.001 #	<0.001 #
Q5	0.869 *	0.884 *
	<0.001 #	<0.001 #

* Pearson’s correlation coefficients; # p values.

).

We observed that all patients with NT1 but two had a negative SNS-TR score. None of the patients with NT2 or idiopathic hypersomnia and no healthy controls had an SNS-TR score < 0 points, which is schematically shown in Figure 1, in comparison with the ESS points of the study population. The sensitivity and specificity of the SNS-TR were found to be 90.5% and 100%, respectively, for diagnosing NT1. The positive and negative predictive values for the initial SNS-TR were 0.85 and 0.94, respectively, while they were calculated as 1.0 and 0.96 for the second SNS-TR.

The Cronbach’s alpha based on the standardized items was found to be 0.976, showing a very reliable level of internal consistency for the SNS-TR scale. The inter-rater reliability of the questions, tested by Cohen’s kappa, showed that the level of agreement was moderate for Question 2; substantial for Questions 1, 3, and 4; and excellent for Question 5 (

Table 3. The validity and reliability of SNS-TR in whole study population.

Questions	Kappa	p Value	ICC	ICC (Min–Max)	F Value	p Values
1	0.661	<0.001	0.918	0.869–0.948	12.139	<0.001
2	0.548	<0.001	0.923	0.877–0.951	12.906	<0.001
3	0.663	<0.001	0.910	0.856–0.943	11.063	<0.001
4	0.788	<0.001	0.955	0.929–0.972	22.442	<0.001
5	0.858	<0.001	0.989	0.983–0.993	92.219	<0.001

). The ICCs testing the validity and reliability between two evaluations with 1-month intervals demonstrated almost perfect agreement for all questions (Table 3).

The area under ROC curve analyses of the first and second SNS-TR showed fair performance compared to that of ESS, which showed a fail response (Figure 2).

4. Discussion

We aimed to establish the Turkish validity and reliability of the SNS, which was the first study to perform a validity and reliability study of the SNS in another language. In this study, we developed a Turkish version of the SNS and investigated the discriminative power of the SNS-TR among patients with NT1, NT2, and idiopathic hypersomnia, as well as healthy controls. Pearson’s correlation coefficients showed that all five questions were above the critical value and highly significant. A score of SNS-TR lower than 0 was suggested as the ‘narcolepsy score’ [5]. In our study population, all patients but two with NT1 had a negative SNS-TR score. None of the patients with NT2 or idiopathic hypersomnia had an SNS-TR score < 0 points, nor did the healthy controls (as shown in Figure 2). Except for two patients with NT1, all had a negative score in SNS-TR. These two patients (one female, 21 years old; one male, 20 years old) had positive HLA-DQB1*0602, while CSF hypocretin could not be measured. In a close look, it was observed that the answers to the 4th and 5th questions regarding cataplexy were in accordance with NT1, while the first three questions regarding the night-time sleep pattern and sleep quality were not; this may in part explain this inconsistency. Nevertheless, we observed that the sensitivity and specificity values of the SNS-TR were very high for NT1 (90.5% and 100%, respectively). The positive and negative predictive values were also highly suggestive of the correct diagnosis (1.0 and 0.96, respectively). Based on the standardized items, Cronbach’s alpha showed a high level of internal consistency for the SNS-TR scale. The validity and reliability between the two evaluations with 1-month intervals were observed to have almost perfect agreement for all questions.

We demonstrated that the SNS-TR allowed the diagnosis and discrimination of NT1 from other disorders of hypersomnolence with very high sensitivity and specificity. Although we cannot rule out the possibility that some patients classified as NT2 may be in fact patients with NT1 but without cataplexy—that may or may not develop several years after EDS—due to the lack of CSF hypocretin measurements, none of the patients with NT2 had a SNS-TR score below zero. In addition, the area under ROC curve analyses [15] showed that both the first and second SNS-TR results had fair performances in diagnosing NT1 (opposed to clinical, PSG, and MSLT results), while ESS showed a fail response. Similar to SNS, UNS was developed as a screening tool for the diagnosis of narcolepsy by Hublin et al. in 1994 [16]. The sensitivity and specificity of UNS in recognizing NT1 were shown to be around 85% and 88%, respectively, against other central disorders of hypersomnolence, including NT2 and idiopathic hypersomnia [17]. Moreover, a moderate correlation was observed between the UNS and hypocretin levels in CSF. While the SNS was demonstrated to be superior to the UNS in narcoleptics with cataplexy [6], the UNS has been suggested to be rather good at detecting not only NT1 but also combined types 1 and 2 with a sensitivity and specificity of 80% and 88%, respectively [17]. The Turkish validation of the Ullanlinna Narcolepsy Scale should be encouraged as another screening tool for narcolepsy.

Recently, a reappraisal for the diagnosis of central disorders of hypersomnolence was formed by European experts [18]. The division of NT1 and NT2 was substituted with “narcolepsy” to improve diagnostic clarity, and narcolepsy was divided into “definite”, “probable”, “familial”, and “symptomatic or secondary” groups. The main tools used in the diagnostic criteria of narcolepsy were clinical symptomatology, PSG and MSLT findings, CSF hypocretin and human leukocyte antigen (HLA) measurements, and other specific tests used in the detection of secondary causes. The use of narcolepsy-specific tests such as SNS or UNS was not placed in this proposal, but ESS was suggested to be used in measuring the severity of EDS. In the same proposal, the distinction between “typical” and “atypical” cataplexy was made to be used in the new classification of narcolepsy. A questionnaire was previously designed by Overeem and his colleagues [19] for a broad description of the clinical aspects of cataplexy, which was not suggested as a diagnostic tool but enabled the distinction of “typical” versus “atypical” forms of cataplexy among a large phenotypical diversity. As cataplexy is the only diagnostic marker for NT1 in the absence of CSF hypocretin measurements, grading cataplexy will be necessary with different levels of diagnostic confidence.

Other than cataplexy, which is of vital importance in diagnosing NT1, the MSLT features are also very important to differentiate narcolepsy from other central disorders of hypersomnolence or from idiopathic excessive sleepiness [18,20]. Because the diagnostic value of MSLT is greatly altered by other sleep disorders such as sleep apnea, shift work, or chronic sleep deprivation, a delicate and detailed evaluation is a requisite for patients with EDS. A long-term actigraphic evaluation of the pre-MSLT period may be helpful to rule out false-positive results. In addition, multiple SOREMPs are prevalent in the general population, which question the distinguishing role of MSLT between narcolepsy without cataplexy and idiopathic hypersomnia and need to be solved by developing new validated tests [20,21]. In another study, a mean sleep latency of <2 min and the presence of ≥3 SOREMPs in MSLT were found to be highly specific (95%) to predict hypocretin deficiency at the expense of sensitivity (39%) [22]. The transition from wakefulness, or the N1 sleep stage, to REM sleep in MSLT before the N2 sleep stage commenced was also suggested to increase the yield of MSLT in differentiating narcolepsy from other disorders of hypersomnolence, while it had no role in the differentiation of NT1 and NT2 [23]. On the other side, a short latency of REM sleep (≤15 min) at nocturnal PSG was proposed to be a highly specific positive predictive value for NT1 [24]. Similarly, a recent study showed that the percentage of SOREMP in nocturnal PSG was significantly higher in NT1 (37.1%) than in NT2, together with a shorter mean sleep latency in MSLT in patients with NT1 [25]. In sum, as proposed by European experts [18], clinical assessment, PSG, and MSLT findings are important tools in the evaluation of patients with EDS, in addition to CSF hypocretin and HLA measurements, if feasible. However, narcolepsy-specific scales, including SNS and UNS, have also demonstrated utility in the identification of NT1, though further reliability and validation studies are needed [26]. Questionnaires to identify NT2 and distinguish it from other hypersomnolence, on the other hand, are lacking in clinical practice.

In light of these data, the implementation of a symptom-based screening tool is needed to improve the differential diagnosis of patients complaining of EDS. Our results have shown that the Turkish version of the SNS, SNS-TR, may reliably be used in the Turkish population for the diagnosis of NT1. One of the strengths of this study is that it is a multi-centric validation study from five different centers. To the authors’ knowledge, this is the first validation study for a narcolepsy scale in Turkish as well as in a foreign language. On the other hand, we are aware that the low sample size as well as the moderate-poor results of the ROC analysis remain important limitations of the manuscript. This should be taken into consideration when using the Turkish version of the scale, for which a larger sample is needed to improve this scale and better delineate the feasibility of the SNS-TR. Nevertheless, the use of such a practical scale as a screening clinical method for the NT1 will lessen the delay in the diagnosis of narcolepsy due to a lack of awareness and the complexity of its symptomatology. Another important limitation of our study is the lack of CSF hypocretin and HLA measurements in all patients, which may change the subgroups of patients with narcolepsy.

5. Conclusions

In conclusion, the SNS-TR, which obviously cannot replace the need for a detailed interview and objective assessments, is a very useful screening tool for narcolepsy and can be suggested for improving the referral of patients with hypersomnolence to specialized sleep centers.