Turkish Version of the New Instrument for Orthorexia Nervosa—Test of Orthorexia Nervosa (TON-17): A Validity and Reliability Study

This study aimed to adapt the Test of Orthorexia Nervosa- (TON-17) into Turkish and verify its validity and reliability. The study included 539 adults with a mean age of 30.2 ± 12.26 years. A reliability analysis was performed, along with a confirmatory factor analysis to test its construct validity. The time-invariance of the scale was examined by test–retest analysis, and its convergent validity was evaluated by a correlation analysis conducted to test the relationships between the scale and Eating Attitudes Test-26 and Obsessive Beliefs Questionnaire-9. Analyses were conducted using SPSS Version 28 and the AMOS-24 software. The Cronbach’s α internal consistency coefficient of the total scale was found to be 0.82, suggesting a strong internal consistency. The Cronbach’s α values of its factors were 0.68 for the “control of food quality”, 0.64 for the “fixation on health and a healthy diet”, and 0.73 for the “disorder symptoms”. In addition, the test–retest reliability was found to be 0.87 for the total scale, suggesting excellent reliability. Most of the fit indices (CMIN/df, RMSEA, AGFI, NFI, and TLI) of the scale were acceptable, and the GFI indicated a good model fit. This study shows that the Turkish version of TON-17, which is a new tool with a three-factor structure to evaluate orthorexia, is a valid and reliable scale.


Introduction
Orthorexia nervosa (ON) was first defined by Bratman in 1997 as "unhealthy obsession with healthy eating" [1]. In the current literature, ON is defined as "pathological obsession, fixation or preoccupation with healthy foods" [2]. ON is characterized by excessive concentration on food quality, food preparation, and nutrition [3]. ON has several symptoms, including avoidance of food additives such as preservatives, colors, flavorings, pesticides, excessive fat, sugar, and salt, or of genetically modified organisms [3,4]. In addition, individuals with ON may be obsessed with cooking methods and tools in the food preparation process, and they may feel guilt and fear when they go beyond their norms in this process [1,5]. Thoughts on healthy nutrition, food preparation, and labeling of foods can negatively affect both quality of life and social life in individuals with ON [6]. ON is not yet considered to be a psychiatric eating disorder, and it is not included in the 11th revision of the International Classification of Diseases (ICD-11) [7] or in the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) [8]. While there is no consensus on the definition of ON so far, a multidisciplinary cohort of experts (47 eating disorder researchers and multidisciplinary treatment specialists) published a consensus document on the definition and diagnostic criteria for ON in 2022 [9]. In this consensus document, ON was characterized by "a strong preoccupation with one's eating behavior and with self-imposed rigid and inflexible rules which are strictly controlled and include spending an excessive amount of time for planning, obtaining, preparing and/or eating one's food" [9]. Although a consensus has been published on 27 criteria 2 of 12 in the definition and diagnosis of ON, it seems that there is no gold-standard scale used in diagnosis.
In Turkey, the ORTO-11 scale-a short version of ORTO-15-is used frequently in studies to determine ON tendencies [25]. However, there are contradictions in the results of these studies in terms of validity and reliability, with a suggestion of high ON frequencies.
In addition, the ONI has recently been adapted into Turkish by Kaya et al. [26], and it was stated to be a valid and reliable scale. The ONI contains 24 questions and has no cutoff point [26]. Based on this, it was thought that TON-17 could be a useful tool for clinicians in academic research and practice, especially since it is a short-form scale and has a cutoff point. This study will contribute to the literature by evaluating the validity and reliability of the Turkish version of TON-17, which was developed in line with the current literature.

Participants
A questionnaire created using Google Forms was used as a data collection tool and sent to individuals via e-mail or WhatsApp, and the individuals were selected by snowball sampling. Those who agreed to participate in the study completed the questionnaire online after giving their consent. All participants were Turkish citizens who could speak Turkish and lived in Turkey. This study was conducted with healthy adults, and individuals diagnosed with eating disorders or psychiatric diseases were not included in the study, as this may have had a confounding effect. Validity and reliability studies have reported that the sample size should be 5-10 times the number of scale items [27]. The scale used in this study consists of 17 items; therefore, we aimed to reach at least 85-170 individuals for conducting the study. A total of 539 adults aged between 18 and 65 years, including 131 males (24.3%) and 408 females (75.7%), participated in this study.

Design and Procedure
To adapt TON-17 into Turkish, permission was obtained from Aleksandra Rogowska [24] (one of the researchers who developed the scale) via e-mail. TON-17 was translated according to the guide created by Beaton et al. [28]. The process of cross-cultural adaptation of TON-17 was as shown in Figure 1.
individuals for conducting the study. A total of 539 adults aged between 18 and 65 years, including 131 males (24.3%) and 408 females (75.7%), participated in this study.

Design and Procedure
To adapt TON-17 into Turkish, permission was obtained from Aleksandra Rogowska [24] (one of the researchers who developed the scale) via e-mail. TON-17 was translated according to the guide created by Beaton et al. [28]. The process of cross-cultural adaptation of TON-17 was as shown in Figure 1. The reliability of the questionnaire was tested with test-retest and internal consistency analyses. To examine its test-retest reliability, the final version of the Turkish TON-17 was administered to 30 adult volunteers twice, with an interval of 15 days.
The questionnaire, which was prepared with the final version of the Turkish TON-17 (see the Supplementary Materials), was applied online. The questionnaire collected information about the participants' sociodemographic characteristics (such as age, gender, and educational status). Their body weight (kg) and height (cm) were self-reported. To evaluate the convergent validity of TON-17, the Eating Attitudes Test Short-Form (EAT-26) [29,30] and the Obsessive Beliefs Questionnaire Short-Form-9 (OBQ-9) [31,32] were used. The eating behaviors of the participants were evaluated with EAT-26, and obsessive disorders were evaluated with OBQ-9.

Test of Orthorexia Nervosa (TON-17)
The Test of Orthorexia Nervosa (TON-17) is a self-report scale developed by Rogowska et al. [24] in 2021 to evaluate ON. Based on the scientific literature review and interviews with people at risk of orthorexia, it was first developed as 40 items (TON-40) to test ON, and the number of items was reduced to 17 as a result of structural analysis. TON-17 is considered to be a useful tool to assess the risk of ON. It has three sub-factors (control of food quality (items 1, 4, 7, 10, 13, and 16), fixation on health and healthy diet (items 2, 5, 8, 11, and 14), and disorder symptoms (items 3, 6, 9, 12, 15, and 17)) and a general factor (total of 3 sub-factors). Each item is rated on a five-point Likert-like scale, The reliability of the questionnaire was tested with test-retest and internal consistency analyses. To examine its test-retest reliability, the final version of the Turkish TON-17 was administered to 30 adult volunteers twice, with an interval of 15 days.
The questionnaire, which was prepared with the final version of the Turkish TON-17 (see the Supplementary Materials), was applied online. The questionnaire collected information about the participants' sociodemographic characteristics (such as age, gender, and educational status). Their body weight (kg) and height (cm) were self-reported. To evaluate the convergent validity of TON-17, the Eating Attitudes Test Short-Form (EAT-26) [29,30] and the Obsessive Beliefs Questionnaire Short-Form-9 (OBQ-9) [31,32] were used. The eating behaviors of the participants were evaluated with EAT-26, and obsessive disorders were evaluated with OBQ-9.

Test of Orthorexia Nervosa (TON-17)
The Test of Orthorexia Nervosa (TON-17) is a self-report scale developed by Rogowska et al. [24] in 2021 to evaluate ON. Based on the scientific literature review and interviews with people at risk of orthorexia, it was first developed as 40 items (TON-40) to test ON, and the number of items was reduced to 17 as a result of structural analysis. TON-17 is considered to be a useful tool to assess the risk of ON. It has three sub-factors (control of food quality (items 1, 4, 7, 10, 13, and 16), fixation on health and healthy diet (items 2, 5, 8, 11, and 14), and disorder symptoms (items 3, 6, 9, 12, 15, and 17)) and a general factor (total of 3 sub-factors). Each item is rated on a five-point Likert-like scale, indicating the degree of compliance with the sentence (1 = strongly disagree, 2 = disagree, 3 = undecided, 4 = agree, 5 = strongly agree). The scale has no reverse scoring. A higher score is associated with a greater risk of ON. The cutoff value for ON is a general factor score equal to or above 61. The internal consistency (Cronbach's α) was 0.79 for the general factor, 0.80 for the 1st factor (F1) (control of food quality), 0.81 for the 2nd factor (F2) (fixation on health and a healthy diet), and 0.74 for the 3rd factor (F3) (disorder symptoms).

Eating Attitudes Test Short-Form (EAT-26)
This is a 26-question version of the Eating Attitudes Test-40 developed by Garner and Garfinkel [33] in 1979, which was revised and shortened by Garner et al. [30]. The scale Nutrients 2023, 15, 3178 4 of 12 was adapted into Turkish by Ergüney-Okumuş and Sertel-Berk [29]. In addition to its ease of use and scoring, this scale provides advantages in terms of psychometric properties, economy, and practicality. The Cronbach's α internal consistency coefficient of the scale was found to be 0.84 in the Turkish adaptation study [29]. In this study, the Cronbach's α of the scale was determined to be 0.80.
EAT-26 is used to detect and define eating disorder behaviors in healthy individuals and to diagnose anorexia nervosa and bulimia nervosa, which are also called eating disorders. All scale items except for the 26th item are scored as "3 = Always, 2 = Very often, 1 = Often, 0 = Sometimes, 0 = Rarely, 0 = Never". However, the 26th item is scored in reverse as "1 = Sometimes, 2 = Rarely, and 3 = Never", where other options are worth 0 points. A scale score of 20 or above indicates deterioration in eating behavior, and a higher score is associated with more evident eating disorder [29].
Obsessive Beliefs Questionnaire Short-Form-9 (OBQ-9) The Obsessive Beliefs Questionnaire (OBQ), which was developed by an international working group to evaluate cognitive biases specific to obsessive compulsive disorder and consisted of 87 items in its initial version, was later used as OBQ-44 with 44 items [34,35]. Gagné et al. [31] developed a 9-item short version in 2018. The Turkish validity and reliability study of the Obsessive Beliefs Questionnaire Short-Form was conducted in 2019 by Yorulmaz et al. [32]. The 9-item short form of the OBQ, which has 3 subscales (perfectionism and intolerance to uncertainty (Cronbach's α = 0.75), responsibility and threat perception (Cronbach's α = 0.74), and importance and control over thoughts (Cronbach's α = 0.70)), was used in this study. Both the total score and the 3 sub-dimension scores of the scale can be used separately [32]. In this study, the Cronbach's α was found to be 0.76 for the total OBQ-9.

Body Mass Index (BMI)
The participants' body weight (kg) and height (cm) values were self-reported. BMI was calculated by dividing the body weight in meters by the square of the height in meters

Statistical Analysis
The data were analyzed using IBM SPSS version 28 and the AMOS-24 software. Descriptive statistics were used to analyze the general characteristics of the sample. Skewness and kurtosis tests were used to determine whether the data were normally distributed. Cronbach's alpha and McDonald's omega coefficients were used to determine the reliability and internal consistency of the Turkish version of TON-17, where values above 0.60 and 0.70 were considered to show acceptable and good fits, respectively [27]. Test-retest reliability was evaluated with the correlation coefficient calculated using the data obtained from a subsample of 30 participants who filled out TON-17 twice. A confirmatory factor analysis (CFA) was performed using the AMOS-24 software to determine the validity of the scale and to analyze the compatibility of its subscales. Maximum likelihood was used to observe parameter estimation for CFA. The multiple fit indices, including CMIN/df (chi-squared goodness of fit), RMSEA (root-mean-square error of approximation), GFI (goodness-of-fit index), AGFI (adjusted goodness-of-fit index), NFI (normed fit index), and TLI (Tucker-Lewis index), were evaluated within the scope of the CFA [37]. The convergent validity was examined using Pearson's or Spearman's correlation coefficients between TON-17 and other scales, depending on whether the data were normally distributed or not. The significance level was accepted as 0.05 in all statistical analyses.

Results
This study evaluated the validity and reliability of the Turkish version of TON-17, using data obtained from a total of 539 adults, including 131 males (24.3%) and 408 females (75.7%). Like the original TON-17, a hierarchical bi-factor model was found to be suitable for the Turkish version, with three lower-order factors (control of food quality, fixation on health and healthy diet, and disorder symptoms) and one general higher-order factor (orthorexia). The Cronbach's α internal consistency coefficient was found to be 0.82 and McDonald's omega was found to be 0.86 for the general factor, suggesting a strong internal consistency. The Cronbach's α was found to be 0.68 for factor 1, 0.64 for factor 2, and 0.72 for factor 3; similarly, McDonald's omega was found to be 0.68 for factor 1, 0.63 for factor 2, and 0.70 for factor 3. In the analysis performed to evaluate the test-retest reliability of the Turkish version of TON-17, a significant positive correlation was found for both the general factor and the sub-factors (p < 0.005) ( Table 2). The mean scores of the items and factors of TON-17 are presented in Table 3. The mean score for the first factor was 17.9 ± 3.92, that for the second factor was 17.0 ± 2.92, that for the third factor was 13.3 ± 3.78, and that for the general factor was 48.1 ± 8.55.  Table 4 presents the fit indices of the model, and Figure 2 shows the three-factor model fit diagram. Among the goodness-of-fit indices, CMIN/df (χ 2 /df) was found to be 3.81, RMSEA was 0.07, GFI was 0.90, AGFI was 0.87, NFI was 0.80, and TLI was 0.81 (Table 4). On the basis of CFA, the standard factor loading of the scale varied between 0.37 and 0.58 for factor 1, between 0.37 and 0.60 for factor 2, and between 0.31 and 0.66 for factor 3. Accordingly, the factor loading of TON-17, which consists of 3 factors and 17 items, was found to be at an acceptable level ( Figure 2). Table 5 presents the correlations of TON-17's general factor and sub-factors with the EAT-26 and OBQ-9 scales. The general factor of TON-17 had a significant low-level positive correlation with EAT-26 (r:0.29) and OBQ-9 (r:0.35) (p < 0.001). CMIN/df: chi-squared goodness of fit, RMSEA: root-mean-square error of approximation, GFI: goodness-of-fit index, AGFI: adjusted goodness-of-fit index, NFI: normed fit index, TLI: Tucker-Lewis index. * Values of thresholds for acceptable and good fit [37].  Table 5 presents the correlations of TON-17's general factor and sub-factors with the EAT-26 and OBQ-9 scales. The general factor of TON-17 had a significant low-level positive correlation with EAT-26 (r:0.29) and OBQ-9 (r:0.35) (p < 0.001).  The mean EAT-26 score was 8.5 ± 7.35, and the mean OBQ-9 score was 38.8 ± 8.73. In addition, 7.4% of the individuals were found to have ON, considering the cutoff score for TON-17 as ≥61, and 8.4% of the individuals were found to have eating disorders, considering the cutoff score for EAT-26 as ≥20. In addition, the EAT-26 and OBQ-9 scores of individuals with and without ON were evaluated, as shown in Table 6. The EAT-26 total score, OBQ-9 total score, and OBQ-9 subscale scores were found to be significantly higher in adults with ON (p < 0.001). The observed effect sizes were medium (>0.50) or large (>0.80).

Discussion
ON is characterized by consuming healthy foods and focusing on their content and quality [22], and it can be briefly defined as an obsession with healthy eating [38]. This behavior may include an unbalanced diet due to beliefs about the "purity" of food, strict avoidance of unhealthy foods, feelings of guilt and anxiety after eating violations, or intolerance to other people's food beliefs [22,39]. Extreme orthorexic behaviors may cause physical health to deteriorate due to malnutrition and cause social and academic failures due to obsessive thoughts and behaviors focused on beliefs about healthy eating [39]. Donini et al. [9] also stated in the consensus report that ON-related behaviors negatively affect health status and quality of life. Therefore, the diagnosis and treatment of ON may be important in improving health and quality of life.
In the literature, different scales (BOT, ORTO-15, EHQ, DOS, TOS, BOS, and ONI) have been used to evaluate ON [3], and their positive and negative aspects have been discussed [3,4,38]. A decrease in the total score on ORTO-15-one of the most frequently used scales in the literature-indicates higher orthorexic behavior [38]. While the individuals with the lowest scores are defined as having ON, some of them have been observed to have healthy eating behaviors and normal personalities, and this has not been discussed by researchers [3,38]. In the following years, several versions of ORTO-15 with different item numbers, factor numbers, and cutoff scores were developed [25,[40][41][42]. Therefore, the usability of ORTO-15 has been questioned by many studies due to the high frequency of ON in the groups studied, the lack of information about how the items are created, and the lack of a clear, validated tool [3,4,38]. The use of ORTO-15 is considered dubious due to these limitations, and its use is not recommended, although it is popular.
In a systematic review to examine the psychometric properties of scales assessing ON, it was stated that there is no gold standard for assessing ON [2]. Thus, future studies will be important for establishing a gold standard for determining the risk of ON [24]. In this regard, the validity and reliability studies of TON-17 in different societies will contribute to the literature. The TON-17 scale was developed based on qualitative methods (e.g., interviews) and a reconceptualization of ON using cutting-edge technology. In addition, statistical techniques such as EFA, CFA, and CCA supported the structural construction of TON-17. TON-17 consists of three sub-factors, and this structure for ON has been used previously in DOS, EHQ, and ORTO-15 [2]. In TON-17, sub-factors F1 and F2 are both associated with healthy orthorexia, while F3 is associated with unhealthy Nutrients 2023, 15, 3178 9 of 12 orthorexia [24]. More recently, it has been proposed to distinguish between ON, as defined here, and "healthy orthorexia", which refers to a non-pathological interest in healthy eating and food culture [43][44][45].
This study was conducted to determine the validity and reliability of the Turkish version of TON-17 [24]. Similar to the original TON-17 (Cronbach's α 0.79 for the general factor), the reliability analysis of the Turkish version of TON-17 suggested good internal consistency (Cronbach's α 0.82) for the general factor. The Cronbach's α coefficients for the three sub-factors of the scale were lower than those in the original study (0.80, 0.81, and 0.74 for F1, F2, and F3, respectively), but still at an acceptable level (0.68, 0.64, and 0.72 for F1, F2, and F3, respectively). The Cronbach's α coefficient of the ORTO-15 scale, which is the most frequently used for evaluating orthorexia in Turkey, is 0.62, and the internal consistency of TON-17 was found to be higher than that of ORTO-15 [25]. On the other hand, the internal consistency of the Turkish ONI (Cronbach's α 0.91) [26] was higher than that of TON-17 (Cronbach's α 0.82). However, the lack of test-retest reliability of the Turkish ONI may be a disadvantage, and it also has no cutoff point. In this study, the test-retest reliability was found to be 0.87 for the general factor, suggesting excellent reliability for the Turkish TON-17. Most of the fit indices (CMIN/df, RMSEA, AGFI, NFI, and TLI) obtained by CFA were found to be at acceptable or good (GFI) levels. Accordingly, TON-17 can be considered to be a valid and reliable tool to determine the risk of ON in Turkish adults, and it is associated with both eating disorders and obsessive behavior.
Clinical observation and existing research have indicated that ON may share important traits with both eating disorders and obsessive-compulsive disorder (OCD) [46]. In a recent meta-analysis examining the associations between ON, eating disorders, and obsessive-compulsive symptoms, ON symptoms were shown to be more associated with eating disorders (r:0.36, p: < 0.01) than with OCD (r:0.21, p < 0.01) [47]. However, in this study, parallel to the original study [24], it was found that the TON-17 general score was more correlated with general OCD scores (r:0.35, p < 0.01) compared to the EAT-26 scores (r:0.29, p < 0.01) ( Table 5). The fact that the relationships were not strong is a recurring theme throughout both this study and the meta-analysis. On the other hand, it should be considered crucial that the EAT-26 total score, OBQ-9 total score, and OBQ-9 subscale scores were found to be significantly higher in adults with ON in this study, since studies using different ON scales have shown similar results in the literature [48][49][50].
Niedzielski et al. [3] have reported that there are no reliable data on the prevalence of ON, and that there are differences in the frequency of ON depending on the diverse cutoff scores and tools used in the existing studies. Due to the lack of standardized diagnostic tools, there is a large variability in the prevalence of ON in the literature, ranging from 0% to 97% [51,52]. The prevalence of ON varies by country of study, participant profile, and the tool used to assess ON in the study. While the prevalence of ON was 6.9% [53] in one study using ORTO-15, it was 75.2% in another study [54]. It has been reported that the use of ORTO-15 to diagnose ON should be questioned due to the high rate of false positive results [3]. The prevalence of ON was found to be 5.5% in the study of Rogowska et al. [24]. Also, the prevalence of ON was found to be 7.5% in the present study. Future studies of TON-17 in diverse cultures will contribute to the literature.
The Turkish validity and reliability of a scale for orthorexia, which is the cutoff point, is one of the strengths of this study. In addition, the retest was performed in the study, and the results were statistically significant, constituting another strength of this study. Although it was conducted with a large sample, this study has some limitations that should be noted. One of the limitations of this study is that the majority of the participants were women. It is thought that this may be related to the fact that women show more interest in studies related to nutrition. In addition, the high level of education of most of the participants makes it difficult to generalize the findings to the adult population.

Conclusions
In conclusion, this study has shown the validity and reliability of the Turkish version of TON-17-a new tool with a three-factor structure to evaluate orthorexia. This scale will contribute to the literature, allowing future studies to better examine the prevalence, risk, and diagnosis of ON.