Digital Healthy Diet Literacy and Self-Perceived Eating Behavior Change during COVID-19 Pandemic among Undergraduate Nursing and Medical Students: A Rapid Online Survey

Assessing healthy diet literacy and eating behaviors is critical for identifying appropriate public health responses to the COVID-19 pandemic. We examined the psychometric properties of digital healthy diet literacy (DDL) and its association with eating behavior changes during the COVID-19 pandemic among nursing and medical students. We conducted a cross-sectional study from 7 April to 31 May 2020 at 10 public universities in Vietnam, in which 7616 undergraduate students aged 19–27 completed an online survey to assess socio-demographics, clinical parameters, health literacy (HL), DDL, and health-related behaviors. Four items of the DDL scale loaded on one component explained 71.32%, 67.12%, and 72.47% of the scale variances for the overall sample, nursing, and medical students, respectively. The DDL scale was found to have satisfactory item-scale convergent validity and criterion validity, high internal consistency reliability, and no floor or ceiling effect. Of all, 42.8% of students reported healthier eating behavior during the pandemic. A 10-index score increment of DDL was associated with 18%, 23%, and 17% increased likelihood of healthier eating behavior during the pandemic for the overall sample (OR, 1.18; 95%CI, 1.13, 1.24; p < 0.001), nursing students (OR, 1.23; 95%CI, 1.10, 1.35; p < 0.001), and medical students (OR, 1.17; 95%CI, 1.11, 1.24; p < 0.001), respectively. The DDL scale is a valid and reliable tool for the quick assessment of digital healthy diet literacy. Students with higher DDL scores had a higher likelihood of healthier eating behavior during the pandemic.


Introduction
The unprecedented coronavirus disease (COVID-19) pandemic has created vast socioeconomic burdens [1], morbidity, and mortality [2,3]. The pandemic has also changed eating behaviors for the worse [4][5][6][7]. To deal with this challenge, the USA National Institute of Health Nutrition Research Task Force developed a 2020-2030 strategic plan to improve health and prevent or combat diseases and conditions that have been affected by food and nutrition [8].
Healthy dietary intake has shown protective effects on the immune systems and health outcomes during the COVID-19 crisis [9,10]. There are no data available, and rigorous clinical trials conducted to elaborate and confirm the benefits of a healthy diet in the current COVID-19 pandemic. Diversified and balanced diets improve the immune response to viral infection (e.g., SARS-CoV-2) [11], helping to reduce the severity [12] and any complications of COVID-19 [13]. Yet, unhealthy eating behaviors rapidly increased worldwide [14,15], including in Vietnam [16]. In ASEAN countries, the proportions of disordered eating attitudes among university students were 10%-20.6% [17].
The COVID-19 pandemic has also caused an "infodemic" of diverse information and sources [18,19]. Health literacy [20] and eHealth literacy [21] are highly recommended as strategic approaches to contain the pandemic. Nursing and medical students were mobilized to respond to the global health crisis [22][23][24][25], and because these future health workers can help combat misinformation and disinformation [26,27], their digital literacy should be promoted [28,29].
It is critical to promote healthy dietary behaviors to improve health outcomes [30] and contain the COVID-19 disease and its consequences [31][32][33]. The assessments of eating behaviors and nutritional status are critical to identifying comprehensive approaches for managing COVID-19 [34] and suggesting sustainable food intake [35]. To identify behavior changes and develop strategies to improve students' eating behaviors, their ability to access, understand, appraise, and apply the healthy diet information on the internet during the pandemic requires a short, valid, and reliable survey tool.
We examined the psychometric properties of an expanded digital healthy diet literacy (DDL) domain of health literacy to investigate associations between DDL and eating behavior changes during the COVID-19 pandemic among nursing and medical students at 10 public universities in Vietnam.

Study Design
We conducted a cross-sectional online survey from 7 April to 31 May 2020. Participants from 10 universities across Vietnam were recruited, including those from six universities in the nation's north, two in the south, and two in the country's central region. The study settings were all public universities that train and provide the nation's healthcare workforce.

Study Sample
Undergraduate medical and nursing students were invited as they are the medical university's major focus and the country's future healthcare providers. The data collection procedure was similar to a previous study [36]. In brief, researchers as university lecturers invited students to voluntarily participate, and there were no direct benefits or compensations for participating in the survey. Lecturers sent the survey link to student leaders via email, Messenger, and Zalo. The student leaders sent the link to other students. The online consent forms were signed before taking the survey. Students filled in the form with their name and phone number to avoid duplications, while data were coded and analyzed confidentially.
A total sample of 7616 students (out of 32,632 possible participants) aged 21 to 27 years completed the survey. The studied and possible students at each university are listed in Table 1.

Sociodemographics and Clinical Parameters
Students reported sociodemographic information, including age, gender, academic year (1 to 6 for the medical field, 1 to 4 for nursing field), academic field (medical vs. nursing), and ability to pay for medication (very or fairly difficult vs. very or fairly easy). Clinical parameters were assessed including self-reported body height (cm) and weight (kg). Body mass index (BMI, kg/m 2 ) was also calculated. Students also reported health problems that resembled symptoms of COVID-19, which were recorded as suspected COVID-19 symptoms (S-COVID-19-S) [37]. These included common symptoms of fever, cough, and dyspnea along with less common symptoms of myalgia, fatigue, sputum production, confusion, headache, sore throat, rhinorrhea, chest pain, hemoptysis, diarrhea, and nausea/vomiting. If students had any of those symptoms, they were classified as having S-COVID-19-S or symptoms like COVID-19. Chronic health problems were assessed using Charlson comorbidity index items [38,39].

Health Literacy and Digital Healthy Diet Literacy
Health literacy (HL) was evaluated using a 12-item short-form health literacy questionnaire (HLS-SF12) that has been widely used in Asian countries [40], including in Vietnam [41][42][43]. The HLS-SF12 questionnaire was used to measure comprehensive health literacy, including four stages of information processing (e.g., accessing, understanding, appraising, and applying), and three domains of health (e.g., health care, disease prevention, and health promotion) [44,45].
During the COVID-19 pandemic, digital healthy diet literacy (DDL) guides people towards healthier eating behaviors that can improve immune resistance [9,10]. Therefore, as an expanded concept of health literacy, DDL refers to the ability to access, understand, judge, and apply digital healthy-diet-related information to improve healthy eating behavior and health outcomes that are critical to contain the pandemic. We adapted the HL conceptual framework and expanded a DDL domain by adding four more items to assess the information processing ability, including the ability to (1) . . . find reliable and accurate healthy diet information on the internet, (2) . . . understand healthy diet information and dietary guidelines on the internet, (3) . . . judge whether healthy diet information on the internet is applied for individuals, and (4) . . . apply healthy diet information from the internet into individuals' daily lives to eat healthily.
Participants reported difficulty in performing each task item based on 4-point Likert scales from 1 = very difficult to 2 = fairly difficult, 3 = fairly easy, and 4 = very easy. The HL and DDL indices were standardized to a unified metric from 0 to 50 with higher scores representing better HL or DDL [46] using Formula (1): where Index is the specific index calculated, Mean is the mean of all participating items for each individual, 1 is the minimal possible value of the mean (leading to a minimum value of the index of 0), 3 is the range of the mean, and 50 is the chosen maximum value. Both HLS-SF12 and DDL are subjective measures.

Eating Behavior and Other Lifestyle Changes
Students reported their current health-related behaviors compared with before the pandemic [36]. We did not assess the actual dietary intake or eating habits in this study. Students simply self-reported the perception of eating behaviors. Students rated their eating behavior as less healthy, unchanged, and healthier. They also ranked their smoking, drinking, and physical activity on a scale ranging from never to stopped, less, unchanged, and more. During the pandemic, healthier eating, continued or additional physical activity, and decreases in or the cessation of both smoking and drinking are crucial, along with the risk management strategies of hand-washing, mask-wearing, and social distancing [31]. On the one hand, we did not investigate whether the "unchanged" eating behavior during the pandemic was considered "unhealthy" or "healthy" behavior. On the other hand, in health promotion, people are suggested to comply with healthy behaviors on every basis. These healthy behaviors are even more important during the pandemic; people should have healthier eating behaviors that can improve immune function and prevent viral infection. Therefore, we aimed to compare the "healthier diet" and other behavioral categories, including "unchanged" and "less healthy". Similarly, because physical activity and cessation/reduction of smoking/drinking are suggested, "never, stopped, or less" were grouped into one category, and "unchanged or more" was another category. "Healthier diet" and "unchanged or more physical activity" were positive behaviors; "unchanged or more" smoking or drinking were negative behaviors.

Ethical Consideration
The study was reviewed and approved by the Institutional Ethical Review Committee of Hanoi University of Public Health, Vietnam (IRB No. 133/2020/YTCC-HD3). Students voluntarily took the survey.

Psychometric Properties of Digital Healthy Diet Literacy
Principal component analysis (PCA) with the oblique rotation (Promax) method was utilized to assess the construct of the DDL scale. Correlations between the DDL scale and its four items were estimated using Spearman's correlation test which provided evidence of item-scale convergent validity. In addition, the correlation between DDL and HLS-SF12 was estimated using the Pearson correlation test that provides evidence of criterion validity [47]. Cronbach's alpha test was used to check the internal consistency of the DDL scale. The floor and ceiling effects of the DDL scale were assessed by calculating percentages of the possibly lowest and highest DDL index scores.

Health Literacy, Digital Healthy Diet Literacy, and Eating Behavior Changes
The frequency of eating behaviors (less healthy/unchanged vs. healthier) in different categories of socio-demographics, clinical parameters, and other health-related behaviors were explored using a Chi-square test. In addition, a one-way ANOVA test was used to check the distribution of HL and DDL scales by two categories of eating behavior. Multivariable logistic regression models were used to assess the associations of HL and DDL along with eating behavior change. Adjusted variables in the multivariable model were those demonstrating the associations with eating behavior change at p < 0.20 in the univariable analysis [48]. In order to exclude colliders which may cause multicollinearity, Spearman's correlation test was used to check correlations between them. If the case of a moderate or high correlation, a representative variable was selected in the multivariable analysis.
Data analysis was conducted using IBM SPSS Version 20.0 for Windows (IBM Corp., Armonk, NY, USA). p < 0.05 was set for statistical significance.

Psychometric Properties of Digital Healthy Diet Literacy
The Kaiser-Meyer-Olkin values (KMO) for the whole scale of the overall sample, nursing, and medical students were 0.78, 0.77, and 0.79, respectively. In addition, ranges of KMO values for individual items of the overall sample, nursing, and medical students were 0.75-0.83, 0.74-0.81, and 0.75-0.83, respectively. These values were higher than 0.6, which was set for measuring the sampling adequacy [49]. Furthermore, Bartlett's Test of Sphericity values of the overall sample, nursing, and medical students were less than 0.05 which indicated data suitability for the PCA [49].
Four items were strongly loaded on a single component and explained 71.32%, 67.12%, and 72.47% of the scale variance in the overall sample, nursing, and medical students, respectively ( Table 3). Ranges of correlations between the DDL scale and its four items were 0.80-0.83, 0.76-0.80, and 0.81-0.84 for the overall sample, nursing, and medical students, respectively. This provides adequate evidence of the item-scale convergent validity [50]. The DDL correlated with HL at a rho value of 0.68 for all, which provided satisfactory evidence of the criterion's validity [47]. Table 3. Construct, convergent, criterion validity, internal consistency, floor, and ceiling effects of digital healthy diet literacy scale (N = 7616). Cronbach's alpha values of the DDL scale for the overall sample, nursing, and medical students were 0.86, 0.83, and 0.87, respectively, which were larger than 0.70, indicating the satisfactory reliability [51]. Percentages of the lowest and highest score in the overall sample (0.20% and 12.00%), nursing students (0.30% and 7.90%), and medical students (0.20% and 13.30%) were smaller than 15% which were reflected no floor and ceiling effects [52].

Associations of HL and DDL with Eating Behavior Changes
Age highly correlated with the academic year (rho = 0.95), gender moderately correlated with the academic field (rho = 0.36), and smoking moderately correlated with drinking (rho = 0.50; Table S1). Therefore, age, gender, and drinking were selected with medical payment ability, BMI, chronic health conditions, and physical activity for the overall sample; with BMI, and physical activity for the sample of nursing students; and with medical payment ability, chronic health conditions, and physical activities for the sample of medical students in the multivariable analysis. After adjusting for these mentioned confounders, a 10-score increment of HL and DDL indices were associated with the increased likelihood of healthier eating behavior of 23% and 18% for the overall sample, 24% and 23% for the sample of nursing students, and 23% and 17% for the sample of medical students (Table 4).

Discussion
The current study shows the digital healthy diet literacy (DDL) scale is a brief, valid, and reliable tool with satisfactory construct, criterion, and convergent validity, with a high level of internal consistency reliability, and with no floor or ceiling effect. The psychometric properties of the expanded DDL domain of HL were similar to the HLS-SF12 scale [40,41,53] and e-healthy diet literacy (e-HDL) scale [54]. Moreover, the performance of the DDL scale was similar in medical and nursing students. In a previous study, the e-HDL scale was developed with inconsistent response options for eleven survey questions. The number of questions was not equally distributed in different subscales [54]. Therefore, a shorter DDL questionnaire with the same concept and response options of the original health literacy that is needed for research and practice [44,45].
In this study, our findings illustrate that higher scores of health literacy and digital healthy diet literacy were positively associated with healthier eating behavior during the pandemic for both nursing and medical students. College students with higher eHealth literacy engaged better in positive health-promoting behaviors [29,55]. A previous study also showed that higher e-HDL scores were associated with better health status and health-related behaviors in the general population in Taiwan [54]. In addition, food literacy was found to be associated with healthier food consumption in the public in the Netherlands [56] and Canada [57]. In university settings, it is strategic to create opportunities (e.g., academic courses, media, and the internet) and motivators (e.g., social responsibility, and personal development) for students to develop food literacy and apply them in improving healthy eating behaviors and health outcomes [58]. Digital healthy diet literacy should be emphasized and taken into account for better decision making and health outcomes [59].
Medical school students play important roles in responding to the pandemic [22,23,60] and can promote healthy behaviors (e.g., healthy eating, preventive behaviors) to their peers, communities, and patients using social media or other available modalities [61,62]. Improving digital literacy focusing on a healthy diet for nursing and medical students is a strategic approach [28,29] that can influence public behavior in positive ways.
We conducted an online cross-sectional study in which the findings cannot be interpreted as causal relationships. The category of "unchanged" eating behavior during the pandemic does not distinguish "unhealthy" from "healthy" behaviors that may potentially bias the analysis. In addition, we did not use questions about actual nutritional habits or behaviors and diet. The actual dietary intake was not assessed in the current study. Future studies are suggested to recruit questionnaires investigating dietary intake, such as a food frequency questionnaire or a 24 h food record form. Another limitation was that respondents were not informed about the meaning of "healthy eating behaviors" that may cause reporting bias and potentially affect the association between DDL and eating behaviors. Furthermore, the test-retest reliability of the DDL scale was not evaluated. However, the findings from the large sample analysis can possibly be generated to nursing and medical students. Despite the limitations, the online survey is suggested in conducting the study with fewer resources required and providing timely evidence for research and practice, further contributing to COVID-19 containment. Future studies are suggested to compare DDL, eating behaviors, and their association in students with those in general populations and healthcare workers. A longitudinal design is recommended to assess the long-term effect and causality.

Conclusions
The digital healthy diet literacy scale with four items (DDL-4) was an extended domain of a comprehensive health literacy framework. This scale was found to be a valid and reliable tool for the quick assessment of students' ability to access, understand, appraise, and apply healthy diet information found on the internet. Nursing and medical students with higher HL or DDL scores had a higher likelihood of healthier eating behavior. The findings of our study provide a brief, valid, and reliable tool for research and practice during and after the COVID-19 pandemic. Funding: This work was supported by the Taipei Medical University (108-6202-008-112; 108-3805-022-400).

Acknowledgments:
We would like to thank all experts, research assistants, and students who participated in the study.

Conflicts of Interest:
The authors declare no conflict of interest.