Comparison of Self-Administered Web-Based and Interviewer Printed Food Frequency Questionnaires for Dietary Assessment in Italian Adolescents

Innovative tools for assessing food and nutrient intakes in adolescence are essential to uncover the long-term effects of diet on chronic diseases. Here, we developed and compared a web-based self-administered food frequency questionnaire (web-FFQ) with a traditional interviewer printed FFQ (print-FFQ) among 174 Italian adolescents (aged 15–18 years). To assess the reliability of the web-FFQ compared with the print-FFQ, we used Spearman’s rank correlation coefficients, Wilcoxon rank test, quartile misclassification analysis, Cohen’s weighted kappa and the Bland–Altman method. Correlation coefficients ranged from 0.14 (i.e., pizza) to 0.67 (i.e., raw vegetables) for foods, and from 0.45 (i.e., monounsaturated fatty acids, MUFA) to 0.62 (i.e., zinc) for nutrients. Results from the Wilcoxon rank test showed that food and nutrient intakes were comparable between two FFQs, except for nuts, shellfish, fruit juices and MUFA. Adolescents classified into the same or adjacent quartiles ranged from 68.2% (i.e., tea) to 89.1% (i.e., raw vegetables and dipping sauces) for foods, and from 77.2% (i.e., vitamin C) to 87.2% (i.e., folate and calcium) for nutrients. Except for pizza, the weighted kappa indicated moderate to substantial agreement for other foods and nutrients. Finally, we demonstrated that the web-FFQ significantly overestimated shellfish and fruit juice intakes, while it underestimated nuts, canned fish, olive oil, total energy intake, fatty acids and calcium. The limits of agreement analysis indicated moderate to wide individual differences for all groups. In conclusion, our self-administered web-FFQ represents an easy, suitable and cost-effective tool for assessing food and nutrient intakes in adolescents. However, the wide individual differences in level of agreement suggest that additional refinements and calibrations are necessary to investigate the effects of absolute nutrient intakes at the individual level.


Introduction
Although adolescents-young people between 10 and 19 years old-are often considered a healthy group, most of them suffer chronic ill health or disability [1]. Several population-based studies from Western countries showed that 20-30% of adolescents had a chronic illness [2]. In the last decades, global reduction in the burden from infectious disease has been accompanied by a growing importance of non-communicable diseases (NCDs), such as mental and substance use disorders, cancer, congenital anomalies and hemoglobinopathies [1]. However, the leading causes of death and disability vary with age, sex and socioeconomic status, so challenges for each country may be different [1]. In addition, several NCDs in adulthood have their roots in childhood and adolescence [3]. Indeed, behaviors underpinning the major NCDs-tobacco and alcohol use, unhealthy diet and physical inactivity, overweight and obesity-usually start or are reinforced during the second decade of life [4][5][6]. Due to the rising number of obese infants, children and adolescents, the World Health Organization set up the Commission on Ending Childhood Obesity, which in turn proposed a set of recommendations to tackle the childhood and adolescent obesity epidemic [7]. This Commission recognized that a multidisciplinary approach is crucial for sustained progress. For instance, countries should survey prevalence and trends in childhood and adolescent obesity at national, regional and local levels. They should also collect data on diet and physical activity of children and adolescents across different socioeconomic groups and settings. Although some data are collected, a significant lack that needs to be filled remains for children over 5 years and adolescents [8]. This data might guide the development of appropriate priorities and strategies, providing a benchmark against which the efficacy of policies and programs can be measured [8]. Large-scale epidemiologic studies and surveys often use food frequency questionnaires (FFQs) to assess food and nutrient intakes over an extended period of time. Although food diaries and 24 h dietary recalls are considered more precise than FFQs [9,10], it has been demonstrated that the latter correctly rank individuals according to their dietary intakes [11]. While different FFQs have been validated in adolescents [12][13][14][15][16], the need of innovative methods, such as a web-based FFQ, has been recently highlighted for uncovering the long-term effects of diet on chronic diseases [17]. In 2014, researchers from the Adolescents and Surveillance System for Obesity prevention (ASSO) project reported on the relative reproducibility of a web-based FFQ among Italian adolescents [17]. Despite this progress, more efforts are needed to develop web-based tools that are easier, more suitable and more cost-effective than traditional methods for assessing food and nutrient intakes [17]. To fill this gap, in the present study we aim to compare a self-administered web-based FFQ (web-FFQ) with an interviewer printed FFQ (print-FFQ) for dietary assessment in Italian adolescents aged 15 to 18 years.

Study Design and Data Collection
The present study included adolescents (aged 15 to 18 years), attending three high schools in the urban area of Eastern Sicily. Ethical approval for the study protocol was given by the school boards of involved institutions, and informed consent was signed by parents or by the adolescents themselves if they were 18 years old or older. More details on protocols and characteristics of study population were reported elsewhere [18]. During classroom time and under the supervision of trained teachers, all the adolescents were invited to participate in this comparative study and were provided with information sheets. We used data from adolescents who completed both FFQs with plausible total energy intake [19]. A structured ad hoc questionnaire was administered to collect data on lifestyles (i.e., physical activity and smoking status) and sociodemographic factors of parents. Physical activity was assessed using the International Physical Activity Questionnaire for adolescents (IPAQ-A) [20]. Self-reported body weight and height were used to calculate body mass index (BMI) according to the World Health Organization criteria for adolescents [21].

Dietary Assessment
We first adapted the interviewer print-FFQ and the self-administered web-FFQ from a 95-item interviewer print-FFQ, which was previously used in the research on the effect of dietary habits on women's health [22][23][24][25]. The latter, in turn, was updated from a 46-item FFQ validated for the assessment of folate intake in Italian women of child-bearing age [26]. The print-FFQ has already been used to assess the association between dietary patterns and school performance in the present study population of adolescents [18].
Both FFQs included questions on 95 food items categorized into 39 food groups and eight food categories. In particular, data on frequency of consumption (from "never" to "more than twice per day") and the portion size (small, half a medium portion size; medium; large, 1.5 times or more than the medium portion size) for each food item were collected. Portion sizes were assessed by using images extracted from a photographic atlas. First, adolescents were invited to fill the web-FFQ following information sheets and instructions by trained epidemiologists. Next, the print-FFQ was administered by trained interviewers within the week after the web-FFQ administration. From both FFQs, daily food intakes were calculated by multiplying the daily frequency of consumption with the portion size of each food group. Daily nutrient intakes were calculated using the table of food composition of the US Department of Agriculture (http://ndb.nal.usda.gov/). Prior to further analysis, nutrient and food intakes were adjusted for total energy intake using the residual method [27].

Statistical Analysis
Prior to analysis, we excluded print-FFQs and web-FFQs in which the total energy intake per day was implausible (total energy intake <700 kcal or >4500 kcal), as previously recommended by Rockett et al. [19]. Moreover, the normal distribution of variables was checked using the Kolmogorov-Smirnov test. Adolescent characteristics were reported as proportions, mean or standard deviation (SD), or median and interquartile range (IQR), and compared between sexes using the Chi-squared test for categorical variables or the Mann-Whitney U test for skewed continuous variables.
Several methods were used to assess the reliability of the web-FFQ for the assessment of food and nutrient intakes, in comparison with the print-FFQ. Although several lines of evidence suggested that the correlation coefficient might be a misleading indicator of agreement [28][29][30], we first calculated Spearman's rank correlation coefficients to allow comparisons with previous studies that did not use the other methods. Wilcoxon rank test was used to compare differences between the two FFQs. The quartile misclassification analysis and Cohen's weighted kappa were applied to assess the correct ranking ability of the web-FFQ compared with the print-FFQ. According to Masson and colleagues, at least 50% of subjects should be classified into the same quartile, no more than 10% should be classified into the opposite quartile and Cohen's weighted kappa should be above 0.4 [31]. The Bland-Altman method was used to assess the agreement between absolute food and nutrient intakes estimated by each FFQ [32]. This method also helps identify the direction and consistency of any bias across the range of intakes [32]. Prior to analysis, food and nutrient intakes were log-transformed due to their skewed distributions (i.e., a constant value of 0.01 was added to food or nutrient intakes equal to zero). For each food group and nutrient, mean agreement was calculated as the average of all individual differences between FFQs, according to the following formula: ( log(web-FFQ)-log(print-FFQ))/n. The limits of agreement (LOA) were calculated as mean agreement ± 1.96 * (standard deviation of differences between methods). Mean agreement indicates how well the web-FFQ and print-FFQ agree on average, while the LOA designate the interval within which 95% of all individual differences between FFQs are expected to fall [32]. Finally, we plotted individual differences in intakes between methods (log(web-FFQ-log(print-FFQ)) against their average. To evaluate whether agreement between the web-FFQ and print-FFQ was constant across the range of intakes, we estimated the regression slope (β) of the average of the two methods on their differences [32]. Statistical analyses were performed using the SPSS software (version 22.0, SPSS, Chicago, IL, USA) and p-values less than 0.05 were considered statistically significant.

Results
A total of 179 adolescents, aged 15 to 18 years (median = 16 years), completed both FFQs, out of which five were excluded due to implausible energy intakes. Thus, 174 adolescents were included in this comparative study (median age =16 years; IQR = 2). No significant differences between included and excluded adolescents were evident. Mean BMI was 20.4 kg/m 2 (SD = 3.1 kg/m 2 ), with boys (47.8%) reporting higher weight, height and BMI than girls (p-values < 0.001). According to BMI, 91% of adolescents were normal weight (88.2% boys and 93.5% girls) and only 1.7% were obese (2.4% boys and 1.1% girls). Moreover, boys spent more time in moderate-to-vigorous physical activity than girls (p < 0.001). No differences in sociodemographic characteristics of parents and smoking status were evident between sex. Table 1 displays Spearman's correlation coefficients for food groups that ranged from 0.14 (i.e., pizza) to 0.67 (i.e., raw vegetables). Results from the Wilcoxon rank test indicated that food intakes were comparable between the two FFQs, except for nuts, shellfish and fruit juices. The percentage of subjects classified into the same or adjacent quartiles by the two FFQs ranged from 68.2% (i.e., tea) to 89.1% (i.e., raw vegetables and dipping sauces). In contrast, the web-FFQ misclassified from 0% (i.e., dipping sauces) to 13.1% (i.e., fish) of subjects into the opposite quartile compared to the print-FFQ. The weighted kappa values indicated substantial agreement for raw vegetables, yoghurt, eggs, coffee, dipping sauces and soup (weighted kappa from 0.6 to 0.8), and low agreement for pizza (weighted kappa below 0.2).
As an example, Figure 1 shows Bland-Altman plots of the average between the web-FFQ and print-FFQ versus their difference, the mean agreement, LOA and regression slope for raw vegetables, fruit, red meat and white bread. The mean agreement between the web-FFQ and print-FFQ was significantly different from zero for five food groups: on one hand, the web-FFQ significantly underestimated nuts, canned fish and olive oil intakes; on the other hand, it overestimated shellfish and fruit juice intakes. However, the LOA indicated wide individual differences between the web-FFQ and the print-FFQ for all food groups. In particular, the mean agreement between the web-FFQ and print-FFQ was significantly poorer at high levels of intake (i.e., significant positive regression slope) for offal, shellfish and fruit salad (β = 0.514, 0.258 and 0.170, respectively) ( As an example, Figure 1 shows Bland-Altman plots of the average between the web-FFQ and print-FFQ versus their difference, the mean agreement, LOA and regression slope for raw vegetables, fruit, red meat and white bread. The mean agreement between the web-FFQ and print-FFQ was significantly different from zero for five food groups: on one hand, the web-FFQ significantly underestimated nuts, canned fish and olive oil intakes; on the other hand, it overestimated shellfish and fruit juice intakes. However, the LOA indicated wide individual differences between the web-FFQ and the print-FFQ for all food groups. In particular, the mean agreement between the web-FFQ and print-FFQ was significantly poorer at high levels of intake (i.e., significant positive regression slope) for offal, shellfish and fruit salad (β = 0.514, 0.258 and 0.170, respectively) ( Table 2).      Table 3 displays Spearman's correlation coefficients that ranged from 0.45 (i.e., monounsaturated fatty acids, MUFA) to 0.62 (i.e., zinc). Results from the Wilcoxon rank test indicated that nutrient intakes were comparable between the two FFQs, except for MUFA. The percentage of subjects classified into the same or adjacent quartiles by the two FFQs ranged from 77.2% (i.e., vitamin C) to 87.2% (i.e., folate and calcium). In contrast, the web-FFQ misclassified from 1.7% (i.e., zinc) to 13.1% (i.e., MUFA) of subjects into the opposite quartile compared to the print-FFQ. The weighted kappa values indicated moderate agreement for all nutrients (weighted kappa from 0.4 to 0.6). As an example, Figure 2 shows Bland-Altman plots of the average between the web-FFQ and print-FFQ versus their difference, the mean agreement, LOA and regression slope for total energy intake, saturated fatty acids (SFA), MUFA and folate. The mean agreement between the web-FFQ and print-FFQ was significantly different from zero for total energy intake, SFA, MUFA, polyunsaturated fatty acids (PUFA) and calcium, which were underestimated by the web-FFQ. The LOA indicated moderate to wide individual differences between the web-FFQ and the print-FFQ for all nutrients ( Table 2). In particular, the mean agreement between the web-FFQ and print-FFQ was significantly poorer at high levels of intake for SFA, (β = 0.179); on the contrary, the difference in MUFA intake was lower at lower levels of intake (β = −0.427) ( significantly poorer at high levels of intake for SFA, (β = 0.179); on the contrary, the difference in MUFA intake was lower at lower levels of intake (β = −0.427) ( Table 4).  Abbreviations: 95% CI, 95% confidence interval; LOA, limits of agreement. *** p < 0.001; * p < 0.05.

Discussion
Despite the need for innovative tools for gathering dietary data in childhood and adolescence, few studies have developed and validated web-based tools for dietary assessment. To the best of our knowledge, only a web-based FFQ for dietary assessment in Italian adolescents has been developed thus far [17] and validated [33] within the Adolescents and Surveillance System for the Obesity prevention (ASSO) project.
In the current study, we adapted a self-administered web-FFQ from a validated intervieweradministered print-FFQ, which used the previous month as a reference period [23,24]. Next, we reported on correlations, ranking ability and mean agreement between food and nutrient intakes in Italian

Discussion
Despite the need for innovative tools for gathering dietary data in childhood and adolescence, few studies have developed and validated web-based tools for dietary assessment. To the best of our knowledge, only a web-based FFQ for dietary assessment in Italian adolescents has been developed thus far [17] and validated [33] within the Adolescents and Surveillance System for the Obesity prevention (ASSO) project.
In the current study, we adapted a self-administered web-FFQ from a validated intervieweradministered print-FFQ, which used the previous month as a reference period [23,24]. Next, we reported on correlations, ranking ability and mean agreement between food and nutrient intakes in Italian adolescents, estimated by the self-administered web-FFQ in comparison with the interviewer print-FFQ. To allow comparison with previously reported tools, we first assessed Spearman's rank correlation coefficients between the web-FFQ and print-FFQ. Except for pizza, correlation coefficients of food and nutrient intakes were in line with previous studies using a dietary reference method. As reviewed by Cade and colleagues and Tabacchi and colleagues, a good correlation between two methods is generally considered for a coefficient value of 0.4 [34,35]. Specifically, the correlation coefficients ranged from 0.14 (i.e., pizza) to 0.67 (i.e., raw vegetables) for foods, and from 0.45 (i.e., MUFA) to 0.62 (i.e., zinc) for nutrients. Results from the Wilcoxon rank test showed that food and nutrient intakes were comparable between the two FFQs, except for nuts, shellfish, fruit juices and MUFA. Moreover, the ranking ability of our web-FFQ was adequate, as demonstrated by quartile misclassification analysis and Cohen's weighted kappa. In fact, the percentage of subjects classified into the same or adjacent quartiles ranged from 68.2% (i.e., tea) to 89.1% (i.e., raw vegetables and dipping sauces) for foods, and from 77.2% (i.e., vitamin C) to 87.2% (i.e., folate and calcium). These results were in line with those reported by previous studies using self-administered traditional or web-based FFQs [17,33,36,37]. According to Masson and colleagues [31], misclassification was higher for fish and MUFA, with 13.1% of subjects classified into the opposite quartile. For food groups, the weighted kappa values indicated substantial agreement for raw vegetables, yoghurt, eggs, coffee, dipping sauces and soup, moderate agreement for other foods and low agreement for pizza. For all nutrients, the weighted kappa values indicated moderate agreement. Finally, we used the Bland-Altman method, which assesses the agreement between absolute intakes estimated by each FFQ [32]. We demonstrated that the mean agreement between FFQs was significantly different from zero for five foods and five nutrients: while the web-FFQ significantly overestimated shellfish and fruit juice intakes, it underestimated nuts, canned fish, olive oil, total energy intake, SFA, MUFA, PUFA and calcium. Although a good distribution of values around the mean difference was reported in general, the LOA analysis indicated moderate to wide individual differences for all food and nutrient groups [32]. This was in line with previous validation studies of web-based FFQs [33,[38][39][40]. In particular, the mean agreement between the web-FFQ and print-FFQ was significantly poorer at high levels of intake for offal, shellfish, fruit salad and SFA, while it was lower at lower levels of MUFA intake.
We acknowledge the limitations of our study. Since FFQs rely on memory, the web-FFQ and print-FFQ share measurement errors [35]. Weighed record or food record are the first methods for validating FFQs, followed by 24 h recalls. However, similar to FFQs, they are prone to a degree of misreporting [35]. Accordingly, Cade and colleagues reviewed dietary assessment tools which were used as a reference measure. They showed that almost half the validation studies used weighed record or food record, about a fifth used 24 h recall and more than one-tenth used another FFQ [34]. Potential measurement errors might partially explain the low total energy intake observed among our study population. Indeed, we reported median values of~1900 kcal, which are lower than those reported by a previous study [17]. However, the IQR of energy intake assessed by the web-FFQ was 1328-2618, similar to that reported by Tabacchi et al. using a weighted food record [33]. Since we cannot completely exclude misleading estimates, we only included adolescents with plausible total energy intake, and adjusted nutrient and food intakes for total energy intake. We considered implausible total energy intake as <700 kcal or >4500 kcal according to a previous study [19]. Rockett and colleagues defined outliers or implausible responses to the questionnaire as energy intake less than 500 kcal per day or greater than 5000 kcal per day [19]. In our study we used more stringent values to reduce the risk of potential measurement errors. The present study also has several strengths, including the number of respondents, which exceeds most other reliability studies. Our sample size of 175 adolescents allowed correlations, ranking ability and Bland-Altman analyses, which require a sample size of at least 100 subjects [34]. Taking into account its strengths and weaknesses, in our opinion, the proposed web-FFQ could be used for gathering dietary data in adolescence in future large-scale research against the increasing burden of NCDs.

Conclusions
In conclusion, our self-administered web-FFQ represents an easy, suitable and cost-effective tool for assessing food and nutrient intakes in adolescents. Although mean agreement between absolute intakes was less than ideal, the web-FFQ showed moderate to substantial ability to rank adolescents into the same or adjacent quartile as the interviewer print-FFQ. However, the wide individual differences in the level of agreement for all food and nutrient groups suggest that additional refinements and calibrations are needed to investigate the effects of absolute nutrient intakes at the individual level.