Reproducibility and Validity of a Food Frequency Questionnaire for Dietary Assessment in Adolescents in a Self-Reported Way

Tools to assess diet in a reliable and efficient way are needed, particularly in children and adolescents. In this study, we assess the reproducibility and validity of a semiquantitative food frequency questionnaire (FFQ) among adolescents in Spain. We analyzed data of 51 male adolescents aged 15–17 years from a prospective birth cohort study. Participants answered the FFQ twice in a self-administered way over a 12-month period. Reproducibility was assessed by comparing nutrient and food intakes from the FFQs, and validity by comparing nutrient intakes from the average of two FFQs and the average of two 24-Hour Dietary Recalls obtained in the period. Pearson correlation coefficients were calculated. The average of reproducibility correlation coefficients for food group intakes was 0.33, with the highest correlation for vegetable intake (r = 0.81); and the average for nutrient intake was 0.32, with the highest coefficients for α- and β-carotene (r = 0.65). Validity correlation coefficients ranged from 0.07 for carbohydrates to 0.53 for dietary fiber. The average of the validity correlation coefficients was r = 0.32. This study suggests that our FFQ may be a useful tool for assessing dietary intake of most nutrient and food groups among Spanish male adolescents in a self-administered way, despite reproducibility and, particularly validity, being low for some nutrients and food groups.


Introduction
Adolescence is a transitional stage of development from childhood to adulthood in which numerous biological and behavioral changes occur. Adolescents may be confronted with high-risk nutritional behaviors such as restrictive or excessive diets, or health problems such as overweight and obesity, most of them closely related to diet [1][2][3]. Thus, precise and accurate dietary assessment methods to investigate diet-disease relationships in adolescence are needed. In this regard, the food frequency questionnaire (FFQ) has been considered the most convenient dietary assessment method in most epidemiologic studies and populations, mainly due to the ease of administration, coding and processing (ref. 5 Willett). Consequently, FFQs have been validated in many different populations worldwide, mostly in adult populations [4,5], and though they have been used to explore diet-disease relationships in adolescents [2,[6][7][8], the number of reproducible and valid FFQs for adolescents is still low [9,10]. Like in adults, FFQs in adolescents should also be validated for each specific population for which they will be used, due to factors that may vary depending on the geographical area, culture and age [11,12].
Although a few FFQs have been developed for adolescents [13][14][15][16][17], we are not aware of FFQs specifically validated for adolescents in Spain. In the context of the INfancia y Medio Ambiente (Environment and Childhood) Project, a multicenter population-based birth cohort study in Spain [18], we have previously developed and validated several FFQs for dietary assessment in pregnant women and their children at the ages of 4-5 and 7-9 years [19][20][21]. In accordance with the needs of the follow-up phases of the INMA study, the aim of this study was to evaluate the reproducibility and validity of a semiquantitative FFQ designed to assess diet in a self-administered way in male adolescents aged 15-17 years in the INMA Project, using two 24-Hour Dietary Recalls (24hDR) as the validation method.

Study Population
We analyzed data from 51 male adolescents aged 15-17 years enrolled in the INMA-Granada cohort, one of the seven cohorts participating in the INMA Project. The INMA project is a population-based multicenter mother-child prospective cohort study designed to investigate the effect of environmental exposures and diet during pregnancy and childhood using a common protocol [18]. Briefly, 668 mother-son pairs were recruited at delivery between October 2000 and July 2002, and 152 children participated in the follow-up visit at the age of 15-17 years at the San Cecilio University Hospital (HUSC) of Granada [22]. The final sample for this validation study was based on 51 adolescents that agreed to complete all questionnaires in a self-administered way. This sample size provided the required statistical power to detect statistically significant correlation coefficients [12]. The scheme of the validation study is displayed in Figure 1. Participants received written guidelines to follow standardized procedures in order to complete the two FFQs and two 24hDRs in a self-administered way or with their parents' help over a period of 9-12 months. All parents provided written informed consent. The study followed the principles of the declaration of Helsinki and were approved by the Ethics Committee of HUSC of Granada and Miguel Hernández University. in most epidemiologic studies and populations, mainly due to the ease of administration, coding and processing (ref. 5 Willett). Consequently, FFQs have been validated in many different populations worldwide, mostly in adult populations [4,5], and though they have been used to explore diet-disease relationships in adolescents [2,[6][7][8], the number of reproducible and valid FFQs for adolescents is still low [9,10]. Like in adults, FFQs in adolescents should also be validated for each specific population for which they will be used, due to factors that may vary depending on the geographical area, culture and age [11,12]. Although a few FFQs have been developed for adolescents [13][14][15][16][17], we are not aware of FFQs specifically validated for adolescents in Spain. In the context of the INfancia y Medio Ambiente (Environment and Childhood) Project, a multicenter population-based birth cohort study in Spain [18], we have previously developed and validated several FFQs for dietary assessment in pregnant women and their children at the ages of 4-5 and 7-9 years [19][20][21]. In accordance with the needs of the follow-up phases of the INMA study, the aim of this study was to evaluate the reproducibility and validity of a semiquantitative FFQ designed to assess diet in a self-administered way in male adolescents aged 15-17 years in the INMA Project, using two 24-Hour Dietary Recalls (24hDR) as the validation method.

Study Population
We analyzed data from 51 male adolescents aged 15-17 years enrolled in the INMA-Granada cohort, one of the seven cohorts participating in the INMA Project. The INMA project is a populationbased multicenter mother-child prospective cohort study designed to investigate the effect of environmental exposures and diet during pregnancy and childhood using a common protocol [18]. Briefly, 668 mother-son pairs were recruited at delivery between October 2000 and July 2002, and 152 children participated in the follow-up visit at the age of 15-17 years at the San Cecilio University Hospital (HUSC) of Granada [22]. The final sample for this validation study was based on 51 adolescents that agreed to complete all questionnaires in a self-administered way. This sample size provided the required statistical power to detect statistically significant correlation coefficients [12]. The scheme of the validation study is displayed in Figure 1. Participants received written guidelines to follow standardized procedures in order to complete the two FFQs and two 24hDRs in a selfadministered way or with their parents' help over a period of 9-12 months. All parents provided written informed consent. The study followed the principles of the declaration of Helsinki and were approved by the Ethics Committee of HUSC of Granada and Miguel Hernández University.

Dietary Assessment: Semi-Quantitative Food Frequency Questionnaire (FFQ)
We used a semiquantitative FFQ of 104 food items to assess the usual daily intake of food and nutrients. This FFQ was an adaptation of other FFQs previously validated, one among children aged 4 to 5 years [20], and another among pregnant women of the INMA Project [19]. These FFQs had a similar structure to the Harvard questionnaire [23]. To adapt the new FFQ to adolescents, we included food items and portion sizes appropriate for adolescents aged 15 to 17 years. This FFQ was designed to be self-administered by the adolescents although it was also recommended the parents help when needed, and in all cases, a final supervision was conducted by the interviewer during the visit at the hospital. Adolescents were requested to report the average frequency of consumption for the specified serving or portion size for each food item of the FFQ in the previous year. The questionnaire had nine possible frequency options, from "never or less than once a month" to "six or more times a day". To estimate nutrient values and total energy intake, we used the published food composition tables of the US Department of Agriculture (USDA) and other published sources for specific Spanish food and portion sizes [24][25][26]. In order to estimate average daily nutrient intakes for each adolescent, we multiplied the frequency of consumption of each food item by the nutrient composition of the serving size specified in the FFQ and summed the results for all foods. Additionally, we estimated the mean daily intake of 17 food groups as shown in Table 1.

24-Hour Dietary Recall (24hDR)
Two 24hDRs were collected from each adolescent in a self-reported way and in nonconsecutive days. The first 24h DR was completed when the first FFQ was completed. In order to stimulate the recall on the foods and beverages consumed in the previous day and to reduce the potential recall bias in fulfilling the 24hDR, adolescents were provided written instructions, based on the USDA Automated Multiple-Pass Method [27]. At total of 9-12 months after the first FFQ and 24hDR were completed, participants were also provided with the second FFQ and 24hDR, and were instructed to complete and return both questionnaires to the investigators by email or post mail ( Figure 1). A reminder was sent to participants when a delay was noticed.
A nutritionist performed the coding of all food items based on the recorded portion size, brand name and the method of food preparation as collected in the 24hDR. The Food Processor II ® software was used to estimate the nutrient intakes. This software primarily uses the food composition tables from the USDA although we added information for specific Spanish foods available from Spanish food composition tables [24][25][26]. We used the average of two 24hDRs as the reference method to represent the individual intakes.

Statistical Analysis
All statistical analyses were performed using the statistical R software version 3.4.2 (R Foundation for Statistical Computing, Vienna, Austria; http://www.r-project.org). Means and standard deviations were calculated for nutrients and food groups from the FFQ and from the average of two 24hDRs. Student's t-test was used to compare means of nutrients and food group intake between the two periods. Nutrient and food group intakes were log-transformed to reduce skewness and to improve their normality. Energy-adjusted intakes were computed using the residual method proposed by Willett [23], where each nutrient is regressed on total calories and the population mean is then added back to the calculated residuals.
For the reproducibility analysis, we calculated Pearson correlation coefficients to compare nutrients and food groups from the two FFQs, the first completed at baseline and the second 9-12 months later. We also used Pearson correlation coefficients to evaluate validity of the FFQ, with the average of nutrient and food intakes of the two 24hDRs, that served as the reference FFQs (FFQav) with those of the average of the two 24hDRs (24hDRav) that we used as the reference method. We estimated correlation coefficients for log transformed intakes and energy-adjusted intakes that were also deattenuated to reduce the within-person variation found in the 24hDRs. The formula to calculate the deattenuated correlations was: where S2w represents within-person variance and S2b between-person variance for each nutrient and n is the number of replicated 24hDRs, in our case n = 2 [23].
We also estimated Spearman correlation coefficients although the results were very similar to those observed for parametric correlations. Therefore, only Pearson correlations are presented. We calculated the percentage of agreement as the proportion of individuals who were classified correctly into the same or adjacent quintile for the reproducibility and validity analysis.

Results
The characteristics of participant adolescents in the validation study were very similar to nonparticipants although mothers from participant adolescents were two years older ( Table 2). The mean age was 16.2 years and the mean body mass index 23.3 Kg/m 2 . The prevalence of overweight and obesity was 20.8 and 9.4% respectively.

Reproducibility
The mean daily nutrient intakes and Pearson correlation coefficients between the nutrient intakes estimated by the two FFQ are presented in Table 3. Estimates of energy and nutrients intakes were in general slightly higher in the first FFQ than in the second FFQ, except for the intakes of β-cryptoxanthin, lycopene, vitamin B6 and vitamin C. Statistically significant correlations were observed for all log-transformed nutrients, ranging from r = 0.31 for sodium to r = 0.64 for trans-fatty acids. The average of log-transformed correlation coefficients was 0.43. When nutrient intakes were adjusted for total energy, correlation coefficients tended to decrease for most nutrients, ranging from r = 0.11 for zinc to r = 0.65 for αand β-carotene; the average of energy-adjusted coefficient correlations was 0.36. Regarding the classification of nutrient intakes by quintiles as estimated by the two FFQs, the percentage of agreement (i.e., adolescents classified in the same or adjacent quintile) ranged from 58.5% for sodium to 79.2% for β-carotene; the average percentage of the agreements for nutrient intakes was 67.4%.  Regarding the reproducibility of the FFQ for food group intake, Table 4 shows the mean daily intake for the 17 foods and food groups estimated by the two FFQ. Most of the food intakes estimated by the first FFQ were higher than those estimated by the second, although only the intake of dairy products and sweet and sugar were statistically different (p < 0.05). White and red meat, fruit, potatoes and bread intakes estimated by the second FFQ were slightly higher than those estimated by the first. The correlation coefficients of food group intakes between the two FFQ showed a wider range of variation than that observed for nutrients, ranging from r = 0.10 for eggs and r = 0.81 for vegetables, although the average correlation coefficient was slightly lower than that observed for nutrients, r = 0.35 (energy-adjusted intakes). 1 FFQ1 and FFQ2, the same FFQ was firstly administered at baseline (FFQ1) and secondly (FFQ2), between 9 to 12 months later. 2 p-value from paired t-tests; 3 r, coefficient correlations after food groups intakes were log-transformed; 4 r adj, correlation coefficient using food groups intakes adjusted for total energy; FFQ, food frequency questionnaire. Table 5 shows mean daily nutrient intakes and Pearson correlation coefficients from the average of two FFQs and two 24hDRs. Intakes from the FFQs were on average between 15 and 20% higher than that estimated by the two 24hDRs (p < 0.001). For the log-transformed intakes, the average of the correlation coefficient was 0.21, although this average increased to 0.28 when the analysis was based on energy-adjusted nutrient intakes. The magnitude of correlation coefficients was, in general, higher for energy-adjusted correlation coefficients ranging from 0.07 for total carbohydrates to 0.44 for magnesium. The average of deattenuated Pearson correlation coefficients was r = 0.32. The average percentage of agreement for validity was 59.4%, ranging from 51.0% (lycopene) to 68.6% (protein). Figure 2 shows the Bland-Altman plots for energy, total fat, trans fatty acids, and iodine. When we examined the Bland-Altman plots to explore the validity of the FFQ, the discrepancies between the two dietary assessment methods were equally likely in either direction, although for energy and total fat intake, the difference in absolute intakes increased with increasing average intake, with the mean of FFQ producing systematically higher estimates than the average of two 24hDRs. 1 FFQav, average of FFQ1 and FFQ2; 2 24hDRav, average of the two 24hDRs; 3 p-value from paired t-tests; 4 r, coefficient correlations after nutrient intakes were log-transformed; 5 r adj, correlation coefficient using energy-adjusted nutrient intakes; 6 deattenuated correlation coefficients after nutrient intakes were log-transformed and energy-adjusted; 7 percentage of children classified into the same or an adjacent quintile. FFQ, food frequency questionnaire; 24hDR, 24 hours dietary recall; SFA, saturated fatty acids; MUFA, monounsaturated fatty acids; PUFA, polyunsaturated fatty acids.

Discussion
This study shows that the FFQ is a reliable method to assess diet in a self-administered way among male adolescents. The correlation coefficients of reproducibility for nutrient and food group intakes may be considered as moderate. The correlations for validity of the FFQ when compared with two 24hDRs were in general higher than 0.20 (average for deattenuated correlations, 0.32) which may be considered in a range between low and moderate validity when compared to other studies in adults [4,12]. However, to our knowledge, this is the first validation study of an FFQ to assess the nutrient and food intake among adolescents in Spain, and one of the few studies based on an FFQ for

Discussion
This study shows that the FFQ is a reliable method to assess diet in a self-administered way among male adolescents. The correlation coefficients of reproducibility for nutrient and food group intakes may be considered as moderate. The correlations for validity of the FFQ when compared with two 24hDRs were in general higher than 0.20 (average for deattenuated correlations, 0.32) which may be considered in a range between low and moderate validity when compared to other studies in adults [4,12]. However, to our knowledge, this is the first validation study of an FFQ to assess the nutrient and food intake among adolescents in Spain, and one of the few studies based on an FFQ for self-administered use. Although the sample size of the study was generally small, it was sufficient to detect statistically significant correlations of interest for validity (r > 0.20).
The Pearson's correlation coefficients of reproducibility for most of the nutrients and foods groups were slightly higher than those obtained in a validation study assessing diet in a five-month period with children and adolescents in Australia where the average of correlation coefficients for reproducibility was 0.32 [28]. Other studies in adolescents have shown higher correlation coefficients in reproducibility than those shown in this study [29][30][31][32]. One reason for the higher correlations observed in those studies could be the much shorter interval used between the two FFQ administrations, which ranged from two weeks to one month. On the other hand, this study showed higher correlation coefficients compared to a study carried out in 52 adolescents in Peru in a six-month period [11], in which the average of correlation coefficients for reproducibility was 0.17. We used a 9-12-month period between the first and the second FFQ in order to control the potential influence of seasonal variability, although such a long period could also be a cause of poorer correlations due to changes in diet during that timeframe. Nonetheless, the range of reproducibility of correlation coefficients for most nutrients and food groups observed in this study may be considered according to Cade et al. [4] and Willett [12] as adequate, and it would suggest that our FFQ is an acceptably reproducible tool to assess the nutrient and food intake in Spanish male adolescents.
The validity of this FFQ was assessed by comparing the nutrient intake estimates from the average of two 24hDRs with the average of the first and second FFQ. We used two 24hDRs as the most feasible reference method given the general characteristics of the study population, the available resources and because they have been also considered adequate to estimate the intake of energy and various nutrients [33]. Other alternative reference methods like the use of food diaries would have been more demanding and cumbersome for adolescent participants. FFQs tend to estimate higher nutrient intakes when compared to 24hDRs and Food Records [12] which has also been shown in validity studies with adolescents [28][29][30]. This was also shown in the current study, with the FFQ providing higher nutrient intake estimates than those from the 24hDRs for all nutrients. A possible explanation for the lower estimates from the 24hDRs could be that adolescents did not take into account all the foods eaten out-of-home, or the self-administered way of passing the 24hDR. On the contrary, it has been suggested that overestimation may also occur when FFQ with many food items are used [4], although this 104-item FFQ was similar to others we validated with adult Mediterranean populations in our area [19,34].
The correlation coefficient of validity in our study for energy intake between the FFQs and the 24hDRs was 0.28, higher than the observed in a study with 35 adolescents in the USA (r = 0.08) that used 3 day food records as a reference method [35]. The deattenuated correlations improved with respect to those based on energy-adjusted nutrient intakes for all nutrients due to correction of the within variation in nutrient intakes (average 0.32); the range for energy-adjusted correlation coefficients for validity observed for energy and nutrient intakes ranged from 0.07 to 0.44, higher than those reported by Rodriguez et al. [11], with their results ranging from −0.62 to 0.59, and within the range of 0.20 to 0.60 reported in a systematic review of dietary assessment methods for micronutrient intakes in adolescents [10]. The cross-classification of intakes by the two methods also showed a satisfactory level of agreement in nutrient and food intakes. Thus, the range of validity correlation coefficients observed for most nutrients in our study would suggest that the FFQ is also a reliable tool to assess diet and rank Spanish male adolescents according to their nutrient and food intake.
The Bland-Altman diagrams showed no evidence of serious bias for any of the 25 nutrients examined, except for energy and total fat intakes, where some evidence that the FFQ overestimated intake compared with the 24hDR and that this overestimation increased with increasing average intake. The Bland-Altman diagrams only allow a graphical interpretation and the results should be considered together with other results (e.g., correlation coefficients, cross-classification into quantiles). Thus, our study shows an acceptable level of agreement in intake ranking between the two dietary methods used.
We acknowledge the limitations of our study. While 24hDRs have been considered an adequate reference method to validate the FFQ, they are not a perfect measure of the true dietary intake due to the high degree of intrapersonal variability. Moreover, the misreporting of energy intake is a common problem among adolescents [36]. In order to minimize these problems, parents and adolescents were given instructions to complete the 24hDRs following the USDA Automated Multiple-Pass Method to minimize under-reporting of consumed food and beverages due to memory lapse. Another limitation could be the use of a 9 to 12 month period, which could result in lower reproducibility and validity than desired, if real changes in diet occurred during that study period. However, we use a 9-12-month period to validate the FFQ in part to control seasonality. It should be also taken into account that the FFQ was validated only in male adolescents and this may be a limitation if the questionnaire was to be used in female adolescents. In addition, due to limited resources, we were not able to obtain more than two 24hDRs, including one weekend day, in order to control weekend variation.
On the other hand, a major strength of the present study may be that we followed standardized protocols during the data collection of the INMA study fieldwork [18]. Although 51 participants represent a small sample size, it has been previously suggested that a sample size of 50 participants is acceptable (4). It should be noted that recruiting adolescents, especially older ones, is particularly challenging [37].
Moreover, the participants of the study were a subsample of a population-based birth cohort and the results could be more generalizable to adolescents of the same age range and gender.

Conclusions
This study suggests that our FFQ may be a reliable tool to assess diet in a self-administered way and rank Spanish adolescent males according to the dietary intake of a wide range of nutrients such as protein, dietary fiber, PUFA, omega 6, trans-fatty acids, alpha-beta-carotene, and also food groups such as dairy products, processed meat, fish, vegetables, fruits, cereals and sweets and sugar. The low validity observed for some nutrients should be further investigated in order to improve the overall validity of the FFQ and extend its use in studies with adolescents in Spain. In addition, testing the FFQ in a mixed-gender population is warranted.