Reproducibility of A Non-Quantitative Food Frequency Questionnaire (62-Item FFQ-6) and PCA-Driven Dietary Pattern Identification in 13–21-Year-Old Females

The aim of this study was to evaluate the test–retest reproducibility of a non-quantitative food frequency questionnaire (acronym: 62-item FFQ-6) and the possibility of identifying dietary patterns (DPs) in 13–21-year-old females. The study involved 97 females within three age groups: 13–15, 16–18, and 19–21 years, including 31, 38, and 28 subjects, respectively. The questionnaire was completed twice with a two-week interval (test and retest). For the total sample, using a principal component analysis (PCA), two similar PCA-driven DPs (DP1 and DP2) were identified separately from test data and retest data, considering two sets of input variables. 60-item-DP1 and 60-item-DP2 were identified after excluding two items—vegetables and fruits in general—due to including single items of various kinds of vegetables and fruits. After an aggregation of some items of the questionnaire, 25-item-DP1 and 25-item-DP2 were identified. The kappa statistic (test vs. retest) in the total sample averaged at 0.52 (0.32–0.72 for food items), while within age groups, it averaged at 0.41, 0.53, and 0.65, respectively. The percentage of subjects classified into the same food frequency category (test vs. retest) in the total sample averaged at 68% (51%–89% for food items), while within age groups, it averaged at 60%, 68%, and 77%, respectively. The Spearman correlations between dietary pattern scores (test vs. retest) in the total sample were: 0.84 (within age groups 0.83, 0.81, and 0.78, respectively) for 60-item-DP1, 0.68 (within age groups 0.24, 0.79, and 0.76, respectively) for 60-item-DP2, 0.76 (within age groups 0.56, 0.82, and 0.89, respectively) for 25-item-DP1, and 0.48 (within age groups 0.40, 0.57, and 0.53, respectively) for 25-item-DP2 (p < 0.05 for all). In conclusion, the test–retest reproducibility of the 62-item FFQ-6 was good or very good for most food items, with a tendency to be higher in older age groups of females under study. Due to the acceptable-to-good reproducibility of dietary pattern identification, the use of a 62-item FFQ-6 to describe the overall diet of young Polish females can be recommended.


Introduction
An assessment of dietary intake is still a topic of interest due to the causal relation of many diseases with diet and the possibility of identifying individuals and subpopulations at risk of inadequate food consumption [1]. There is no gold-standard method for dietary intake assessment. The most commonly used are three methods: food record, 24-hour dietary recall, and food frequency method using food frequency questionnaires (FFQs) [2][3][4]. These methods are characterized by various levels of validity and reproducibility since the validity and reproducibility of any dietary method is a function of the

Ethical Approval
The study was approved by the Bioethics Committee of the Faculty of Medical Sciences, University of Warmia and Mazury in Olsztyn in June 17, 2010 (resolution no. 20/2010). Informed consent was obtained from adult study participants and from parents/legal guardians of underage girls (<18 years old).

Study Design and Sample Collection
A pilot study was carried out to examine if the questions included in the 62-item FFQ-6 questionnaire were understandable and properly formulated. The main study was carried out in October and November 2012. The questionnaire was completed by each respondent twice with a two-week interval (test and retest). The data were collected by researchers in a face-to-face situation. The researchers described the aim of the study and answering manner in detail to the respondents before starting an interview. The same researcher conducted both the test and the retest with the same respondent (each respondent was assigned an identification number to identify the respondent during the second administration of the questionnaire).
Participants were recruited by contacting students of middle and high schools located in North-Eastern Poland. The intention was to achieve the same proportion of subjects in each age category (13-15, 16-18, and 19-21 years). The inclusion criteria were female gender, age ≥ 13 and ≤21 years and the use of the Polish language in speech. A total of 100 females were recruited. Three girls did not complete at least one questionnaire and they were excluded from the study. The final sample consisted of 97 females aged 13-21 years, including 31 subjects aged 13-15 years, 38 subjects aged 16-18 years, and 28 subjects aged 19-21 years.

A Food Frequency Questionnaire
The 62-item FFQ-6 consists of a list of 62 food items and refers to the usual frequency of food consumption over the last 12 months. The grouping of foods into 62 food items (Table S1) was created based on the authors experience and the Food Intake Variety Questionnaire (FIVeQ) with a similar food list whose reproducibility was tested previously [21]. The 62-item FFQ-6 contains two additional questions regarding the consumption frequency of vegetables and fruits in general (questions Q40 and Q29, respectively; Table S4). These two questions can be used by researchers in two ways: (1) to interpret the consumption frequency of vegetables and/or fruits in general (without considering single items), (2) to adjust the consumption frequency of single items of vegetables and fruits, collected with separate questions (Q41-47 and Q30-37, respectively) and then to interpret the consumption frequency of single items of vegetables and fruits in detail. A manual for the adjustment of consumption frequency of single items of vegetables and fruits is included in the Supplementary Materials (Table S4). This study shows the results from crude data, without adjustment for single items of vegetables or fruits.
When answering, to indicate frequency of food consumption, respondents could choose one of six categories (next converted by researcher into daily frequency): never or very rarely (0 times/day), once a month or less (0.025 times/day), several times a month (0.1 times/day), several times a week (0.571 times/day), daily (1 time/day), or a few times a day (2 times/day) [36]. Some food items were aggregated into 25 food items by summing up their daily consumption frequencies (in times/day), and this data set was subjected to further analysis (Table S1).

Statistical Analysis
Categorical variables were presented as a sample percentage (%), and continuous variables as means and standard deviation (SD) [37,38]. The distribution of continuous variables was examined using the Kolmogorov-Smirnov normality test. The reproducibility of a questionnaire was measured by comparing the results of the first interview (test) and the second administration of the questionnaire (retest) in the total sample and by age groups [5]. To comprehensively compare test data and retest data, several statistical measures and tests were used: (i) the Wilcoxon signed rank test for two dependent samples-to verify differences in means of food consumption frequency between the test and the retest; since all continuous variables (as expected) lacked a normal distribution, a non-parametric test was chosen, (ii) Spearman's correlation coefficient (SCC)-to compare the daily frequency of food consumption (times/day) between test-data and retest-data, (iii) the Fleiss kappa statistic, cross-classification analysis, and chi-square test-to evaluate the agreement of the subject distribution by the same food frequency categories in the test and the retest. The strength of the correlation was interpreted as fair (<0.3), moderate (0.3 to <0.5), good (0.5 to <0.7), or very good (≥0.7). The agreement (measured with the Fleiss' kappa) was interpreted as poor (≤0.2), fair (0.21-0.40), moderate (0.41-0.60), good (0.61-0.80), or very good (≥0.81) [39]. Two-tailed tests were applied and p-values < 0.05 were considered as significant.
The principal component analysis (PCA) with varimax normalized rotation was used to identify PCA-driven dietary patterns [40]. For the total sample (n = 97), four separate PCA for test data and retest data including a different number of input variables were performed. The input variables were consumption frequencies (in times/day) of: (i) all 60 food items, except for vegetables and fruits in general, (ii) 25 food items after aggregating some items (see Section 2.3). Factor loadings ≥ |0.40| were considered as having a significant contribution toward identifying DPs. Eigenvalues of at least 1.00, scree plot, and variance explained were considered when choosing the best solution. Based on tertile distribution of factor scores of dietary patterns, subjects were divided into three categories within each DP as follows: bottom, middle, and upper tertile. Dietary pattern scores reflecting adherence of each subject to each DP were established. The dietary pattern scores were calculated as a sum of the products of the food consumption frequency and a factor loading for 60 or 25 food items. To evaluate the agreement of DPs identified from data collected with the test and the retest, Spearman correlations for dietary pattern scores were calculated. All analyses were performed with STATISTICA software (version 13.3 PL; StatSoft Inc., Tulsa, OK, USA; StatSoft, Krakow, Poland).

Mean Frequency of Food Consumption
There were no significant differences in mean frequency of food consumption for most food items reported in the test and the retest in the total sample (57 out of 62) and within age groups (60, 58, and 60 for 13-15, 16-18, and 19-21 years, respectively; Table 1). Significant differences (p < 0.05) in mean frequency of food consumption (test vs. retest) were found for olives, wine and cocktails, chocolates, honey, milk and milk beverages-natural in the total sample; for baked confectionery, vegetables, and vegetable-fruit juices in females aged 13-15; for olives, sweetened beverages, honey, and milk and milk beverages-natural in females aged 16-18; and for high-quality cured meats and milk and milk beverages-natural in females aged 19-21. The Spearman correlations (SCCs) between the frequency of food consumption reported in the test and the retest for food items ranged from 0.24 to 0.86 in the total sample, from 0.09 to 0.84 in females aged 13-15, from 0.41 to 0.91 in females aged 16-18, and from 0.52 to 0.98 in females aged 19-21 ( Table 1). The SCC was more than 0.50 (good and very good) for 57 out of 62 food items (92% of total) in the total sample and for 42, 59, and 62 (68%, 95%, and 100%) within the age groups, respectively.
The percentage of subjects classified into the same food frequency category (test vs. retest) in the total sample was on average 68% and ranged for food items from 51% ('root vegetables and others') to 89% (venison), while within age groups it averaged at 60%, 68%, and 77%, respectively (Table 2).

Dietary Patterns Identified from 25 Food Items
Considering 25 food items as input variables, two similar dietary patterns (DP1 and DP2) were identified from both test data and retest data (Table S3, Figure 1). The total variance explained for two DPs derived from test data was 26.6%, including 16.1% for the 25-item-DP1 and 10.5% for the 25-item-DP2, and from retest data 28.2%, including 17.7% for the 25-item-DP1 and 10.5% for the 25-item-DP2.

General Reproducibility
It has been found that the reproducibility of the questionnaire was good or very good for most food items regardless of the statistical approach used. In the total sample, the Spearman's correlation coefficient was on average 0.68, which is considered a good result obtained for FFQ [1]. The same interpretation can be drawn for test-retest reproducibility measured with the kappa statistic (on average 0.52), the percentage of compatible classification into the food frequency category (on average 68%) and the comparison of mean of food consumption frequency (no significant differences for 92% of items). These results are in line with those previously reported. Based on the literature review, Cade et al. [5] stated that correlation coefficients in the range of 0.5-0.7 between test and retest of the FFQs were often reported. For the example, such correlations (0.50-0.70) were found among Brazilian, Danish, Norwegian, and Chinese adolescents [41][42][43][44][45] and American adults (on average 0.70) [46], while a slightly higher correlation (on average 0.78) was noted in Polish young females [22]. In Belgian adolescents, the percentage of compatible classification between test and retest ranged from 37% to 87% [47].
Better reproducibility of the questionnaire was found in older than younger age groups of females (19-21 vs. 16-18 vs. 13-15 years). The Spearman's correlation coefficient and the kappa statistic tended to be higher in older age groups. For all food items, the highest percentages of compatible classification into the food frequency category were found for the oldest females (19-21 years), while the lowest was for the youngest females (13-15 years). There are a few explanations for the higher reproducibility of the questionnaire in older females (19-21 years). Older females could better identify foods based on its name (given in the questionnaire without a photo gallery or 3D models) and they could better distinguish various types of foods and accurately determine the frequency of consumed food. Older females (as with most adults, in comparison with adolescents) tended to have more stable dietary habits, which are easier to report. Difficulties in precisely assessing dietary intake by adolescent respondents were previously reported [48][49][50]. This was explained by adolescents lower knowledge related to food perception and preparation compared to adults and also in rapid changes in dietary habits during adolescence, less eating at home and less supervision by adults. These changes result from a growing sense of independence, peer influence and awareness of social acceptance, greater emotional and financial autonomy, limited time of concentration and attention, and also a lack of interest and motivation to monitor one s own diet [49,50].
Regardless of the statistical approach used, test-retest reproducibility was the highest for foods consumed occasionally or never (e.g., spirits, olives, venison), and the lowest for foods consumed often (e.g., root vegetables and others, cruciferous vegetables, cheese curds). It can be speculated that some of the foods with lower reproducibility could be consumed as a component of complex dishes (e.g., root vegetables and others in vegetable salad) which could cause difficulties in determining the consumption frequency. A bias in the reported food consumption frequency should be also considered, including overestimation and underestimation. Young people, especially females, may overestimate the consumption of food considered healthy (e.g., vegetables), and to the opposite, can underestimate the consumption of foods considered unhealthy (e.g., fast-foods, soft drinks, sweets, and salty snacks) [51]. In adolescence, requirements for increasing energy supply, concerns related to self-image, and following food fashions may all contribute to poor compliance in dietary reporting [49].

Reproducibility of the Identification of Dietary Patterns
Regardless of the data set used (with 60 or 25 food items), two dietary patterns were identified in the total sample. The first dietary pattern was characterized by the frequent consumption of various types of fruit, sweetened milk products, nuts and seeds, juices, dried fruit, fruit preserves and fruit condiments. This dietary pattern, having a fruit-vegetable-milk profile, can be classified as pro-healthy, although sweet. A similar dietary pattern (the fruit and vegetables) was previously found in a representative sample of Polish females aged 13-21 years [52], in which a positive attitude towards health and natural product interest were revealed. Females chose fruits presumably because of the desire to be healthy and enjoying the good taste of food [53]. Vegetables are also perceived by females as healthy, fashionable, and low-calorie [27,54,55]. Across the world and various subpopulations, healthy or prudent dietary patterns consisting of fruit and vegetables as well as low-fat dairy products, whole grains, legumes, fish and seafood, nuts, and vegetable oils were identified more often [56][57][58].
The second dietary pattern was characterized by the frequent consumption of processed meats, refined grain products, potatoes, butter, sugar and sweets. This dietary pattern can be classified as Polish traditional with a westernized profile. Traditional westernized dietary patterns were identified across the world, showing a universal trend toward diet westernization, i.e., a shift from traditional foods towards highly processed, high-fat, high-sugar, and low-fiber foods [13,14,59]. It was discussed that the taste, food availability, and also following food fashions are important factors of food choice, especially for young people [60][61][62]. Karimi-Shahanjarini et al. [61] showed that young females aged 12-15 years were eating unhealthy snacks because of the taste, easy access, and high price of healthy snacks, and the potential risk of disease in the future was not important to them.
It was found that the dietary patterns identified from retest data were similar to those identified from the test data. This was documented by considering dietary patterns scores in the test and retest. In the total sample, the Spearman's correlation coefficients were good to very good (0.68-0.84) except one was slightly lower (0.48), and the agreement of subjects' distribution by tertiles of dietary patterns was acceptable to good (39% to 59%). Based on this, the use of the questionnaire to identify dietary patterns in young Polish females can be recommended. It is possible that the 62-item FFQ-6 can also be used in other European countries with similar dietary habits and food availability, although further investigation is needed. There are limited data regarding the reproducibility of the dietary patterns identified with the FFQs. Such an analysis was previously conducted for American, Spanish, and Japanese adults [46,63,64]. Among American men, two dietary patterns were identified from the 131-item FFQ [46]. The correlations between first and second (1 year apart) administration of the FFQ were good: 0.70 for the Prudent pattern and 0.67 for the Western pattern. In Japanese adults, the reproducibility of dietary patterns was reported based on a systematic review of PCA-derived dietary patterns [64]. The reproducibility of dietary patterns was assessed using a congruence coefficient (CC). When high quality data, i.e., coming from a validated FFQ or multiple-day dietary records and sample size ≥ 200, were included, the median CC was high for Healthy pattern (0.89), Prudent pattern (0.86), and Japanese pattern (0.80), while it was low for the Traditional pattern (0.59), Western pattern (0.44), and Traditional Japanese pattern (0.31). This systematic review has shown that there are some dietary patterns that are relatively reproducible in different populations in a given country. The reproducibility of data-driven dietary patterns was assessed in different Spanish samples extracted from similar populations [63] using congruence coefficients similarly to the Japanese study [64]. The median of the CC was 0.90 for Western pattern, 0.77 for the Mediterranean pattern, and 0.76 for Prudent pattern. Due to a lack of similar studies covering respondents at the same age (13-21 years), and taking into account that reporting of food consumption is more biased in adolescents than adults [49], the current findings (i.e., Spearman's correlation 0.48-0.84) cannot be directly compared with those cited above. However, it can be speculated that reproducibility of pro-healthy dietary patterns (e.g., Prudent) is better than non-healthy dietary patterns (e.g., Western).
It is difficult to comprehensively discuss the reproducibility of dietary patterns identified across the age groups (13-15 vs. 16-18 vs. 19-21 years) due to the small number of subjects (31 vs. 38 vs. 28, respectively). However, showing great caution in reasoning, it can be suggested that better reproducibility of dietary patterns identification was found in older than younger age groups of females. A possible explanation was discussed in Section 4.1.

Strengths and Limitations
The main strength of the study is applying several methods of statistical analysis as recommended, all suitable for an evaluation of the FFQs' reproducibility [5,65]. Such an approach-the application of multiple statistical tests-allows for gaining comprehensive insights, reduces the chance of an over-interpretation of research findings, strengthens the conclusions, and increases the possibility of comparing these results with others. Secondly, using two data sets, i.e., with 60 and 25 food items, provided the possibility of using the questionnaire to identify dietary patterns from non-aggregated and aggregated food items and, indirectly, the usefulness of the questionnaire in terms of processing the dietary data obtained. This indicates the possibilities offered by this questionnaire and may be an inspiration for less advanced researchers. To facilitate future use of this questionnaire by other researchers, the manual (Table S4) is attached.
The limitation of the study is the relatively small number of subjects (97). The findings related to the age groups should be particularly interpreted with caution (as tendency) due to the small number of respondents (31, 38, and 28). However, in FFQ validation studies, similarly numbered samples (in total 48-90) were previously reported in adolescents from Denmark [43], Brazil [41], New Zealand [11], and Norway [44], and also in young Polish females [17,20]. Furthermore, similarly numbered sub-samples  were analyzed across the age, sex, or ethnicity in children and adolescents [66] and adults [39,67]. When a principal component analysis (PCA) was performed from 60 food items, the subject-to-item ratio was low (1.6:1, i.e., 97 subjects to 60 items), but for 25 food items it can be considered as sufficient (3.9:1, i.e., 97 subjects to 25 items) because it was slightly below the lower border of the recommended range of the subject-to-item ratio [68,69]. Regardless of the limitations in the PCA, it should be emphasized that the PCA was performed to assess repeatability in identifying dietary patterns, and not to interpret them comprehensively. Since only one gender group of Polish residents was selected with a narrow age range (13-21 years), these findings cannot be applied to people of different age or gender. Even so, it may be supposed that similar relations can be found in other European females of a similar age. Since dietary interviews conducted in adults are less burdened by measurement errors compared to adolescents [49], it can be speculated that this questionnaire can be used in adults of both sexes as they provide relatively reliable dietary data. In the present study, the reproducibility was assessed for a questionnaire administrated by trained interviewers, so it can be assumed that the reproducibility will be lower for a self-administered questionnaire. Previously, the better reproducibility of the interviewer-administered KomPAN®questionnaire than its self-administered version was revealed in Polish adolescents and adults [24].

Conclusions
The test-retest reproducibility of the 62-item FFQ-6 was good or very good for most food items, with a tendency to be higher in older age groups of females under study. Due to the acceptable-to-good reproducibility of identification of dietary patterns derived using the principal component analysis, the use of 62-item FFQ-6 to describe the overall diet of Polish young females can be recommended.

Supplementary Materials:
The following are available online at http://www.mdpi.com/2072-6643/11/9/2183/s1, Table S1: Questionnaire food items (62 items) description and foods aggregation into 25 food items, Table S2: Factor loading matrix for the two major dietary patterns identified by principal component analysis with 60 food items as input variables (excluding all kinds of fruits and all kinds of vegetables), Table S3: Factor loading matrix for the two major dietary patterns identified by principal component analysis with 25 food items as input variables, Table S4: Food Frequency Questionnaire (62-item FFQ-6) including a manual for the adjustment of consumption frequency of single items of vegetables and fruits.
Author Contributions: E.N. and L.W. were involved in the conceptualization, the study design and methodology. E.N. and L.W. developed the Food Frequency Questionnaire (62-item FFQ-6); E.N., L.W. and J.K. were involved in the development of the manual for the adjustment of consumption frequency of single items of vegetables and fruits. E.N. was involved in the data acquisition. E.N. and J.K. statistically analyzed the data; E.N. and L.W. were involved in the data visualization; E.N., L.W. and J.K. interpreted the data; E.N. wrote the manuscript. L.W. and J.K. were involved in critically revising the manuscript. L.W. was involved in the funding acquisition. All authors have given their approval to the manuscript submitted.

Funding:
The project was financially supported by the Ministry of Science and Higher Education in the range of the program entitled "Regional Initiative of Excellence" for the years 2019-2022, Project No. 010/RID/2018/19, amount of funding 12.000.000 PLN.