Measuring dietary intake in large population groups can be difficult when time and resources are limited. Traditional methods of assessment may have a high participant burden, and also take time for researchers to manage the data entry and conduct the analysis. Adapting traditional methods to incorporate technology is one way researchers have tried to reduce the burden of measuring dietary intake. Most technology assisted methods reduce the time and costs associated with data collection by automating some aspects of the data collection, coding, and analysis. Removing the face to face element of traditional methods may also allow for a larger sample of individuals to be assessed. However, this application of technology does not usually overcome the measurement biases commonly associated with assessing dietary intake. Technology is also being used to develop new methods which may address some of these biases, such as using photography to capture intake instead of self-report [1
], or physiological measurements as a proxy to intake [2
]. However, these applications are generally not ready for use in large population studies.
There are some examples of where traditional methods have been transferred online and made available to researchers and the general public to assess dietary intake. One example is the Automated Self-Administered 24-h recall (ASA-24), which is an automated, self-administered 24-h recall available in the USA [3
], and is currently being adapted and validated for use in the Australian population. The online platform removes the role of an interviewer in a 24-h recall, and the system provides prompts and cues to assist individuals’ in reporting all food and beverages consumed on a day, or across multiple days. By tradition, the primary output from these recalls is nutrient intake, although more recently with the use of an online platform and advances in statistical methods, multiple recalls are being administered more frequently and used to estimate usual food consumption [4
Food frequency questionnaires (FFQs) also collect information about usual or habitual intake of a selection of foods. They are often designed to be self-administered, accommodate larger samples, and are well suited to an online delivery. The length of FFQs can vary depending on the researcher’s focus. Longer FFQs aim to capture the total food and nutrient intake but shorter versions tend only to characterise intakes of one, or a few, nutrients or food groups. The validation of FFQs has traditionally been in relation to energy and nutrient intakes but not food intake [5
]. A recent review of short FFQs and questions to measure food intake in children and adolescents identified 30 tools that had undergone scientific validation. Only six of these short tools measured a wide range of food groups [7
]. The challenge remains whether short questionnaires can be used to capture intake of a range of foods, total diet, diet quality, and/or whole dietary patterns.
The Australian Dietary Guidelines provide information on the types and amounts of food to eat from a range of food groups, and a dietary pattern to promote health and wellbeing. In recent years, the interest in methods to assess whole diet and eating patterns has increased as a way to monitor population intakes, and compare population intakes against such food based dietary guidelines. The Short Food Survey, is a short questionnaire which assesses intake of all five core food groups, beverages, discretionary choices, food choices, and variety, allowing for compliance with the Australian Dietary Guidelines to be estimated as a diet quality score [8
]. The survey has been validated against 24 h recalls in adults [9
] and children [10
], and found to provide a reliable and adequate estimate of overall diet quality. Given the lack of scientifically developed tools available online [7
], there was an opportunity to provide the Short Food Survey to Australians to assess their own dietary intake, and for researchers to find out how Australians’ diets comply with dietary guidelines. Conducting national nutrition surveys is a big financial undertaking, and as such it was almost 20 years between surveys in Australian adults [11
]. We envisaged that a freely available tool would attract a large audience to complete the survey, and have the potential to provide a unique and contemporary insight into what Australians are eating. This information would be most useful to fill the gap in the intervals between national surveys. In May 2015, the Commonwealth Scientific and Industrial Research Organisation (CSIRO) Healthy Diet Score survey was launched online. This paper describes the reach and user experience of the survey, and summarises how compliant Australians self-reported diets are with the Australian Dietary Guidelines.
2. Materials and Methods
The CSIRO Healthy Diet Score was developed in partnership by the Commonwealth Scientific and Industrial Research Organisation (CSIRO) and SP Health Co. (Sydney, Australia)—an Australian business who deliver online health solutions. The CSIRO Healthy Diet Score survey is an extension of the Short Food Survey which was developed by CSIRO and the University of South Australia [9
]. The Short Food Survey is a series of 38 short questions which were purposefully compiled to allow for the calculation of a Dietary Guideline Index score [13
], referred to here as “Diet Score”. The Short Food Survey has undergone validation in a sample of Australian adults [9
] and separately for parents reporting children’s intake [10
], and been shown to provide a valid estimate of overall diet quality.
The Australian Dietary Guidelines provide Australians with specific advice on the quantities of core and discretionary (noncore) food and beverages to consume on a daily basis, based on their age and gender [14
]. To reflect these guidelines as best as possible, the Diet Score scoring system assesses the quantity, quality, and variety of foods consumed. A detailed description of the questions and scoring algorithm are published elsewhere [9
]. Briefly, for the quantity components individuals report their usual intake, in serves, per day, week or month, from which serves per day is calculated. The scoring algorithm compares the daily amount of fruit, vegetables, breads and cereals, meat and alternatives, dairy and dairy substitutes, and discretionary foods to age and gender specific recommendations. For the quality components, the frequency of wholegrains consumption, reduced fat dairy consumption, frequency of trimming meat, fat type of spreads used, and water consumption (as a proportion of total fluids) are assessed. The variety of foods consumed within each core food group is also scored. Each component of diet quality is calculated as a score out of 10, except for discretionary foods which is out of 20 points. Achievement of a maximum score indicates that an individual’s intake met the recommendation (i.e., had an optimal intake). Minimum scores were generally assigned to zero intakes, and in between scores were allocated as a proportion of the recommendation. All components are summed to give an overall Diet Score out of 100, where an individual’s Diet Score reflects their overall compliance with the Australian Dietary Guidelines, and a higher score reflects greater compliance with guidelines and higher diet quality. To aid interpretation and comparison of scores between these components, we have expressed the component scores (originally out of 10, except for discretionary foods which was out of 20) as a score out of 100 in the tables and description of results.
The CSIRO Healthy Diet Score survey contains the 38 questions asking about the frequency and amounts of different foods consumed, food choices, and variety—the same as the Short Food Survey. In addition, the CSIRO Healthy Diet Score survey contained coloured images of example foods included in each question. The portion size of the images were selected to be as realistic as possible, however were not intended to be exact serve sizes in all instances (Figure 1
). The survey was programed into a readily available survey platform. On completion, individuals receive an immediate Diet Score on completion of the survey, as well as three personalized suggestions for areas to improve.
The CSIRO Healthy Diet Score survey is currently a live website, meaning data collection is continuous and ongoing. The survey went live on the 21 May 2015 and CSIRO and SP Health coordinated four media releases: (i) a small scale media on 21 May associated with the launch; (ii) a major release about the first results of the survey conducted on 10 August 2015; (iii) a second small scale release marking one year since launch; and (iv) a second major release of findings on 26 September 2016. The two major releases received national television, print, and radio coverage across a range of free to air television channels (e.g., news and current affairs programs), Australian print newspapers and internet stories, and local and national radio programs. The peaks in visitation to the survey corresponded with these publicity campaigns (Figure 2
). This paper describes data collected from individuals who completed the survey from when it went live on the 21 May 2015 through to the 17 October 2016. During this period, the survey was commenced 190,667 times. Duplicates were identified using an ID variable (n
= 9046) and individuals’ first attempt of completing the survey was included in the present analysis. Of the 181,621 surveys by unique users, 34,323 surveys were only partially completed, leaving 147,298 complete surveys. Outliers were identified and removed based on extreme age (less than 18 and older than 100 years removed), Body Mass Index (less than 13 and greater than 97 removed), height (less than 1m and greater than 3 m), and weight (less than 13 kg and greater than 250 kg). In total 1323 surveys were removed based on the above criteria, leaving 145,975 surveys for analysis (76.6% of those that commenced the survey, Figure 3
). According to the summary statistics provided by the survey platform, the highest drop off between pages of the survey was 4% between page 2 about fruit and page 3 about vegetables. The drop between the remaining pages was 0%–2%, and there was no pattern to this drop off.
Descriptive analysis (means, standard deviations, and frequencies) were used to describe the sample included in the analysis, and their scores on each component of the Diet Score as well as the Diet Score overall. Due to the large sample size, discussion of results favoured the most meaningful differences (considered to be a difference of 5% or more between groups), as opposed to a statistically significant difference. This study was approved by the CSIRO Health and Medical Human Research Ethics Committee Low Risk Review Panel (LR 29/2016).
This study found that the average Diet Score for a sample of over 145,000 Australians was 59 out of 100, meaning that, at large, Australians self-reported diets are not consistent with the recommendations in the Australian Dietary Guidelines. While women scored better than men, the poorest performing areas for all adults were discretionary foods, dairy foods, and healthy fats. The overconsumption of discretionary foods is problematic for a number of reasons. Firstly, discretionary foods are energy dense, and overconsumption of these foods can lead to overconsumption of energy and weight gain [14
]. Secondly, discretionary foods, which are high in saturated fat, salt, and sugar but low in essential nutrients, can displace nutrient-dense core foods—meaning the overall nutritional adequacy of the diet may be compromised.
It is well established that women have healthier dietary patterns than men [15
], and here we found that women scored better than men on nearly all aspects of diet quality, but in particular vegetables and discretionary foods. On average women scored 17%–20% better on these two dietary components. Vegetables are a key marker of a healthy diet [17
] and the focus of leading public health nutrition messages globally [14
]. Discretionary foods traditionally have not received the same attention, but given they account for about 35% of total energy intake [22
] and in the light of the current rates of obesity, an alternative approach to address healthy eating may be through starting from a message of eating less discretionary foods [23
It is assumed, but not always the case in survey data, that overweight and obese individuals consume more energy and have poorer diets than adults of a normal weight. Using short questions delivered in an online format, we found overweight and obese adults reported lower Diet Scores than normal weight adults. It is unusual to see this finding in large nutrition surveys within Australia [11
] and internationally [24
]. The inability of traditional surveys to show increases in energy intake with weight status is often associated with potential underreporting of intake, and the likelihood that underreporting is more frequent and of a greater magnitude in obese individuals [25
]. It has been estimated that Australian adults may under report their energy intake by up to 20% [11
], and this is more likely due to an under reporting of unhealthy discretionary foods. Misreporting of intake can be the result of issues associated with recall—that is remembering what and how much you consumed; or social bias—that is adjusting what you report to eat because you think you might be judged [28
]. The anonymity of online completion may remove some of this perceived judgement, resulting in a reported intake closer to true intake. In addition, the presence of pictures to assist the recall of foods in this survey may also have partially addressed some of misreporting, but we cannot know the extent to which each of these have helped. Also, there will still be some degree of error associated with individuals’ ability to aggregate intake into groupings as a result of the style of questioning used in this survey tool. Like all dietary assessment methods, we will continue to refine the methods used to minimize bias and develop a tool which estimates food intake as accurately as is possible.
The context in which one lives and works can impact on health and health outcomes [29
]. While we found little variation in Diet Score by Australian state of residence, there were interesting patterns in Diet Score by occupation. Adults working in the health industry and those who were retired scored between 62 and 63 out of 100, compared to construction workers and those who were unemployed who score 54 out of 100—which was 5 points lower than the Australian average. Occupation, like income, has been used as a proxy for socioeconomic status, and poorer dietary intake and health outcomes [31
] have been associated with lower socioeconomic status. Variation in nutrition knowledge, beliefs and food skills might also differ between groups of different occupations, which may also explain some of these differences in intake. Tailored workplace based nutrition interventions can address some of the challenges faced by people in different workplaces such as access to healthy food on site or on the road and are largely unexplored. These may be an effective approach to reaching population groups who are known to be at greater risk of poor intake.
The media releases for the Survey received national coverage, and attracted a lot of traffic to the website, resulting in Australia’s largest ever survey of dietary intake. The sample analysed here was about 15 times larger than the National Nutrition Survey conducted as part of the 2011–2012 Australian Health Survey [15
], however the media attention was not able to attract a nationally representative population—which is a key feature of the sophisticated sampling techniques used in national surveys [11
]. The National Nutrition Survey also applies a population weighting to data to allow statements about the intake of the Australian population to be made. There is potential to apply population weighting to CSIRO Healthy Diet Score survey sample in future analyses. As is commonly observed with health related surveys, the majority of the current sample were women (71% compared to 51% within the Australian population) [32
]. In addition, our sample had a higher proportion of young adults than the Australian population (18–30 years: ~30% vs. ~19% nationally) [32
]. Normal weight adults made up the greatest proportion of our sample, but still almost 50% of adults in this survey were overweight or obese (compared to 63% nationally) [15
]. While there were a range of occupations represented in our sample, the distribution may have also influenced the overall results.
Given our finding that obese adults had poorer diet quality than normal weight adults, and the overweight and obese adults were underrepresented relative to national estimates, it is likely that the overall Diet Score of all Australians would actually be lower than what has been reported here. Regardless, we found that at a national level compliance with dietary guidelines is poor. To close this gap, there is a need for nutrition promotion, practical support, and even tailoring of nutrition education initiative to specific groups to improve the overall dietary intake of the Australian population.
This project also demonstrates that readily available survey tools are suitable for use when collecting data from large samples and their automated downloads make it easy to extract large datasets. Interestingly, despite the advances in technology and the ubiquitous nature of computers and electronic devices, a recent review of short dietary assessment tools found only four tools were administered via a computer—the majority were still administered in a paper format [7
]. A second advantage of the online environment is the cost effectiveness of scalability. Once established, there is little to no additional cost associated with collecting large amounts of data. Conversely, traditional methods of dietary assessment are resource intensive and each administration generally takes time, which equates to money. Data cleaning becomes an important process for large, online surveys. We removed partially completed surveys; duplicate administrations for the same user; and excluded surveys based on erroneous age, BMI, height, and weight values. Time associated with data analysis is generally constant regardless of the amount of data collected, so overall online delivery of dietary assessment tools is a cost effectiveness mechanism.
The online environment was an asset in this study, allowing individuals to receive their Diet Score and personalized feedback immediately on completion of the survey. This type of feedback, at a food group level, is not always possible with traditional approaches to dietary assessment or even with short FFQ type surveys delivered in traditional ways. We are unsure of how this feedback changed dietary behavior, however this is an opportunity for future research. Another area for future research is to define a ‘good’ score. The average Diet Score of the highest quintile (top 20%) of this sample was about 75, which might be a benchmark for a ‘good’ score, but we cannot substantiate this with health outcomes, other that weight status. A recent examination of diet quality using the Australian Health Survey data reported higher diet quality was associated with lower odds ratio of hypertension in Australian men but not women [16
]. More longitudinal research is need to establish whether diet quality, in this case Diet Score, predicts health and risk of disease.
There are other Australian diet quality indexes which have been developed primarily for research purposes that can be applied to FFQ [33
] and 24-h recall data [8
] or state-based health surveys [34
]. There are also two Australian online FFQs which have been validated to provide estimates of usual nutrient intake [35
], and one of these also offers immediate analysis of results and comparisons with food-based recommendations [37
]. However there is a cost associated with using both of these online surveys, and only one, the Australian Eating Score, is available to individuals interested in evaluating their own diet [37
]. The advantage of making the Diet Score survey freely available is that we have reached an unprecedented number of individuals, as well as a national audience. However, this may also be considered a limitation as it attracts motivated, health interested individuals. Regardless, this survey has provided some interesting insights about the food intake of many Australians, and will continue to provide these insights on a more regular basis than the National health and nutrition surveys are able to.