Review of Short-Form Questions for the Evaluation of a Diet, Physical Activity, and Sedentary Behaviour Intervention in a Community Program Targeting Vulnerable Australian Children

Childhood obesity is associated with low socioeconomic status in developed countries, and community programs can deliver cost-effective obesity interventions to vulnerable children and adolescents at scale. Evaluating these programs in a low-cost, time-efficient, and culturally appropriate way with valid and reliable measures is essential to determining their effectiveness. We aimed to identify existing valid and reliable short-form instruments (≤50 items for diet, ≤15 items for physical activity) suitable for the assessment of change in diet, physical activity, and sedentary behaviour in an Australian obesity intervention program for children and adolescents aged 7–13 years from low socioeconomic groups, with a focus on Aboriginal and Torres Strait Islander children. Relevant electronic databases were searched, with a focus on Australian literature. Validity and/or reliability studies using diet instruments (5), physical activity/sedentary behaviour instruments (12), and diet and physical activity/sedentary behaviour instruments used with Aboriginal and Torres Strait Islander (3) children were identified. Seven questions on diet, one question on physical activity, and no questions on sedentary behaviour were recommended. These questions can be used for evaluation in community-based obesity programs among Australian children and adolescents, including those from low socioeconomic groups and Aboriginal and Torres Strait Islander children.


Introduction
Obesity in childhood and adolescence is associated with low socioeconomic status in developed countries [1][2][3]. Children who are overweight or obese are more likely to experience health problems, including higher metabolic and cardiometabolic risk factors, asthma, negative psychological outcomes, poorer dental health, and sleep issues [4,5]. They also have a greater likelihood of being overweight

Results
There were 18 unique validity and/or reliability studies meeting the inclusion criteria identified through two searches of the literature (see Section 3) and from experts in the fields: 5 short diet questions, 12 physical activity and/or sedentary behaviour measures, and 1 additional study identified from the search specific to Aboriginal and Torres Strait Islander children. Two of the papers identified in the search for non-indigenous specific studies were also identified in the search specific to Aboriginal and Torres Strait Islander children. A detailed narrative of each study included in the current review and the project-specific report is provided in Flood et al. [25]. A final list of the short-form questions recommended from the review can be found in the Supplementary Materials.

Diet Questionnaires
Five studies [26][27][28][29][30] were identified. Of these, one conducted reliability testing only [26], and the remaining studies conducted both validity and reliability testing [27][28][29][30]. Most of the studies did not indicate ethnicity and only half indicated weight status of the participants. Socioeconomic status was generally not indicated. The details of these studies are included in Table 1.
The review identified questions on fruit and vegetable intake, water and sugary drink consumption, discretionary food intake, breakfast consumption, and eating in front of the television as common. The questions we recommended for use in evaluation included frequency of consumption of fruit, vegetables, sugar sweetened beverages, water, and discretionary foods [29]. These have been tested in Aboriginal and Torres Strait Island groups (see Section 2.3). An additional question on eating the dinner meal in front of the television [26] was also recommended.

Physical Activity and Sedentary Behaviour Questionnaires
Twelve validity and/or reliability studies on physical activity and/or sedentary behaviour measures were identified. Four studies were on physical activity only [31][32][33][34], four were on sedentary behaviour only [35][36][37][38], and four combined physical activity and sedentary behaviour measures [26,[39][40][41]. There was a wide age range across the studies, and most of the studies did include details on ethnicity but not on the weight status of the participants. Only three studies involved parents answering the questionnaire [37,38,41]. Inclusion of children with low literacy was indicated in the pilot phase of one of the studies only [26], one included rural Aboriginal and Torres Strait Islander and non-Indigenous children [32], and three studies indicated mixed socioeconomic status (two used maternal education as an indicator) [33,37,41]. The details of these studies are included in Table 2.
Frequency and duration of different physical activity and sedentary behaviour domains were commonly reported. One question that had been tested for validity and reliability in a range of child/adolescent profiles [34] and used in the Australian setting [42] was recommended to evaluate physical activity. A suitable short-form question was not found for the evaluation of sedentary behaviour for the Go4Fun ® program, however a single question assessing children's television screen time could be considered [26].

Diet and Physical Activity/Sedentary Behaviour Questionnaires in Aboriginal and Torres Strait Islander Groups
Only one dietary intake questionnaire [29] and two physical activity questionnaires [32,43] were identified that had been validated with Aboriginal and Torres Strait Islander children. There were no sedentary behaviour questionnaires identified. There was a narrow age range across the studies-none of the studies involved parental response, and only one study included the weight status of the participants [32]. Socioeconomic status and literacy levels were not indicated in these studies. The details of these studies are included in Table 3. The Short Food Survey (SFS) consisting of 38 items on "usual" intake, including 35 on food and three on beverages.
The survey was completed online by the parent.

Reliability
The ICC was 0.43-0.94 for food groups/beverages, and 0.92 for the total diet index score (all p < 0.01).

Validity
The ICC was 0.04-0.44 for food groups/beverages and 0.44 for the total diet index score (p < 0.01). Percentage agreement across tertiles of index scores was 84% between the administrations and 43% when comparing the SFS with the mean of the recalls. Bias values were within the 95% CI.   The recommended measure had two recall assessing frequency of past seven days and "usual" activity performed for a total of at least 60 min per day.
Self-administered by children, supervised by research staff.  Three items on time spent in sedentary behaviour (watching TV, playing electronic games, and using the computer) were presented for a typical week (Monday to Friday) and a typical weekend (Saturday and Sunday).

Reliability
Self-administered by children and parents.

Reliability *
The ICC of the proxy-reported time (minutes per day) spent on each of these screen based behaviours ranged from 0.6 to 0.8.

Validity *
The ICC of the proxy-reported time (minutes per day) spent on each of these screen-based behaviours ranged from 0.44 to 0.61. * Report states that "Because proxy-reported sedentary time was more reliably reported, these items were used in analyses rather than the children's self-reports." (p. 1942).      Figure 1 summarises the quality rating for studies.  Figure 1 summarises the quality rating for studies. The reporting of studies was generally adequate, however many studies did not describe the characteristics of participants with missing, incomplete, and/or invalid data. The external validity of both diet and physical activity studies in terms of the representativeness of those invited to participate and those participating was often not able to be determined or was not adequate. However, the mode of administration of instruments was usually representative of similar study designs. Some aspects of internal validity such as attempts to minimise altered behaviour, appropriate statistics to test reliability (where applicable), planning of analyses, and having sufficient power were adequate across the studies.

Quality Ratings
For validity studies, the reference measure was generally deemed to be more accurate than the test method and assessed behaviour in the same timeframe, however while studies on dietary measurement used appropriate statistics to assess agreement, this was not always the case for studies measuring physical activity/sedentary behaviour. For those studies that assessed reliability, statistics were mostly assessed to be appropriate. Clear exposition of compliance was frequently not provided, and blinding of research staff was either not able to be determined or not done. The reporting of studies was generally adequate, however many studies did not describe the characteristics of participants with missing, incomplete, and/or invalid data. The external validity of both diet and physical activity studies in terms of the representativeness of those invited to participate and those participating was often not able to be determined or was not adequate. However, the mode of administration of instruments was usually representative of similar study designs. Some aspects of internal validity such as attempts to minimise altered behaviour, appropriate statistics to test reliability (where applicable), planning of analyses, and having sufficient power were adequate across the studies.
For validity studies, the reference measure was generally deemed to be more accurate than the test method and assessed behaviour in the same timeframe, however while studies on dietary measurement used appropriate statistics to assess agreement, this was not always the case for studies measuring physical activity/sedentary behaviour. For those studies that assessed reliability, statistics were mostly assessed to be appropriate. Clear exposition of compliance was frequently not provided, and blinding of research staff was either not able to be determined or not done.

Discussion
Valid and reliable short-form questions to measure dietary intake, physical activity, and sedentary behaviours are ideal for routine monitoring and evaluation of community programs to treat child and adolescent obesity. Despite the general availability of questionnaires, many of the articles reviewed in the current study did not have information on ethnicity, weight status, socioeconomic status, and literacy levels to determine their suitability for Go4Fun ® and similar programs, and many were not tested with a parent proxy. There were few studies specifically conducted with Australian Aboriginal and Torres Strait Island children. Although the general representativeness of the samples recruited across reviewed studies was not optimal, some recommendations were still able to be made based on question validity and reliability, suitability to address the targeted outcomes of Go4Fun ® and similar programs, and potential to be administered in different ways and among different population groups. Additionally, the quality of the studies from which questions were recommended [29,34] were satisfactorily rated by reviewers.
The dietary factors evaluated by the recommended short-form dietary questions from the current review align with public health concerns and are similar to those identified by Golley et al. in their recent systematic literature review [44]. These questions demonstrated good reliability, however, satisfactory validity was not consistently demonstrated. Golley et al. similarly found that short-form food questions were more likely to be reliable than valid, and seldom both [44]. Responsiveness (ability to detect change) to an intervention compared with an alternative diet assessment at both time points was not identified in studies included in the current review. The recommended dietary intake questions may therefore be useful to indicate pre-post change in program interventions that target these behaviours, but not the magnitude of change. We found that frequency versus quantity of intake was generally found to be a superior measure. Children under 12 years old may be poor at conceptualising portion size even when prompts are provided [45]. Additionally, questionnaires with prespecified portion sizes may rely on serving sizes that more closely represent population food selection guides, however both adults and children may typically consume portion sizes that vary from guidelines [46,47].
The physical activity questions identified in the current review tended to perform poorly for validation of activity. Accurate assessment of activity in children is known to be difficult and may reflect the cognitive ability of this group in recalling different aspects of physical activity (e.g., intensity, frequency and duration) [48][49][50], as well as the sporadic nature of some activities, particularly of younger children [50][51][52]. In fact, in their systematic review and appraisal of studies of self-administered and proxy-reported physical activity questionnaires in youth, Chinapaw et al. determined that there were no questionnaires available with acceptable validity and reliability [53]. In any case, where program evaluation includes elements across the whole program, the inclusion of longer, more complex physical activity questionnaires [31,41] would impose an unacceptable burden, particularly for low literacy groups. Short-form questions which have demonstrated adequate reliability and validity in the USA as well as having been evaluated across different ethnicities [34] have been recommended for national monitoring in Australia [42], and one of these questions was recommended from our current review for evaluation of physical activity in community programs for children and adolescents. The value of using objective measures of physical activity (e.g., activity trackers such as pedometers and accelerometers) for children and adolescents is often discussed in the physical activity literature, however these may not be suitable. Gwynn et al. found that around 20% of children may remove these devices for various reasons [32]. If they are used in programs where assessment of change is important, it is recommended that a standardised protocol be used across timepoints [54].
Sedentary behaviour occurs across a range of activities in children and adolescents, for example inactive transport, desk-based schoolwork, and various forms of screen time. Australian national guidelines for children and adolescents aged 5-17 years recommend minimising the time spent on sedentary behaviour, and specifically limiting the use of electronic media (including television watching and computer use) to less than two hours a day [55]. The Adolescent Sedentary Activities Questionnaire (ASAQ) [36] was identified in the current review as having reliability and face validity, and does include questions on screen time; however, it is likely to be too lengthy for community program settings. We could not recommend any short-form questions to assess sedentary behaviour more broadly, however a single question assessing children's television screen time [26] was considered to be potentially suitable for use in community programs if validated in the target population. Although a number of sedentary behaviours are associated with reduced energy expenditure and passive consumption of food [56], the most common measure of sedentary behaviour in this children and adolescents is television watching [57]. In their systematic review of over 200 studies, Tremblay et al. found that watching television for more than 2 h per day was associated with a range of adverse health outcomes, including unfavourable body composition, decreased physical fitness, and poorer scores on psychosocial and academic measures [57]. However, in community programs that target multiple forms of sedentary behaviour, a more global measure would be required. An additional consideration is that screen devices are constantly evolving [58,59], which may require modification of questions that address screen time behaviour.
Few studies were available that were specifically tested in Australian Aboriginal and Torres Strait Islander groups, representing a gap in the literature. In addition to the three studies identified by our review [29,32,43], Thurber et al. recently evaluated the relationship of screen time and dietary factors such as sugar-sweetened beverage and discretionary food intake reported by carers to body mass index trajectories in Aboriginal and Torres Strait Islander children [60]. However, the instrument used was not validated. Healthy physical activity and eating may be experienced differently among Australian indigenous children, as explained by Crowe et al. in their qualitative study of 40 Australian Aboriginal and Torres Strait Islander children aged 5-12 years recruited from the southeastern coast of Australia [61]. They found that healthy lifestyle behaviours were connected and influenced by cultural connections and activities [61], which may need further consideration in future questionnaires that measure diet, physical activity, and sedentary behaviours in this group.
A strength of this study is the consideration of short-form questions suitable for vulnerable child and adolescent populations, including Australian Aboriginal and Torres Strait Island groups. Children from Aboriginal, Torres Strait Islander, and low socioeconomic groups have higher rates of obesity than in the general population [16,17] and there is a need for measures to evaluate suitable programs for these children. A further strength of this study is the inclusion of only short-form questions since many available questionnaires for use in research are lengthy, burdensome, and not suited to a community intervention setting. A limitation of the review is that psychometric assessment of some of the included questionnaires may have favoured recruitment of children from less disadvantaged backgrounds, and as such it may be less applicable for very disadvantaged children, however most of the dietary questions identified were tested for validity and reliability in "priority funded" (disadvantaged) schools. A further weakness of the current work is the timeframe of the review. The searches were completed in mid-2016 due to the requirements of the commissioning body.

Materials and Methods
This review was conducted by a research team with expertise in: diet and physical activity interventions with Australian Aboriginal and Torres Strait Islander children as well as non-Indigenous children, the development and interpretation of relevant measurement instruments, and in associated validity/reliability studies.

Search Criteria
Searches were devised to locate: (1) validity/reliability studies on diet, physical activity, and sedentary behaviour measurement instruments suitable for Australian children and adolescents aged 7-13 years; and (2) studies with a focus on Australian Aboriginal and Torres Strait Islander children; hence, Australian papers were the focus of the search. The search strategy was developed by the review team, and one researcher (J.A.G.) conducted a systematic literature search to identify studies addressing the review questions. English language studies published between 1 January 2005 and 18 April 2016 were identified from the following electronic databases: Medline, CINAHL, EMBASE, and ATSIhealth. Search terms are shown in Table 4.

Selection and Inclusion Criteria
The literature search predominately comprised Australian studies in peer-reviewed journal articles, however selected publications from the international literature were also included if they met the inclusion criteria. Diet, physical activity, and sedentary behaviour questions were also sourced from the Parenting, Eating and Activity for Child Health [62] and GRx Active Families [63] as these were known childhood obesity programs in Australia and New Zealand respectively.
Studies were included where the following items were described: • components of diet, physical activity or sedentary behaviour questions relevant to current Australian nutrition and physical activity/sedentary behaviour policies for those aged 5-12 and 13-17 years, or that make a significant contribution to components of concern identified in policy documents; and • short questionnaires with ≤50 items for diet [44] and ≤15 items for physical activity (expert opinion); and • validity or reliability information in the population of interest (7-13 year old Australians); and • questionnaire administration details indicating completion by children/adolescents or parent proxy.
One reviewer (J.A.G.) screened the titles and abstracts of studies identified from the searches following removal of duplicates. Studies not meeting inclusion criteria were removed by the same reviewer. The same inclusion/exclusion criteria were applied to the full text of the remaining studies. Validity/reliability studies from the reference lists of relevant intervention studies and systematic reviews which met the inclusion criteria were included (if publication date was prior to 2005, inclusion was based on expert opinion). Additional references were included as advised by the review team. Figure 2 illustrates the flow of information through the different phases of the review according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) process [64].

Data Extraction
Data were extracted by two researchers (J.A.G., J.D.G.) in standardised tables that included: author, year and country of study, program setting and name (if applicable), design, characteristics of the participants (sample size, ethnicity, age, sex, weight status, literacy details, comparison group), tool type and number of items, response variables, recall period, administration method, respondent (child or parent), respondent burden, duration of the study, period between administration (for reliability), reliability statistics, reference method (for validity), and validity statistics.

Assessment of Quality
The methodological quality of each paper that met inclusion criteria was independently assessed by two of four members of the review team (J.A.G., J.D.G., L.L.H., V.M.F.) using a modified version of the Hagströmer-Bowles Physical Activity/Sedentary Behaviour Questionnaire Checklist (HBQC) [65], which is based on the scale devised by Downs and Black [66]. Members of the review team did not rate studies for which they were also an author.
The HBQC was modified to assess dietary validation/reliability papers simply by inserting relevant terminology in place of the physical activity/sedentary behaviour terminology. Additionally, in order to assess reliability in the studies, one question was added as follows: "Were the statistical tests used appropriate to assess reliability for the main physical activity constructs between tests for the self-report measure?" (The statistical techniques used must be appropriate to the data e.g., intra-class correlation co-efficient, weighted kappa).
While the HBQC scores items numerically, questions were simply assigned a value of yes, no, or unable to be determined/unsure for the current review. A "partially" option was available for the final question on statistical power, as per the HBQC. To accommodate studies that reported either validity or reliability but not both, a not applicable (N/A) option was included for relevant questions. A decision was made not to score the papers numerically because some individual questions may have more or less perceived importance qualitatively, and some methodological areas may have more or less questions; these two factors may bias the impression of the overall quality of the paper for low or high numerical scores.
All four raters met to discuss the ratings and settle differences in ratings at an interim stage to assist with consistency by checking interpretation on quality items. When all papers had been rated individually, pairs met by phone or in person to discuss any differences. Differences that were not resolved by discussion were shared with the full team for adjudication. A record of decisions on interpretations was kept and shared with the team for review of past decisions to ensure all quality items were rated consistently within and across pairs of raters.

Recommendations on Questions
Four members of the review team (J.A.G., J.D.G., L.L.H., V.M.F.) were involved in the final recommendations on the questions. Deliberations were made by group discussion following (and based on) data extraction of included studies. Factors considered by reviewers included the questions' validity and reliability, suitability to address the objectives (targeted outcomes) of the Go4Fun ® program, and potential to be administered in different ways and among different population groups, in particular, Aboriginal and Torres Strait Islander children. Acceptable (statistically significant) validity and reliability were required. Specific outcomes measured in the Go4Fun ® program included daily servings of fruit, dairy foods, vegetables, sugar-sweetened beverages, and discretionary foods, as well as hours in physical activity and sedentary behaviour (screen time and non-active transport).

Conclusions
In conclusion, there were some valid and reliable questionnaires that were considered useful for evaluation of our community-based obesity intervention targeting healthier diet, physical activity, and sedentary behaviour in Australian children and adolescents. Questionnaires selected for evaluation of programs need to capture the objectives of the intervention. The questions identified in this rapid review can provide information on the primary factors involved in child obesity prevention, that is, consumption of fruit and vegetables, sugar-sweetened beverages, and energy-dense nutrients, poor eating habits, time spent in physical activity, and screen time. Culturally appropriate support must be provided for Aboriginal and Torres Strait Islander children completing the survey questions. Central to this is ensuring a key role for their community members in survey administration and in interpretation of results.