Early Experience Analyzing Dietary Intake Data from the Canadian Community Health Survey—Nutrition Using the National Cancer Institute (NCI) Method

Background: One of the underpinning elements to support evidence-based decision-making in food and nutrition is the usual dietary intake of a population. It represents the long-run average consumption of a particular dietary component (i.e., food or nutrient). Variations in individual eating habits are observed from day-to-day and between individuals. The National Cancer Institute (NCI) method uses statistical modeling to account for these variations in estimation of usual intakes. This method was originally developed for nutrition survey data in the United States. The main objective of this study was to apply the NCI method in the analysis of Canadian nutrition surveys. Methods: Data from two surveys, the 2004 and 2015 Canadian Community Health Survey—Nutrition were used to estimate usual dietary intake distributions from food sources using the NCI method. The effect of different statistical considerations such as choice of the model, covariates, stratification compared to pooling, and exclusion of outliers were assessed, along with the computational time to convergence. Results: A flowchart to aid in model selection was developed. Different covariates (e.g., age/sex groups, cycle, weekday/weekend of the recall) were used to adjust the estimates of usual intakes. Moreover, larger differences in the ratio of within to between variation for a stratified analysis or a pooled analysis resulted in noticeable differences, particularly in the tails of the distribution of usual intake estimates. Outliers were subsequently removed when the ratio was larger than 10. For an individual age/sex group, the NCI method took 1 h–5 h to obtain results depending on the dietary component. Conclusion: Early experience in using the NCI method with Canadian nutrition surveys data led to the development of a flowchart to facilitate the choice of the NCI model to use. The ability of the NCI method to include covariates permits comparisons between both 2004 and 2015. This study shows that the improper application of pooling and stratification as well as the outlier detection can lead to biased results. This early experience can provide guidance to other researchers and ensures consistency in the analysis of usual dietary intake in the Canadian context.


Ratio Within-Between Variance ≤ 10
Step 5A: Run MIXTRAN macro with the AMOUNT model option to obtain estimates of the model parameters with the root survey weight Step 6A: Run DISTRIB macro to obtain estimates of mean, percentiles and proportion above or below a cutoff value from the AMOUNT model using root survey weight Step 7A: Using the bootstrap weights, run MIXTRAN and DISTRIB macro for the AMOUNT model Optional: 1-Use the estimates of the model parameters from the first run of MIXTRAN macro as starting values for bootstrap runs.
2-Use the λ from the first run of MIXTRAN macro in the bootstrap runs.
Step 8A: Estimate standard error of desired parameters (e.g. mean, percentiles, proportion above or below a cut-off value) using the bootstrap runs

Yes No
Step 6B: Identify and remove outliers: (See below for details of Outliers Detection Methods)

Go to
Step 5A Go to Step 4B Or Step 4C

Yes No
Step 5B: Run MIXTRAN macro with CORR model option to obtain estimates of the model parameters with the root survey weight Note: NOCORR model will also be automatically run by MIXTRAN Step 7C: Estimate the Fisher's transformation of the correlation coefficient parameter (ρ) and its standard error from Step 6C to test the significance of the correlation coefficient between the probability of consumption and the amount consumed Step 6C: Test for correlation: run MIXTRAN with the CORR model using at least the first 50 bootstrap weights.

Note:
Use the λ from Step 5B in the bootstrap runs to ensure testing from a similar model

Optional:
Use the estimates of the model parameters corresponding to CORR model from Step 5B as starting values for bootstrap runs.

Ratio Within-Between Variance ≤ 10
Model convergence

Go to
Step 8B

Go to
Step 8C

Go to
Step 6D

No
No No

Yes
Step 6B: Identify and remove outliers: (see below for details of Outliers Detection Methods)

Go to
Step 5B Go to Step 4B Or Step 4C

Yes No
Step 6D: Run MIXTRAN macro with NOCORR model option to obtain estimates of the model parameters with the root survey weight

Go to
Step 5A

Yes
Ratio Within-Between Variance ≤ 10

No
Step 6B: Identify and remove outliers: (see below for details of Outliers Detection Methods)

Go to
Step 8B

Go to
Step 4B or Step 4C Caution: If the Amount model is selected, the estimates of usual intakes may be biased upward, as the zero amount will be replaced by one-half the minimum amount

Go to
Step 6D

Go to Step 4B
Or Step 4C

Yes No
Step 8B: Using parameter estimates from MIXTRAN corresponding to the NOCORR model option, obtained either as part of the Step 5B or from Step 6D, run DISTRIB macro to obtain estimates of mean and percentiles with the NOCORR option in the MODELTYPE parameter using the root survey weight.
Step 9B: Run MIXTRAN and DISTRIB macros with the NOCORR model option using bootstrap weights (analysis of Step 6C may need to be re-run)

Optional:
1-Use the estimates of the model parameters from the MIXTRAN macro, obtained either as part of the Step 5B or from Step 6D, as starting values for bootstrap runs.
2-Use the λ from the first run of MIXTRAN macro in the bootstrap runs.
Step 10B: Estimate standard error of desired parameters (e.g. mean, percentiles, proportion above or below a cut-off value) using the bootstrap runs UNCORRELATED MODEL

END
Step 8C: Using parameter estimates from MIXTRAN corresponding to the CORR model option, obtained from Step 5B, run DISTRIB macro to obtain estimates of mean and percentiles with the CORR model option using root survey weight Step 9C: Run MIXTRAN macro with the CORR model option with remaining bootstrap weights from Step 6C.

Note:
Use the λ from Step 5B in the bootstrap runs to be consistent with Step 6C

Optional:
Use the estimates of the model parameters corresponding to CORR model from Step 5B as starting values for bootstrap runs.
Step 11: Estimate standard error of desired parameters (e.g. mean, percentiles, proportion above or below a cut-off value) using the bootstrap runs.

CORRELATED MODEL
Step 10C: Run DISTRIB macro with the CORR model option for all the bootstraps considered in Step 6C and Step 9C.

Method I: Large Within-Between Variance Components
• When the ratio of within/between variation is greater than 10, consider the mean distribution of the difference between Day 1 and Day 2 recalls.
• Values were identified as possible outliers if they fell ±3, ±2.5 or ±2 SD away from the mean distribution of difference between Day 1 and Day 2 values • Day 2 recalls were removed as Day 1 recalls are considered to be less biased • The scenario which first resulted in the within-between variance ratio less than 10 and excluded the fewest second 24hr recalls was retained