Health Reports
Accounting for misreporting when comparing energy intake across time in Canada

View the most recent version.

Archived Content

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please "contact us" to request a format other than those available.

Archived

This page has been archived on the Web.

by Didier Garriguet

In 2015, Statistics Canada collected national data on detailed dietary intake for the first time in over a decade.^{Note 1} Previous data were collected in 2004.^{Note 2} Initial estimates^{Note 3} show that in 2015, estimated energy intake was lower than in 2004^{Note 4} by 250 kcal, on average.

Any change observed in estimated energy intake should be reflected in corresponding changes to characteristics associated with energy requirements, such as age, height, weight, body mass index (BMI) and physical activity. For example, the data would be expected to show lower national estimates of BMI or physical activity in 2015 compared with 2004 as a consequence of a decrease of 250 kcal in estimated energy intake. Depending on age and sex, a decrease in weight varying from 8 kg to 27 kg would be expected to explain such a change in energy intake using the Institute of Medicine equations for predicted energy requirements.^{Note 5} However, the 2015 CCHS – Nutrition shows little change in measured overweight and obesity rates in both children^{Note 6Note 7} and adults.^{Note 8} The Canadian Health Measures Survey (CHMS) also shows similar trends.^{Note 9Note 10} Physical activity measurements changed between the 2004 CCHS – Nutrition and the 2015 CCHS – Nutrition and cannot be compared; however, CHMS data indicate that measured physical activity levels have remained stable in children^{Note 11} and adults^{Note 12} over the last decade. An average daily decrease of 60 minutes of walking from 2004 to 2015 would be required to explain a decrease of 250 kcal.

Self-reported dietary intake is subject to misreporting (i.e., overestimating or underestimating dietary intake). In general, under-reporting tends to be more common than over-reporting in Europe, North America and Australia.^{Note 13Note 14Note 15Note 16} If the direction (under or over) and magnitude of misreporting remained consistent from 2004 to 2015, this systematic bias would potentially cancel itself out. However, if the direction or magnitude of misreporting changed, it may explain some of the differences observed in estimated energy intake.

Sophisticated measurement of energy intake using techniques such as doubly labelled water^{Note 17} is not feasible within the context of population health surveys. This precludes any comparisons between “true” and “reported” intake. Energy intake is estimated in the CCHS – Nutrition using self-reported information about what respondents drank and ate the day prior to their interview. A “reasonable approach to characterize underreporting in the absence of objective measures of total energy expenditure requirements (TEE)”^{Note 17} is to use a method to assess misreporting based on TEE and derived from formulas using age, height, sex, weight, BMI and physical activity. Such methods include the Goldberg method^{Note 18} or the method proposed by McCrory,^{Note 19} which was used previously with the 2004 CCHS – Nutrition.^{Note 20Note 21}

Other nutrition surveys have compared misreporting through time. In Australia,^{Note 16} the prevalence of low-energy reporters has increased from 1995 to 2011/2012. In the United States, under-reporting also changed from 1971 to 2010,^{Note 22} but this was found to be mostly the result of methodological differences.

The objective of this study is to estimate the change in misreporting from the 2004 CCHS – Nutrition to the 2015 CCHS – Nutrition and to determine how this change may affect further analysis of the data. In particular, this study examines the effects this change has on estimates of total energy intake, the main sources of the change and the potential introduction of a bias.

Methods

Data sources

The 2004 CCHS – Nutrition and the 2015 CCHS – Nutrition collected information on food and beverage consumption with a 24-hour dietary recall. The target population was residents of private dwellings aged 1 or older (people younger than 1 year of age were also included in the 2004 CCHS – Nutrition). Both surveys excluded members of the regular Canadian Forces; people living in the territories, on Indian reserves, in institutions and in some remote regions; and all residents (military and civilian) of Canadian Forces bases.

A total of 35,107 and 20,487 respondents, respectively, took part in the initial 24-hour dietary recalls in 2004 and 2015. In addition, 10,786 and 7,608 respondents completed a second recall 3 to 10 days later. Data were mainly collected in person for the first recall, and through telephone interviews for the second. The 2004 response rates were 76.5% for the first recall and 72.8% for the second recall. The corresponding 2015 response rates were 61.6% for the first recall and 68.6% for the second. Height and weight were measured for 20,739 respondents aged 2 or older in 2004, and for 13,713 respondents aged 2 or older in 2015. The item response rate for that component was 62.5% in 2004 and 70.6% in 2015.

To be representative of the Canadian population at the national and provincial levels, both surveys were weighted to account for the sample design and non-response. Additional weights were provided to account for the additional non-response to the height and weight component. Details about the design, sample and collection are available online.^{Note 1Note 2}

Energy intake

Respondents were asked to report everything they ate and drank in the 24 hours before the interview. To maximize recall, both surveys used the Automated Multiple Pass Method,^{Note 23} which consists of the following five elements: (1) a quick list of easily remembered foods; (2) probes for commonly forgotten foods; (3) time and occasion to group foods consumed at the same time; (4) detailed questions on the previously reported foods, including serving size; and (5) a final review.

A food booklet was used to show respondents pictures of plates, bowls, glasses and mugs to increase accuracy in the reporting of food and beverage sizes: in 2004, dishes were represented by drawings, but were replaced by pictures in 2015. Standard amounts in 2015 were, in general, smaller than standard amounts in 2004, especially for bowls, glasses and mugs.^{Note 1}

Canadian Nutrient File

A food database is required to assign energy and other nutrient values to the foods and beverages reported in the 24-hour recall. The Canadian Nutrient File (CNF) is maintained and continually updated by Health Canada.^{Note 24} Each CCHS – Nutrition uses a fixed version of the CNF.

The CNF used for the 2004 survey was the 2001b Supplement version; for the 2015 survey, the 2015 version was used. The CNF reflects foods available on the market at the time of the survey.

Energy requirements

TEE requirements can be estimated with a series of equations developed by the Institute of Medicine (IOM).^{Note 5} These equations are specific to age, sex and body mass index (BMI) category (normal weight compared with overweight or obese). In addition, physical activity level (sedentary, low active, active, very active) is required to estimate TEE.

Height, weight and body mass index category

Trained interviewers measured the height and weight of CCHS respondents aged 2 or older, following the same protocols in 2004 and 2015.^{Note 1Note 2} The same model of scales (LifeSource Scales Model US-321) was used in both 2004 and 2015. To minimize reporting mistakes, the measurement tape was changed to a metric-only tape in 2015 after a metric and imperial tape was used in 2004.

BMI is calculated by dividing weight in kilograms by height in metres squared. According to Health Canada’s guidelines for adults,^{Note 25} a BMI from 18.5 kg/m² to 24.99 kg/m² is normal weight, from 25 kg/m² to 29.99 kg/m² is overweight, and 30 kg/m² or more is obese. For respondents aged 17 or younger, the categories defined by Cole et al.^{Note 26} were used, since the World Health Organization categories^{Note 27} were not published when the IOM equations for TEE were published.

Physical activity

Physical activity was assessed differently in 2004 and 2015, precluding the use of physical activity estimates from the survey itself for comparison purposes. In the absence of comparable physical activity levels, a fixed level of physical activity is assumed for the entire population.^{Note 28} In this study, children younger than 14 years of age were assumed to be low active, while teenagers aged 14 years or older and adults were assumed to be sedentary. These levels are consistent with what was observed in directly measured physical activity among children^{Note 11} and adults^{Note 12} from 2007 to 2015. Sensitivity analyses were conducted assuming all groups were either sedentary or low active.

Misreporting

Since neither true energy intake nor biomarkers for true energy intake are available in the CCHS – Nutrition, it is reasonable to use predicted TEE to characterize misreporting and identify under-reporters, plausible reporters and over-reporter.^{Note 17} To do so, thresholds must be established to create a confidence interval for the ratio of energy intake to energy requirement. Goldberg^{Note 18} suggested the original method, which Black^{Note 29} and McCrory^{Note 19Note 28} further modified. The latest technique was previously used with the 2004 CCHS – Nutrition^{Note 21}.

To estimate the thresholds, a standard deviation (SD) is required that accounts for the coefficients of variation of multiple components according to the following formula:

where $C{V}_{rEI}^2$ represents the intra-individual variation of energy intake, d represents the number of days of recall, $C{V}_{pER}^2$ represents the error in predicted energy requirements, and $C{V}_{mTEE}^2$ represents the day-to-day variation and the measurement error for TEE based on doubly labelled water.

Following previous work,^{Note 21} Black and Cole^{Note 30} estimate $C{V}_{mTEE}^2$ at 8.2%. The other coefficients are estimated for each CCHS – Nutrition: $C{V}_{rEI}^2$ from the average individual variation of respondents who provided two dietary recalls and $C{V}_{pER}^2$ from predictions of energy expenditure. Since only the first recall is used in this study, d was set to 1. Table 1 shows the SD values estimated for the CCHS – Nutrition by dietary reference intakes (DRIs), for the entire population. For ease of use, a uniform SD value of 35% was used for both surveys.

Finally, the confidence interval for the ratio is built in the log scale to account for the skewness of the estimated energy intake distribution. A multiplicative factor can be used with the SD, but the factor is set to 1 in this study. The resulting interval for the ratio EI:TEE is then (0.70, 1.42). Respondents are classified based on the percentage of their TEE that they reported as energy intake: less than 70%, under-reporters; between 70% and 142%, plausible reporters; and over 142%, over-reporters.

Sources of estimated energy intake

The CCHS – Nutrition contains food categories based on the Bureau of Nutritional Sciences (BNS) classification. These categories were used to estimate sources of energy intake. Basic food and recipe ingredients are used for this classification, not the recipes themselves. Categories are listed in Appendix 1.

Bias

In addition to height, weight, age and BMI categories, other categories used to estimate potential bias among plausible reporters included being an immigrant or a daily smoker. The highest level of education in the household was also used based on the categories available in 2004 (postsecondary diploma) and 2015 (bachelor’s degree or above bachelor’s degree). These variables were previously identified^{Note 21} to be correlated with the ratio EI:TEE.

Statistical analysis

Descriptive statistics were used to present estimated energy intake, TEE, types of reporters, sources and potential bias. Only the first recall was used at this stage, since only averages are presented, and average daily intake is the same as average usual intake. Estimates were weighted using respondents aged 2 or older with a specific sample weight, accounting for the lower response rate among people with measured height and weight. Energy intake estimates using the full sample were not different than energy intake estimates restricting the sample to respondents with measured height and weight (data not shown). The bootstrap method was used to estimate confidence intervals, since it takes into account the complex nature of the survey. Comparisons were done using t-tests. The significance level was set at p<0.05.

Results

Estimated energy intake and requirements

Table 2 shows that, on average for the full sample, estimated energy intake was 250 kcal lower in 2015 compared with 2004, while predicted TEE was 1 kcal higher in 2015 compared with 2004. Differences in estimated energy intake ranged between -60 kcal and -512 kcal, depending on the age and sex group, with 2015 estimates always lower than 2004 estimates. Significant differences in TEE were observed for males aged 9 to 13 (lower TEE in 2015) and 31 to 50 (higher TEE in 2015). These differences are mostly explained by the average weight being 2.5 kg lower in 2015 for males aged 9 to 13 and 3.7 kg higher for males aged 31 to 50 (data not shown).

Misreporting

Under-reporters, plausible reporters and over-reporters were identified using the ratio EI:TEE and applying the 0.7 and 1.42 thresholds. The percentage of plausible reporters was stable from 2004 to 2015 (Table 3). However, among age and sex groups in 2015, the percentage of over-reporters was lower in 11 out of 14 groups, and the percentage of under-reporters was higher in 9 groups (Table 3).

Comparing estimated energy intake in 2004 and 2015

Respondents categorized as under-reporters or over-reporters had similar estimated energy intake in 2004 and 2015. In 2015, the estimated energy intake for under-reporters was 21 kcal lower than in 2004, and 2 out of 14 DRI groups had significantly lower estimates in 2015 (Table 4). Although the total estimated energy intake for over-reporters was significantly lower (181 kcal) in 2015, only 2 out of 14 DRI groups had significantly lower estimates in 2015 (Table 4).

The difference in estimated energy intake was 84 kcal for plausible reporters. Children aged 2 to 13 years, male teenagers (aged 14 to 18 years) and females aged 19 to 50 years had significantly lower estimated energy intake in the plausible reporter category (Table 4).

Sources of energy

Table 5 lists the top caloric sources that each contribute to at least 10 kcal of the difference in estimated energy intake by BNS group. The top sources are based on the entire sample and the absolute value of the difference, representing around 75% of all estimated energy intake for both years (data not shown). For the entire sample, two-thirds of the 250 kcal difference in estimated energy intake is explained by sugar-sweetened beverages; pasta, rice, cereal grains and flours; meats; and milk. For plausible reporters, estimated energy intake differed very little by BNS group, except for sugar-sweetened beverages, explaining most of the difference. For the smaller sample of over-reporters, differences were more volatile.

Estimated energy intake from sugar-sweetened beverages and milk was lower in 2015 than in 2004, regardless of the category of reporters. On the contrary, estimated energy intake from nuts and pastries was higher in 2015 for all categories of reporters.

Bias

Comparisons between the plausible reporters and the entire population are shown in Table 6. Bias is small between the plausible reporters and the entire population. Body weight and the percentage of the population who are overweight or obese are lower for plausible reporters, especially in 2015. Obesity prevalence was 3% lower for plausible reporters in 2015.

Discussion

Reported energy intake in 2015 was substantially lower than reported energy intake in 2004 by 250 kcal on average. This decrease cannot be explained by a change in predicted total energy expenditure (a difference of 1 kcal, on average) because of the lack of change in weight and physical activity during that period.

If misreporting in the CCHS – Nutrition was similar in 2004 and 2015, the systematic bias for each year would cancel itself out, and any differences observed between cycles would reflect a change in the estimated energy intake of the population. This study found that misreporting did change from 2004 to 2015. Compared with 2004, in 2015, the proportion of under-reporters was higher and the proportion of over-reporters was lower. When only the plausible reporters from each survey are considered, the difference in estimated energy intake is 84 kcal, compared with 250 kcal for the entire population. An alternative to limiting the analysis to plausible reporters is to adjust the energy intake estimate according to reporting status. This is similar to age standardization, only using reporting status instead. To do this, a reporting status distribution must be fixed. In this case, it is fixed to 2004. Average estimated intake by reporting status in 2015 is then multiplied by the proportion of under-reporters, plausible reporters and over-reporters observed in 2004. This would lead to an adjusted estimate for reporting status. With this approach, the energy intake decrease from 2004 to 2015 is 85 kcal, the same as was observed when only plausible reporters were considered.

Detailed analyses of food sources were not the main objective of this study. However, these analyses provided insight on the potential effect of a change in misreporting between the 2004 CCHS – Nutrition and the 2015 CCHS – Nutrition. For example, they highlight the observation that, no matter the category of respondents, estimated energy intake from beverages was lower in 2015 and estimated energy intake from pastries and nuts was higher in 2015. For many other sources, such as meat, sugars and oil, the decrease in estimated energy intake results from the decrease in the category of over-reporters.

Few studies have been published on the change in misreporting between time points of nutrition surveys. Archer^{Note 22} showed that in the United States, under-reporting also increased from 1999 to 2010, but to a lesser extent than in Canada. Some assumptions of this analysis have been criticized.^{Note 31} In Australia, under-reporting increased from 1995 to 2011/2012.^{Note 16} With the Goldberg method, the difference in estimated energy intake among respondents aged 10 and older was 347 kcal for the entire Australian sample, compared with 244 kcal for the plausible reporters. The reported difference in energy intake includes over-reporters and may explain why the impact was smaller in Australia. By default, potential over-reporters were classified as plausible reporters in Australia, and it is not possible to know whether the change in under-reporters is a change from plausible reporters or over-reporters. In Canada, the proportions of both under-reporters and over-reporters changed between surveys.

Limitations

Many assumptions were made in this study that could affect the estimated change in energy intake. Physical activity was assumed to be the same from 2004 to 2015 and among all respondents of a given age group. Therefore, the variability in physical activity cannot be taken into account. Assuming the entire population is low active or sedentary would change misreporting for a given year, but would not affect assessments of the change in misreporting between two time points (data not shown).

Although a different CNF was used in each CCHS – Nutrition, it was assumed in this analysis that this difference had no impact. However, changes to the CNF reflect changes in the food industry, recipes and food formulations, as well as changes to the way that certain foods have been amalgamated. For example, energy in Italian dressing has decreased threefold from 2004 to 2015. At the same time, energy from baking chocolate has gone down by 20%. Preliminary analysis has shown that this change in food profiles did not contribute significantly to the observed change in overall estimated energy intake (a decrease of 20 kcal for the entire population, on average).

Demographics were taken into account when comparing TEE requirements. The predictive TEE equations are published by the IOM. These equations are not necessarily representative of the Canadian or U.S. population, but they are the best that are currently available. Other changes in sociodemographic characteristics have not been taken into account. For example, many immigrants have come to Canada in the last 11 years.^{Note 32} Overall, the estimated energy intake of recent immigrants was not different from that of the rest of the population (data not shown).

The analysis of top sources of energy intake used the BNS groups at the basic food and ingredient level. For example, “milk” includes milk as a beverage and milk as an ingredient in a cake recipe. This first analysis is sufficient to highlight where further investigation should be made to separate certain foods in their basic form or in a recipe. The analysis of top sources is also influenced by some of the changes to the food booklet. Changes in the standard reporting options in 2015 could have affected the amount of beverages consumed.^{Note 1} For example, the 2004 version included four drinking glass options. A full glass represented a quantity of 148 mL, 311 mL, 325 mL and 429 mL. The 2015 version included three pictures, representing quantities of 200 mL, 270 mL and 390 mL. For the two biggest options, this represented a decrease of 9% (429 mL in 2004 to 390 mL in 2015) and 16% (325 mL in 2004 to 270 mL in 2015). A respondent consuming the same large glass in 2004 and 2015 would select the biggest option, which would automatically assign a smaller amount in 2015.

Excluding under-reporters and over-reporters from the analysis could potentially exclude plausible intakes of under-eaters or overeaters for that specific day. Other techniques adjust for reporting status without excluding respondents.^{Note 33Note 34} However, this particular source of bias was minimal in this study for both 2004 and 2015.

This particular analysis used only the first recall to estimate average energy intake. To account for day-to-day variability, usual intake would have to be calculated. Tooze suggested accounting for misreporting by adding an indicator variable of misreporting to the usual intake modelling.^{Note 34}

Conclusion

The objective of this analysis was to estimate misreporting change from 2004 to 2015 and how these changes affect the analysis of the data. Misreporting did change and does affect analysis of the CCHS – Nutrition data, particularly when comparisons are made with 2004.

In the presence of misreporting, and in agreement with various other sources,^{Note 15Note 31Note 33Note 34} misreporting should at least be acknowledged. A comparable category of plausible reporters can be used, although with the potential of creating a bias. Adjusting for type of reporters or for estimated energy intake can be done to avoid excluding respondents.