Validation of the Food Security Module in the 2018 Longitudinal and International Study of Adults

Skip to text

Text begins

Executive Summary

Background: The Longitudinal and International Study of Adults (LISA) is a socio-economic longitudinal survey conducted every two years covering subjects ranging from labour market activity to retirement. Food security was added in Wave 4 of the LISA and comprise four derived variables: Adult Scale, Child Scale, Household Scale, and Modified Household Scale (including marginally food insecure).

Objective: This technical report is intended to validate the LISA Food Security (FSC) module and provide recommendations for analytical use.

Key Findings: The food security derived variables may be used to study person-level or household-level populations. There are three distinct food security measures: adult food security, household food security and child food security. The recommended survey and bootstrap weights to use for the adult food security measure are the all waves responding person weights (AWRPW) and bootstrap weights; wave-t enumerated person weights (EPW) and bootstrap weights are for the household food security measure and the child food security. The adult food security measure may be used to study the adult LISA population (individuals 18 years of age and older or individuals 16 or 17 years of age who live alone). This measure also has an additional requirement – the data user must assign all adult household members the food security status of the person most knowledgeable (PMK). The PMK is the sole-respondent of the FSC module and so for adult-level analysis, one must assume that all adult members experience the same food security status as the PMK. Without this step, the sample would only consist of the PMK respondent, not representative of the adult population and so will lead to incorrect statistical analyses. Since the PMK also provides answers on behalf of the household, these responses are representative of the household food security status and therefore may be used for household-level food security analysis.

The household food security measure may be used to study the LISA population of households and the child food security measure may be used to study LISA population of households with children. Households with children include households of adults and children ages 0 to 15, and households with individuals 16 or 17 years of age who are living with an adult relative. The household food security, modified household food security and child food security measures should use the enumerated person weights (EPW) and bootstrap weights in order to produce reliable statistical analyses. The first step to the household-level analyses is to generate household-level weights. This is done by taking the average of the person-level enumerated weights for all persons living in the household and then assigning these average weights to the respective households to represent the household-level weights. This same step will need to be applied to the bootstrap weights.

In spite of the fact that LISA is a longitudinal survey with a reference population at 2012, the reported incidences of LISA household food insecurity in 2018 are quite comparable to those of the Canadian Community Health Survey (CCHS).

Recommendations: Before performing any statistical analyses with the LISA food security data, the data user must follow the methodological guidelines provided. First, the user must identify the population of interest and the food security measure that may be required to study the identified population. Second, the user must select the appropriate LISA survey weights and bootstrap weights for the analyses of interest as indicated in the Key Findings above. For longitudinal studies, the recommended weights are the all waves responding person weights (AWRPW) and for contemporary cross-sectional analyses, the recommended weights are the enumerated person weights (EPW). Once these checks are completed, one can conduct valid estimates with the LISA FSC module.

1. Introduction

This technical report is intended to validate the Longitudinal and International Study of Adults (LISA) Wave 4 (2018) Food Security (FSC) module and provide recommendations for analytical use. Section 2 of this report provides an overview of the LISA data. Section 3 provides some background information of food security measures in national surveys and why it is significant in today’s literature. Section 4 analyzes FSC data by presenting key descriptive statistics and logic checks using LISA methodology as well as outside researcher information. In section 5, certification validation was done by comparing other Canadian national surveys that have used a FSC module to the one used by LISA. Finally in section 6, key findings and their implications with regard to LISA are outlined.

2. Data source

The Longitudinal and International Study of Adults (LISA) began in 2011/2012 and was designed to gather information every two years on core topics^Note , feature topics^Note , and rotating content^Note . Sponsored by Employment and Social Development Canada (ESDC), LISA interviewed 32,133 Canadian men, women and children in its initial wave (2012). The original sample^Note  was selected from a population frame based on the 2011 Canadian Census. This sample will grow over time to include descendants of the original sample members. For longitudinal studies using the LISA data, this “base year” will represent the Canadian population in 2012. For cross-sectional studies at any subsequent wave using the LISA data, the population at that wave will represent the “base year” population plus the population of the descendants. For instance, the LISA Wave 4 (2018) cross-sectional population is not representative of the 2018 Canadian population, but the 2012 Canadian population plus their descendants since 2012. This means that the LISA cross-sectional population in 2018^Note  excludes a population such as new immigrants to Canada since 2012.

For each wave, LISA produces four products accompanying the survey data; (i) four survey weights; (ii) four sets of 1000 bootstrap weights; (iii) four administrative datasets; and (iv) a set of intergenerational files^Note . The survey weights and bootstrap weights are essential to produce reliable estimates for two distinct types of studies namely longitudinal studies and LISA-defined cross-sectional-type studies. The respective weights for these studies are (i) the all waves weights; and (ii) the wave-t weights. The two LISA all waves weights are: all waves responding person weights and the all waves ISA (PIAAC) responding person weights. The two LISA wave-t weights are the enumerated person weights and the responding person weights (Benhin, 2018). Administrative data is amalgamated to further enhance the analytical depth of LISA and includes the T1 Family File (T1FF), Statement of Remuneration Paid (T4 tax slip), Pension plans in Canada (PPIC), and Longitudinal Immigration Database (IMDB). Four waves of LISA data have been released, most recently Wave 4 (2018). Using these LISA datasets, and an appropriate research design, data users can identify causality, analyze gross change, and measure time-related characteristics of socioeconomic circumstances; these are some of the major benefits unique to a longitudinal survey. Since its inception as a survey program, LISA has served as an important tool for researchers, policy makers, and government departments.

3. Food Security Overview

3.1. Background of Food Security in Canada

The first use of a food security questionnaire module in a Canadian national survey was in the 1998/1999 National Population Health Survey (NPHS)^Note  sponsored by Human Resources Development Canada. However, the earlier use of food security-related questions can be traced back to the National Longitudinal Survey of Children and Youth^Note  in 1994/1995 where questions were asked about experiences and coping mechanisms of individuals on how to deal with hunger as a result of financial constraints. In 2001, Health Canada commissioned a discussion paper on Household and Individual Food Insecurity^Note , which led to the implementation of the Household Food Security Survey Module (HFSSM)^Note  in the Canadian Community Health Survey (CCHS) Cycle 2.2, Nutrition (2004). HFSSM serves as a general baseline structure^Note  of food security modules that are adapted by other Canadian national surveys, including the Longitudinal and International Study of Adults (LISA).

At the time of this report, the food security module or versions of it have been implemented in a number of Statistics Canada surveys, including: (1) Canadian Community Health Survey - Annual (2000^Note , 2007, 2008, 2011, 2012, 2017, and 2018), (2) Canadian Community Health Survey - Nutrition (2004 and 2015), (3) Canadian Health Survey on Children and Youth (2019), (4) Canadian Health Measures Survey (2007-2009, 2009-2011, and 2014-2015), (5) Aboriginal Peoples Survey (2012), (6) Survey of Household Spending (2010 cycle) and (7) The Longitudinal and International Study of Adults 2018 (Wave 4).

The HFSSM, implemented in the CCHS Cycle 2.2 in 2014, consisted of an 18-question module (Health Canada, 2007). These questions were about the household food security situation over the previous 12 months. This 18-question module is a variation of the food security measurement developed in the United States for their annual household food security in 1995. The goal of the HFSSM was to identify households with deficient or uncertain access to food due to constraints on financial resources which may undermine eating habits. The CCHS version of the HFSSM was adopted by LISA and is simply referred to as Food Security (see Appendix A for LISA FSC module).

3.2. The need for Food Security as part of LISA

Food security is classified as one of the 12 Dimensions of Poverty, one of many actions in Canada’s Poverty Reduction Strategy to monitor progress on poverty reduction for Canadians and their households (Statistics Canada, 2019; Employment and Social Development Canada, 2018). The Poverty Reduction Strategy is one response to the 2030 Agenda for Sustainable Development, which was ratified by Canada and all other United Nations Member States in 2015 (Employment and Social Development Canada, 2018). The Sustainable Development framework has 17 goals, one of which is to end hunger, achieve food security and promote nutrition and sustainable agriculture (United Nations Statistics Division, 2019).

As a result of consultations between Statistics Canada and Employment and Social Development Canada (ESDC) as well as data users, it became apparent that a module on food security would complement the other subjects covered by LISA and provide a unique insight for monitoring poverty in Canada. A decision was then made to include, for the first time, the FSC module in 2018 (Wave 4) of LISA. Along with the current roster of subjects on low income, family history, health, labour market activities, and education, researchers can use LISA data to investigate associations among these subjects and their impact on food security. The longitudinal character of LISA will make it valuable for longitudinal statistical analyses of these subjects and how they influence food security and poverty in Canada. The unique ability for longitudinal data to determine causality will provide a deeper understanding on the Canadian population that enters, remains, and exits low income.

3.3. Food Security Measures

There are four derived LISA food security measures: 1) a three-category Adult food security, 2) a three-category household food security, 3) a four-category household food security and 4) a three-category child food security. The three-category food security measures indicates households (or the adult or child members of the household) as being food secure, food insecure (moderate) and food insecure (severe). The four-category household food security measure, originally added to the CCHS at the specific request of researchers/organizations, includes being food secure, food insecure (marginal), food insecure (moderate) and food insecure (severe). Marginal food insecurity occurs when households are concerned they do not have a sufficient amount of food and/or an inadequate food variety due to financial constraints (PROOF Food Insecurity Policy Research, 2018a). Moderate food insecurity is when households suffer from compromised quality and/or quantity of food as a result of deficiency in finances (PROOF Food Insecurity Policy Research, 2018a). Severe food insecurity, the harshest of the three categories, is when households experience patterns of disruptive eating habits, such as missed meals, and reduced food intake (PROOF Food Insecurity Policy Research, 2018a). At the most extreme levels of severe food insecurity, households can go day(s) without food (PROOF Food Insecurity Policy Research, 2018a). Food secure households are households with no or only one “affirmative” response to the food security questions for the three-category food security measure or when there is no “affirmative” response for the four-category food security measure (see section 3.4 for more details).

3.4. Deriving the Food Security Measures

According to Health Canada (2007), of the 18 food security questions, 10 are specific to the food security experiences of the adults in the household and eight are specific to the food security experiences of children under 18 in the household, if any are present (see Table 1A of Appendix A and Table 1B of Appendix B for more details). The household-level food security measures were derived using all 18 questions. The four derived LISA food security measures were adopted from HFSSM. They are: adult food security measure (FSC_AFS), child food security measure (FSC_CFS), a three-category household food security measure (FSC_HHFS), and a four-category household food security (FSC_HHF2); see Table 1B of Appendix B. LISA FSC questions were asked of a person 18 years of age and older and who is the most knowledgeable for the household. To determine if the child food security experience questions should be asked, household members aged 16 or 17 years old living with an adult relative were considered children. Household members aged 16 or 17 years old were considered adults if they were not living with an adult relative.

The creation of these food security measures are similar in principle (see Table 2B, Appendix B). First, there was the need to recode responses to each question as either "affirmative" (coded as 1) or "negative" (coded as 0). Affirmative responses correspond to questions with answers “yes” and negative responses correspond to question with answers “no”. Questions with responses such as "Often true", "Sometimes true", "Almost every month", and "Some months but not every month" were recoded as "affirmative" and responses such as "Never true" and "Only 1 or 2 months" were recoded as "negative".

Second, the recoded responses (1 or 0) were then combined depending on the number of questions involved to create the specific food security measure (see Table 2B, Appendix B).

• The adult food security measure was created by combining the recoded responses of the 10 adult experience FSC questions as follows: food secure (0 or 1 affirmative responses), moderately food insecure (2 to 5 affirmative responses), or severely food insecure (greater than or equal to 6 affirmative responses).
• The child food security measure was created by combining the eight child experience FSC questions as follows: food secure (0 or 1 affirmative response), moderately food insecure (2 to 4 affirmative responses) and severely food insecure (5 or more affirmative responses).
• The household-level food security measures were derived from the adult and child food security measures: If both adult and child security measure outcomes are food secure, then the household food security is food secure. If either adults or children, or both adults and children in the household are moderately food insecure, and neither are severely food insecure then the household food security is moderately food insecure. If either the child or adult measures are coded as severely food insecure, then the household is considered severely food insecure.
• The four-category household food security measure is slightly different from the three-category household food security measure; the household is food secure when there is zero affirmative response, moderately and severely food secure categories remain unchanged. The marginally food insecure category is applied when the combined child and adult food security measure results in exactly one affirmative response.

4. Descriptive Statistics

4.1. Food Security Statistical Analysis

Longitudinal and International Study of Adults (LISA) food security measures may be used to analyze three distinct populations and their corresponding sub-population domains (Table 4.0). These populations include LISA population of households^Note , LISA population of adults (individuals 18 years of age and older and individuals 16 or 17 years of age not living with an adult relative) and LISA population of households with children (individuals 0 to 15 years of age, and individuals 16 or 17 years of age who are living with an adult relative). The food security measures corresponding to these populations are household food security measure, adult food security measure and child food security measure, respectively. One must ensure that when producing estimates using each of these food security measures, the appropriate populations of interest are correctly specified.

LISA produces two distinct sampling weights to be used for person-level statistical analyses: all waves weights and wave-t weights. The all waves weights are for longitudinal statistical analyses and the wave-t weights are for LISA contemporary cross-sectional statistical analyses. Household-level analyses are only possible for cross-sectional type analyses^Note  and person-level analyses can be done as longitudinal and cross-sectional type analyses. For the household-level analyses, the recommended set of weights to use are the wave-t enumerated person weights and for the person-level analyses, the recommended weights to use are the all waves responding person weights (for longitudinal statistical analyses) and wave-t responding person weights (for cross-sectional type statistical analyses)^Note . Since LISA does not produce household weights, approximate household weights may be derived: average of the enumerated person weights of all persons in a household are used as approximate LISA wave-t household weights.

Table 4.1 presents preliminary unweighted frequency counts for the four food security measures. These measures were derived based solely on responses from a selected member of the household - the person most knowledgeable about the household (PMK). For LISA, in general, all members of a household who are 15 years of age and older are eligible for the LISA questionnaire. However, for the food security module, only one person, a PMK (18 years of age and older or an individual 16 or 17 years of age who does not live with an adult relative) from a household, was eligible to provide answers to this module. The “Valid skip” in Table 4.1 represents responding adult members in the household who were not selected to provide responses to the food security questions. It should be noted that, valid statistical analyses and inferences cannot be made without appropriately recoding these valid skips to have the same responses as the PMK for the adult food security.

In order to produce valid statistical analyses and inferences (using LISA sampling and bootstrap weights) for any of these four LISA food security measures, analysts/data users would have to first do the following: For household-level analysis, in order to use the two household food security measures and the child food security measure, the PMK response on behalf of the household will represent the household response. For adult-level analysis, in order to use the adult food security measure one must assign responses from the person most knowledgeable to all adult members of the household. This process will appropriately eliminate the “Valid skip” category before starting any statistical analyses.

Adult Food Security Measure

Table 4.2 shows the unweighted frequency counts using the adult food security measure (FSC_AFS) for cross-sectional type and longitudinal statistical analyses. As mentioned earlier, for valid statistical analyses, the responses from the person most knowledgeable in the household are first assigned to all adult members of the household.

Table 4.3 presents sample counts, weighted frequency counts (of totals), weighted proportions and their associated coefficients of variation (CVs) for adult food security measure in the context of longitudinal analyses. Overall, the estimates for the various categories of the food security measure are of good quality for publication based on the CVs. The table also shows that for the adult LISA population in 2018, 93.9% were food secure, 3.4% were moderately food insecure, 1.9% were severely food insecure, and 0.9% were not stated.

The “Not stated” category represents missing response. These non-responses could bias the overall estimates depending on the type and size. These type of non-responses are usually treated by a method referred to as imputation. The treatment of this type of non-response is beyond the scope of this report.

Household Food Security Measure

The LISA survey was designed primarily for longitudinal analyses at the person level. Therefore, all LISA sampling and bootstrap weights are generated at the person level. The food security module is primarily a household driven module. However, analyses can be done at both the person and household levels.  Unfortunately, there are no LISA longitudinal or cross-sectional household sampling and bootstrap weights to allow for direct analyses of LISA data at the household level. For contemporary cross-sectional statistical household-level analyses, a good approximation is to use LISA wave-t enumerated person sampling and bootstrap weights as follows: assign the average of all the persons-level weights to the household to represent the household sampling and bootstrap weights. Note that the LISA population in 2018 is the Canadian population in 2018 not accounting for new immigrants to Canada since 2012. Using these approximate household weights, household food security sample counts, weighted frequency, weighted proportions and the corresponding CVs were produced.

Table 4.4 shows the sample counts, weighted frequency counts (of totals) and percentages (proportions) for the three-category food security measure for the LISA population in 2018. For the household food security measure, the estimates for all the categories demonstrate good quality based on the smaller CVs. The table shows that for the Canadian (LISA) population in 2018, 91.3% were food secure, 4.2% were moderately food insecure, 2.0% were severely food insecure, and 2.5% were unknown (not stated).

Table 4.5 is similar to Table 4.4, except that the food secure category in Table 4.4 was divided into two categories, food secure and marginally food insecure. In Table 4.5, 87.7% of the Canadian (LISA) population in 2018 were food secure and 3.6% were marginally food insecure.

Table 4.6 shows the sample counts, weighted frequency counts (of totals) and percentages (proportions) for the three-category household food security measure for the LISA population of households in 2018. The estimates are based on household sampling and bootstrap weights derived from LISA wave-t enumerated person weights (as explained in section 4.1 of this report). The table shows that for the Canadian (LISA) households in 2018, 91.1% were food secure, 4.5% were moderately food insecure, 2.6% were severely food insecure and 1.8% were unknown (not stated).

Table 4.7 is the same as Table 4.6 except the food secure category is split into two categories to include the marginally food insecure category. In Table 4.7, 87.5% of the Canadian (LISA) households in 2018 were food secure and 3.6% were marginally food insecure.

Child Food Security Measure

Table 4.8 shows the sample counts, weighted frequency counts (of totals) and percentages (proportions) for the child food security measure of LISA households in 2018 with young and older children. The household-level estimates are based on household sampling and bootstrap weights derived from LISA wave-t enumerated person weights (as explained in section 4.1 of this report).  The CVs show good quality estimates at the Canada level except for the severely food insecure category which is not publishable. The table shows that of the Canadian (LISA) households with children in 2018, 93.1% were food secure, 4.0% were moderately food insecure and 2.7% were unknown. Since the severely food insecure category has insufficient responding households, it may be advisable to combine the moderately and severely food insecure categories into one category to allow for publishable estimates.

For all the statistics reported in the tables in section 4 and related ones, it should be noted that, for smaller population domains such as age groups by sex, province by sex and so on, the number of responding persons/households will decrease and this may affect the quality of the estimates that may be produced.   In general, the smaller the sample size (i.e., the number of responding persons/households), the lower the quality of the estimates that may be produced.

5. LISA Food Security Module versus Other Modules

The LISA Food Security (FSC) module was adopted from the Canadian Community Health Survey (CCHS) version of the Household Food Security Survey Module (HFSSM)^Note . So in essence, it is practically no different from the modules used by CCHS, and other surveys that adopt this module. Hence, this may allow for some comparisons of the data among these surveys. CCHS is a cross-sectional survey, and is therefore not adequate for longitudinal statistical analyses. LISA on the other hand was designed for longitudinal statistical analyses. An added benefit to LISA is the potential to use for some LISA-defined cross-sectional-type statistical analyses. This means that food security measures can benefit from both longitudinal and cross-sectional analyses, as shown in section 4 of this report.

Table 5.1 compares estimates of household food insecurity by province between LISA household population in 2018 and four cycles of CCHS in 2007-2008, 2011-2012, 2015-2016, and 2017-2018. It should be noted that LISA was designed primarily for longitudinal studies. However, estimates for LISA in the table are cross-sectional. Moreover, LISA households in 2018 does not account for new immigrant households to Canada since 2012 whereas the CCHS households in the respective cycles represent all the Canadian households for those specific cycle years. In spite of these conceptual differences in population, the table shows that LISA estimates are reasonably comparable to those of CCHS.

6. Conclusion

This report provided a general description of the Longitudinal and International Study of Adults (LISA) survey and its overall content. Section 3 summarized the importance of adding the Food Security (FSC) module in 2018 (Wave 4) as part of the rich array of subjects covered by LISA. The FSC module may benefit from the longitudinal character of the survey through primarily longitudinal statistical studies as well as some cross-sectional studies. The descriptive statistics illustrated in section 4 also show that overall, LISA data on food security produced good quality and publishable estimates, especially for population domains with reasonably large sample sizes (i.e., sufficient number of responding persons/households). Moreover, the section also explained how to correctly use responses collected from only a single person (person most knowledgeable, PMK) from a household for statistical analyses. Note that in general, LISA data are collected from all persons 15 years of age and older but the FSC module was collected from only a PMK. Furthermore, the section also discussed how to correctly use LISA longitudinal (referred to as “all waves”) weights and cross-sectional (referred to as “wave-t”) weights to produce valid statistical inferences. The cross-sectional analyses may also allow for some comparisons with other surveys, as illustrated in section 5.

In all of the illustrated tables on food security measures provided in section 4, there is a category referred to as “Not stated” that is, missing data. This type of missing data is generally known as item non-response, i.e., missing data for the particular food security measure. Item non-response could bias the overall estimates and so are generally treated by a method referred to as imputation so as to reduce bias. Imputation was implemented for only a limited number of financial variables for LISA. Therefore, the treatment of item non-response for most of the LISA variables was left for the data users to decide on the best approach to handle the missing data to meet their particular analytical studies’ needs. It must be mentioned that the smaller the number/percent of missing data as is the case here, the less likely for it to have a significant impact on the statistical inferences.

7. References

Benhin, E. (2018). Data Quality Report. Longitudinal and International Study of Adults, Wave 3. Internal Document.

Bickel G, Nord M, Price C et al. (2000). Guide to Measuring Household Food Security, Revised 2000. Alexandria, VA: Food and Nutrition Service, United States Department of Agriculture. Available at: https://fns-prod.azureedge.net/sites/default/files/FSGuide.pdf.

Employment and Social Development Canada. (2018). Opportunity for All – Canada's First Poverty Reduction Strategy. Retrieved from https://www.canada.ca/en/employment-social-development/programs/poverty-reduction/reports/strategy.html

Health Canada. (2007). Canadian Community Health Survey, Cycle 2.2, Nutrition (2004): Income-Related Household Food Security in Canada. Retrieved from https://www.canada.ca/content/dam/hc-sc/migration/hc-sc/fn-an/alt_formats/hpfb-dgpsa/pdf/surveill/income_food_sec-sec_alim-eng.pdf

Health Canada. (2012). Household Food Insecurity in Canada: Overview. Accessed on October 4, 2019

PROOF Food Insecurity Policy Research. (2018a). Household Food Insecurity in Canada. Retrieved from https://proof.utoronto.ca/food-insecurity/.

Statistics Canada.  Table 13-10-0462-01. Household food insecurity, by presence of children in the household and food insecurity status.

Statistics Canada. (2018a). Canadian Community Health Survey - Annual Component (CCHS). Retrieved from http://www23.statcan.gc.ca/imdb/p2SV.pl?Function=getSurvey&SDDS=3226.

Statistics Canada. (2018b). Longitudinal and International Study of Adults (LISA). Retrieved from http://www23.statcan.gc.ca/imdb/p2SV.pl?Function=getSurvey&SDDS=5144.

Statistics Canada. (2018c). Longitudinal and International Study of Adults (LISA) - excluding question flow - 2018 (Wave 4) (LISA). Retrieved from http://www23.statcan.gc.ca/imdb/p3Instr.pl?Function=getInstrumentList&Item_Id=424498&UL=AV.

Statistics Canada. (2019). Dimensions of Poverty Hub. Retrieved from https://www.statcan.gc.ca/eng/topics-start/poverty

Tarasuk, V. (2001). Discussion Paper on Household and Individual Food Insecurity. A report prepared for the Office of Nutrition Policy and Promotion, Health Canada. Available at: www.hc-sc.gc.ca/fn-an/nutrition/pol/food_sec_entire-sec_aliments_entier_e.html.

United Nations Statistics Division. (2019). Global indicator framework for the Sustainable Development Goals and targets of the 2030 Agenda for Sustainable Development. Retrieved from https://unstats.un.org/sdgs/indicators/Global%20Indicator%20Framework%20after%202019%20refinement_Eng.pdf

Appendix A: LISA Food Security Module

Appendix B: Food Security Measures