National Apprenticeship Survey: Canada Overview Report 2015
Appendix C Survey methodology National Apprenticeship Survey: Canada Overview Report 2015
Appendix C Survey methodologys

View the most recent version.

The 2015 National Apprenticeship Survey (NAS) is a cross-sectional survey designed to collect data directly from Canadian apprentices. The apprentices were contacted by Statistics Canada between September 2015 and March 2016 and responded to a voluntary telephone survey. The survey sample consists of two distinct apprentice groups rather than the entire apprentice population. The survey results are based on a representative sample, which provides a snapshot of each group at one point in time.

Target population

For the purposes of the NAS, a selected person was considered in scope for the survey if he or she had engaged in apprentice activities between 2011 and 2013. The NAS targeted apprentices registered in the 10 provinces and three territories by apprenticeship status. These respondents do not represent all apprentices. The two groups of apprentices targeted were as follows:

Completers: individuals who had been registered apprentices and had completed their apprenticeship program between 2011 and 2013.
Discontinuers: individuals who had been registered apprentices and had discontinued their apprenticeship program between 2011 and 2013.

The target population was defined at the frame creation stage using the above definitions. Respondents were asked to confirm the apprentice activities for which they were registered between 2010 and 2013. If they confirmed that their apprenticeship status did not fall within the above target groups, they were considered out of scope. Furthermore, trade qualifiers (i.e., individuals who obtained their certificate of qualification without completing their apprenticeship programs) were also considered out of scope.

Sampling frame

The survey sampling frame was based on lists of registered apprentices provided by provinces and territories for the reference years 2011, 2012 and 2013. These lists contain all the information needed to stratify and select the sample, such as the apprentices’ status, registration year, trade, and age and sex. The lists also contain the apprentices’ contact information, such as their address and phone number. A second source of contact information was also provided for some jurisdictions.

The data provided by the 13 provinces and territories were evaluated for coverage, quality and uniformity. The data for the three reference years were grouped together in order to classify each apprentice in the appropriate group (completers or discontinuers) and to eliminate duplicates within and across provinces and territories.

Sample design

Four variables were used to stratify the survey sample: province or territory, apprentice status, main trade groups and the year the status was obtained. There are 13 provinces and territories, two apprentice statuses, 11 main trade groups and two groups for the years in which the status was obtained (for data collection purposes, the years 2011 and 2012 were grouped). These variables produced a total of 501 strata.

A national sample size of at least 30,000 respondents was needed to provide reliable estimates for each stratum. A minimum sample was allocated to each stratum and the remaining sample was allocated proportionally to the number of apprentices in each stratum. In several strata, a census of apprentices was selected. Moreover, in small provinces and territories, this resulted in selecting a census of apprentices within the same province or territory.

Within each stratum, a random sample of apprentices was selected according to a five-step process. In the first step, the sample was sorted based on the year the apprentice’s status was obtained. The second step consisted in assigning a minimum number of cases to each of the strata and determining the take-all strata. In the third step, the remaining cases were allocated proportionally to the number of apprentices in each stratum. The last two steps involved making corrections for tracing and response rates as well as an augmentation for cases with no useful contact information.

The table below shows the total number of cases, allocated by province and territory and by frame status, sent to the different regional offices of Statistics Canada to do the survey. It is from this sample that the targeted sample of 30,000 respondents was collected in order to reach a minimum precision for all domains of interest (a coefficient of variation of approximately 16.5% for an estimated proportion of 12%).

A much higher than expected out-of-scope rate was observed in some strata during the first half of collection. As a result, sample sizes were adjusted to ensure the desired number of respondents was obtained.

Response rates

Survey response rates help to measure the effectiveness of the population being sampled and the collection process, as well as being good indicators of the quality of the estimates produced. The table below shows the response rate at collection and estimation of the NAS, at the national, provincial and territorial level.

Estimation and weighting

The principle behind estimation in a probability sample such as the NAS is that each person in the sample “represents,” besides himself or herself, several other persons not in the sample. In order to have estimates produced from survey data that is representative of the target population, a weight is given to each person who responded to the survey questions. This weight, calculated at the weighting stage for each record, corresponds to the number of persons represented by the respondent for the target population. This weight appears on the micro data file and must be used to derive meaningful estimates from the survey.

For weighting purposes, the NAS can be seen as a two-phase survey: selection of the sample and the responding units correspond to the second-phase sample. The first phase weight is the inverse of the probability of selection of the apprentice. The first-phase weight is the multiplied by a second-phase adjustment factor. For the purpose of the second-phase adjustment, response homogeneity groups (RHG) are created based on the characteristics of the respondents and non-respondents. The adjustment factor simply corresponds to the inverse of the observed weighted response rate in each RHG.

Bootstrap weights were created and used for variance estimation.

Data accuracy

While considerable effort is made to ensure high standards throughout the collection and processing of date, the resulting estimates are inevitably subject to error. There are two major types of error: non-sampling and sampling.

Non-sampling errors may result from frame imperfections and non-responses. A large proportion of apprentices (11.7%) in the sample were found to be out-of-scope (no apprentice activities during the reference period due to the frame imperfection). They were out of scope because they said they had never been an apprentice or they had been an apprentice but not within the reference years. Provincial and territorial out-of-scope rates ranged from 3% to 19%. The out-of-scope rate was 3.2% for completers and 20.0% for discontinuers.

There is an important coverage difference for Quebec compared with other provinces and territories as its automobile sector is not represented on the NAS frame. This should be considered when comparing estimates for Quebec with other provinces.

Another major source of non-sampling errors is the effect of non-response on the survey results. The extent of non-response varies from partial non-response (failure to answer just one or some questions) to total non-response. Total non-response occurred because the interviewer was either unable to contact the respondent, no member of the household was able to provide the information, or the respondent refused to participate in the survey. Total non-response was handled by adjusting the weight of individuals who responded to the survey to compensate for those who did not respond.

In most cases, partial non-response occurred when the respondent did not understand or misinterpreted a question, refused to answer a question or could not recall the requested information. In partial non-response cases, donor imputation was performed for certain variables. The variables imputed were the wage- and salary-related variables of the labour force and most recent job modules.

The basis for measuring the potential size of sampling errors is the standard error of the estimates derived from survey results. Because of the wide variety of estimates that can be produced from a survey, the standard error of an estimate is usually expressed relative to the estimate to which it pertains. This resulting measure, known as the coefficient of variation (CV) of an estimate, is obtained by dividing the standard error of the estimate by the estimate itself and is expressed as a percentage of the estimate.

• Previous
• Main page
• Next