Factors related to literacy skills and skill loss

View the most recent version.

We now expand the regression analysis in an attempt to achieve a more accurate estimate of skill loss, and possibly explain some of the observed loss. The argument is that the literacy skills of a cohort are related to people's demographic characteristics and their experiences at home and at work. We therefore introduce these factors into our model for document literacy skills such that our two cohorts are statistically comparable. This work builds on previously published analysis, which has revealed that skill loss is concentrated in adults from lower socio-economic backgrounds, a fact that raises concerns about the quality of tertiary education for these individuals and about their ability to compete in a labour market that rewards literacy skill to a high degree (Statistics Canada and Organisation for Economic Cooperation and Development, 2005).

Our model includes the following controls: gender, age, age-squared, level of education (coded in four categories, with high school completion as the base category), people's engagement in general literacy activities at work, people's engagement in technical literacy activities at work, and people's engagement in literacy activities at home. The regression analyses were conducted separately for each cohort. We found that skill loss (the cohort effect) varied by the level of respondent's education, and therefore a separate model tested whether the relevant interaction terms were statistically significant.

The most immediate finding is that the estimate of average skill loss from 1994 to 2003 was 12.7 points, which is approximately the same as for the base model, which was 11.6 points. This means that controlling for these various factors, which improves our synthetic cohort analysis by ensuring the two cohorts are statistically comparable, yields an estimate of the skill loss that is approximately the same.

A by-product of this analysis is that it provides estimates of the effects associated with level of education, and engagement in literacy activities. The results indicate that compared with high school graduates, adults who did not complete secondary school scored about 48 points lower on the document literacy scale, while those who had completed some post-secondary scored 15 points higher than high school graduates, and those who had completed university scored about 30 points higher. An important finding, though, is that skill loss interacted with level of education. For dropouts, there was no skill loss (i.e., -12.7 + 14.6), while for high school grads, and those who completed some post-secondary and university graduates, the skill loss was the same as in the general population (i.e., the interaction terms were small and not statistically significant).

The results also indicate that adults who had participated in further education and training in the previous twelve months scored about 16 points higher than those who had not participated. Adults who were regularly engaged in literacy activities at work or at home had higher literacy scores¹. The effects associated with general literacy engagement at work and at home were 11.1 and 14.3 points respectively. The effect associated with engagement in technical activities at work was positive, but not statistically significant.

We now ask whether the factors found to be related to literacy skills in the above analysis can account for the variation among provinces in their literacy skills. To address the question we fit an hierarchical linear model to the data, with adults nested within provinces². As in Table 2 we can also get an estimate of the skill gain after controlling for the various factors that are in the model. The results are presented in Table 3.

The first model simply estimates the variation in skill levels among provinces in their 1994 literacy scores, and in their skill gain or loss from 1994 to 2003, with statistical adjustment for sex and age. We refer to this as the base model³. The results indicate, as one would expect from Table 1, that provinces vary significantly in their literacy skills. The variance is 173.7, and the standard deviation is 13.2. The amount of skill loss from 1994 to 2003 also varies among provinces: the variance is 38.4, and the standard deviation is 6.2. Considering our earlier results, which suggested that there is a significant skill loss for adults at age 40, these results confirm that the amount of skill loss varies among provinces and regions of the country.

The second model adds the variables describing level of education to the base model. The variance in 1994 provincial means is reduced from 173.7 to 103.5, or by about 40%, when respondents' level of education is included in the model. Recall that this model yields a better estimate of the skill loss or gain than the base model, as the 1994 and 2003 populations have been equated for level of education. The variance among provinces is slightly greater with this model, with a variance in skill loss of 43.4.

The third model extends the base model with the work and home literacy engagement variables, as well as the indicator denoting whether a person was not in the labour force. This model reduces the inter-provincial variance from 173.6 to 82.6, or by about 52%. The variance in skill loss is 43.3. This is how much variance we observe when the provinces are equated for levels of literacy engagement.

The final model includes both the level of education and the literacy engagement variables. The inter-provincial variance in means is reduced slightly, to 76.3, while the variance in skill loss estimates increases to 49.9.

The increase in the variance in skill gains or losses deserves comment, as one might expect the models to help explain why some provinces had bigger or smaller skill gains or losses than others. The base model, which includes age and sex, essentially equates the two synthetic cohorts within provinces. When level of education is added to the model, the "equating" of the synthetic cohorts within provinces is improved, such that we get better estimates of the skill gain or loss in each province. However, this should not have any material effect on the variation in the estimates among provinces. The same argument applies when we add the engagement variables.

We attempted to examine whether changes in average levels of engagement, from 1994 to 2003, were related to the observed levels of skill gain or loss. This involves entering mean levels of engagement, and measures of changes in engagement, into the second level of the model. However, the results were inconclusive because with only 10 provinces there was not enough statistical power to discern an effect. In the next section, we examine the extent to which provinces' scores varied on the key factors found by ALL to relate to literacy scores.

In summary, the statistical models employed in these analyses are able to explain a little more than half of inter-provincial differences in literacy skill, leaving almost half of these differences unexplained. However, the models fail to explain inter-provincial differences in skill loss identified in the previous analysis.

---

Notes

1. The scores on the scales for literacy engagement range from 0 to 2, with 0 indicating little or no engagement to 2 indicating daily engagement in literacy activities. To simplify the interpretation of the results in this section and the next section, dummy variables were created using a score of greater than 1.5 as the cut-off. Using the dichotomous indicators versus the continuous measures did not substantially change the results.

2. For the analyses reported in Table 1, results for the Prairie and Atlantic provinces were aggregated into regions. For these analyses, the second level of analysis is provinces, not regions, as we are interested in assessing the extent of variation among provinces. The hierarchical linear modeling technique weights results according to how accurately they are estimated.

3. Because the age and sex distributions do not differ substantially among provinces, these variables explain only a small percentage of the null model variances in average literacy scores and in skill loss. (Variance in means for the null model is 150.8, and the variance in skill loss is 46.1).