Problem-solving Skills and Labour Market Outcomes – Results from the Latest Adult Literacy and Life Skills Survey (ALL)

View the most recent version.

Disentangling problem-solving and prose literacy skills
Problem-solving skills and prose literacy
Educational attainment and years of schooling
Gender
Occupation
Problem-solving skills and labour market outcomes
Conclusion

Recent discussions of lifelong learning point to problem solving as one of the major competencies to be fostered in a lifelong learning process. Yet, compared to other domains of learning, problem-solving has received less attention in the literature, in part because of a lack of information on this topic. A new report, based on the Adult Literacy and Life Skills Survey (ALL), provides information that helps to address that gap.¹

The Adult Literacy and Life Skills Survey assessed four foundation skills thought to be essential for social, professional and economic success – prose literacy, document literacy, numeracy and problem solving. Eleven countries, including Canada, participated in the most recent Adult Literacy and Life Skills Survey, which was conducted in two main waves between 2002 and 2008.²

This article summarizes the key findings reported in that report, focusing on problem-solving skills, their definition, distribution in the labour force and related labour market outcomes.

The development of the analytical and measurement frameworks for the skills domains assessed in ALL took account of observations of skills made in the workplace. Figure 1 illustrates one of many models that were developed to understand which skills matter economically. The triangle identifies three hierarchical levels of skill, starting at the bottom with basic skills that are thought to be required by all occupations and hence are considered to be portable between jobs and employers. According to this model, both literacy and numeracy fall into this category. The second layer of the triangle identifies a set of skills that are used in the workplace which are still portable, but which vary across broad industry and occupational groups. Problem-solving skills are positioned in this level. The third level concerns skills that are specific to a particular job or even firm and that are considered to be non-portable in the labour market.

Figure 1.
Hierarchy of skills

Description for Figure 1

Source: Ontario. Premier's Council. 1990. People and skills in the new global economy: Premier's Council Report. Ontario: Premier's Council as cited in Organisation for Economic Co-operation and Development and Statistics Canada. 2011. Literacy for Life: Further Results from the Adult Literacy and Life Skills Survey. Ottawa and Paris. Statistics Canada Catalogue no. 89-604-XWE-2011001, Figure 1.3.

Disentangling problem-solving and prose literacy skills

The construction of test items for measuring problem-solving skills poses a major challenge. Such assessments are typically carried out using written materials assembled in paper-and-pencil booklets. Hence, some facility with written language is necessary for completion of the test items, since the materials must be read and understood by respondents before they can use their problem-solving skills to try to find solutions to the questions. Consequently, performance on the test items depends on a minimum level of prose literacy.

In order to minimize the effect of prose literacy on the measurement of problem-solving skills, most of the analysis of those skills in the ALL report excludes individuals below Level 2 on the prose literacy scale.³ The authors of the report note that the literacy demands of the problem-solving items are relatively constant for those at or above prose literacy Level 2. As a result, variations in prose literacy skills for this group have only a relatively minimal effect on variations that are observed in their problem-solving scores.

A number of factors help to explain variation in the distribution of problem-solving skills. These variables include prose literacy skills, educational attainment, age, gender and occupation.

Problem-solving skills and prose literacy

Prose literacy is considered to be a cornerstone of instruction in the formal education system. Hence, there is an expected positive relationship between educational attainment and prose literacy, so that population sub-groups without advanced formal education rarely exhibit high levels of prose literacy. In contrast, problem solving is considered to be a generic, but also higher-order, skill that can be developed in formal as well as in informal learning settings. Therefore, since problem-solving skills are not closely tied to prose literacy skills (outside of the measurement context noted above), it is, in theory, possible for individuals to have strong problem-solving skills in the absence of strong literacy skills.

In Canada, as in the other countries, there is a high correlation between the two, with most individuals possessing similar levels of prose literacy and problem-solving skills.

Educational attainment and years of schooling

Most countries displayed a similar pattern in the relationship between educational attainment and problem-solving skills. The international average indicates two plateaus: the first upon completion of high school and the second upon completion of a college diploma or a bachelor's degree.

Although the increases in problem-solving skills associated with high school completion and with completion of college or a bachelor's degree appear to be relatively large, these levels of education, along with advanced research degrees, are also associated with the longest periods of study. When the different levels of educational attainment are expressed in terms of number of years of education, the results indicates a gradual leveling off of skill gain with each additional year of schooling.

Two patterns of diminishing returns are evident. The first, characterizing Canada, Switzerland (French and German), Italy and the Netherlands, suggests small incremental gains per additional year of schooling, followed by a rapid rise in skill gains in the first few years of college or university. After those initial years in college or university, however, gains in problem-solving skills level off and there are no notable increases in problem-solving skills associated with additional years of formal education.

The second pattern, shared by Norway, Bermuda, New Zealand and Hungary, is characterized by a steeper initial increase in problem-solving skills to the end of high school, followed by gradually decreasing gains with additional education; however, in this case, gains in problem-solving skills do not level off completely.

Gender

The previous international ALL report,⁴ along with other studies using data from the OECD Programme for International Student Assessment (PISA), consistently find that, on average, females have an advantage in reading proficiency, whereas males have an advantage in numeracy.

However, the findings with respect to gender differences in problem-solving skills are more ambiguous. In the recent ALL data, few significant differences were found in the average problem-solving scores of men and women. Furthermore, among the countries where there were significant male-female differences in problem-solving skills, the advantage was small and did not always favour one over the other. Evidence from PISA presents a similar story for 15 year-olds; the few differences found were as much in favour of boys as they were of girls. The authors of the PISA report suggest that problem-solving scores tend to be gender neutral since they rely as much on analytical reasoning, which is closely related to mathematical literacy, as they do on reading skills.⁵

At first glance, three countries appeared to have statistically significant differences in the problem-solving scores of women and men, namely Bermuda, Hungary and the Netherlands. However, once educational attainment and occupation were taken into account, women had a slight skill advantage over men in all countries, though the only statistically significant differences were in the Netherlands and New Zealand.

These results suggest that although men appear to have an advantage over women in problem solving in many countries, this advantage is related in part to gender differences in educational attainment and occupational distribution. For example, in some countries, women may be underrepresented among college and university graduates or women may have much less presence than men in knowledge-intensive occupations. Where women are not disadvantaged in these respects, they tend to have higher scores in problem solving than men.

Occupation

In all countries surveyed, occupations that are considered to be more 'knowledge intensive' are associated with higher levels of problem-solving skills, while length of time in the workforce after completing initial education is associated with a steady decline in average problem-solving skills. The latter findings are consistent with other studies which have found that older adults typically perform at lower levels when compared to middle-aged adults on problem-solving tasks.

Isolating the effects of workplace learning and other informal learning in everyday life is difficult because of two opposing factors. On the one hand, ongoing learning after the initial period of formal education should result in increased skill levels. On the other hand, the absence of learning opportunities and the effects of aging will contribute to skill loss over time.

In order to examine the relationship between initial education and subsequent learning and their impacts on problem-solving skills, four combinations of education and occupation were created:

1. High education, high-knowledge occupation;
2. High education, low-knowledge occupation;
3. Low education, high-knowledge occupation;
4. Low education, low-knowledge occupation.

By comparing the distribution of problem-solving skills, along with age, for these four education and knowledge groups, the importance to skill maintenance of workplace and other learning experiences relative to formal education can be studied.

Conventional knowledge in the field of literacy suggests that the high-education groups will outperform the low-education groups in problem-solving skills throughout the lifespan, due to the positive and cumulative effects of initial education. However, the results of the analysis suggest a more complex explanation for the development and maintenance of problem-solving skills.

In most countries, age-related decreases in problem-solving skills are moderated by both education and occupation. The combination of high education and high-skill occupation is associated with the greatest degree of skill maintenance. High initial education is associated with significantly higher problem-solving skills for younger people, but this does not remain true through the life course. In fact, over time, the scores of individuals with low education in high-skill occupations tend to be higher than those of individuals with high education in low-skill occupations (Chart 1).

Chart 1
Maintenance of problem-solving skills, by education, occupation and age, population scoring at prose literacy Level 2 or above, 26 to 65 year-olds, 2003 and 2008

Description for Chart 1

Note: The data analysis underlying the chart was produced using information from only four countries with sufficient sample sizes in each category: Canada, New Zealand, Norway and Switzerland (French and German).
Source: Organisation for Economic Co-operation and Development and Statistics Canada. 2011. Literacy for Life: Further Results from the Adult Literacy and Life Skills Survey. Ottawa and Paris. Statistics Canada Catalogue no. 89-604-XWE-2011001, Figure 5.9.

Overall, the effects of initial educational attainment on the long-term maintenance of problem-solving skills seem to be of similar or lesser magnitude than the effects of skill use in high-knowledge occupations – although individuals with high education tend to pursue occupations with high skill intensity, high education in itself does not guarantee skill retention over time. In other words, while initial problem-solving skill levels are mainly determined by educational attainment, the maintenance of skills is tightly related to on-going development and application of those skills, with the latter often being a function of occupation type.

Problem-solving skills and labour market outcomes

In general, individuals with high scores in problem solving are more likely to be in the labour force and are even more likely to be employed than persons with low skills. The findings also show that that the strength of the effect decreases as the overall employment rate in a country increases – that is, when employer demand for labour is very high, only those with the weakest problem-solving skills have a lower chance of being employed. Conversely, when the supply of labour is plentiful, those with the strongest problem-solving skills are the most likely to be employed.

Of related interest is the question of whether there is an earnings premium associated with problem-solving skills, once an individual has acquired a job. In other words, even though the ability to solve problems is highly valued by many employers, does that value translate into higher pay for individuals with higher problem-solving skills?

The analysis finds that, in general, where there is a reward for higher problem-solving skills, it is likely to be for individuals in occupations with greater knowledge intensity. This pattern is evident in Canada, Bermuda, New Zealand and the Netherlands. However, there is a great deal of variation across countries. In most countries, there are no significant differences in wage income associated with problem-solving skills and in some, the relationship is not as expected in that individuals with lower skills tended to earn more than those with higher skills in occupations with similar knowledge intensity.

It should be noted that wage income responds to many factors outside of skills and occupation type and that individual characteristics such as job tenure, education and age can play an important role in determining wage income. Other factors such as economic climate, labour market structure and regulation also play important roles.

Conclusion

Unlike many other skills which often depend on previously-acquired knowledge, the fluidity of problem solving makes it more sensitive to ongoing experiences and behaviours. That is reflected in part in the weak relationship observed between problem-solving skills and other demographic characteristics, such as gender. While education plays a role in developing problem-solving skills, this is not as significant as the daily use of those skills, particularly in the workplace.

Countries with higher proportions of high-scoring problem solvers, as well as high average problem-solving skills, are likely to be better equipped in a global context to deal constructively with rapid changes in work environments and to use technology in order to enhance efficiency and productivity.

The contribution of problem-solving skills to individuals' economic outcomes is likely a product of labour market dynamics, rather than the result of an intrinsic appreciation of problem-solving skills on the parts of employers. In Canada, both employment and earnings tend to be higher for Individuals with stronger problem-solving skills. This is especially the case for knowledge-intensive occupations. These results suggest that, in general, problem-solving skills are related to important individual labour market outcomes, such as employment and earnings. They also suggest that the strength of this relationship rests on a complex set of interactions between labour market structure and individual characteristics.

Notes

1. Organisation for Economic Co-operation and Development and Statistics Canada. 2011. Literacy for Life: Further Results from the Adult Literacy and Life Skills Survey. Ottawa and Paris. Statistics Canada Catalogue no. 89-604-XWE-2011001.

2. Participation in the Adult Literacy and Life Skills Survey took place in two waves. The first wave was conducted in 2003 in seven countries: Bermuda, Canada, Italy, Norway, Nuevo Leon (northern Mexico), Switzerland, and the United States. The second wave was conducted between 2006 and 2008 in Australia, Hungary, New Zealand, and the Netherlands.

3. For each domain, proficiency is denoted on a scale ranging from 0 to 500 points. Each score denotes a point at which a person has an 80 per cent chance of successfully completing tasks that are associated with a similar level of difficulty. For the problem-solving domain, experts have defined four broad levels of difficulty, where Level 1 is the lowest level of proficiency and Level 4 the highest. Since each level represents a progression of knowledge and skills, individuals within a particular level not only demonstrate the knowledge and skills associated with that level but the proficiencies associated with the lower levels as well.

4. Organisation for Economic Co-operation and Development and Statistics Canada. 2005. Learning a living: First Results of the Adult Literacy and Life Skills Survey. Ottawa and Paris. Statistics Canada Catalogue no. 89-603-XWE.

5. Organisation for Economic Co-operation and Development. 2004. Problem Solving for Tomorrow's World – First Measures of Cross-Curricular Competencies from PISA 2003. Paris: OECD.