The analytical framework

View the most recent version.

In this section we describe the analytical framework used in the study, explain why this approach is used, and then set out the key definitions relating to persistence employed in the analysis: graduating, continuing at the initial institution, switching to a new program at a different institution, and leaving PSE. We also discuss the dynamic of returning to school among those who are observed to leave as well as those who continue with their studies after graduating from a PSE program.

3.1   The hazard approach

This paper uses what is variously called a hazard, survival, or duration analysis set-up (the terms are used interchangeably here). This approach is appropriate for studying persistence in PSE because it is meant to capture what are essentially time-related processes where there is a certain final outcome of interest (in this case graduation from a given PSE program), with the focus being on the time taken to reach that outcome (i.e., years to graduation), or alternatively, whether at each point in time a person achieves that outcome or otherwise remains on the path towards it (i.e., continues in their studies) or instead makes one of the transitions that takes them off that path (switching programs/institutions, leaving PSE).¹ 

Analyses of this sort can be carried out in a modelling (regression-type) framework, or (as here) by using simpler hazard rate calculations.² Either way, the approach consists of calculating the relevant transition (or hazard) rates at each point in time following the individual student's entry into the state of being "at risk" within the PSE institution for making one those transitions.

In the present case, the analysis takes the form of estimating the probability that a student continues, graduates, switches, or leaves PSE on a year by year basis from their point of entry into a new program. More specifically, we begin to track students at the point they are observed to start a PSE program at a given institution, and measure at the end of each year whether at that time they are continuing their studies at the initial institution, have graduated, have switched to another program at a different institution, or have left PSE entirely.

Once one of the relevant transitions has been made the student is no longer generally followed since the process in question (i.e., what happens after a student starts a given PSE program) has been identified, and one of the relevant transitions has been made.³ We do, however, subsequently follow those particular students who leave PSE to see how many return, and also look at how many of those who graduate then pursue further studies, as described below.

The reason we look at persistence at the institutional level rather than at the program level (Finnie and Qiu, 2008, adopt the latter perspective) is principally due to the difficulty of identifying program changes at a given institution within the PSIS data, as discussed further below. The terms "program change" and "change of institution," along with other related terms, are used interchangeably in our discussions, and in all cases they refer to situations where a student moves from one institution to another, and thus starts a new program.

We use a similar methodological approach to look at two other processes, or dynamics. The first of these is the rate of returning to school among those who are observed to leave PSE after entering a program (i.e., the first process described above). How many return to PSE after being out one year, after two years, and so on. In this case, the state of "risk" – using the hazard analysis terminology – begins when the student leaves PSE (before graduating) and may therefore be in a situation to subsequently decide to return to school. Unfortunately, the number of years we can follow them through this dynamic within the scope of this particular study is quite limited, as discussed below.

Finally, we also use the same general approach to look at how many of those observed to graduate from a program continue their studies, either immediately or after a short break.

Rather than trying to track the vast number of different pathways that a person might take as they move through PSE for which any associated "decision tree" would be equally complex, this hazard model approach allows us to reduce the overall analysis to a few key well defined processes which lay at the heart of the general set of PSE persistence dynamics. We believe that this methodological approach, based on the transition-hazard analytical framework, is the only one well suited to exploiting the PSIS data to tell us what we want to know about PSE pathways.

3.2   The hazard approach and censoring

As indicated above, one of the principle reasons to adopt the hazard/duration/survival approach of the various processes to be investigated – continuing in a first program, returning to school after leaving PSE, starting a new program after graduating from a prior one – is that it is well suited to analysing the underlying dynamics which, taken together, capture the fundamental elements of persistence in PSE. In short, the inherently dynamic properties of the survival approach suit the PSE persistence processes in which we are interested in this study, and the PSIS data.

The second (and related) reason for adopting the survival approach is that the PSIS data result in many students' records being "censored". "Censoring" is the general term that is used to indicate a situation where we are able to follow the given process, or relevant "spells" (e.g., a student's trajectory in a given PSE program) for at least some individuals for only a certain length of time (one year, two years, three years). The spells are in this sense incomplete – or "censored" – i.e., we run out of data before any of the possible transitions are made.

For example, some students can be observed only for one year after entering PSE (those who enter in 2004/2005), whereas we are interested in the process beyond that point. Censoring is a general, inherent issue in duration studies of this type, precisely because they rely on longitudinal data which often follow individuals for uneven and generally limited lengths of time (as here).

The specific reason for censoring here, given the nature of the PSIS data, is that many of the "spells" that underlie the analysis representing students' pathways after entering PSE are still in progress in the final year of the PSIS data (2004/2005). For example, in the first dynamic of what happens after students enter a new program, many individuals are still in that first program – having not graduated, not switched, not left PSE. And this may be after having been followed just one year, two years, or three years, depending on the calendar year (2001/2002 through 2004/2005) in which they started their program (as explained further below).

The standard solution (in hazard analyses in general as well as in this specific case) is to include spells in the analysis up to the point they are censored in this way. By doing so, all the information available in the data is used in the most efficient manner possible, and the sample used in the analysis is as general as possible, rather than being restricted to only those individuals who are observed over the longest period of time available (at the end of which many records would still be censored in any event).

It should be noted that another typical reason for censoring in longitudinal analyses is sample attrition. In survey data, this usually occurs when individuals who are initially included in the analysis (e.g., they are observed to start a PSE program) cannot be located, refuse to be interviewed, or otherwise do not have useable records in subsequent interviews in later years. Given the administrative nature of the PSIS data used here, however, this should not be a problem for this analysis, since the data should at least theoretically cover all persons in all years covered by the PSIS data. We return to this issue below.

3.3   Spell time and the dynamics in question

Consistent with the general hazard approach, the time frame of the analysis is spell time, not calendar time. So, although individuals enter PSE in different calendar years corresponding to the 2001/2002 through 2004/2005 reporting years currently covered in the PSIS, we define the beginning year for anybody starting a spell (i.e., when they are observed to enter PSE) as , regardless of the calendar (reporting) year in which that spell started.

We then observe individuals after one year (t₁), after two years (t₂), and after three years (t₃), depending on when the spell started. Those in the 2001/2002 cohort (defined as all individuals who start their program in that year) are followed over the longest interval, through to 2004/2005 or three years in total. The 2002/2003 cohort can be followed for a period of two years, still out to 2004/2005 of course – but in this case only two years after entry. And the 2003/2004 cohort can be followed for just one year. The analysis is organized around these event-based intervals, and the data organised accordingly (i.e., by "event year" rather than calendar year).

Figure 1 below presents the framework graphically. Individuals start their PSE programs at time . After one year, at time t₁, they are classified according to the four possible outcomes: "continuers", "graduates", "switchers", and "leavers". For "continuers", a solid arrow depicts their progression to the next time period t₂, since they did not, by definition, make any of the relevant transitions in the first year. For "graduates", "switchers", and "leavers", a dashed arrow indicates that these individuals are excluded from further analysis of the persistence dynamic because they have in fact made one of the relevant transitions during the year in question. Those who cannot be followed any further in the data because they are censored, as described above, are simply deleted from the analysis as of the relevant spell year.

Figure 1 Conceptual framework

We can, however, follow those who leave PSE through any further PSE experiences by defining a new state, and a new process, which is what in fact we do for leavers. That is, a similar hazard set-up then characterises the re-entry process among leavers, for whom t₀ is now defined for this second kind of spell as of the point they leave PSE, and the possible outcomes, in any given year, are either that the individual re-enters PSE (the transition of interest), or does not. (Or again the spell may be censored, for the same reasons as discussed above.) Once this basic dynamic is established, we then probe the return-to-PSE process a little deeper to look at where they return – same or different institution, same or different level of study, etc.

Finally, we also look at graduates to see how many start new programs after finishing a prior one. This dynamic should probably be considered separate from the "core" persistence dynamics, however, and may perhaps best be defined as those students who continue in their studies towards graduation after starting a program, including those who return to school after dropping out.

Taken together, we believe these processes capture the key elements of the persistence dynamics, while also being analytically well defined and therefore suitable for analysis using the established hazard approach.

3.4   Tracking students and defining the transition states

The analysis requires us to track individuals over time and match their PSE enrolment status and related information from one year to the next in order to identify when students start a program. We then need to identify the subsequent pathways for this sample of PSE program starters as represented by our dynamics of interest (who continues, who graduates, who switches, and who leaves) on a year by year basis. And we need to do all this using an "event"-oriented time frame as opposed to a calendar year basis, which is how the PSIS data are organised.

Furthermore, the data must be organised in this way in a context where individuals can have multiple records in any given year, possibly at the same or different institution; they may have programs that overlap or run concurrently in a given year or across years, again possibly within a given institution or across different institutions; they may have programs that stop and then re-start; and more.

A further challenge is that although one of the fundamental – and extremely useful – characteristics of the PSIS is the uniformity of the data across institutions, different institutions do have somewhat different program structures and different reporting methods related to those different structures. For example, at some institutions, when individuals move into second or third year they declare a major (and not before), and are at this point considered to have started a new program, and this will be indicated in the person's record, while these very same dynamics will be considered as the continuation of the same program in other institutions.

The main point here is that the data are detailed and complex, reflecting the underlying reality of individuals' extremely varied PSE profiles, the differences in the classification and organisation of the information across different institutions, and the gathering of this information into a defined set of variables by Statistics Canada. This extremely complex set of pathways, and associated complexity in the PSIS dataset must, however, be organised to fit the analytical framework. This organisation is the first and, in many senses, the greatest challenge for the analysis.

The first step in the analysis was, therefore, to link a given individual's records longitudinally across all years. This was done by Statistics Canada, and more is said about this below.

Once this was done, the next step was to check, for each individual, all programs in all years for which they had a record in order to identify the point of entry into PSE for those observed to make such a start. From this point, we then tracked the person over time, checking all subsequent programs in order to identify the various dynamics and transitions of interest: continuing in a given program, graduation, a change to a different institution, and leaving PSE in the case of our first dynamic of interest.

In addition, precise dates had to be attached to all program information in each year: when the person started the program, when they stop attending (if that happened), the date of any graduation that occurred, etc. This was required in order to track the person's outcomes on a precise year by year basis: When exactly did they start PSE? What was their situation one year – i.e., precisely 12 months (give or take a month) – after starting their program? And for those who continued in their studies after that first year, what were they doing after two years (24 months), after three years (36 months)?⁴

And recall that in doing all this, we essentially strip away the calendar/reporting year basis of the data – which is of course how the data are organised in PSIS – and use instead a "spell interval" time frame, where the relevant concept stems from the starting date of an individual's program, as described above.

This obviously requires complex programming, even if the end result is conceptually fairly simple – as is often the case with analyses undertaken with longitudinal data.

Based on this treatment of the data, we define the following outcomes. First, graduation is captured by a variable in the PSIS that explicitly marks this event. In our treatment, a student was counted as having graduated in the year in question if they did so at any point up to the relevant anniversary date (or the following month).

A student is, alternatively, defined as a continuer in a given year if he or she had not graduated but was identified as still being enrolled at the original institution at the end of the reporting year in question.

Given the difficulty of accurately identifying program changes within a given institution in a consistent manner in the PSIS, either within a given faculty (say, from History to English), or across faculties (e.g., from Humanities to Engineering), we take the more tractable route of defining "continuing" with respect to the institution rather than a given program. That is, if a person was enrolled in the same institution one, two, or three years after starting, they were classified as a continuer at that point.⁵

Thirdly, a switcher is defined as someone who left the initial institution (without graduating) and was enrolled at a new one as of the year-end dates used to parameterise the analyse.⁶

Finally, a leaver is defined as someone who either had no record in the relevant year, thus – given the comprehensive nature of the PSIS file – implying no enrolment in an Atlantic Canada PSE institution, or had a PSIS record in the year in question but was not enrolled as of the relevant one year anniversary date (again give or take a month).

Note that those who leave Atlantic Canada but stay in PSE are classified as leavers, rather than switchers, but an analysis of the YITS data for Atlantic Canada suggests that switching rates would only be about 5 percent higher and leaving rates commensurately lower, were inter-regional switchers to be taken into account (i.e., about 95 percent of all Atlantic Canada switchers appear to go to another institution in that region). What is, therefore, potentially an important limitation in the data does not, in practice, appear to be very significant.

---

Notes

1. In the health literature, where this approach was developed extensively before economists generally discovered it, the classic example is time to death, typically modelled from the point of becoming ill or otherwise being diagnosed with a malady (hence the term "hazard rate" – i.e., of death). In economics, outcomes analysed using this general approach include things such as time spent unemployed or in poverty, time to a job change, or time to marriage or child birth. The defining characteristic of the approach is that a person enters a state of "risk" (falling ill, becoming unemployed, starting a new job, moving into poverty, etc.) where the event in question (death, escaping unemployment or poverty) and the focus of the analysis is the probability of the relevant outcome (death, employment, a new job, escaping poverty) occurring at any given point after into the state of being at risk or, alternatively, the time it takes for the event to happen. The extension to a "competing risk" framework, where more than one type of transition is possible – as used here – is straightforward.
2. See Finnie and Qiu (2008) for an analysis that combines both approaches.
3. The student can then be followed in a new process from that point, as described below (i.e., returning to school among those who leave, going on to a new program among those who graduate).
4. This one-year framework is somewhat arbitrary, but corresponds to how the persistence dynamic is often framed.
5. Further attempting to identify program changes in a given institution, perhaps for a subset of institutions, would be one potential route for future research to go.
6. Someone who switched and then immediately dropped out and was thus not in school at the relevant year end date was classified as a leaver, not a switcher. This kept the analysis tractable, and consistent. There are, in any event, relatively few transitions of this type.