Revisions analysis, 1981 to 2007

Warning View the most recent version.

Archived Content

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please "contact us" to request a format other than those available.

[an error occurred while processing this directive]13-604-m[an error occurred while processing this directive] [an error occurred while processing this directive]

Revision summary statistics
Distribution and statistical inference
Revisions and the economic cycle

The following analysis will focus on the seasonally adjusted, quarter-to-quarter real GDP growth rate for Canada.1 Using the real GDP growth rate in revisions analysis is advantageous compared to using the level or nominal growth rate, because: 1) it eliminates the trend problem whereby revisions to GDP tend to grow over time as economic growth and inflation raise the GDP level at current prices; and 2) it reduces the need to account for the impact on revisions of the significant changes in concepts, sources, and methods, including scale changes due to rebasing.

GDP and its components are revised up to seven times over a four-year revision period as per the revisions policy outlined in Appendix A. GDP can be revised again at the time of a historical revision. Chart 1 illustrates five different vintages of real GDP growth rate for the period from the first quarter of 1981 to the first quarter of 2010.2

Chart 1 Real gross domestic product, five different vintages
Description for Chart 1

Chart 1 Real gross domestic product, five different vintages

Table 4 presents summary statistics for the quarterly real GDP growth rate over the time period from the first quarter of 1981 to the fourth quarter of 2007. Revisions have had an impact on GDP growth rates as the average growth rate increased with each vintage and the spread of those growth rates became larger (the mean and standard deviation). The volatility of revisions can be seen in the variability of the range of growth rates for the different vintages. The average GDP growth rate of the initial estimate was 0.64%; with subsequent revisions, the average growth rate of the final estimate was 0.69%. The preliminary growth rates ranged from -1.8% to 2.0%, increasing to a range of -1.5% to 2.5% for the final estimate.

Table 4 Summary statistics, real gross domestic product growth rate, first quarter 1981 to fourth quarter 2007

During this time period, the statistical system changed significantly. Selected annual services surveys were added in the early 1980s, and the redesign of business surveys occurred in 1989. The historical revision that took place in 1997 introduced new international standards from the SNA 1993. PIPES introduced higher-quality business survey frames; this allowed for comprehensive coverage of industries, particularly services. These improvements, which increased the level of GDP, were phased in over a number of years, from 1998 to 2000. The capitalization of software in 2001, which was an international recommendation from the SNA 1993, raised the level of GDP back to 1981 and affected growth rates in the late 1990s due to the high-tech boom.

Revision summary statistics

The following table (Table 5) provides summary statistics often used in revisions analysis.3 Measures of reliability include those for bias, such as the mean revision, and those for dispersion, such as the mean absolute revision and the relative dispersion. The chart that follows illustrates the initial and final GDP growth rates and the amount of the total revision.

There were 108 quarters over the time period from the first quarter of 1981 to the fourth quarter of 2007. The revisions ranged from -1.47 percentage points in the third quarter of 1984 to 0.93 percentage points in the second quarter of 1984. The low value of -1.47 is the only extreme outlier in the series. If this value were ignored, the lowest value would be -0.94, from the first quarter of 1986. With the exception of this extreme outlier, all revisions were within 1 percentage point. The range in the first ten years is larger than the range in the following fifteen years. The range that 90% of revisions lie within provides an indication of the normal range of revisions not influenced by unusually large revisions or outliers. This range is larger in the earlier periods, excluding the first ten years (1981 to 1990); the range of 90% of revisions for 1991 to 2007 is -0.52 to 0.53.

The mean revision of 0.05 percentage points is quite small. The small positive average indicates that the earlier releases have been slightly under-estimated. This can be seen in Chart 2, where most revisions have been positive. As revisions of opposite signs will have a tendency to cancel out, it is beneficial to examine the mean absolute revision (MAR). The MAR is a measure of dispersion between the initial estimates and the corresponding final estimates; it also provides an indication of the magnitude of revisions. The MAR over the entire period of 0.32 percentage points reflects the higher MAR of 0.42 percentage points for the 1980s. The MAR was only 0.31 percentage points for the 1990s and has been 0.20 percentage points for the last seven years.

The standard deviations, which are a measure of the volatility of revisions, are smaller in the more recent periods. The lower standard deviations, combined with the lower MARs, suggest that revisions have gotten smaller over time, a trend that can also be observed in Chart 2.

Table 5 Revision summary statistics, quarterly real gross domestic product growth rate

Chart 2 Real gross domestic product growth rates and total revision
Description for Chart 2

Chart 2 Real gross domestic product growth rates and total revision

To examine the behaviour of revisions over time, the analysis period was further decomposed into five-year periods of 20 quarters. Table 6 and Chart 3 illustrate the summary revision statistics for the interval periods. The standard deviation, or spread of revisions, is large in the first interval, at 0.63, and is much smaller in the final interval, at 0.26. The MAR is half a percentage point in the first five years, but decreases in the following years.4 When one excludes the extreme outlier in the third quarter of 1984, the first five-year interval still has the largest MAR (0.45 percentage points) and the largest standard deviation (0.52 percentage points).

Table 6 Selected revision summary statistics for five-year intervals

Chart 3 Measures of bias and dispersion for five-year intervals
Description for Chart 3

Chart 3 Measures of bias and dispersion for five-year intervals

The mean revisions for the 1981-to-1985 and 1996-to-2000 time periods are larger than those occurring in the mid-1980s to the mid-1990s and in the 2000s. This may be partly the result of statistical system improvements that took place during those two time periods. As mentioned previously, the early 1980s saw the introduction of new services surveys; as well, there was a historical revision and the implementation of PIPES in the late 1990s. The PIPES implementation resulted in more widespread coverage of a given industry and in the number of industries covered. These statistical system improvements can also be seen in the following chart, which is the cumulative sum of revisions. The cumulative sum is larger in those two time periods as well: 2.9 percentage points for the 1981-to-1985 time period and 4.0 percentage points for the 1996-to-2000 time period.

Chart 4 Cumulative sum of revisions to real gross domestic product growth rate
Description for Chart 4
Chart 4 Cumulative sum of revisions to real gross domestic product growth rate

Revision vintages

Revisions to the estimates occur with each new vintage in the normal four-year revision cycle. Previous revisions analysis (de Zilva 2004) noted that the revision is larger when the Input-Output benchmarking takes place. To understand this revision pattern a little further, a subset of years was examined, namely the quarters for years 1993 to 2007, for which all vintages are readily available.5 For this period, seven vintages were included in a decomposition analysis. Table 7 presents the summary statistics for quarterly real GDP for each vintage of data within this time frame.

Table 7 Summary statistics, real gross domestic product growth rate, first quarter 1993 to fourth quarter 2007

The mean absolute growth rate for the initial estimate of quarterly real GDP was 0.74 percentage points; with subsequent revisions, the mean absolute growth rate for the final estimate was 0.81 percentage points. Examining the mean absolute growth rate provides a more precise indication of direction because this avoids the offsetting that can occur with positive and negative growth rates. The largest increase in the mean absolute growth rate appears to be between the estimate one year later and the estimate two years later.

Chart 5 illustrates the revisions from each vintage, with the total revision as the bar and the various revisions by vintage as the line for first quarter of 1993 to the fourth quarter of 2007. The first graph shows the revision between the initial estimate and the estimate one quarter later. For the most part, this new estimate reflects late survey returns and more up-to-date seasonal factors. The second graph shows the revision between the estimate one quarter later and the estimate one year later. This estimate is from the first annual revision and reflects benchmarking to the Input-Output Tables for two years previous. Input-Output benchmarking may revise the GDP level for the Input-Output year; this may impact the current-year growth rate.

The third graph is the revision between the estimate one year later and the estimate two years later, and reflects the second annual revision. This estimate includes results from the Capital Expenditure Survey and T4 data for wages and salaries, which led to a large revision to the series. This revision also reflects benchmarking to the Input-Output Tables for one year previous.

The fourth graph shows the revision as a result of the third annual revision. This is also a large revision and is due to the Input-Output benchmarking. Annual surveys that cover the entire economy are incorporated at this time. The fifth graph illustrates the revision between the estimate three years later and the estimate four years later (at the fourth annual revision). This estimate is considered the final estimate within the regular revisions cycle and contains updated health and education sector data. The fifth graph contains the revisions resulting from the historical revisions in which new concepts and methods were carried back to 1981.

Chart 5 Revisions to real gross domestic product at different vintages
Description for Chart 5
Chart 5 Revisions to real gross domestic product at different vintages

Table 8 provides a more detailed breakdown of the revision vintages. From this table, it is easier to identify that the largest revision occurs between the estimate one year later and the estimate two years later. This revision accounts for approximately 25% of the total revision over the 1993-to-2007 period. This vintage represents, among other things, the incorporation of the Capital Expenditure Survey and T4 data for wages and salaries.

Table 8 Selected revision summary statistics for different vintages, first quarter 1993 to fourth quarter 2007

Distribution and statistical inference

The following chart (Chart 6) shows the frequency histogram of revisions to quarterly real GDP growth and the normal distribution curve fitted to that histogram. The histogram graphically summarizes the distribution of the data set; users can see the spread of the data, how the data may be skewed, and the presence of any outliers. This revisions data set has a slight negative skew, since the median (0.09) is larger than the mean (0.05). This can be seen with the longer tail on the left of the histogram. When the extreme outlier is excluded, a negative skew still remains. The lack of symmetry means that this distribution is not exactly the same as a normal distribution but is nevertheless very close.

As noted by de Zilva,6 a t-test can be used to conduct statistical inference. The t-test will determine whether the mean revision is significantly different from zero. If initial estimates were consistently, or in the majority, underestimating the final estimates, the revisions would tend to be positive, and vice-versa. For the t-test to be valid, no significant serial correlation between the quarters should be present. The coefficient of determination is 0.026; this means that 2.6% of the variation in the revisions for a certain quarter is due to its linear relationship with revisions in the previous quarter. The correlation coefficient of -0.161 indicates that the relationship is a low, negative one. Where the null hypothesis is 'the mean revision is zero,' this would be rejected if the mean of the revisions were too high or too low. The alternative hypothesis is 'the mean is not zero'. The rejection region is t < -1.984 or t > 1.984, with 0.05 level of significance and a sample size of 108. The test statistic is computed as 1.321; this result does not fall in the rejection region. The test failed to reject the null hypothesis that the set of revisions is representative of a population with mean zero. This indicates that there is no significant bias; in other words, the initial estimates are a reliable estimate of the final estimates.

Chart 6 Histogram of revisions and normal distribution curve, real gross domestic product growth rate, 1981 to 2007
Description for Chart 6
Chart 6 Histogram of revisions and normal distribution curve, real gross domestic product growth rate, 1981 to 2007

Revisions and the economic cycle

During times of accelerated or decelerated growth and around cyclical turning points (peaks and troughs), the reliability of GDP is especially important. This reliability can be investigated by examining the behaviour of revisions with respect to the direction of this growth and in relation to the economic cycle.

A regression analysis can help determine whether there is a correlation between revisions and the economic cycle (GDP growth rate). The coefficient of determination is 0.141; this means that 14.1% of the variation in the revisions is due to its linear relationship with the economic cycle. The correlation coefficient of 0.379 indicates that the relationship is a low, positive one. This relationship is illustrated in the scatter plot (Chart 7), which shows the correlation between GDP growth rate revisions and the economic cycle. When GDP is expanding, the initial estimate tends to underestimate the growth. When GDP is contracting, the initial estimate is more likely to overestimate the growth.

Chart 7 Real gross domestic product growth rate revisions and the economic cycle, first quarter 1981 to fourth quarter 2007
Description for Chart 7
Chart 7 Real gross domestic product growth rate revisions and the economic cycle, first quarter 1981 to fourth quarter 2007

It is of interest to know how often the final growth rate is in the opposite direction to the initial growth rate. A weakness of this measure is that it does not record the total magnitude of the revision. For example, a change from a small negative to a small positive, or vice versa, may not be that substantial. Over the analysis period, estimates switched direction in only five quarters, or 4.6% of the time. The average revision for these five quarters was small, at -0.26 percentage points. The revision to two of these five quarters was a result of the mini-historical revision that took place in the first quarter of 2001. There were no cases throughout the time period when the sign of the growth rate switched away from, and then back to, the original sign of growth.

Changes in trend

The initial GDP estimate ideally needs to correctly indicate the direction of economic growth and to capture shifts in economic activity. A question to ask in determining reliability is: How often do initial estimates capture changing economic trends?

From 1981 to 2007, a peak or trough was missed 17.6% of the time (19 out of 108 quarters). Of those 19 quarters, 8 quarters experienced the turning point within one quarter of the reference quarter. The average revision for a missed peak was 0.36 percentage points, and the average revision for a missed trough was -0.51 percentage points. For the purposes of this analysis, a peak at quarter t occurs when the growth rate of real GDP at that quarter is larger than the growth rate in the previous two quarters and in the following two quarters. A trough occurs when the growth rate is smaller than that in the previous two quarters and in the following two quarters. The missed changes in economic trends are highlighted in Chart 8.

Chart 8 Missed changes in trend from the initial to final estimate, real gross domestic product level and growth rate
Description for Chart 8
Chart 8 Missed changes in trend from the initial to final estimate, real gross domestic product level and growth rate

Acceleration and deceleration

Preliminary statistical estimates based on surveys and methods used to project the components of GDP are sometimes based on assumptions of little change in the structure of components of GDP. This may lead to an overestimate of growth when there is a slowdown or to an underestimate when there is acceleration. An examination of acceleration and deceleration (AC/DC) in the initial versus final estimates can shed light on whether the initial estimate correctly estimated growth of economic activity. An acceleration of growth is what takes place when the change in growth rate from the previous quarter is positive, and a deceleration is what results when that change is negative.

AC/DC represents the number of times that the initial estimate of GDP growth indicated acceleration of economic activity from the previous period while the final estimate indicated deceleration, or vice versa. Over the analysis period, the AC/DC was incorrectly captured 36 out of 108 times. The relative AC/DC, which is the AC/DC as a percent of the total number of quarters, is 33.3%. Chart 8 identifies the quarters in which AC/DC existed. The average revision for a missed deceleration was -0.29 percentage points, whereas the average revision for a missed acceleration was 0.31 percentage points.

Chart 9 illustrates the growth rate of real GDP during periods of acceleration and deceleration and the corresponding revision for that quarter. An upward revision during accelerated growth is an indication that the initial estimate is underestimating growth. Conversely, downward revisions may mean that the initial estimate is overstating growth. During the analysis period, 55 quarters were found to have experienced acceleration, and 53 quarters were found to have experienced deceleration. The average revision for accelerating periods was 0.18 percentage points whereas the average for decelerating periods was -0.08 percentage points. In both cases, the initial estimate tended to flatten out the rate of change in GDP. As can be seen in Chart 9, most revisions during periods of acceleration are upwards, with only 29% of quarters experiencing a downward revision. For decelerating growth, 53% of revisions are upwards, but 44% are downwards. This may indicate that, during periods of slowing growth, there is a tendency to underestimate the slowdown in GDP.

Chart 9 Revisions to real gross domestic product growth rate during periods of acceleration and deceleration, first quarter 1981 to fourth quarter 2007
Description for Chart 9
Chart 9 Revisions to real gross domestic product growth rate during periods of acceleration and deceleration, first quarter 1981 to fourth quarter 2007

Periods with negative growth rates

Table 9 shows the revisions and standard deviations for different categories of growth rates, periods of acceleration, periods of deceleration, and periods of negative growth. Although the MARs are similar, they are slightly larger for periods of slowing growth and negative growth; the standard deviations for those periods are larger as well.

Table 9 Real gross domestic product growth rate, average estimate and revision, first quarter 1981 to fourth quarter 2007

Fifteen quarters over the analysis time frame had negative final growth rates. When one ignores these negative growth rates and looks at the same chart showing only positive growth rates (Chart 10), it is quite evident that, during periods of acceleration, revisions are mostly upwards. A downward revision was made in only 16 of the 52 quarters (31%). During slowing GDP growth, there are more negative revisions, in 21 out of 41 quarters (this translates as 51% being downward revisions).

Chart 10 Change in real GDP growth rate during periods of acceleration and deceleration with positive growth rates and the revision, first quarter 1981 to fourth quarter 2007
Description for Chart 10
Chart 10 Change in real GDP growth rate during periods of acceleration and deceleration with positive growth rates and the revision, first quarter 1981 to fourth quarter 2007

Table 10 contains summary statistics of the real GDP growth rate for quarters with negative growth. The mean growth rate is similar throughout the different vintages, but the standard deviation, or spread, is much smaller for the final estimate. By the final estimate, the range is much tighter than for the initial estimate, and this is the first time that the growth rates for all quarters are negative. Prior to the final estimate, the preceding vintages had positive values for the upper range.

Table 10 Summary statistics, real gross domestic product growth rate, quarters with negative growth, first quarter 1981 to fourth quarter 2007

The mean revision for quarters with negative growth was -0.04 percentage points, which is quite small. The MAR was 0.35 percentage points; this is higher than the MAR for all quarters (0.32 percentage points). The larger MAR for negative growth quarters suggests that there is greater volatility in revisions to estimates with negative growth.

Significant economic events

Significant economic events can have a considerable impact on GDP and its components. It is worthwhile to investigate the behaviour of revisions during these times of uncertainty to see whether a larger revision occurs during these periods. Table 11 lists some recent major economic events in Canada.

Table 11 Recent significant economic events

The recent major economic events set out in Table 11, along with other past economic events and the total revision for the quarter in which they occurred are highlighted in Chart 11 (Chart 2 reproduced with economic events indicated). For the most part, the total revisions for the quarter in which each of these events occurred were quite small. The only time when the total revision was higher than the average total revision for all quarters was for the first quarter of 1998, when the 1998 ice storm occurred; the total revision for this quarter was 0.4%. These results would suggest that the statistical system is as good at times of significant economic events as at other times.

Chart 11 Real gross domestic product growth rates and total revision
Description for Chart 11
Chart 11 Real gross domestic product growth rates and total revision

Although the total revisions during the above economic events were relatively low, it is of interest to see how those revisions progressed over different vintages. Chart 12 illustrates the growth rates for different vintages during the quarters that experienced these economic events. The revisions to the quarters affected by the ice storm of 1998, the high-tech crash of 2000, and the SARS and mad cow epidemics of 2003 fluctuated more between revision vintages.

Chart 12 Real gross domestic product during recent major economic events for different vintages
Description for Chart 12
Chart 12 Real gross domestic product during recent major economic events for different vintages


Notes

  1. Quarter-to-quarter growth rate of GDP is used in the analysis rather than the annualized rate, which is calculated by means of a compound growth formula. Annualized rate is useful with comparisons to the U.S.
  2. Final estimate available for the fourth quarter of 2007. Data for the following years are data available as of the date of this analysis. The chart contains data that were readily available for five vintages of real GDP back to 1981.
  3. Formulas for these summary statistics are included in Appendix C.
  4. Earlier periods have undergone more revision intervals in comparison to later periods.
  5. Data vintages readily available from the Statistics Canada Real-time Data Analysis Tool (RTDAT).
  6. de Zilva, D. 2004. Toolkit for Revision Studies. 2003-04 Revision Project, System of National Accounts. Paper presented to the OECD-ONS Workshop on Assessing and Improving Statistical Quality – Revisions Analysis for the National Accounts, held on October 7 and 8. Ottawa: Statistics Canada.
Report a problem on this page

Is something not working? Is there information outdated? Can't find what you're looking for?

Please contact us and let us know how we can help you.

Privacy notice

Date modified: