Growth Rates Preservation (GRP) temporal benchmarking: Drawbacks and alternative solutions
Section 3. Two problems with GRP benchmarking

• Previous
• Next

3.1  Time reversibility

Time reversibility means that it does not matter whether a method is applied forward or backward in time. This property can be of interest in many application areas.

In physics, it means that if time would run backwards, all motions are reversed. In index number theory, time reversibility was introduced in a classical work of Fisher (1922, page 64). It is stated that “if taking 1913 as a base and going forward to 1918, we find that, on the average, prices have doubled, then, by proceeding in the reverse direction, we ought to find the 1913 price level to be half that of 1918”. The motivation of this principle is that the direction of time can be considered arbitrary; it does not have any naturally preferred direction.

Time reversibility can also be applied in the context of benchmarking. It means that if we would reverse a time series, apply benchmarking, and reverse the benchmarked series back again, we get exactly the same results as for benchmarking the original series. In other words: from the benchmarked results it cannot be seen whether benchmarking has been applied forward or backward in time.

Benchmarking a reversed time series, according to GRP and Denton PFD, respectively, is equivalent to minimizing the following objective functions

$$f_{\text{B}}^{\text{GRP}}\left(x\right)={\displaystyle \sum }_{t=2}^n{\left(\frac{x_{t-1}}{x_t}-\frac{p_{t-1}}{p_t}\right)}^2(3.1)$$

and

$$f_{\text{B}}^{\text{PFD}}\left(x\right)={\displaystyle \sum }_{t=2}^n{\left(\frac{x_{t-1}}{p_{t-1}}-\frac{x_t}{p_t}\right)}^2,(3.2)$$

where subscript “B” stands for backwards. These objective functions are obtained from the forward objective functions by interchanging $t$ and $t-1.$ From now on, the minimization of (3.1) or (3.2) will be called “backward benchmarking”, as opposed to standard, forward benchmarking.

As mentioned above, a benchmarking method satisfies the time reversibility property if forward and backward benchmarking lead to the same results. It can be easily seen that $f_{\text{F}}^{\text{GRP}}\left(x\right)\ne f_{\text{B}}^{\text{GRP}}\left(x\right),$ while $f_{\text{F}}^{\text{PFD}}\left(x\right)=f_{\text{B}}^{\text{PFD}}\left(x\right).$ From this it follows that Denton PFD satisfies the time reversibility property, but GRP does not.

More practically, in many production processes “forward” benchmarking is applied, for example for the reconciliation of the Dutch Supply and Use tables (Bikker et al., 2013). However, after a revision, revised time series may be constructed “back in time”, by using backward objective functions. It is highly undesirable that there are any differences in outcomes that can be purely attributed to a difference in “time direction”. Practitioners who are unaware of the time reversibility property, may apply forward and backward benchmarking and mistakenly assume that both methods lead to the same results.

Although it is true that any benchmarking application can be restricted to preserving forward growth rates, it is undesirable that results are affected by the irrelevant property of time direction. Therefore, any benchmarking method should preferably satisfy time reversibility. Moreover, Subsection 3.3 illustrates that a benchmarking method that is not symmetric in time may change the timing of the most important economic events, e.g., the peaks and troughs that demark the start and end of a crisis.

3.2  Singularity

A second problem of GRP is the singularity of its objective function. If $x_{t-1}$ approaches to zero in case of forward benchmarking (or $x_t$ for backward benchmarking) the objective function value tends to infinity. This causes several problems.

One complication is that the optimization problem becomes unstable, a small change in preliminary values can lead to a large shift in benchmarked values. Consequently, undesirably large revisions can be obtained when benchmarking updated data.

Another complication is that, since a correction of near zero values can be heavily penalised, growth rates of such values are strongly preserved. This may however come at the expense of relatively large corrections of other growth rates. On the other hand, one may argue that growth rates do not contain much information for extremely small (close-to-zero) values. Hence, growth rate preservation can be deemed inappropriate in this case. Subsection 5.3 shows a real-life example of this problem.

A third complication is that, as close-to-zero benchmarked values may cause a large objective function value, GRP methods tend to avoid such values. Consequently, irregular correction patterns can be obtained. In particular, negative benchmarked values may be obtained for a problem in which all preliminary values are positive. Consider an example in which two consecutive values are 100. Then, an adjustment of the first value from 100 to -100 is less costly in terms of GRP’s objective function value than a correction from 100 to 30. The corresponding objective function values are ${\left(\left(100/-100\right)-\left(100/100\right)\right)}^2=4$ and ${\left(\left(100/30\right)-\left(100/100\right)\right)}^2=\text{5}\text{.44}.$ A value that goes from a large positive to a large negative will however usually not be considered good movement preservation. Therefore, the example also demonstrates the questionability of the use of growth rates when positive and negative values occur.

For this reason, it can be advisable to avoid negative outcomes by inclusion of non-negativity constraints, see Subsection 4.1 for more details. For Denton PFD negative values are less likely obtained. In the previous example, an adjustment from 100 to 30 is preferred to an adjustment from 100 to -100. A real-life example of this problem is shown in Subsection 5.3.

Although singularity of GRP’s objective function may trigger negative benchmarked values, it is not the only cause. Denton PFD may also yield negative values. In general, there is a risk of negative benchmarked values, when the (relative) change from one benchmark to another significantly differs from the (relative) change from the underlying annualised preliminary values.

A fourth complication of GRP’s singular objective function is that irregular peaks and throughs may occur in a benchmarked time series. The explanation is that in standard GRP a correction of large positive value to a close-to-zero value is less costly in terms of the objective function value than an opposite correction from close-to-zero to a large positive. That is, a correction of a growth rate $g$ with a factor $c,$ where $c>1,$ corresponds to a larger objective function value than a correction with $1/c,$ especially if $c$ is large. The objective function values are ${\left(\left(c-1\right)g\right)}^2$ and ${\left(\frac{\left(c-1\right)}{c}g\right)}^2$ respectively. Since large upward corrections from a close-to-zero value are relatively costly, these are avoided as much as possible. Thus, the GRP’s benchmarked values move more gradually from a close-to-zero value than Denton’s results do. To compensate for this, larger peaks may be necessary for the following time-periods to fulfill the temporal aggregation constraint. As benchmarking usually aims at as smooth as possible corrections over time, irregular peaks can be considered undesirable. Related to the relatively slow growth from a close to zero value is that the peaks tend to turn up later in time than for a time-symmetric method like Denton PFD. For the backward variant of GRP the opposite occurs, benchmarked time series move relatively quickly from a close to zero value, which gives rise to relatively early peaks. The example in Subsection 3.3 illustrates this problem.

3.3  Example

Below we present an example that illustrates the problems of GRP methods. In this example, a time series consisting of 15 months is reconciled with five quarterly values. The monthly series is constant: each monthly value is 10. The quarterly values are: 80, 250, 80, 400 and 100, respectively. Figure 3.1 compares the results of Denton PFD, GRPF and GRPB.

As the largest differences occur between both GRP methods, time reversibility is obviously not satisfied. The highest and lowest points appear at different months. The example clearly shows that the use of a different benchmarking method may lead to substantially different conclusions.

In accordance with Subsection 3.2, GRPF leads to relatively late peaks, i.e., at the last month of each quarter, while GRPB results in early peaks, i.e., at the first month of each quarter. Denton PFD’s results are in between, peaks and troughs occur at the middle month of each quarter.

It needs however to be noted that the example cannot be considered representative for real life applications. In general, benchmarking methods are not meant to be used for reconciling large differences and for constant sub annual series. To explain the latter, a main assumption of Denton PFD is that the sub annual series provides information about short-term change. Constant series however cannot be considered very informative. Nevertheless, the problem of reconciling constant term series does occur in problems that are closely related to benchmarking, like interpolation and calenderization (see e.g., Dagum and Cholette, 2006 and Boot, Feibes and Lisman, 1967). The reason for choosing this example is purely educational. It provides good insight into properties of the different types of objective functions. The reader is referred to Subsection 5.3 for more realistic examples.

• Previous
• Article main page
• Next