Dealing with small sample sizes, rotation group bias and discontinuities in a rotating panel design 8. DiscussionDealing with small sample sizes, rotation group bias and discontinuities in a rotating panel design 8. Discussion

• Previous

National statistical institutes widely apply GREG estimators to produce official statistics. The advantage of these estimators is that they are robust against model misspecification, reduce the design variance, and correct at least partially for selection bias in the case of well-specified weighting models. Furthermore, they result in domain estimates which are consistent by definition, and their use in production processes is relatively straightforward since only one set of weights is required to estimate all possible output tables in a multipurpose survey.

GREG estimators, however, have unacceptably large design variances in the case of small sample sizes and do not handle measurement bias in an effective way. The Dutch LFS is an example where these problems require additional estimation procedures. The sample size is too small to produce sufficiently precise monthly labour force figures with the GREG estimator. The rotating panel design and the major redesign of the survey process make differences in measurement bias visible and compromises comparability of outcomes over time. These problems are solved simultaneously with a multivariate structural time series model that uses the series with GREG estimates for the different panels as input. The time series method combines strong points of the GREG estimator with the advantages of a model-based approach. Since time series of GREG estimates as well as their standard errors are used as input series, the method accounts for the complexity of the sample design and corrects for unequal selection probabilities and selective non-response. The time series model accounts for small sample sizes by taking advantage of sample information observed in previous periods, the autocorrelation in the survey errors, the rotation group bias by benchmarking the estimates to the level of the first panel, and discontinuities that arise from a major survey redesign.

We discussed how the model can be extended with a strongly correlated auxiliary series, which is the number of people formally registered at the employment office in this application. Auxiliary information further decreases the standard error of the filtered trend and signal. Also the levels of the filtered estimates are affected by the auxiliary variable. Since there are strong indications that the evolution of the auxiliary series is affected by factors other than economic cycles, and that this improperly affects the monthly filtered trend of the unemployed labour force, it was decided not to use this information in the ultimately selected model. In this application, the auxiliary series hardly influences the estimated discontinuities. This conclusion, however, cannot be generalized. If e.g., the moment of the change-over coincides with a real break in the evolution of the variable of interest, then auxiliary series should contain similar breaks and can provide valuable additional information to disentangle discontinuities from real developments correctly.

If no parallel run is conducted, then discontinuities are estimated through an intervention variable with a regression coefficient initialized with a diffuse prior. In the case of a parallel run, direct estimates for the discontinuities provide additional information that can be used in the time series model. One possibility is to use the direct estimate with its standard error as an exact prior to initialize the regression coefficient of the intervention variable. Another approach is to assume that the regression coefficient is equal to the direct estimate. This approach treats the external information about the discontinuities as if it is observed without error. A well-conducted parallel run has the advantage that it provides a direct estimate for the discontinuities and therefore does not rely on the assumption that, at the moment of the change-over, the evolution of the variables of interest is captured by the time series components other than the intervention variable.

A consequence of modelling discontinuities is that the standard errors of the filtered trend and signal increase each time the new design enters another panel. This illustrates the importance of keeping the survey process unchanged as long as possible and of limiting the number of redesigns.

In conclusion, a time series model is proposed that simultaneously solves problems with small sample sizes, RGB in a rotating panel, and discontinuities due to a redesign. It enables Statistics Netherlands to publish real monthly figures about the labour force, instead of the rolling quarterly figures that are often used as a second best approximation. During the redesign, the model avoids distortion of real developments of the monthly labour force indicators with sudden changes in measurement bias. The method is flexible and of general interest, since most national statistical institutes apply rotating panels for labour force surveys. Furthermore, redesigns of survey processes aimed to reduce administration costs or to improve outdated methods remain inevitable, resulting in loss of comparability of the outcomes over time. Finally there is an increasing interest for small area estimates while there is always pressure to reduce sample sizes due to budget constraints and lowering the response burden.

Acknowledgements

The authors wish to thank the referees and the Associate Editor and Rita Gircour (Statistics Netherlands), for careful reading of the first draft of this paper and providing constructive comments. The views expressed in this paper are those of the authors and do not necessarily reflect the policy of Statistics Netherlands.

Appendix

With the structural time series model (3.1), monthly estimates for the employed, unemployed and the total labour force are computed for the national level and for a breakdown in the six domains. These 21 population parameters are notated by ${\theta }_{t,l,m},$ where $l=1,2,3$ denotes respectively the employed, unemployed and total labour force, $m=1$ the national level, and $m=2,\dots ,7$ the six domains. For the population parameters, the following consistency requirements hold:

$${\theta }_{t,1,m}+{\theta }_{t,2,m}-{\theta }_{t,3,m}=0,m=1,\dots ,7(\text{A}\text{.1})$$

$${\displaystyle \sum _{m=2}^7{\theta }_{t,l,m}}={\theta }_{t,l,1},l=1,2,3.(\text{A}\text{.2})$$

Subscript $m$ runs within $l,$ which in turn runs within $t.$ Because time series model (3.1) is applied to each population parameter separately, requirements (A.1) and (A.2) do not hold for the model estimates. Therefore, they are restored with a Lagrange function. The model estimates for the national level are changed as little as possible, because they are based on considerably larger samples than the six domains. Therefore, the consistency is achieved in two steps. Both steps are specified for the filtered trends. Consistent filtered signals can be computed in a similar way.

Let $L_{t,l,m}$ denote the filtered trend for ${\theta }_{t,l,m}.$ In the first step, the requirements of equation (A.1) for the national level $\left(m=1\right)$ are considered. The consistency requirement can be written as ${\Delta }^{\left[1\right]}{L}_t^{\left[1\right]}=0$ with $L_t^{\left[1\right]}={\left(L_{t,1,1},L_{t,2,1},L_{t,3,1}\right)}^T$ a vector with the model estimates for the three trends at the national level and ${\Delta }^{[1]}=\left(1,1,-1\right)$ a $3\times 1$ matrix that specifies requirement (A.1). Adjusted estimates that fulfil (A.1) are computed with the Lagrange function

$$L_{t,\text{adj}}^{\left[1\right]}=L_t^{\left[1\right]}-V_t^{\left[1\right]}{\Delta }^{\left[1\right]T}{\left({\Delta }^{\left[1\right]}{V}_t^{\left[1\right]}{\Delta }^{\left[1\right]T}\right)}^{-1}{\Delta }^{\left[1\right]}{L}_t^{\left[1\right]}(\text{A}\text{.3})$$

with $L_{t,\text{adj}}^{\left[1\right]}={\left(L_{t,1,1,\text{adj}},L_{t,2,1,\text{adj}},L_{t,3,1,\text{adj}}\right)}^T$ the adjusted filtered trends. In the ideal case $V_t^{\left[1\right]}$ is the variance-covariance matrix of the trend estimates $L_t^{\left[1\right]}.$ The covariances of the model estimates, however, are not known. Therefore the diagonal matrix of the variances is used instead.

In the second step, $L_{t,\text{adj}}^{\left[1\right]}$ is not changed anymore. Now the vector of domain estimates $L_t^{\left[2\right]}={\left(L_{t,1,2},L_{t,1,3},\dots ,L_{t,1,7},L_{t,2,2},\dots ,L_{t,2,7},L_{t,3,2},\dots ,L_{t,3,7}\right)}^T$ is adjusted according to equation (A.1) for $m=2,\dots ,7$ and to equation (A.2) for $l=1,2.$ Equation (A.2) for $l=3$ is redundant and therefore left out. Again, the consistency requirements for the filtered trends of the domains are written as ${\Delta }^{\left[2\right]}{L}_t^{\left[2\right]}=C_t^{\left[2\right]},$ with

$${\Delta }^{\left[2\right]}=\left(\begin{array}{ccc}{I}_6& I_6& -I_6\\ 1_6& 0_6& 0_6\\ 0_6& 1_6& 0_6\end{array}\right),$$

$C_t^{\left[2\right]}={\left(0_6,L_{t,1,1,\text{adj}},L_{t,2,1,\text{adj}}\right)}^T,I_6$ the six dimensional identity matrix, and $1_6$ and $0_6$ six dimensional row vectors with each element equal to one or zero respectively. Consistent domain estimates are computed with the Lagrange function

$$L_{t,\text{adj}}^{\left[2\right]}=L_t^{\left[2\right]}-V_t^{[2]}{\Delta }^{\left[2\right]T}{\left({\Delta }^{\left[2\right]},V_t^{\left[2\right]},{\Delta }^{\left[2\right]T}\right)}^{-1}\left({\Delta }^{\left[2\right]}{L}_t^{\left[2\right]}-C_t^{\left[2\right]}\right),$$

similarly to (A.3). In this case $V_t^{\left[2\right]}$ is the diagonal matrix of the variances of the estimates of $L_t^{\left[2\right]}.$

References

Bailar, B.A. (1975). The effects of rotation group bias on estimates from panel surveys. Journal of the American Statistical Association, 70, 23-30.

Binder, D.A., and Dick, J.P. (1990). A method for the analysis of seasonal ARIMA models. Survey Methodology, 16, 2, 239-253.

Doornik, J.A. (2009). An Object-oriented Matrix Programming Language Ox 6. London: Timberlake Consultants Press.

Durbin, J., and Koopman, S.J. (2001). Time Series Analysis by State Space Methods. Oxford: Oxford University Press.

Harvey, A.C., and Durbin, J. (1986). The effects of seat belt legislation on British road casualties: A case study in structural time series modelling. Journal of the Royal Statistical Society, Series A, 149, 187-227.

Isaki, C.T., and Fuller, W.A. (1982). Survey design under the regression superpopulation model. Journal of the American Statistical Association, 77, 89-96.

Koopman, S.J. (1997). Exact initial Kalman filtering and smoothing for non-stationary time series models. Journal of the American Statistical Association, 92, 1630-1638.

Koopman, S.J., Shephard, N. and Doornik, J.A. (2008). SsfPack 3.0: Statistical Algorithms for Models in State Space Form. London: Timberlake Consultants Press.

Pfeffermann, D. (1991). Estimation and seasonal adjustment of population means using data from repeated surveys. Journal of Business & Economic Statistics, 9, 163-175.

Pfeffermann, D., Feder, M. and Signorelli, D. (1998). Estimation of autocorrelations of survey errors with application to trend estimation in small areas. Journal of Business & Economic Statistics, 16, 339-348.

Rao, J.N.K. (2003). Small Area Estimation. New York: John Wiley & Sons, Inc.

Robinson, P.M., and Särndal, C.-E. (1983). Asymptotic properties of the generalized regression estimator in probability sampling. Sankhyā B, 45, 240-248.

Särndal, C.-E., Swensson, B. and Wretman, J. (1992). Model Assisted Survey Sampling. New York: Springer Verlag.

U.S. Census Bureau (2009). X-12-ARIMA Reference Manual. Washington DC.

Van den Brakel, J.A., and Krieg, S. (2009). Estimation of the monthly unemployment rate through structural time series modelling in a rotating panel design. Survey Methodology, 35, 2, 177-190.

Van den Brakel, J.A., and Roels, J. (2010). Intervention analysis with state-space models to estimate discontinuities due to a survey redesign. Annals of Applied Statistics, 4, 1105-1138.

• Previous