Dealing with small sample sizes, rotation group bias and discontinuities in a rotating panel design 5. Redesign of the Dutch Labour Force Survey
The LFS was redesigned in 2010, as described in Section 2. Discontinuities induced by this redesign were quantified by conducting the first panel under the old and new design in parallel for a period of six months, from January through June 2010. Each month two separate samples with the regular monthly sample size were drawn from the target population according to the sample design of the LFS. One sample was assigned to the old and one to the new LFS design. This made a direct estimate possible for the discontinuities for the main parameters in the first panel.
Mainly due to budget constraints, the subsequent panels were not conducted in parallel under the old and the new design. Possible discontinuities were quantified using the intervention approach described in Section 3. In the time series model, the outcomes of the subsequent panels are benchmarked to the level of the first panel. It is therefore crucial that the first panel is measured as accurately as possible, including possible discontinuities due to a redesign. Therefore it was decided to conduct a sufficiently large parallel run for the first panel, and use the intervention approach for the remaining panels. The estimates for the discontinuities from the parallel run as well as the intervention variables of the time series model are the effect of all factors that changed simultaneously in the redesign of the survey.
In the parallel run, 19,150 responding households under the old design and 16,906 responding households under the new design were obtained. Table 5.1 compares the field work results of the new and old design, both for households with and without a listed phone number. Overall, the response rate is lower for households without a listed phone number. This can be explained by the fact that this part of the population typically consists of hard to reach groups like young people and migrants. Furthermore, the response rate is lower under CATI than under CAPI for households with a listed phone number. Both the percentages of no contact and of frame errors increase substantially when using CATI instead of CAPI. Frame errors under CAPI are mostly non-existing or unoccupied addresses, under CATI they are mostly closed phone lines. Other non-response includes, for example, illness.
Table 5.2 summarizes the estimation results of the parallel run for the unemployed labour force. At the national level, the change-over to the new design resulted in an increase of about 55,000 in the monthly unemployed labour force figures. The differences fluctuated considerably over the six months of the parallel run, probably caused by the large sampling errors of the GREG estimates. A strong increase in the differences was observed in the last two months of the parallel run, particularly at the national level. This can be explained partially by the low response under the new design during these two months.
The decision was made to produce official monthly figures using the data obtained under the old design until June 2010. After completion of the parallel run, all the available data obtained under the new design were used to compile official monthly figures. So since July 2010, the data in the first panel have been based on the new design from January 2010, while the data in the second panel are based on the new design from April 2010, and the data in the third panel are based on the new design from July 2010 and so on.
Category | CAPI - phone | CAPI – no phone | total | |||
---|---|---|---|---|---|---|
households | % | households | % | households | % | |
Total | 20,813 | 10000.0 | 14,469 | 100.0% | 35,282 | 100.0% |
Frame errors | 769 | 370.0 | 1,039 | 7.2 | 1,808 | 5.1 |
Not approached | 618 | 3.0 | 463 | 3.2 | 1,081 | 3.1 |
Language problems | 390 | 1.9 | 878 | 6.1 | 1,268 | 3.6 |
Refusal | 4,909 | 23.6 | 3,112 | 21.5 | 8,021 | 22.7 |
No contact | 889 | 4.3 | 1,455 | 10.1 | 2,344 | 6.6 |
Other non-response | 921 | 4.4 | 689 | 4.8 | 1,610 | 4.6 |
Complete response | 12,317 | 59.2 | 6,833 | 47.2 | 19,150 | 54.3 |
To analyse differences in response distributions between the old and the new design, results must be compared column-wise. |
Category | CATI | CAPI | total | |||
---|---|---|---|---|---|---|
households | % | households | % | households | % | |
Total | 20,234 | 100.0 | 13,345 | 100.0 | 33,579 | 100.0 |
Frame errors | 1,539 | 7.6 | 982 | 7.4 | 2,521 | 7.5 |
Not approached | 1 | 0.0 | 428 | 3.2 | 429 | 1.3 |
Language problems | 317 | 1.6 | 788 | 5.9 | 1,105 | 3.3 |
Refusal | 4,545 | 22.5 | 2,903 | 21.8 | 7,448 | 22.2 |
No contact | 2,233 | 11.0 | 1,333 | 10.0 | 3,566 | 10.6 |
Other non-response | 963 | 4.8 | 641 | 4.8 | 1,604 | 4.8 |
Complete response | 10,636 | 52.6 | 6,270 | 47.0 | 16,906 | 50.3 |
To analyse differences in response distributions between the old and the new design, results must be compared column-wise. |
National level |
Men 15-24 |
Women 15-24 |
Men 25-44 |
Women 25-44 |
Men 45-64 |
Women 45-64 |
|
---|---|---|---|---|---|---|---|
Monthly unemployed labour force new design | |||||||
Mean over January – June | 475 | 67 | 56 | 103 | 101 | 80 | 68 |
Difference new and old design monthly unemployed labour force | |||||||
Mean January – June | 55*(17) | 19*(6) | 7 (6) | -1 (9) | 20*(8) | 6 (8) | 4 (7) |
Difference per month | |||||||
January | 56 (39) | 13 (14) | 1 (14) | -15 (21) | -16 (18) | 52*(18) | 22 (15) |
February | 38 (42) | 41*(16) | 9 (17) | -10 (22) | 24 (21) | -41*(18) | 15 (18) |
March | 1 (41) | -2 (15) | -11 (13) | -18 (21) | 29 (21) | 6 (19) | -4 (14) |
April | 55 (40) | -2 (13) | 17 (17) | 17 (21) | 36 (20) | 0 (17) | -13 (16) |
May | 70 (44) | 20 (15) | 17 (13) | 12 (27) | 14 (21) | 4 (20) | 3 (15) |
June | 110*(41) | 41*(15) | 10 (14) | 6 (21) | 35 (18) | 13 (20) | 5 (17) |
- Date modified: