4. Case study

Jeroen Pannekoek and Li-Chun Zhang

Previous | Next

4.1 Imputation and adjustment of pasture data

The population for the “main questionnaire” of the Norwegian Agriculture Census 2010 contains about 45,000 units. Questions 22 - 24 deal with pasture area:

• Question 22 inquires the units that possess productive pasture.

• Question 23 inquires the total productive pasture area in 2010.

• Question 24 inquires the composition of pasture area by the last time it was seeded: (1) 2006 - 2010, (2) 2001 - 2005, and (3) 2000 or earlier.

Denote by $x_{\mathrm{0,1}},x_{\mathrm{0,2}}$ and $x_{\mathrm{0,3}}$ the three reported categories of pasture area in Question 24. Let $x_0={\displaystyle {\sum }_{j=1}^3{x}_{\mathrm{0,}j}}$ be the sum that is the subject of Question 23. Now, this total is available from the government agency that administers the relevant subsidy. In editing the reported $x_0$ is overwritten by the administrative figure, denoted by $\tilde{x},$ and held as fixed afterwards. Next, Question 22 can be inferred given $\tilde{x}$ and held as fixed afterwards, so that only Question 24 remains to be handled.

Below we describe the treatment of the 34,480 units that have productive pasture area according to their respective observation patterns (Table 4.1, where the unit index $i$ of all the variables was omitted for ease of presentation).

• 10,378 units reported a total pasture area that is consistent with the administrative source: these are the potential donors; no adjustment is needed.

• 11,827 units have a reported total that is greater than the known value: these have a micro-level inconsistency problem. Of course, missing values can also be the case if ${\displaystyle {\sum }_j{r}_j}<3,$ but the chance is small, so we shall assume that there are no missing values among these units. All the observed values are adjustable, such that the accounting equation is given by

  $${\displaystyle \sum _{j;r_j=1}{\tilde{x}}_j=\tilde{x}}.$$

  The GR approach simply yields the proportional adjustment $\tilde{x}/{\displaystyle {\sum }_{j;r_j=1}{x}_{0,j}}.$ The same adjustment is given by the WLS-approach with $w_j=1/x_{0,j}$ if $r_j=1,$ as well as by the KL approach. We notice that there is no particular motivation for considering additive adjustments for these data.

• 3,876 units have no reported pasture area of any kind, despite they have productive pasture area according to the administrative source: these constitute unit-missing records. The nearest-neighbour (NN) donor is found according to $\tilde{x},$ within each of the 12 “farming forms”, which is a classification known for the whole population. In the case of multiple NN donors, we choose the one with the shortest physical distance, which make the NN-imputation completely deterministic, given all the $\tilde{x}-$ values. Finally, a proportional adjustment of the donor values is carried out in order to satisfy the accounting equation

  $${\displaystyle \sum _{j;r_j^{\ast }=1}}{\tilde{x}}_j=\tilde{x}$$

  where $r_j^{\ast }$ is the observation/reporting indicator associated with the donor.

• 3,019 units have reported pasture areas of all the three kinds, but their sum is less than the known total: these have a micro-level inconsistency problem. A proportional adjustment is applied to all the reported values w.r.t. the accounting equation ${\displaystyle {\sum }_{j=1}^3{\tilde{x}}_j}=\tilde{x}.$

• The last two groups are the 2,703 units with one kind of reported pasture area and the 2,677 units with two kinds of reported pasture area. Obviously, that the reported total is less than the known value here may be caused by inconsistency and/or missing values. To avoid introducing systematic pattern through editing, we let the decision depend on the donor. Take a unit with only one reported pasture area. Firstly, the potential donors are limited to those from the same “farming form”, as well as having at least the same kind of pasture area. The NN donor is then selected among these to minimize

  $$\max \left(\left|{\tilde{x}}^{\ast }/\tilde{x}-1\right|,{\left|x_j^{\ast }/{\tilde{x}}^{\ast }-x_{0,j}/\tilde{x}\right|}_{j;r_j=1}\right)$$

  where $\left(x_1^{\ast }\mathrm{,}{x}_2^{\ast }\mathrm{,}{x}_3^{\ast }\right)$ and ${\tilde{x}}^{\ast }$ are the values of the potential donor. In other words, the NN donor is selected both w.r.t. the relative difference between the total pasture area as well as the proportion of the reported kind of pasture area to the corresponding total. Let the NN donor be associated with $x^{\ast }$ and $r^{\ast }.$ If ${\displaystyle {\sum }_j{r}_j^{\ast }}>1={\displaystyle {\sum }_j{r}_j},$ then we assume that there are missing values where $r_j^{\ast }=1$ but $r_j=0;$ whereas, if ${\displaystyle {\sum }_j{r}_j^{\ast }}={\displaystyle {\sum }_j{r}_j},$ then we assume that there is only an inconsistency problem. The remaining imputation and adjustment actions are straightforward. The same treatment is applied to the units with two reported pasture areas, with obvious modifications due to ${\displaystyle {\sum }_j{r}_j}=2.$

Table 4.1
Observation pattern among units with productive pasture area: $r_j=1$ if $x_{0,j}$ is reported, $r_j=0$ otherwise; $j=1,2,3$ for three categories of pasture area
Table summary
This table displays the results of Observation pattern among units with productive pasture area: $r_j=1$ if $x_{0,j}$ is reported. The information is grouped by Total (appearing as row headers), ${\displaystyle {\sum }_j{r}_j{x}_{\mathrm{0,}j}}=\tilde{x}$ and ${\displaystyle {\sum }_j{r}_j{x}_{\mathrm{0,}j}}>\tilde{x}$, and ${\displaystyle {\sum }_j{r}_j{x}_{\mathrm{0,}j}}<\tilde{x}$ (appearing as column headers).

Total	${\displaystyle {\sum }_j{r}_j{x}_{\mathrm{0,}j}}=\tilde{x}$	${\displaystyle {\sum }_j{r}_j{x}_{\mathrm{0,}j}}>\tilde{x}$	${\displaystyle {\sum }_j{r}_j{x}_{\mathrm{0,}j}}<\tilde{x}$
			${\displaystyle {\sum }_j{r}_j}=0$	${\displaystyle {\sum }_j{r}_j}=1$	${\displaystyle {\sum }_j{r}_j}=2$	${\displaystyle {\sum }_j{r}_j}=3$
34,480	10,378	11,827	3,876	2,703	2,677	3,019

The sub-population and population totals based on imputation with adjustments are given in Table 4.2, in comparison with raw data totals and the census file totals. We notice the following. (a) The census file had been edited in a ‘traditional’ way that involves much clerical work (about 1.5 man-year in total). In contrast, the editing procedures here are fully automated, and everything (i.e., exploratory analysis, decision of the treatments, programming and processing) was done in less than two days. Although the questions concerning pasture areas are only 3 out of a total of 36 questions of the “main questionnaire”, it is obvious that the potential saving in time could be enormous. (b) The differences between the imputed totals and the census totals are small for all sub-populations, compared to those between the raw data and the census totals. All the changes from the raw data are in the ‘right’ direction, judged by the census results. One may conclude that the automated editing procedures have achieved most of the census editing results. (c) It is possible to introduce benchmark constraints in addition. An an illustration, we used the census file sub-population totals for the 3,876 unit-missing records, in addition to the known pasture area total for each of them. Convergence was reached in 23 iterations with the WLS criterion. (d) For the 5,380 units where partial missing may be the case, imputation of ‘missing’ values was carried for about 25% of them in the census processing, whereas it is about 75% by the editing procedure here. The number of cases for partial missing is probably under-estimated in the census file because it is based on selective manual checks. In any case, not withstanding the differences in the individual treatments, the edited totals are fairly close to each (Table 4.2, under $0<{\displaystyle {\sum }_j{r}_j}<3).$

4.2 Approximate mean squared error estimation

As the measure of uncertainty for the pasture area data here, we use the mean squared error of prediction (MSEP) given by

$${\text{MSEP}}_j=E\left\{{\left({\tilde{X}}_j-X_j\right)}^2|R_U\mathrm{,}{\tilde{X}}_U\right\}$$

where $X_j={\displaystyle {\sum }_{i\in U}{x}_{ij}}$ is the target population total and ${\tilde{X}}_j={\displaystyle {\sum }_{i\in U}{\tilde{x}}_{ij}}$ is the corresponding total based on imputation with adjustments, for $j=1,2,3.$ Moreover, ${\tilde{X}}_U={\left({\tilde{x}}_i\right)}_{i\in U}$ contains the known pasture area totals in the population, and $R_U$ is the matrix of missing indicators whose $i^{\text{th}}$ row is given by $\left(r_{i1}\mathrm{,}{r}_{i2}\mathrm{,}{r}_{i3}\right).$

Now, while it is customary that adjustments due to inconsistency in the micro data are referred to as imputation in statistical data editing, the eventual uncertainty associated with this is generally ‘ignored’ afterwards. This amounts to assume that ${\tilde{x}}_{ij}=x_{ij}$ if $r_{ij}=1.$ What remains to be accounted for is the uncertainty associated with the imputation of the missing values and the subsequent adjustment of the donor values, under the assumption that neither imputation nor adjustment introduces bias to the final value. This amounts to assume that $E\left({\tilde{x}}_{ij}-x_{ij}\right)=0$ if $r_{ij}=0.$ Under these two assumptions, we have

$$\begin{array}{lll}{\text{MSEP}}_j\hfill & =\hfill & E\left\{{\left({\displaystyle \sum _{i\in U}}\left(1-r_{ij}\right){\tilde{x}}_{ij}-{\displaystyle \sum _{i\in U}}\left(1-r_{ij}\right)x_{ij}\right)}^2\right\}\hfill \\ \hfill & =\hfill & V\left({\displaystyle \sum _{i\in U;r_i=1\mathrm{,}{d}_{ij}\ge 1}{d}_{ij}{\delta }_{ij}{x}_{ij}}\right)+V\left({\displaystyle \sum _{i\in U;r_{ij}=0}{x}_{ij}}\right)\hfill \\ \hfill & \approx \hfill & {\displaystyle \sum _{i\in U;r_i=1\mathrm{,}{d}_{ij}\ge 1}{d}_{ij}^{2}V}\left({\delta }_{ij}{x}_{ij}\right)+{\displaystyle \sum _{i\in U;r_{ij}=0}V}\left(x_{ij}\right)\hfill \end{array}$$

where $d_{ij}$ is the number of times $x_{ij}$ is used as a donor value for imputation of missing data, and the decomposition of variance holds provided the distributions of the units are independent of each other. Moreover, provided $d_{ij}\ge 1,$

$${\delta }_{ij}={\displaystyle \sum _{k\in U;x_{kj}^{\ast }=x_{ij}}{\tilde{x}}_{kj}}/\left(d_{ij}{x}_{ij}\right)$$

where $x_{kj}^{\ast }=x_{ij}$ means that $x_{ij}$ is used as the donor value for $x_{kj},$ and ${\tilde{x}}_{kj}$ is the final value after adjustment. In other words, ${\delta }_{ij}$ is the combined adjustment made to $d_{ij}{x}_{ij},$ where $d_{ij}{x}_{ij}$ would have been the contribution of $x_{ij}$ to ${\tilde{X}}_j$ through imputation if it had been donor imputation without adjustment. Notice that $d_{ij}$ can be treated as a constant in the last (approximate) equation as long as the donor identification depends only on $R_U$ and ${\tilde{X}}_U.$ This is true for the 3,876 unit-missing records, but not exactly for the 5,380 units that may have partial missing. As explained in Section 4.1, the NN-identification in fact also depends on the observed $x_{ij}-$values. For this reason, the last equation holds only approximately.

A ratio model for the conditional variance of $x_{ij}$ seems natural here, i.e.,

$$x_{ij}={\beta }_j{x}_i+{\epsilon }_{ij}\text{ where }E\left({\epsilon }_{ij}\right)=0\text{ and }V\left({\epsilon }_{ij}\right)={\sigma }_j^2{x}_i^{{\alpha }_j}$$

where $\left({\beta }_j\mathrm{,}{\sigma }_j^2\mathrm{,}{\alpha }_j\right)$ may vary according to the composition of the pasture areas, denoted by $q=\left(1,1,1\right),\left(1,1,0\right),\left(1,0,1\right)\text{ and }\left(0,1,1\right),$ where $q_{ij}=1$ if unit $i$ has the $j^{\text{th}}$ type pasture and $0$ otherwise. Notice that, in the case of ${\displaystyle {\sum }_j{q}_{ij}}=1,$ we have $x_{ij}=\tilde{x}$ if $q_{ij}=1,$ so that the conditional variance is zero. The parameters of this ratio model can be estimated from the 10,378 potential donors satisfying ${\displaystyle {\sum }_j{r}_j{x}_{\mathrm{0,}j}}=\tilde{x}.$ Exploratory data analysis shows that ${\alpha }_j=2$ is a reasonable choice in all the cases, so that in the calculations below only ${\beta }_j$ and ${\sigma }_j^2$ vary according to the observation pattern, denote by $\left({\beta }_{j;h}\mathrm{,}{\sigma }_{j;h}^2\right)$ for $h=1,\mathrm{...},4.$ Notice that, as a result of ${\alpha }_j\equiv 2,$ the same ${\widehat{\sigma }}_{j;h}^2$ will be obtained regardless of $j$ whenever ${\displaystyle {\sum }_j{q}_{ij}}=2.$ Take e.g., $q={\left(\mathrm{1,1,0}\right)}^T,$ we have ${\widehat{\beta }}_1+{\widehat{\beta }}_2=1,$ such that the ‘standardized’ fitted residuals are given by ${\widehat{\epsilon }}_{i1}/{\tilde{x}}_i=x_{i1}/{\tilde{x}}_i-{\widehat{\beta }}_1$ and ${\widehat{\epsilon }}_{i2}/{\tilde{x}}_i=x_{i2}/{\tilde{x}}_i-{\widehat{\beta }}_2=$  $\left({\tilde{x}}_i-x_{i1}\right)/{\tilde{x}}_i-\left(1-{\widehat{\beta }}_1\right)=-{\widehat{\epsilon }}_{i1}/{\tilde{x}}_i.$ In any case, we obtain ${\widehat{V}}_h\left(x_{ij}\right)={\widehat{\sigma }}_{j;h}^2{\tilde{x}}_i^2$ for unit $i$ with composition $h.$

The adjustment factor ${\delta }_{ij}$ seems difficult to model in advance. But its mean and variance can be estimated empirically after imputation and adjustment have been carried out, denoted by ${\mu }_{\delta }=E\left({\delta }_{ij}\right)$ and ${\sigma }_{\delta }^2=V\left({\delta }_{ij}\right),$ respectively. Moreover, we assume ${\delta }_{ij}$ to be independent of $x_{ij}$ conditional on ${\tilde{x}}_i.$ This seems a plausible assumption, since the former depends mostly on how $x$ is distributed in the ‘neighbourhood’ of $x=\tilde{x},$ whereas the latter depends on the variation across $j$ given that the sum is equal to $\tilde{x}.$ For instance, asymptotically as the chance of finding a donor in any arbitrarily close neighbourhood tends to unity, the adjustment factor ${\delta }_{ij}$ tends to 1 in probability, irrespective of the values of $x_{ij}.$ It now follows that, given composition $h,$ an estimate of the corresponding $V_h\left({\delta }_{ij}{x}_{ij}\right)$ is given by

$${\widehat{V}}_h\left({\delta }_{ij}{x}_{ij}\right)={\widehat{\sigma }}_{j;h}^2{\tilde{x}}_i^2{\widehat{\sigma }}_{\delta }^2+{\left({\widehat{\beta }}_{j;h}{\tilde{x}}_i\right)}^2{\widehat{\sigma }}_{\delta }^2+{\widehat{\sigma }}_{j;h}^2{\tilde{x}}_i^2{\widehat{\mu }}_{\delta }^2.$$

Finally, combining all the above, we obtain an approximate MSEP estimate as

$${\widehat{\text{MSEP}}}_j\approx {\displaystyle \sum _h}{\displaystyle \sum _{i\in U_h;r_i=1}}{d}_{ij}^2{\widehat{V}}_h\left({\delta }_{ij}{x}_{ij}\right)+{\displaystyle \sum _h{\displaystyle \sum _{i\in U_h;r_{ij}=0}{\widehat{V}}_h\left(x_{ij}\right)}}.$$

The results of approximate variance estimation are given in Table 4.3. We know in advance that the regression coefficient of the ratio model must vary according to the composition of pasture area, but the estimates of ${\sigma }_{j;h}^2$ suggest that it has been sensible to allow the variance parameter to depend on $h.$ The estimated mean of ${\delta }_{ij}$ is close to unity for all the pasture area types, making no indications that the assumptions regarding the adjustment factors are unreasonable. The variance of ${\delta }_{ij}$ is clearly the largest for $j=2,$ which is also reflected in the fact that the estimated MSEP here has the largest increase compared to NN-imputation without adjustment. The relative root MSEPs are too small to account for the actual differences between the census totals and the imputed totals (given in Table 4.2). This serves to illustrate the following general impression regarding the assessment of uncertainty due to editing. Systematic effects in terms of the first-order moments of the resulting statistics usually dominate the overall uncertainty due to editing. But they are also more difficult to quantify compared to the second-order variance properties. In the case here, this concerns the two ‘first-order’ assumptions made in the beginning, i.e., ${\tilde{x}}_{ij}=x_{ij}$ if $r_{ij}=1$ and $E\left({\tilde{x}}_{ij}-x_{ij}\right)=0$ if $r_{ij}=0.$ More sophisticated assumptions about the error-mechanism of consistency adjustments in editing are needed in order to progress beyond such an ‘optimistic’ approach.

Table 4.3
Approximate variance estimation for imputation with adjustment. RMSEP: Root MSEP. RMSEP by NN-imputation without adjustment in parentheses
Table summary
This table displays the results of Approximate variance estimation for imputation with adjustment. RMSEP: Root MSEP. RMSEP by NN-imputation without adjustment in parentheses x and x (appearing as column headers).

		$j=1$	$j=2$	$j=3$
${\widehat{\beta }}_j$	$q=\left(\mathrm{1,1,1}\right)$	0.312	0.359	0.329
	$q=\left(\mathrm{1,1,0}\right)$	0.346	0.654	-
	$q=\left(\mathrm{1,0,1}\right)$	0.407	-	0.593
	$q=\left(\mathrm{0,1,1}\right)$	-	0.567	0.433
${\widehat{\sigma }}_j^2$	$q=\left(\mathrm{1,1,1}\right)$	0.0248	0.0511	0.0364
	$q=\left(\mathrm{1,1,0}\right)$	0.0478	0.0478	-
	$q=\left(\mathrm{1,0,1}\right)$	0.0464	-	0.0464
	$q=\left(\mathrm{0,1,1}\right)$	-	0.0798	0.0798
	$\left({\widehat{\mu }}_{\delta },{\widehat{\sigma }}_{\delta }^2\right)$	(0.992, 0.0248)	(1.020, 0.0994)	(1.003, 0.0236)
	$\widehat{\text{RMSEP}}$	3,267 (3,134)	4,190 (3,530)	3,111 (2,925)
	$\widehat{\text{RMSEP}}/{\displaystyle {\sum }_{i;r_{ij}=0}{\tilde{x}}_{ij}}$	1.41%	1.79%	0.93%
	$\widehat{\text{RMSEP}}/{\tilde{X}}_j$	0.24%	0.34%	0.15%

Previous | Next