6. Empirical application

Andrés Gutiérrez, Leonardo Trujillo and Pedro Luis do Nascimento Silva

Previous | Next

We first consider an empirical approach in this section, through simulations that will let us assess some statistical properties such as unbiasedness and efficiency of the proposed estimators. Following the modeling proposed by Stasny (1987), we considered a two-stage simulation as follows:

• Allocation of all the individuals in the population to the different cells of a contingency table. In this first stage, we will define the initial probabilities ${\eta }_i,p_{ij}$ and,
• Nonresponse process at two consecutive periods. In this second stage, we will define the initial probabilities $\psi ,{\rho }_{RR}$ and ${\rho }_{MM}$ .

In the first stage, it was necessary to assume some conditions (non observable process) where the group classification probabilities were established at time $t-1$ and the conditional classification probabilities at time $t$ . In this way, every individual in the population was assumed to be classified in any of three categories: E1, E2 and E3. The state vector at time $t$ was given by

$$\eta ={\left({\eta }_1\mathrm{,}{\eta }_2\mathrm{,}{\eta }_3\right)}^{\prime }={\left(\mathrm{0.9,0.05,0.05}\right)}^{\prime }.$$

In this way, there is a classification probability in E1 equals to 0.9 for any individual in the population and classification probabilities in E2 and E3 equal to 0.05. The transition matrix from time $t-1$ to time $t$ is given by

$$P=\left(\begin{array}{c}{p^{\prime }}_1\\ {p^{\prime }}_2\\ {p^{\prime }}_3\end{array}\right)=\left(\begin{array}{ccc}{p}_{11}& p_{12}& p_{13}\\ p_{21}& p_{22}& p_{23}\\ p_{31}& p_{32}& p_{33}\end{array}\right)=\left(\begin{array}{ccc}0.80& 0.15& 0.05\\ 0.30& 0.60& 0.10\\ 0.10& 0.10& 0.80\end{array}\right).$$

We assumed that the population size was $N=100,000$ and its size would not change at the two periods of evaluation. In order to classify the individuals at the periods of time we used the R function rmultinom (R Development Core Team 2012). This way, the distribution of gross flows according to equation (3.2) would be given by the values in Table 6.1.

Table 6.1
Expected values under the model $\xi$ for the population gross flows at two consecutive periods.
Table summary
This table displays the results of Expected values under the model $\xi$ for the population gross flows at two consecutive periods.. The information is grouped by Time $t-1$ (appearing as row headers), Time $t$ (appearing as column headers).

Time $t-1$	Time $t$
	E1	E2	E3
E1	72,000	13,500	4,500
E2	1,500	3,000	500
E3	500	500	4,000

6.1 Methodology

We considered for this empirical exercise, a Monte-Carlo with $L=1,000$ simulations. In order to classify the individuals between respondents and nonrespondents at the two periods of time, we used the function rmultinom from the language R. Dichotomic variables $y_{1ik},y_{2jk},z_{1k}$ and $z_{2k}$ were created using the function Domains from the library TeachingSampling (Gutiérrez 2009).

For each run of the simulation, a sample of size $n=10,000$ was drawn. We considered a simple random sampling design (SI) along with a complex sampling design inducing unequal inclusion probabilities ( $\pi$ PS). The behavior of the different proposed estimators will be assessed according to their relative bias and relative root mean square error, given by

$$RB=L^{-1}{\displaystyle \sum _{l=1}^L}\frac{{\widehat{\theta }}_l-\theta }{\theta }\text{ and }RRMSE=\frac{\sqrt{L^{-1}{\displaystyle {\sum }_{l=1}^L{\left({\widehat{\theta }}_l-\theta \right)}^2}}}{\theta }.$$

respectively. In those situations where the vector of inclusion probabilities was unequal, function S.piPS on the library TeachingSampling was used in order to choose a without replacement sample with inclusion probabilities proportional to an auxiliary characteristic assumed known and following a normal distribution with different parameters. The proposed methodology is compared with two other estimators: an estimator taking into account the functional shape of the model but not taking into account the sampling design and a gross flows estimator not taking into account the sampling design but assuming that the nonresponse is ignorable.

The first estimator, that we call the Design-based estimator, corresponds to the expressions at results 4.2, 4.3 and 4.4. The second estimator, that we shall call as Model-based estimator, correspond to the expressions at result 5.1, being maximum likelihood estimators not considering the sampling weights. Finally, the third estimator, that we call the Naive estimator estimator expands the sampling information to the population and is given by

$${\widehat{\mu }}_{ij\mathrm{,}ING}=\frac{N}{{\displaystyle {\sum }_i{\displaystyle {\sum }_j{N}_{ij}}}}{N}_{ij}.$$

The response probability at time $t-1$ was assumed as $\psi =\mathrm{0.8.}$ The response probability at time $t$ for those individuals responding at time $t-1$ was assumed as ${\rho }_{RR}=\mathrm{0.9.}$ Finally, the nonresponse probability at time $t$ for those individuals not responding at time $t-1$ was assumed as ${\rho }_{MM}=\mathrm{0.7.}$

Based on model $\xi ,$ the expected values of the responses are given in Table 6.2.

Table 6.2
Expected values under the model $\xi$ for the response at two consecutive periods.
Table summary
This table displays the results of Expected values under the model $\xi$ for the response at two consecutive periods.. The information is grouped by Time $t-1$ (appearing as row headers), Time $t$ (appearing as column headers).

Time $t-1$	Time $t$
	Response	Nonresponse
Response	72,000	8,000
Nonresponse	6,000	14,000

Taking into account the dynamics of the respondents in both periods and assuming that is possible to collect all the population information through a census, we get the classifications given in Table 6.3 below.

Table 6.3
Expected values under the model $\xi$ for the population gross flows (observable process) at two consecutive periods.
Table summary
This table displays the results of Expected values under the model $\xi$ for the population gross flows (observable process) at two consecutive periods.. The information is grouped by Time $t-1$ (appearing as row headers), Time $t$ (appearing as column headers).

Time $t-1$	Time $t$
	E1	E2	E3	Row complement
E1	51,840	9,720	3,240	7,200
E2	1,080	2,160	360	400
E3	360	360	2,880	400
Column complement	4,440	1,020	540	14,000

6.2 Results

6.2.1 Simple random sampling: design-based and model-based estimator

In a first empirical approach, we considered a simple random sampling without replacement as the sampling design. This sampling design induces uniform inclusion probabilities and expansion factors. Under this scenario, the design-based and model-based estimators are the same. Under this scenario the approach shows some strength according to the values of the relative biases that can be considered as negligible. This can be appreciated in Tables 6.4, 6.5 and 6.6.

Table 6.4
Relative biases in percentage and relative root mean square errors in percentage (shown in brackets) of the proposed estimator for the population gross flows.
Table summary
This table displays the results of Relative biases in percentage and relative root mean square errors in percentage (shown in brackets) of the proposed estimator for the population gross flows.. The information is grouped by Time $t-1$ (appearing as row headers), Time $t$ (appearing as column headers).

Time $t-1$	Time $t$
	E1	E2	E3
E1	0.24 (0.094)	-0.35 (0.189)	-0.49 (0.474)
E2	-2.89 (0.158)	-1.89 (0.221)	2.00 (0.980)
E3	-0.63 (0.790)	4.54 (0.822)	-0.84 (0.569)

 

Table 6.5
Relative biases in percentage and relative root mean square errors in percentage (shown in brackets) of the transition probabilities $p_{ij}$ .
Table summary
This table displays the results of Relative biases in percentage and relative root mean square errors in percentage (shown in brackets) of the transition probabilities $p_{ij}$ . The information is grouped by Time $t-1$ (appearing as row headers), Time $t$ (appearing as column headers).

Time $t-1$	Time $t$
	E1	E2	E3
E1	0.13 (0.284)	-0.39 (0.537)	-1.00 (3.225)
E2	1.70 (1.296)	-2.29 (0.569)	8.64 (0.347)
E3	-6.6 (3.415)	2.09 (1.992)	0.56 (0.158)

 

Table 6.6
Relative biases in percentage and relative root mean square errors in percentage (shown in brackets) of the initial classification probabilities ${\eta }_i$ .
Table summary
This table displays the results of Relative biases in percentage and relative root mean square errors in percentage (shown in brackets) of the initial classification probabilities ${\eta }_i$ . The information is grouped by Time $t-1$ (appearing as column headers).

Time $t-1$
${\eta }_1$	${\eta }_2$	${\eta }_3$
-0.01 (0.094)	-1.42 (0.980)	1.74 (0.790)

Also, the relative bias in percentage for the response probability $\psi$ was -0.23 and the relative root mean square error in percentage was 0.221; for the response probability ${\rho }_{RR},$ the bias in percentage was 0.055 and the relative root mean square error in percentage was 0.031; for the nonresponse probability ${\rho }_{MM},$ the bias in percentage was -0.192 and and the relative root mean square error in percentage was 0.189. On the other hand, Table 6.7 shows the empirical expected value of the gross flows for the proposed estimator and it can be appreciated that the values are very close to those given on Table 6.1.

Table 6.7
Empirical expected values for the proposed estimator for the population gross flows.
Table summary
This table displays the results of Empirical expected values for the proposed estimator for the population gross flows.. The information is grouped by Time $t-1$ (appearing as row headers), Time $t$ (appearing as column headers).

Time $t-1$	Time $t$
	E1	E2	E3
E1	72,085	13,444	4,454
E2	1,504	2,889	535
E3	474	519	4,092

6.2.2 Simple random sampling: naive estimator

Under this scenario and considering that this estimator does not take into account the nonresponse process, the values of the relative biases cannot be considered as negligible. This can be appreciated in Table 6.8.

Table 6.8
Relative biases in percentage and relative root mean square errors in percentage (shown in brackets) of the naive estimator for the population gross flows.
Table summary
This table displays the results of Relative biases in percentage and relative root mean square errors in percentage (shown in brackets) of the naive estimator for the population gross flows.. The information is grouped by Time $t-1$ (appearing as row headers), Time $t$ (appearing as column headers).

Time $t-1$	Time $t$
	E1	E2	E3
E1	-1.21 (4.4)	10.2 (60.7)	8.34 (25.2)
E2	-0.25 (38.8)	-7.51 (30.9)	1.33 (12.6)
E3	13.7 (43.3)	-8.54 (46.1)	0.92 (6.9)

Table 6.9 shows the empirical expected values for the naive estimator; compared to the expected values for the model given in Table 6.1 these are not even close.

Table 6.9
Empirical expected values for the naive estimator for the population gross flows.
Table summary
This table displays the results of Empirical expected values for the naive estimator for the population gross flows.. The information is grouped by Time $t-1$ (appearing as row headers), Time $t$ (appearing as column headers).

Time $t-1$	Time $t$
	E1	E2	E3
E1	54,628	760	4,507
E2	1,506	2,079	1,175
E3	1,603	905	32,832

6.2.3 Unequal inclusion probabilities: design-based estimator

In a third scenario, we considered a sampling design that induces unequal inclusion probabilities and expansion factors. Under this scenario, the proposed estimators are still unbiased both for the gross flows and for the parameters of the model. The relative biases are shown on Tables 6.10, 6.11 and 6.12.

Table 6.10
Relative biases in percentage and relative root mean square errors in percentage (shown in brackets) of the proposed estimator for the population gross flows.
Table summary
This table displays the results of Relative biases in percentage and relative root mean square errors in percentage (shown in brackets) of the proposed estimator for the population gross flows.. The information is grouped by Time $t-1$ (appearing as row headers), Time $t$ (appearing as column headers).

Time $t-1$	Time $t$
	E1	E2	E3
E1	-0.09 (0.8)	0.25 (3.6)	3.17 (7.9)
E2	0.72 (40.9)	-1.21 (27.2)	-4.62 (71.08)
E3	1.76 (20.4)	-3.19 (22.6)	-0.73 (7.2)

 

Table 6.11
Relative biases in percentage and relative root mean square errors in percentage (shown in brackets) of the transition probabilities $p_{ij}$ .
Table summary
This table displays the results of Relative biases in percentage and relative root mean square errors in percentage (shown in brackets) of the transition probabilities $p_{ij}$ . The information is grouped by Time $t-1$ (appearing as row headers), Time $t$ (appearing as column headers).

Time $t-1$	Time $t$
	E1	E2	E3
E1	-0.05 (0.7)	0.115 (3.6)	0.47 (7.1)
E2	2.39 (36.0)	-0.13 (18.6)	-6.40 (69.1)
E3	1.15 (24.9)	-5.14 (21.7)	0.49 (3.7)

 

Table 6.12
Relative biases in percentage and relative root mean square errors in percentage (shown in brackets) of the initial classification probabilities ${\eta }_i$.
Table summary
This table displays the results of Relative biases in percentage and relative root mean square errors in percentage (shown in brackets) of the initial classification probabilities ${\eta }_i$ . The information is grouped by Time $t-1$ (appearing as column headers).

Time $t-1$
${\eta }_1$	${\eta }_2$	${\eta }_3$
-0.02 (1.1)	-0.70 (19.8)	1.13 (6.9)

For the probability of response $\psi$ , the bias in percentage was -0.46 and the relative root mean square error in percentage was 0.6; for the probability of response ${\rho }_{RR},$ the bias in percentage was -0.21 and the relative root mean square error in percentage was 0.6; for the probability of nonresponse ${\rho }_{MM},$ the bias in percentage was 0.99 and the relative root mean square error in percentage was 1.8. On the other hand, Table 6.13 shows the empirical expected values of the proposed estimator for the population gross flows and these are very close to the values given in Table 6.1.

Table 6.13
Empirical expected values of the design-based estimator for the population gross flows.
Table summary
This table displays the results of Empirical expected values of the design-based estimator for the population gross flows.. The information is grouped by Time $t-1$ (appearing as row headers), Time $t$ (appearing as column headers).

Time $t-1$	Time $t$
	E1	E2	E3
E1	71,910	13,505	4,518
E2	1,523	2,972	470
E3	511	479	4,062

6.2.4 Unequal inclusion probabilities: model-based estimator

A fourth scenario considers a sampling design inducing unequal inclusion probabilities and expansion factors in the same way as the last scenario. However, we consider estimators not taking into account the sampling design only the model $\xi .$ Under this scenario, estimations are biased for both the gross flows and the model parameters as can be appreciated by the relative biases on Tables 6.14, 6.15 and 6.16.

Table 6.14
Relative biases in percentage and relative root mean square errors in percentage (shown in brackets) of the model-based estimator for the population gross flows.
Table summary
This table displays the results of Relative biases in percentage and relative root mean square errors in percentage (shown in brackets) of the model-based estimator for the population gross flows.. The information is grouped by Time $t-1$ (appearing as row headers), Time $t$ (appearing as column headers).

Time $t-1$	Time $t$
	E1	E2	E3
E1	4.7 (6.1)	4.6 (8.9)	6.3 (10.5)
E2	-89.0 (126.6)	-89.5 (125.9)	-88.4 (126.9)
E3	4.1 (23.8)	-3.7 (26.67)	5.3 (10.4)

 

Table 6.15
Relative biases in percentage and relative root mean square errors in percentage (shown in brackets) of the transition probabilities $p_{ij}$ .
Table summary
This table displays the results of Relative biases in percentage and relative root mean square errors in percentage (shown in brackets) of the transition probabilities $p_{ij}$ . The information is grouped by Time $t-1$ (appearing as row headers), Time $t$ (appearing as column headers).

Time $t-1$	Time $t$
	E1	E2	E3
E1	0.03 (0.9)	-0.71 (4.1)	1.63 (8.6)
E2	2.77 (35.5)	-1.50 (19.6)	0.70 (70.6)
E3	4.00 (20.8)	-14.6 (20.1)	1.33 (3.41)

 

Table 6.16
Relative biases in percentage and relative root mean square errors in percentage (shown in brackets) of the initial probabilities of classification ${\eta }_i$.
Table summary
This table displays the results of Relative biases in percentage and relative root mean square errors in percentage (shown in brackets) of the initial probabilities of classifications ${\eta }_i$ . The information is grouped by Time $t-1$ (appearing as column headers).

Time $t-1$
${\eta }_1$	${\eta }_2$	${\eta }_3$
4.74 (6.48)	-89.3 (126.7)	3.95 (11.9)

In the same way, for the response probability $\psi$ , the relative bias in percentage was -0.77 and the relative root mean square error was 1.7; for the response probability ${\rho }_{RR},$ the relative bias in percentage was -0.53 and the relative root mean square error was 0.5; for the nonresponse probability ${\rho }_{MM},$ the relative bias in percentage was 0.11 and the relative root mean square error was 1.8. On the other hand, Table 6.17 shows the empirical expected values for the model-based estimator for the population gross flows (not considering the sampling design) and these are quite far from the values in Table 6.1, especially for the second category.

Table 6.17
Empirical expected values for the model-based estimator for the population gross flows.
Table summary
This table displays the results of Empirical expected values for the model-based estimator for the population gross flows.. The information is grouped by Time $t-1$ (appearing as row headers), Time $t$ (appearing as column headers).

Time $t-1$	Time $t$
	E1	E2	E3
E1	75,438	14,039	4,790
E2	164	315	53
E3	540	443	4,213

6.2.5 Unequal inclusion probabilities: naive estimator

In a fifth scenario, we consider a sampling design with unequal inclusion probabilities and expansion factors. Considering the naive estimator, that does not take into account the sampling design nor the respondent model, Table 6.18 shows the relative bias for each cell in the matrix of gross flows. This estimator would be only recommendable if the nonresponse was ignorable and the sampling design would correspond to a simple random sampling design.

Table 6.18
Relative biases in percentage and relative root mean square errors in percentage (shown in brackets) for the naive estimator of the population gross flows.
Table summary
This table displays the results of Relative biases in percentage and relative root mean square errors in percentage (shown in brackets) for the naive estimator of the population gross flows.. The information is grouped by Time $t-1$ (appearing as row headers), Time $t$ (appearing as column headers).

Time $t-1$	Time $t$
	E1	E2	E3
E1	-28.1 (34.7)	-27.6 (60.0)	-24.5 (41.1)
E2	497.0 (629.2)	570.2 (610.3)	432.7 (686.2)
E3	-40.5 (44.2)	-37.0 (47.4)	-33.0 (33.8)

In order to have a more accurate comparison, it would be possible to calculate the expected values of the gross flows and compare them with the current scenario. Table 6.19 shows the empirical expected values for the naive estimator; compared to the expected values for the model given in Table 6.1 these are not very close and are especially poor for the classifications in the second category.

Table 6.19
Empirical expected values for the naive estimator of the population gross flows.
Table summary
This table displays the results of Empirical expected values for the naive estimator of the population gross flows.. The information is grouped by Time $t-1$ (appearing as row headers), Time $t$ (appearing as column headers).

Time $t-1$	Time $t$
	E1	E2	E3
E1	51,755	9,849	3,297
E2	9,194	19,838	2,823
E3	279	295	2,665

Previous | Next