7 Examples

Jeremy Strief and Glen Meeden

Previous | Next

We believe that standard design based theory over emphasizes the role that the selection probabilities should play in making inferences after the sample has been observed. In this section we consider examples that show how the WDP can make use of objective prior information after the sample has been selected.

7.1 A simulation study

To further understand how using the stepwise Bayes weights in the WDP can work we did a simulation study. We constructed a population with 2,000 units and a single auxiliary variable, $x.$ This variable was a random sample from a gamma distribution with shape parameter 5 and scale parameter 1. The conditional distribution of $y_i$ given $x_i$ was normal with mean $100+{\left(x_i-8\right)}^2$ and standard deviation 20. The correlation for the resulting population was -0.38. We denote this population by quad. Clearly this is a toy example and the particular form of the relationship between $x$ and $y$ is not important to the WDP methods beyond the fact that $x$ does contain some information about $y.$ In what follows we will compare WDP estimators to two standard methods under four different sampling plans.

To construct the CPP we assumed that the $x$ values for the population are known and we use them to construct three strata after the sample has been observed. These strata will not be constructed in the usual way. We did this to underplay the usual role of the design and to emphasize the robustness of our approach against the choice of design. We will have a sample size of $n=60$ and we will construct three post-strata. Let $x_{\left[1\right]}<x_{\left[2\right]}<\dots <x_{\left[60\right]}$ be the order statistic of the $x$ values in the sample. Let $q_{20}$ and $q_{40}$ be the population quantiles of $x_{\left[20\right]}$ and $x_{\left[40\right]}$ respectively. Then the CPP assumes that the total probability assigned to the units in the sample with the 20 smallest $x$ values must be $q_{20}$ and the total probability assigned to the next 20 smallest must be $q_{40}-q_{20}.$ In other words we break the sample into three equal groups and use the information in the $x$ values to get the appropriate population size of the corresponding strata. In addition the CPP assumes that the probabilities assigned to the sample must satisfy the population mean constraint for $x.$

The resulting WDP will be compared to two standard frequentist methods. The first is the post-stratified estimator which makes use of the same strata information as the CPP. The second is the usual regression estimator which assumes that the population mean of $x$is known. Although the regression estimator is not really appropriate for population quad it is included as a comparison. When computing 95% confidence intervals for the population total both frequentist methods will assume simple random sampling even when different sampling designs were used. We will denote these two estimators by STR and REG respectively.

The first sampling design was simple random sampling without replacement. For the second we generated a set of sampling weights by taking a random sample of 2,000 from a gamma distribution with shape parameter 5 and scale parameter 1. We then added 5 to each value to get the vector, $v$ say. Note the values of $v$ and $y$ are completely independent. We then used approximate $\text{pps}\left(v\right)$ where at each step the probability that a unit is selected is proportional to its $v$ value and depends only the unselected units remaining in the population. We call this the Random Weights design. For the third design we used approximated $\text{pps}\left(x\right).$ For the fourth we found the linear function, say $l,$ which maps the range of $y$ onto the the interval $\left[1,2\right].$ We then used approximate $\text{pps}\left(l\left(y\right)\right)$ as the sampling design. We call this the $y$ Dependent design. In this design the selection probabilities depend weakly on the $y$ values and units with large $y$ values are more likely to be selected than those with small values of $y.$ In particular the unit with the largest $y$ value is twice as likely to be selected as the unit with the smallest $y$ value. Clearly the Random Weights design and the $y$ Dependent design are not standard designs and would never be used in practice. They were included to emphasize our belief that in many cases given a sample a good estimate does not depend on how the sample was selected.

For each design we took 500 samples of size 60 and computed the point estimate, its absolute error, the length of its interval estimate and whether or not it contained the true parameter value. The results are given in Table 7.1.

Table 7.1
Simulation results for population quad discussed in section 7.1 for 500 random samples of size 60 for four different sampling plans. The true population total was 227,923.0. The nominal coverage for each method is 0.95.
Table summary
This table displays the results of simulation results for population quad discussed in section 7.1 for 500 random samples of size 60 for four different sampling plans. the true population total was 227. The information is grouped by method (appearing as row headers), ave. value, ave. err, ave. len and freq of coverage (appearing as column headers).

Method	Ave. value	Ave. err	Ave. len	Freq of coverage
SRS
STR	227,856.1	4,165.0	21,332.1	0.950
REG	227,602.1	4,302.7	21,300.3	0.944
WDPNote 1	227,546.9	4,190.6	23,029.7	0.958
Random Weights
STR	227,976.5	4,371.2	21,254.1	0.938
REG	227,715.5	4,462.2	21,305.9	0.934
WDPNote 2	227,721.2	4,420.6	22,901.4	0.950
pps(x)
STR	225,295.8	5,228.9	23,008.4	0.916
REG	224,207.2	5,611.2	21,780.3	0.878
WDPNote 3	227,471.1	4,919.2	22,706.6	0.936
y Dependent
STR	231,590.0	5,229.0	21,170.8	0.892
REG	231,424.4	5,143.4	21,127.9	0.902
WDPNote 4	231,139.1	4,967.6	22,867.0	0.938
Note 1. Ave. min and max of WDP parameters were 0.658 and 1.580. Note 2. Ave. min and max of WDP parameters were 0.651 and 1.583. Note 3. Ave. min and max of WDP parameters were 0.374 and 3.024. Note 4. Ave. min and max of WDP parameters were 0.660 and 1.643.

Remember that in this example the WDP is using information from both the post-stratification and knowing the population mean of $x$ while STR just uses the first and REG just uses the second. Under SRS and the Random Weights design all four methods preform about the same. For the other two designs WDP does the best. Over all four designs its frequency of coverage is closest to the nominal level of 0.95. Using the constraint involving the population mean of $x$ allows it to correct for some of the bias introduced by the sampling plans that STR cannot do. However this constraint can only do so much. If in the $y$ dependent design the range of $l$ was $\left[1,4\right]$ then WDP's average absolute error is 4.5% better then that of STR and the frequency of coverage on the 0.95 nominal intervals were 0.86 and 0.80 respectively. There is just not enough information in $x$ to correct for this much selection bias.

For each design we have included the average of the smallest and largest values of the parameter values defining the WDP which in this case must sum to 60. We see the range is largest for $\text{pps}(x).$

In the simulations we also used the WDP to construct 0.95 credible intervals for the population median of $y.$ For the four designs its respective frequency of coverage was 0.956, 0.950, 0.952 and 0.930.

We did another simulation study where $x$ was generated in the same way but now the conditional distribution of $y_i$ given $x_i$ was normal with $60+x_i$ and standard deviation $2\sqrt{x_i}.$ The correlation between $x$ and $y$ was 0.46. Under all four designs the performances of the point estimators were very similar. The WDP intervals tended to be a bit longer than the rest but over the four designs its average frequency of coverage for the population total was 0.949. Under the $y$ Dependent design its frequency of coverage for the population total was 0.934 while for STR and REG the corresponding coverages were 0.896 and 0.886. Its average frequency of coverage for the population median of $y$ was 0.942.

A frequentist could argue that this is an unfair example since the regression estimator does not make much sense for this population and of course they would be right. If for this problem you assumed a quadratic relationship between $y$ and $x$ and if you assumed that the first two population moments of $x$ were known then the resulting regression estimator would out perform the WDP. In Lazar et al. (2008) there is such an example. Moreover, they show that including a constraint for the second moment of the CPP will hardly change the behavior of the resulting estimates. Hence, when there is good prior information about the model relating $x$ and $y$ this should be used in the analysis. When such prior information is not available we believe the WDP does have certain advantages even though it may not yield dramatic improvements over standard methods. It uses only objective prior information and makes no model assumptions about how the characteristics of interest and the auxiliary variables are related. It can correct for a slight dependency of the selection probabilities on the characteristic of interest. Although the sampling design plays no explicit role in its calculation, information which is often incorporated in the design can be reformulated as a constraint and be used when defining the CPP. Given a sample, inferences based on the WDP use many simulated complete copies of the population which on the average are consistent with the prior information. This makes makes it straightforward to estimate parameters other than a population mean or total.

7.2 Stratification and estimating the median

In many applications only a few observations, sometimes only two, are taken from each stratum. For such problems finding a good confidence interval when estimating the population median can be difficult. Next we will compare the standard method, see for example section 5.11 of Särndal et al. (1992), with the WDP. We will assume simple random sampling without replacement within strata.

For definiteness, assume we have $L$ strata and stratum $j$ contains $N_j$ units. Let $N={\displaystyle {\sum }_{j=1}^L{N}_j}$be the total size of the population. Assume that two observations are taken from each stratum. Then the weight assigned to each sampled unit is one-half of the stratum size from which it was selected. The standard method uses these weights to find its confidence interval.

For this scenario the usual Polya posterior is applied within each stratum, independently across strata. Alternatively, this can be thought of as a CPP where the amount of probability assigned to the two sampled units in stratum $j$ must sum to $N_j/N.$ If $p_j=(p_{j,1},p_{j,2})$ represents the probability assigned to the two sampled units from stratum $j$ then under the CPP $E(p_j)=\left(N_j/\left(2N\right),N_j/\left(2N\right)\right).$ Recalling the notation from Section 5 we see that under the WDP the weight assigned to each of the two sampled units in stratum $j$ is $\left(L{N}_j\right)/N.$Recall that simulating complete copies of the population using the WDP means that individual simulated copies will almost certainly not satisfy the constraints however the constraints will be satisfied when we average over all simulated copies. At first glance this might seem like a bad idea but we will see that when estimating the population median interval estimates based on the WDP behave better than the standard intervals which are too short. We shall see that the extra variability present in the WDP yields longer intervals with better frequentist properties.

The stratified populations we considered were constructed as follows. The strata sizes were a random sample from a Poisson distribution with parameter $\lambda =100.$ The strata means were a random sample from a normal population with the mean $\mu =150$ and with either a standard deviation of $\sigma =10$ or $\sigma =20.$ The strata standard deviations were a random sample from a gamma distribution with scale parameter one and shape parameter $\gamma \sigma$ with either $\gamma =0.10$ or $\gamma =\mathrm{0.25.}$We constructed two versions of each of the four types, one with 20 strata and the other with 40 strata. For each of the eight populations we took 500 samples where each sample consisted of two observations selected at random without replacement from each stratum. For each sample we compared the standard approach with estimates based on the WDP. The results can be found in Table 7.2. We only present the results for the 20 strata populations because the results for the 40 strata population are similar. Both methods are approximately unbiased and the point estimate based on the WDP seems to do just a bit better. But the confidence intervals produced by WDP are clearly superior. Even though in one case the WDP intervals are clearly too long its overall performance is much better than the standard intervals.

Table 7.2
Simulation results from 500 stratified random samples of size two within each strata from populations with 20 strata. The nominal coverage for each method is 0.95.
Table summary
This table displays the results of simulation results from 500 stratified random samples of size two within each strata from populations with 20 strata. the nominal coverage for each method is 0.95.. The information is grouped by method (appearing as row headers), ave. value, ave. err, ave. len and freq of coverage (appearing as column headers).

Method	Ave. value	Ave. err	Ave. len	Freq of coverage
$\sigma$ = 10 and $\gamma$ = 0.10
Stand	148.40	2.37	8.30	0.808
WDD	148.39	2.22	12.20	0.95
$\sigma$ = 10 and $\gamma$ = 0.25
Stand	144.28	5.70	20.59	0.834
WDD	144.18	5.41	28.38	0.950
$\sigma$ = 20 and $\gamma$ = 0.10
Stand	152.75	3.02	10.52	0.828
WDD	152.61	2.78	22.88	0.996
$\sigma$ = 20 and $\gamma$ = 0.25
Stand	155.94	6.72	23.17	0.826
WDD	155.89	6.35	34.96	0.962

What causes the poor performance of the WDP intervals in the one case? Additional simulations indicate that when the strata means vary widely and the strata variances tend to be relatively small then the WDP intervals will tend to be too long. In our simulations the case with $\sigma =20$ and $\gamma =0.10$ leads to a population with such strata. When the sample size was increased to four units per stratum the difference between the two methods is not so dramatic but the story remains much the same. The standard intervals tend to be to short and under cover while the WDP intervals are longer and tend to over cover.

Clearly the choice of a good method for constructing a confidence interval depends not only on the size of the intervals it produces and but on the probability with which those intervals fail to include the true but unknown parameter value. Cohen and Strawderman (1973) and Meeden and Vardeman (1985), among others, have explored the question of admissibility for confidence intervals. Although the results given there are not directly applicable to our case the second paper shows that in some situations certain Bayes procedures can yield almost admissible procedures. These type of arguments along with the fact that the standard interval is way too short gives some circumstantial evidence, we believe, that the WDP intervals in this example are not outrageously too long. To sum up, we believe that in the important special case when the sample sizes are two and the strata are not dramatically different the WDP intervals seem to be a serious competitor for the standard intervals.

7.3 Integrated public use microdata series

The Minnesota Population Center (MPC) is an interdepartmental demography research group at the University of Minnesota. A major goal of the MPC is to create databases and statistical tools which can be utilized in the study of economic and social behavior. One database of interest is the Integrated Public Use Microdata Series (IPUMS), which is a consolidation of U.S. censuses and other national surveys from 1850-present (Ruggles, Sobek, Alexander, Fitch, Goeken, Hall, King and Ronnander 2004). The word microdata is applied in this context because each row of an IPUMS dataset corresponds to one individual or one household; such low-level of detail may be contrasted with a typical Census Bureau publication or online summary table, in which a preset geographic specific tabulation (geography can be the entire country, states, counties, census tracts etc.) of the microdata is given to the data user.

One dataset which offers a rich array of numerical variables is the 2005 American Community Survey (ACS). This Census Bureau product is a large sample survey, and the Census Bureau does not know the true population means for the variables. To conduct simulations with the 2005 ACS, the sample played the role of the population. More specifically, the full population was assumed to be a set of 3,579 Minneapolis residents who are of working age (between 25 and 75), and who earn a yearly wage between $20,000 and $120,000. For our purposes the two variables of interest were:

• $inctot.$ Total pre-tax income from 2004.
• $sei.$ The Duncan Socioeconomic Index. Created in the 1950's, this is a numerical variable which attempts to rate the prestige associated with an individual's occupation. The range of this variable is [1,100].

For our simulations we set $y=\log \left(inctot\right)$ and $x=sei.$ The correlation between $y$ and $x$ is 0.398 and we assume that the mean of $x$ is known. For estimating the population mean of $y$ we considered the estimator based on the WDP and the regression estimator. We used two different designs: simple random sampling and approximate $\text{pps}(x).$In each case we took 300 samples of size 30. The results are given in Table 7.3. We see that although the two methods are comparable the WDP clearly gives the better intervals.

Table 7.3
Simulation results from 300 random samples of size 30 from the IPUMS population. The nominal coverage for each method is 0.95.

Design	Method	Ave. err	Ave. len/2	Freq of coverage
SRS	Reg	0.052	0.128	0.943
	WDP	0.052	0.138	0.947
pps(x)	Reg	0.062	0.132	0.897
	WDP	0.066	0.133	0.937

Previous | Next