Publications

    Survey Methodology

    Browse by

    8 Final remarks

    Jeremy Strief and Glen Meeden

    Previous

    The construction of weights in survey sampling is often more of an art than a science. This is one possible conclusion that can be drawn from the recent paper of Gelman (2007) and the accompanying discussion. He argues for a Bayesian approach to constructing weights using regression models which relate the characteristic of interest to auxiliary variables. Here we argued for a stepwise Bayes approach which will make use of the information present in the auxiliary variables without assuming a model relating the characteristic of interest to the auxiliary variables. The resulting weight for a unit in the sample can be given the usual interpretation as the number of units in the population which it represents.

    A frequentist weight, say w i , MathType@MTEF@5@5@+= feaagKart1ev2aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9LqFf0x e9q8qqvqFr0dXdHiVc=bYP0xb9sq=fFjea0RXxb9qr0dd9q8qi0lf9 Fve9Fve9vapdbaqaaeGacaGaaiaabeqaamaabaabaaGcbaGaam4Dam aaBaaaleaacaWGPbaabeaakiaacYcaaaa@3C18@  is the inverse of an inclusion probability, and this number represents the number of units in the population represented by a particular unit in the sample. So w i ≥1 MathType@MTEF@5@5@+= feaagKart1ev2aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9LqFf0x e9q8qqvqFr0dXdHiVc=bYP0xb9sq=fFjea0RXxb9qr0dd9q8qi0lf9 Fve9Fve9vapdbaqaaeGacaGaaiaabeqaamaabaabaaGcbaGaam4Dam aaBaaaleaacaWGPbaabeaakiabgwMiZkaaigdaaaa@3DE9@  for all i MathType@MTEF@5@5@+= feaagKart1ev2aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9LqFf0x e9q8qqvqFr0dXdHiVc=bYP0xb9sq=fFjea0RXxb9qr0dd9q8qi0lf9 Fve9Fve9vapdbaqaaeGacaGaaiaabeqaamaabaabaaGcbaGaamyAaa aa@3A36@  and ∑ i∈s w i ≈N. MathType@MTEF@5@5@+= feaagKart1ev2aaatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9LqFf0x e9q8qqvqFr0dXdHiVc=bYP0xb9sq=fFjea0RXxb9qr0dd9q8qi0lf9 Fve9Fve9vapdbaqaaeGacaGaaiaabeqaamaabaabaaGcbaWaaabeae aacaWG3bWaaSbaaSqaaiaadMgaaeqaaaqaaiaadMgacqGHiiIZcaWG ZbaabeqdcqGHris5aOGaeyisISRaamOtaiaac6caaaa@43E0@  In Section 6 we saw that for the Horvitz-Thompson estimator the sum of the weights of the units usually fails to equal the population size which can result in a poor estimator except in very special circumstances. Another problem with frequentist weights is that they are often adjusted – MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqefmvESzwyL5 gaiuaajugGbabaaaaaaaaapeGaa83eGaaa@3AAF@  after the sample is collected – MathType@MTEF@5@5@+= feaagKart1ev2aqatCvAUfeBSjuyZL2yd9gzLbvyNv2CaerbuLwBLn hiov2DGi1BTfMBaeXatLxBI9gBaerbd9wDYLwzYbItLDharqqtubsr 4rNCHbGeaGqiVu0Je9sqqrpepC0xbbL8F4rqqrFfpeea0xe9Lq=Jc9 vqaqpepm0xbba9pwe9Q8fs0=yqaqpepae9pg0FirpepeKkFr0xfr=x fr=xb9adbaqaaeGaciGaaiaabeqaamaabaabaaGcbaqefmvESzwyL5 gaiuaajugGbabaaaaaaaaapeGaa83eGaaa@3AAF@  to ensure that the frequentist estimates are in agreement with prior information about the population (Kostanich and Dippo 2002). After making adjustments, the weights may be rescaled so that they sum to a population total. However, the adjusted frequentist weights no longer depend just on the sampling design and they no longer represent inverses of inclusion probabilities. The intuition behind frequentist weights is therefore somewhat confusing. Before adjustments, frequentist weights are functions of the design; but after adjustments, they are now functions of the design and other prior information, which may or may not be related to the design.

    Bayesians think of estimation in survey sampling as a prediction problem. Their predictions are based on an assumed model which can lead to weights being assigned to the units in the sample. See for example the aforementioned Gelman (2007) and Little (2004). As noted by a number of authors (Pfeffermann 1993) performing a weighted analysis for a model using inverses of the inclusion probabilities can protect the sampler from model misspecification. Moreover in certain situations the two approaches may lead to similar results.

    Recently, Rao and Wu (2010) have developed methods which use a pseudo empirical likelihood approach and base their inferences on Dirichlet posterior distributions. The resulting procedures, although formally somewhat similar to some discussed here, use prior information in a different way. For them much of the prior information must be filtered through the design while we believe that prior information which is often included in the design can be used directly to generate good posteriors. For better or worse we are closer to the classical Bayesian scenario where the posterior distribution does not depend on the sampling design.

    Here we have focused on using the CPP to generated a set weights based on the sample and prior information and then making our inferences using the WDP based on these weights. Strief (2007) considered examples where the weights generated by the CPP were instead used in the appropriated frequentist formulas to get an estimate of variance and noted that their performance was similar to standard methods. Alternately one could imagine basing their inferences on the WDP but using frequentist weights, say generated by calibration methods (Särndal and Lundström 2005), instead. Although this deserves further study it is our expectation that such approaches should lead to inferential procedures with good frequentist properties.

    In the design based approach consistency is an important property for an estimator to possess. For an important special case when the design is SRS the CPP estimators are consistent. This is demonstrated in Geyer and Meeden (2013).

    Just as the CPP does, the WDP also has a stepwise Bayes justification. (For more details see Strief (2007).) The weights used in the WDP have a consistent formulation and interpretation. They are always a posterior expectation and always sum to the population size. They represent the average number of times that each unit in the sample appears in a simulated, completed copy of the population under the CPP. This average is with respect to the uniform distribution over all possible copies of the population which just contain the units in the sample and which satisfy the given constraints. These weights depend only on the same kinds of objective prior information about the population which are often used to define and adjust frequentist weights. This allows them to incorporate prior information without explictly specifying a prior distribution.

    In most cases the weight assigned to a unit in the sample will depend on the other units in the sample. We have argued that after the sample has been selected one should argue conditionally. That is, given the sample the weights should depend on all the available prior information about the population but not on how it was selected. (We are assuming that the person selecting the sample and the analyst are one in the same.) Any procedure constructed in this manner should preform well for a variety of sampling designs. For any procedure, be it either frequentist, Bayesian or stepwise Bayes this is the litmus test: it should be evaluated by how it behaves under repeated sampling from the design of interest.

    To implement the methods discussed here one first needs to use the CPP to computed the weights for the observed sample. Then one needs to use the weights in the WDP to simulate complete copies of the population. The first step is the more difficult although the software package polyapost makes it relatively straightforward for anyone familiar with R. Once the weights are known it is easy to simulate from the WDP in many computer packages. This makes our approach more practical for survey datasets (like IPUMS) which are presented with the weights attached and are used by multiple researchers. A more serious limitation is that we have only considered simple single stage sampling designs. More work needs to be done to extend these methods to more complicated multi-stage designs. If the underlying constraints are selected wisely the resulting procedures can have good frequentist properties for a variety of sampling designs. These stepwise Bayes weights can be thought as our best guess for the unknown population given the sampled units and our prior information.

    Acknowledgements

    Research supported in part by NSF Grant DMS 0406169.

    References

    Binder, D. (1982). Non-parametric Bayesian models for samples from a finite population. Journal of the Royal Statistical Society, Series B, 44, 388-393.

    Booth, J.G., Bulter, R.W. and Hall, P. (1994). Bootstrap methods for finite population sampling. Journal of the American Statistical Association, 89, 1282-1289.

    Cohen, A., and Strawderman, W. (1973). Admissible confidence interval and point estimation for translation of scale parameters. Annals of Statistics, 1, 545-550.

    Gelman, A. (2007). Struggles with survey weighting and regression modeling (with discussion). Statistical Science, 22, 153-188.

    Geyer, C., and Meeden, G. (2013). Asymptotics for constrained Dirichlet distributions. Bayesian Analysis, 8, 89-110.

    Ghosh, M., and Meeden, G. (1997). Bayesian Methods for Finite Population Sampling. Chapman and Hall, London.

    Gross, S. (1980). Median estimation in survey sampling. In Proceedings of the Survey Research Methods Section, American Statistical Association, 181-184.

    Hartley, H.O., and Rao, J.N.K. (1968). A new estimation theory for sample surveys. Biometrika, 55, 159-167.

    Hu, M., Zhang, F., Cohen, M. and Salvucci, S. (1997). On the performance of replication-based variance estimation methods with small number of psus. In Proceedings of the Survey Research Methods Section, American Statistical Association.

    Kostanich, D.L., and Dippo, C.S. (2002). Design and methodology: 63rv. Technical report, The U.S. Census Bureau and The Department of Labor Statistics.

    Lazar, R., Meeden, G. and Nelson, D. (2008). A noninformative Bayesian approach to finite population sampling using auxiliary variables. Survey Methodology, 34, 1, 51-64.

    Little, R.J. (2004). To model or not to model? Competing modes of inference for finite poplation sampling. Journal of the American Statistical Association, 99, 546-556.

    Lo, A. (1988). A Bayesian bootstrap for a finite population. Annals of Statistics, 16, 1684-1695.

    Meeden, G., and Vardeman, S. (1985). Bayes and admissible set estimation. Journal of the American Statistical Association, 80, 465-471.

    Pfeffermann, D. (1993). The role of sampling weights when modeling survey data. International Statistical Review, 61, 317-337.

    Rao, J.N.K., and Wu, C. (2010). Bayesian pseudo empirical likelihood intervals for complex surveys. Journal of the Royal Statistical Society, Series B, 72, 533-544.

    R Development Core Team (2005). R: A language and environment for statistical computing. R Foundation for Statistical Computing, www.R-project.org.

    Royall, R., and Cumberland, W. (1981). An empirical study of the ratio estimator and estimators of its variance. Journal of the American Statistical Association, 71, 657-664.

    Royall, R., and Cumberland, W. (1985). Conditional coverage properties of finite population confidence intervals. Journal of the American Statistical Association, 80, 355-359.

    Rubin, D. (1981). The Bayesian bootstrap. Annals of Statistics, 9, 130-134.

    Ruggles, S., Sobek, M., Alexander, T., Fitch, C.A., Goeken, R., Hall, P.K., King, M. and Ronnander, C. (2004). Integrated public use microdata series: Version 3.0 [machine-readable database]. University of Minnesota.

    Särndal, C.-E., and Lundström, S. (2005). Estimation in Surveys with Nonresponse. New York: John Wiley & Sons, Inc.

    Särndal, C.-E., Swensson, B. and Wretman, J. (1992). Model Assisted Survey Sampling. New York: Springer.

    Strief, J. (2007). Bayesian Sampling Weights: Toward a Practical Implementation of the Polya Posterior. Ph.D. thesis, University of Minnesota.

    Zheng, H., and Little, R. (2003). Penalized spline model-based estimation of finite population total from probability-proportional-to-size samples. Journal of Official Statistics, 19, 99-117.

    Zheng, H., and Little, R. (2004). Penalized spline nonparametric mixed models for inference about a finite population mean from two-stage samples. Survey Methodology, 30, 2, 209-218.

    Zheng, H., and Little, R. (2005). Inference for the population total from probability-proportional-to-size samples based on predictions from a penalized spline nonparametric model. Journal of Official Statistics, 21, 1-20.

    Previous

    Date modified: