A method to correct for frame membership error in dual frame estimators
Section 7. Discussion
An important observation from this application was that respondents may have difficulties providing accurate information about domain membership, even when it is defined by a straightforward concept, like having a fishing license. This poses a particular problem for dual frame estimation. When domain membership is available from other sources with minimal cost, there is a simple solution. But when the cost is high, it may be beneficial to use a bias-correction method. Another observation that was important to our application was that the only previously available method that we were aware of required an unstated assumption that units are homogeneous within true domains. This would seem to be a natural and benign assumption for many applications, but appeared not to hold for the key item in our questionnaire.
In this study, the reason for the domain misclassification error was that the respondent was unable or unwilling to provide the correct information. In other cases, it could be that the frame itself contains errors. In either case, identifying the mechanism causing the error could help to determine which assumption about means is most plausible. In our application, it seems obvious in retrospect that a person who admits to frequent fishing might be reluctant to admit to a government agency conducting the survey that they are not licensed.
Many applications of dual frame estimation are for the purpose of improving efficiency rather than coverage, as in our application, where the license frame was included to reduce the cost of contacting anglers. In that case, membership in one of the frames is likely to be predictive of key response variables in the survey and so means of the responses for the subgroups of the population may vary widely. However, this is not always the case. If neither frame is directly related to the topic of the survey itself, it may be more likely that the true domain determines the mean response, or even that all four subgroups of the population have the same mean. (In the latter case, neither bias-correction method would be incorrect.) For example, suppose the two frames for this survey were land-line and cell-phone frames, and a respondent sampled from the cell frame, for example, is asked if they have a land-line phone in order to determine if they are in the overlap domain. Responses to this question are likely to have some measurement error. However, it seems unlikely that whether a respondent says he or she has a land-line is more predictive of angling avidity than whether he or she actually has a land-line. (In fact the latter will probably be related to fishing avidity because both are correlated with age.) Predicting which mean assumption will hold will benefit from the advice of experts on the topic of the survey. However, in the end, examining the data from the survey itself before a decision is made about bias-correction will be necessary.
The cost of bias-correction is increased variance. This penalty is significant, especially if little information is available about the misclassification rates. Therefore, if it can be determined that there are few errors, either because the domains subject to errors are a small fraction of the population or the error rate is very small, then bias-correction may not be worthwhile. Calculating the bias-corrected estimators is straightforward with survey software, once misclassification estimates are available. Their variance estimates, along with the difference in bias-corrected and not bias-corrected estimates themselves can help guide the choice.
Our research was done in the context of Hartley’s dual frame estimator since that was the estimator being used in our application. Many different dual-frame estimators are available, and all require knowledge of domain membership. Some, such as the Fuller-Burmeister estimator, could be adjusted using methods similar to those outlined here. Others, such as the pseudo-maximum likelihood estimators, would require a different approach.
References
Andrews, R., Brick, J.M., Mathiowetz, N. and Stokes, L. (2010). Pilot test of a dual frame two-phase mail survey of anglers in North Carolina. Final report for National Oceanic and Atmospheric Administration.
Bankier, M. (1986). Estimators based on several stratied samples with applications to multiple frame surveys. Journal of the American Statistical Association, 81, 1074-1079.
Fuller, W., and Burmeister, L. (1972). Estimators for samples selected from two overlapping frames. In Proceedings of the Social Statistics Section, American Statistical Association, 245-249.
Hartley, H. (1962). Multiple frame surveys. In Proceedings of the Social Statistics Section, American Statistical Association, 203-206.
Kalton, G., and Anderson, D. (1986). Pilot test of a dual frame two-phase mail survey of anglers in North Carolina. Journal of the Royal Statistical Society, Series A, 149, 65-82.
Lin, D. (2014). Dissertation: Measurement Error in Dual Frame Estimation.
Lohr, S. (2011). Alternative survey sample designs: Sampling with multiple overlapping frames. Survey Methodology, 37, 2, 197-213. Paper available at https://www150.statcan.gc.ca/n1/pub/12-001-x/2011002/article/11608-eng.pdf.
National Research Council (NRC) (2006). Review of recreational sheries survey methods. Technical report.
Skinner, C., and Rao, J. (1996). Estimation in dual frame surveys with complex designs. Journal of the American Statistical Association, 91, 349-356.
- Date modified: