The anchoring method: Estimation of interviewer effects in the absence of interpenetrated sample assignment
Section 4. Simulation study

• Previous
• Next

We first consider an empirical simulation study of the proposed “anchoring” approach. We repeatedly simulated samples of data from a quadrivariate normal distribution, $\left(Y_{1ij}^{*}{Y}_{2ij}^{*}{Y}_{3ij}^{*}{Z}_{ij}\right)~N_4\left(\mu ,\Sigma \right),$ where $j=1,\dots ,J=30$ indexes hypothetical respondents nested within $i=1,\dots ,I=30$ interviewers, $Y_{kij\left(z\text{}\right)}=Y_{kij\left(z\text{}\right)}^{*}+I\left(k=3\right)b_i$ for $b_i~N\left(0,{\sigma }_b^2\right),$ and $Y_{kij\left(z\text{}\right)}^{*}$ is ordered by the values of $Z_{ij}$ prior to assignment of respondents to the 30 interviewers. $Y_{1ij\left(z\text{}\right)}$ and $Y_{2ij\left(z\text{}\right)}$ are the observed values without interviewer-induced measurement error, while $Y_{3ij\left(z\text{}\right)}$ is observed with interviewer-induced measurement error, and $Z_{ij}$ is a (nuisance and unobserved) covariate that induces extraneous variability when the design is treated as interpenetrated. (One might think of $Y_1$ and $Y_2$ as measurement-error free demographic variables and $Y_3$ as a continuous self-reported overall health measure, which is potentially prone to interviewer effects, and $Z$ as amount of time spent at home, which is associated with interviewer scheduling by shift.)

Given this data generating model, we note that a higher correlation of $Z$ with the other measurements will introduce what appears to be interviewer variance because of the ordering of $Y_{kij\left(z\text{}\right)}^{*}$ by the values of $Z_{ij}$ above and beyond the true random interviewer effects on $Y_2$ (given by $b_i).$ This is the lack of interpenetrated assignment that we wish to adjust for with the proposed anchoring method, which aims to isolate the unique interviewer variance ${\sigma }_b^2$ that does not arise from simple assignment of cases to interviewers. For simplicity, we assume that ${\mu }_{Y_1}={\mu }_{Y_2}={\mu }_{Y_3}={\mu }_z=\mu ,$ ${\sigma }_{Y_1}^2={\sigma }_{Y_2}^2={\sigma }_{Y_3}^2={\sigma }_Z^2=1$ and ${\rho }_{Y_1{Y}_2}={\rho }_{Y_1{Y}_3}={\rho }_{Y_{1}Z}={\rho }_{Y_2{Y}_3}={\rho }_{Y_{2}Z}={\rho }_{Y_{3}Z}=\rho .$

We consider four models used to estimate the mean of $Y_3$ and the associated interviewer effect variance:

Unadjusted: $Y_{3ij}~N\left({\mu }_3+b_i,{\sigma }_3^2\right)$

Adjusted: $Y_{3ij}~N\left({\mu }_3+{\beta }_1{y}_{1ij}+{\beta }_2{y}_{2ij}+b_i,{\sigma }_3^2\right)$

Anchoring: $\left(\begin{array}{c}{Y}_{1ij}\\ Y_{2ij}\\ Y_{3ij}\end{array}\right)~N_3\left(\left[\begin{array}{c}{\mu }_1\\ {\mu }_2\\ {\mu }_3+b_i\end{array}\right],\Sigma \right)$

Anchoring-Linear Predictor: $\left(\begin{array}{c}{\widehat{Y}}_{3ij}\\ Y_{3ij}\end{array}\right)~N_2\left(\left[\begin{array}{c}{\mu }_1\\ {\mu }_2+b_i\end{array}\right],\Sigma \right)$

where $y_{kij}$ is the observed realization of $Y_{kij},$ $b_i~N\left(0,{\sigma }_b^2\right),$ and, in the anchoring-linear predictor model, ${\widehat{Y}}_{3ij}={\widehat{\beta }}_0+{\widehat{\beta }}_1{y}_{1ij}+{\widehat{\beta }}_2{y}_{2ij}$ where $\widehat{\beta }$ is obtained from the linear regression of $Y_3$ on $Y_1$ and $Y_2.$ We estimate the mean of $Y_3$ as the REML estimator of ${\mu }_3$ and similarly the associated interviewer effect variance as the REML estimator of ${\sigma }_b^2.$

We consider the power to reject the null hypothesis that the mean of the observed variables is zero (at the 0.05 level) and the empirical bias in the estimation of the variance of the random interviewer effects, ${\sigma }_b^2.$ We evaluated the empirical bias by computing the difference between the mean of the simulated estimates of the variance component and the true value of the variance component specific to a given simulation scenario. Our simulation study design considers a full factorial design where $\mu =\left\{0,\text{0}\text{.5}\right\},$ $\rho =\left\{\text{0}\text{.25},\text{0}\text{.5},\text{0}\text{.75}\right\},$ and ${\sigma }_b^2=\left\{\text{0}\text{.1},\text{0}\text{.5},\text{0}\text{.9}\right\}.$ We generated 200 independent simulations for each of the 18 cross-classifications of values on these parameters. Table 4.1 presents the results of the simulation study.

Several notable patterns emerge from the simulation results in Table 4.1. First, as the values of $\rho$ increase, the anchoring method produces larger reductions in the overestimation of interviewer variance relative to the unadjusted model. Recall that this was expected by design, given the initial ordering of the observations by $Z$ prior to assignment to interviewers, which introduces artificial variance among the interviewers. Similarly, as anticipated, estimation of the interviewer variance using covariate adjustment is similar to the anchoring method when this variance is not large, although there is evidence of a somewhat larger reduction in bias when the variance is large.

In addition, for the non-zero values of $\mu ,$ higher values of $\rho$ yield larger improvements in power when using the anchoring method when compared with the unadjusted estimator, since more of the extraneous variance is correctly allocated. Both the unadjusted and an anchoring method yield higher power than the adjusted estimator, since the adjusted estimator is biased for non-zero means of $Y_{1ij}$ and $Y_{2ij}$ when they are correlated with $Y_{3ij}.$ Smaller values of $\rho$ approximate an interpenetrated design, and as a result, the unadjusted estimation approach does not produce substantially different results from the adjusted or anchoring approach. The empirical bias in the estimation of ${\sigma }_b^2$ is unrelated to the value of ${\sigma }_b^2$ but is entirely a function of $\rho ,$ since that drives the spurious within-interviewer correlation due to the unobserved $Z.$ Finally, we note that replacing the actual values of $Y_{1ij}$ and $Y_{2ij}$ with a summary measure based on their linear prediction of $Y_{3ij}$ yields virtually identical results to their direct use in the anchoring method. This is partly a function of their common normality; we discuss this limitation in the Discussion section below.

• Previous
• Article main page
• Next