2. Background

Yan Lu

Previous | Next

2.1 Chi-squared tests in a single frame survey

Consider a one-way frequency table with $k$ classes and associated finite population proportions $p_1\mathrm{,}{p}_2\mathrm{,}\cdots \mathrm{,}{p}_k$ with ${\displaystyle {\sum }_{i=1}^{i=k}}{p}_i=1.$ Let $n_1\mathrm{,}\cdots \mathrm{,}{n}_k$ denote the observed cell frequencies in a sample falling in each of $k$ categories with ${\displaystyle {\sum }_{i=1}^k{n}_i}=n.$ Under SRS, the Pearson chi-squared statistic for testing simple hypothesis $H_0:p_i=p_{0i}\mathrm{,}\left(i=\mathrm{1,}\cdots \mathrm{,}k\right)$ is given by

$${\tilde{X}}^2={\displaystyle \sum _{i=1}^k}\frac{{\left(n_i-n{p}_{0i}\right)}^2}{n{p}_{0i}}\mathrm{.}\text{ (2}\text{.1)}$$

For complicated designs, ${\tilde{X}}^2$ involve noncentral distributions. It is natural to consider a more general statistic

$$X^2=n{\displaystyle \sum _{i=1}^k}\frac{{\left({\widehat{p}}_i-p_{0i}\right)}^2}{p_{0i}}\mathrm{,}\text{ (2}\text{.2)}$$

where ${\widehat{p}}_i$ is a consistent estimator of $p_i$ under a specified sampling design $p(s).$

Let $\widehat{p}={\left({\widehat{p}}_1\mathrm{,}\cdots \mathrm{,}{\widehat{p}}_{k-1}\right)}^{\prime }$ represent the $k-1$ vector of estimated proportions with ${\widehat{p}}_k=1-\left({\widehat{p}}_1+\cdots +{\widehat{p}}_{k-1}\right);$ $p_0$ be the corresponding $k-1$ vector of hypothesized proportions; $V$ be the $(k-1)\times (k-1)$ covariance matrix of $\widehat{p},$ and $\widehat{V}$ be the estimate of $V$ obtained from the survey data. The generalized Wald statistic

$$X_W^2={\left(\widehat{p}-{\widehat{p}}_0\right)}^{\prime }{\widehat{V}}^{-1}\left(\widehat{p}-{\widehat{p}}_0\right)\mathrm{,}\text{ (2}\text{.3)}$$

is distributed asymptotically as ${\chi }_{k-1}^2$ under $H_0:p_i=p_{0i}\mathrm{,}\left(i=1,\cdots \mathrm{,}k\right)$ for sufficiently large $n.$

Rao and Scott (1981) showed that under $H_0,$ $X^2$ in (2.2) is distributed asymptotically as a weighted sum ${\delta }_1{W}_1+\cdots +{\delta }_{k-1}{W}_{k-1}$ of $k-1$ independent ${\chi }_1^2$ random variables $W_i\mathrm{,}i=1,2,\cdots \mathrm{,}k-1.$ The ${\delta }_i$ s are the eigenvalues of a design effect matrix $P^{-1}V,$ where $P$ is the covariance matrix corresponding to SRS when $H_0$ is true, i.e. $P=n^{-1}\left(diag\left(p_0\right)-p_0{p^{\prime }}_0\right).$ The standard result of Pearson test is recovered under SRS. Let ${\widehat{\delta }}_i$ be an estimate of ${\delta }_i$ and $\widehat{\delta }.=\left({\displaystyle {\sum }_{i=1}^{k-1}}{\widehat{\delta }}_i\right)/\left(k-1\right),$ the Rao-Scott first order corrected test refers $X^2/\widehat{\delta }\mathrm{.}$ to ${\chi }_{k-1}^2.$ When the full estimated covariance matrix $\widehat{V}$ is known, a better approximation to the asymptotic distribution of $X^2$ is to match the first moment and second moment of the test statistic to a ${\chi }^2$ distribution. The Rao-Scott (Rao and Scott 1981) second-order corrected test statistic considers $X_S^2=X^2/\left[\widehat{\delta }\mathrm{.}\left(1+{\widehat{a}}^2\right)\right].$ This statistic is approximately a chi-squared random variable on $v=\left(k-1\right)/\left(1+a^2\right)$ degrees of freedom, where $\widehat{a}$ is an estimate of $a$ with ${\widehat{a}}^2={\displaystyle {\sum }_{i=1}^{k-1}}{\widehat{\delta }}_i^2/\left[\left(k-1\right)\widehat{\delta }{\mathrm{.}}^2\right]-\mathrm{1,}$ and ${\displaystyle {\sum }_{i=1}^{k-1}}{\widehat{\delta }}_i^2=n^2{\displaystyle {\sum }_{i=1}^k}{\displaystyle {\sum }_{j=1}^k}{\widehat{V}}_{ij}^2/p_{0i}{p}_{0j}\mathrm{.}$ If the design effects are all similar, the first and second-order corrections will behave similarly. Otherwise, the second order correction almost always performs better.

2.2 Framework of chi-squared tests and pseudo maximum likelihood estimator in dual frame surveys

The set up in this section follows from Hartley (1962) and Lu and Lohr (2010). Assume there are $k$ categories in both surveys and the same quantities are measured. Let $p_{id}$ be the population proportion of category $i$ in domain $d$ (domain $d$ can be domain $a,$ domain $ab$ or domain $b$ ), with ${\displaystyle {\sum }_{i=1}^k{p}_{id}}=1.$ Let $N_a,N_{ab}$ and $N_b$ denote the population sizes of the three domains respectively, with $N_a+N_{ab}=N_A$ and $N_b+N_{ab}=N_B.$ We consider the common case that $N_{ab}$ is unknown, while $N_A$ and $N_B$ are constants. As a result, ${\displaystyle {\sum }_{i=1}^k{p}_{ia}{N}_a/N_A}+{\displaystyle {\sum }_{i=1}^k{p}_{iab}{N}_{ab}/N_A}=1$ and ${\displaystyle {\sum }_{i=1}^k{p}_{ib}{N}_b/N_B}+{\displaystyle {\sum }_{i=1}^k{p}_{iab}{N}_{ab}/N_B}=1$ (see Figure 2.1 for illustration of the proportions). The vector of proportions $p={\left(p_1\mathrm{,}{p}_2\mathrm{,}\cdots p_{k-1}\right)}^{\prime }$ for the union of the two frames is a function of the parameters $p_{ia}\mathrm{,}{p}_{iab}\mathrm{,}{p}_{ib}\text{ and }{N}_{ab}.$ For example, a natural form of $p_i$ is

$$p_i=\frac{N_a}{N}{p}_{ia}+\frac{N_{ab}}{N}{p}_{iab}+\frac{N_b}{N}{p}_{ib}\mathrm{,}\text{ for }i=1,2,\cdots \mathrm{,}k-\mathrm{1,}\text{ (2}\text{.4)}$$

where $N=N_A+N_B-N_{ab}.$

Figure 2.1: Population proportion in domains and frames

Description for Figure 2.1

In the following, we briefly review the pseudo maximum likelihood estimator that we will use in Section 4 and Section 5. Assume independent simple random samples are taken from frames $A$ and $B$ respectively. The likelihood function is

$$L\left(p_{ia}\mathrm{,}{p}_{iab}\mathrm{,}{p}_{ib}\mathrm{,}{N}_{ab}\right)\propto {\displaystyle \prod _i}{\left(p_{ia}\frac{N_a}{N_A}\right)}^{x_{ia}}\times {\displaystyle \prod _i}{\left(p_{iab}\frac{N_{ab}}{N_A}\right)}^{x_{iab}^A}\times {\displaystyle \prod _i}{\left(p_{ib}\frac{N_b}{N_B}\right)}^{x_{ib}}\times {\displaystyle \prod _i}{\left(p_{iab}\frac{N_{ab}}{N_B}\right)}^{x_{iab}^B}\text{ (2}\text{.5)}$$

where $x_{ia}\mathrm{,}{x}_{ib}$ represent the units falling in category $i$ within domain $a$ and domain $b$ respectively; $x_{iab}^A$ and $x_{iab}^B$ represent the units falling in category $i$ within the overlapping domain $ab$ that are originally sampled from frame $A$ and frame $B$ respectively.

For the estimators of complex surveys, the basic idea is to use a working assumption of a multinomial distribution from a finite population to give the form of the estimators and use a design effect to adjust the cell counts to reflect the complex survey design. The pseudo likelihood function is as follows

$$\begin{array}{cc}L\left(p_{ia}\mathrm{,}{p}_{iab}\mathrm{,}{p}_{ib}\mathrm{,}{N}_{ab}\right)& \propto {\displaystyle \prod _i}{\left(p_{ia}\frac{N_a}{N_A}\right)}^{\frac{{\tilde{n}}_A}{N_A}{\widehat{X}}_{ia}}{\displaystyle \prod _i}{\left(p_{iab}\frac{N_{ab}}{N_A}\right)}^{\frac{{\tilde{n}}_A}{N_A}{\widehat{X}}_{iab}^A}\text{ (2}\text{.6)}\hfill \\ & \times {\displaystyle \prod _i}{\left(p_{ib}\frac{N_b}{N_B}\right)}^{\frac{{\tilde{n}}_B}{N_B}{\widehat{X}}_{ib}}{\displaystyle \prod _i}{\left(p_{iab}\frac{N_{ab}}{N_B}\right)}^{\frac{{\tilde{n}}_B}{N_B}{\widehat{X}}_{iab}^B}\mathrm{,}\hfill \end{array}$$

where design effect is defined as $\left\{v\left(\widehat{\theta }\right)\text{ from complex survey}\right\}/\left\{v\left(\widehat{\theta }\right)\text{ from SRS of same size}\right\},$ ${\tilde{n}}_A=n_A/\left(\text{design effect of }{\widehat{N}}_{ab}^A\right),$ ${\tilde{n}}_B=n_B/\left(\text{design effect of }{\widehat{N}}_{ab}^B\right),$ $n_A$ and $n_B$ are the observed sizes of $S_A$ and $S_B,$ and ${\widehat{X}}_{id}$ denote the estimated counts according to the survey design. The pseudo maximum likelihood estimators (PMLEs), found by maximizing (2.6) are ${\widehat{p}}_{ia}={\widehat{X}}_{ia}/{\widehat{N}}_a\mathrm{,}$ ${\widehat{p}}_{ib}={\widehat{X}}_{ib}/{\widehat{N}}_b\mathrm{,}$ and

$${\widehat{p}}_{iab}=\frac{\frac{{\tilde{n}}_A}{N_A}{\widehat{N}}_{ab}^A{\widehat{p}}_{iab}^A+\frac{{\tilde{n}}_B}{N_B}{\widehat{N}}_{ab}^B{\widehat{p}}_{iab}^B}{\frac{{\tilde{n}}_A}{N_A}{\widehat{N}}_{ab}^A+\frac{{\tilde{n}}_B}{N_B}{\widehat{N}}_{ab}^B}\mathrm{,}\text{ (2}\text{.7)}$$

where ${\widehat{p}}_{iab}^A={\widehat{X}}_{iab}^A/{\widehat{N}}_{ab}^A$ and ${\widehat{p}}_{iab}^B={\widehat{X}}_{iab}^B/{\widehat{N}}_{ab}^B,$ and ${\widehat{N}}_{ab\mathrm{,}PML}$ is the smaller root of the quadratic function

$$\left[{\tilde{n}}_A+{\tilde{n}}_B\right]{\widehat{N}}_{ab\mathrm{,}PML}^2-\left[{\tilde{n}}_A{N}_B+{\tilde{n}}_B{N}_A+{\tilde{n}}_A{\widehat{N}}_{ab}^A+{\tilde{n}}_B{\widehat{N}}_{ab}^B\right]{\widehat{N}}_{ab\mathrm{,}PML}+\left[{\tilde{n}}_A{\widehat{N}}_{ab}^A{N}_B+{\tilde{n}}_B{\widehat{N}}_{ab}^B{N}_A\right]=0.\text{ (2}\text{.8)}$$

The estimators of the population proportions are

$${\widehat{p}}_{i\mathrm{,}PML}=\frac{\left(N_A-{\widehat{N}}_{ab\mathrm{,}PML}\right){\widehat{p}}_{ia}+{\widehat{N}}_{ab\mathrm{,}PML}{\widehat{p}}_{iab}+\left(N_B-{\widehat{N}}_{ab\mathrm{,}PML}\right){\widehat{p}}_{ib}}{N_A+N_B-{\widehat{N}}_{ab\mathrm{,}PML}}\mathrm{.}\text{ (2}\text{.9)}$$

If SRSs are taken in each frame and $k=1,$ these PMLEs reduced to PMLEs in Skinner and Rao (1996).

Previous | Next